This is a hardback picture book we read in the 1980s from the library. One garden is transformed (I think topiary is involved), then gradually other gardens in the town. Drawings are like wonderful aerial maps of the transformation. I recall streets being very rectangular and regular in shape.
Sounds like Chris van Allsburg’s “The Garden of Abdul Gasazi.”