I have stopped and told people how much I enjoy driving by their gardens. You should be flattered, not embarrassed.
We had a business on a busy highway near a big intersection, and I landscaped it with cannas and daylilies and other really gaudy plants. When we moved the business, I dug up some of the plants and left the rest. People came in and asked if they could dig, and some stopped to tell me how sad they were that we were moving.
I was always embarrassed when people honked when I was out there weeding. Don't know if it was because my butt was up in the air or if they knew me or what.