Disinfect tools between plants and wash gloves, too.
Cutting away all finished peony plant material at the end of the season and putting it in bags for the household trash really helps to keep it at bay. This is really the only thing I'm more than a bit fanatic about in the gardens here.
I use an old pair of dish washing gloves for this, and toss them afterward.