I think that they are just a bit too crowded by competing plants - including one another. The planting halo for peonies should be 2' feet. They don't get enough sunlight early enough so get some mold going, and then have a hard time advancing/improving as a result. Peonies are hard to kill, but pretty easy to stall.
Should you spread them out, try building a little mound maybe with a touch of sand for at least one of them. The idea here is to improve drainage a bit to see if that is the direction you need to go.
Top