Look at the area your daylilies are in. Peonies can live in a lot of conditions—extremely acidic/alkaline might be challenging, but the only truly bad as in will kill them is full shade and boggy conditions. I'm sure your garden doesn't thaw out a lot during winter but check the area you had them planted in; does it get really wet and stay that way for a long time in early spring? I used to garden in zone 4b (CO at 6,700 fly altitude) and had some peonies that never thrived, finally realized after being nudged in that direction by a member here that my spring runoff for snowmelt sat right on top of that area! It was fine the rest of the year but super boggy when the peony roots were dormant and just waking for the spring and they were rotting instead of growing.
People are actually advised to put dark blooming peonies in part shade to keep full color longer. Even the 2 hours of full sun combined with partial shade should be enough to provide some blooms. I'd do as Anya suggested, clear roots out of a large area. Make sure they're planted so the top of the root is 2" deep—you don't want them shallow in your zone as you're actually pushing it on the cold side for them to survive already. Make sure the spot never gets boggy—damp for a few days after rain or with spring thaw is fine but routinely staying really wet isn't. Good landscape fabric would provide a small amount of insulation as well as keeping weeds down, know lots of people hate it but it probably would help in your climate. Pick a variety that is known to be vigorous like Sarah Bernhardt or Festiva Maxima for the experiment, I wouldn't start with a hybrid but just straight lactiflora that has been around and thriving for ages. See if you can get them to grow well and bloom before trying to extend the season with different types. Good luck!
