Though I realize you want plants that return every year, treat yourself to a few impatiens to provide you with some blooms. They would help fill in the vacancies as the perennials grow.
This is one of my very small gardens but it's my favorite and it's in shade from noon to the early morning.