I wonder if lack of blooms could be from a nutrient deficiency; have you fertilized your Carolina Allspice (Calycanthus floridus) in the three years since it's been planted? Maybe late fall or early next spring you can give it a good feeding which will possibly help in production of blooms next year.
~ I'm an old gal who still loves playing in the dirt!
~ Playing in the dirt is my therapy ... and I'm in therapy a lot!