Debra, I don't know if you have milkweed bugs in your area, but they drive me nuts. They are red and black, and show up in droves for the seeds and pods. They destroy the pods and seeds. They're not attracted to the plant itself.
See photos here
I don't usually allow mine to set pods for the simple purpose of avoiding thousands of milkweed bugs. Also, if the plant flowers early enough, and seed pods are removed, it should rebloom in the same season.
Karen