Jon, here are some tips I have learned from various hybridizers over the years...
Up here in Ohio, I try to get all my pollenating done by no later than 10 in the morning. 11 for plants that are in filtered shade. A few people have luck later in the day, but most have better luck early in the day. Usually, by 9 a.m. today's pollen is fluffed and viable. Before then, I use pollen from the day before or from the freezer.
I have the best luck using today's pollen on flowers pollinated mid morning in filtered shade.
My understanding is..... There is fluid inside the tube inside the pistle. You want your pollen to be on the tip of the outside of the pistle already when the heat of the day startes to dry the fluid inside the pistle. That drying is what carries the pollen down the tube. If you are waiting until 5 p.m., the fluid is probably already dry. That means there is no way for your pollen to travel down the tube. How hot, humid, and even if it is overcast will affect pollination success.
After 10-11 a.m., I collect pollen to use the next morning. Keep it inside, in air conditioning if possible. Don't keep all the anthers in a clump, make sure they are spread out enough so they can get air and dry out a bit. I like to use disposable contact lens cases that I buy from my eye doctor to put pollen in. That way, if I don't use it fresh, it is ready to pop in the freezer, after it sits a day to dry. I keep just the anthers, not the whole stamen, then I use hemastat tweezers to grasp the anther and apply the pollen.
Another trick, if you want to use today's pollen but it is in the morning, and the pollen is not open and fluffy yet... Go ahead and pick the anthers, put them in a little dish or contact lens cup, and put them under the light of a desk lamp. They need to be close enough to the light so they get a touch of warmth from the light. It takes a combination of the warmth and light to peed up the ripening of the pollen. It won't take long... Maybe a half hour... And the pollen should open up and be ready to use that morning.
After I am done collecting pollen, the next thing I do is use scissors to cut the blooms off that I pollinated the day before. Leave an inch or so of the base of the flower on the scape where the pod forms. Cut right at the base of the flower, but leave the little "stalk" the flower attaches to the scape with. The reason you do this is so that the heavy, old bloom won't pull off a small pod that is forming. Sometimes those old blooms, especially if they are from really large or really full blooms, will get heavy with rain or from watering, or even just the "goo" when they wilt... And they become like glue and are really heavy! If this is confusing, I should have a few blooms open in a couple days, and I can take a photo of how and where to cut for you.
