I don't think so Jai. I think you will be fine. I think sometimes we humans make to much to-do over pickety little things. I look at it this way ~~~ hang the nest boxes; the birds know if they should nest in them or not. They know better than we.
All my nest boxes face south. The bluebirds don't care & both boxes get a lot of sun. We have fledglings in abundance every year.