I'd be freaked out too. I can't find anything that says the leaves are toxic to chickens, but apparently they are very toxic to cows. Cows would tend to eat the leaves, but I doubt chickens would. Especially well fed chickens. Maybe for peace of mind, you can replace the tree? I tend to go overboard myself, when it comes to my dogs. Anything that could possibly hurt them has to go. I'm not sure that there really is anything to worry about here, but if you are going to worry about it all the time, it isn't worth keeping it. Learned something new about those leaves! I had no idea!
Where in Idaho did you move to? Sounds like my house, with the wind!