I would ask the neighbors what happened if they saw anything. It's a bit weird to say the least. Maybe a neighbor thought they were helping even though I agree that they have no business going in your yard and doing all that without permission. The lilies will be fine next year.
