Can someone cut down a tree on my property with just a visitors approval?

My ex husband was at my house painting for me, he doesn`t live here and has never lived here, I have owned this house 12 years, I was sleeping because I work nights and my neighbour saw him outside and came and asked him if he could cut a tree on my property, my ex said sure, the neighbour knows he is not the owner of the house, do I have any recourse for him coming on my property and destroying it or was he entitled to seeing as someone on the property even though they dont live here or own the property said ok?