Neighborhood harassment

Hi, I am a mother of two year old son. He has a habit of hitting children younger than him. May be out of annoyance or boredom or just for fun. I've tried everything I can to get rid of this habit of his. Now my neighbor has a daughter of one and half years. Whenever my son is playing outside if he sees that girl he just runs to her and tries to hit her. But I somehow manage not to let him near her. But the other day that girl was playing all alone on the street with no elder supervising her and I was feeding my son lunch. He is fussy eater and eats only when playing outside. I know that's bad and I am trying to make him eat inside. Anyway, she was playing with a bubble blower gun and my son tried to take it. Once or twice she gave him but third time when he tried to take she let out a shriek and started crying. Thinking that my son had hit her her mother came and picked up started blaming my son that he had hit her. I kept saying that he didn't and I was there watching to make sure he didn't hurt her. But she wouldn't understand and her mother, that is, the girl's grandma also joined and started blaming me. And said that your son usually hits children so that's why she was crying. I kept saying that this time he didn't. But they started blaming my upbringing. Two days later when I was feeding him I don't know when that girl came out and my son went and hit her within a blink of an eye.. This time for real. But this time her grandmother hit my son on his hand and he started crying. I've told my house owner about this but they just say to adjust. What I've noticed is they let the girl out without any supervision only when my son is out and thats on purpose,so that they can blame and insult us. They don't even try to protect their child when my son tries to hit her when I am not around and that is on purpose. Ive talked to my house owner but they didn't respond to this.