New dilema

Im frustrated. I remained on good terms with ex's brother and his wife. The wife and I became close and she was a big support system for me. Ex's brother ended up being treated like crap by ex and their mom even though he didnt do anything and tried to stay neutral.

Now that ex is trying to make his entrance again or I could be wrong. Barbara you may have to correct me on this. He has left me alone the past couple of days. I did see him at a church function and he was nice. To nice. He tried to joke with me. That is how he has been lately other than trying to change a weekend with him because son wanted to go out of town with me to a function. He did start to bring up the old stuff again and I didnt bite. Yeah for me. His mom is being extremely nice to me all of a sudden. She has started asking me what plans son has etc. Ive become very standoffish.

Anyway Im not trying to figure out what he is doing.

My dilema is now that his brother and his wife are having a relationship again with ex and their mom, Im trying to pull back and distance myself. I know they were there for me during a hard time and I appreciate that. They were there for my son. ex's sil and I became very close and really good friends. But I am starting to feel they dont want me to move on either along with ex and his mom. Before she would talk about me meeting someone that was good for me and for son. She thought it was good if ex and i could exchange weekends so son benefited. I have tried to back off and move away from the family and just try to be casual friends. Now its like she has changed. She doesnt think its good for ex and I to exchange weekends and should leave the time alone. If I talk about the future and what I hope for, she gets mad if I talk about dating and making new friends. She got mad that I wanted to do something with another friend about 2 wks ago and told her I couldnt help her. She has made comments about her husband (ex's brother) stating wouldnt it be great if ex and I got back together and they wouldnt have to make up with anybody. I know our divorce affected family to but they arent thinking of whats in mine and sons best interest.

How do you pull away from ex's family and still be nice about it?