Do they set u up to fail?

I always feel like I'm being setup to fail.. Anything u do somehow it's the wrong thing..he's perfect when he's around my kids I love when were all together.. In fact one time he wanted me and the boys to stay the nite and have movie nite so we went to his house ended up staying the nite and the next time we got in s fight??? He threw that in my face and said he would never allow his kids to do that .. He said as a mother he couldn't believe I stayed there with them and asked how many other guys my kids have had to stay with... I was shocked !! Couldn't believe he said that.. I don't understand what I did wrong he set me up to fail..