Am I the A-hole?

So my child's father has been inconsistent since birth. We split when pregnant. He went several months with no contact or seeing baby. After initially refusing, he eventually took part in mediation, and following this, he only turned up to half of the arranged meets with baby. Baby has had hospital appts due to health issues. Dad said he would attend appt. He missed a weekend visit, then no communication for 3 days. Contacted on day of appt to say he couldn't get the day off and had broken his phone, and on the weekend got given loads of work. Plausible excuses it would appear to other people (minus the fact he would have known before the day that he wasn't able to have it off - if he actually booked it off) however, I do not believe they are true, plus there are other ways of contacting me to tell me otherwise. He had suggested going out for mothers day, which I initially agreed to. After the hospital appt I sent a message with an update informing dad baby will need surgery in a couple of years. No response, again no communication for days. Mothers day he sends a message to say happy mothers day and he will pick me up in a bit. I replied and said I was not going to go out, but appreciated the gesture. He got annoyed at me and replied he had booked the f-in table.. (I know he didn't need to put a deposit down) Am I in the wrong? I constantly feel like I'm in the wrong if I try and put a boundary in place or if I don't do something the way he wants or when he wants to.