Should I give up gym for my partner?

My partner seems to have a problem with me going to the gym and is making it really difficult for me. Trying to keep this short, he works in the fitness industry, he’s a bodybuilder, trains 5-6x per week, spends £200+ per month on his own leisure club membership that he spends hours training in and going to the spa after work. We used to go gym together before having kids, it’s the only thing we ever did and bonded over it. Even though we didn’t go gym together anymore after having kids, he used to support me going and training. I made so much progress with my body. I had to stop gym for a while because my new job schedule made it tricky, and being a mom. When I tried to sign back up to a gym to go in my lunch breaks, he had a big problem and told me no. He said my only option is to use the residential gym in our apartment building (it’s very small, minimum equipment and not somewhere I can properly train) but he’s given me no other choice. When I mention that I want to sign up to a gym and get back training again, this gets him very upset and he tells me no, and I can only use the residential gym on the evenings when he is home from work and that’s my only option. Today, I have done EVERY bit of housework, I have looked after the kids all day (it’s Saturday so I’m off work and he was at work, but he trained after work and went to the spa), when he came home at 5:30pm I said I’m going to the gym (I barely get to go to the residential one once or twice a week because he makes it difficult). He was clearly not happy, when I got back he is now very mad and told me ‘my priorities are wrong’. He uses that statement every time I mention or ask to go gym, like I shouldn’t be focusing on gym I should be taking care of my home, doing housework etc. but I don’t understand because every single thing was done. I don’t know what to do because I don’t do ANYTHING else for myself, I’m so unhappy with my body I lost all my gains and I want to get back training and it really helps me mentally. But if this will cause a big problem between us because he doesn’t want me to sign up to a gym, so I really don’t know what to do