Personally if I had a bd that I wasn't with I would inform him when something drastic like bring another child into the home was happening. As it's something that affects his child so it is considered important information.
Maybe just say yes I’m having a baby. It’s not that big a deal he’s gonna know anyway because that’s a sibling to the child you both share. Maybe playing it off and being all mysterious about it is what’s making him question more. For your peace of mind. Yes I’m having a baby and what?!? Run along now and leave me alone! lol! It could be that simple, don’t complicate it for urself.
Enjoy your pregnancy amd ignore this weirdo. Congratulations xx