My ex did that. It robbed me of so much joy during my 1st pregnancy. Years later he got mad that I babied our son and was just plain mean about things. This time I picked a better partner.
Take this as a fellow believer....your husband is not honoring Christ or you. This is his child too. Go seek counseling.
No I would be upset if it did. He needs to help too. Dealing with an infant can get rough sometimes especially alone.