Belgian emergency passport

Well is your child Belgian as I doubt it, you don't get Belgian nationality just because you are born here.

http://diplomatie.belgium.be/fr/Services/services_a_letranger/passeport_...

If your child is Belgian, you buy an ID card at the local maison communale.

You cannot leave Belgium without a passport for your child, even to France of Luxembourg etc you need proof of nationality which you cannot do without a national ID card / passport.

For a British passport, you need to apply to Paris. You would have to plead for an emergency passport in Brussels for a baby, something like a funeral to go to, but I have never ever seen these issued to babies as a first issue.
http://ukinbelgium.fco.gov.uk/en/help-for-british-nationals/