My babys other family?

I am currently 39 weeks pregnant with my first baby, i was in a relationship with my baby's dad for a year but ended it seven months ago. He was violent towards me and caused me emotional abuse, he's very immature for his age he is 21. He takes drugs and i also think he has a behavioural problem as he is incredibly unpredictable and irresponsible. He wont be on my baby's birth certificate and i have absolouty no intention of letting him have contact with my child, for my babys safety. My problem is that his family are very keen to see my baby once he is born, i.e his mum and younger sisters, but i'm not sure whether to let them. I dont think they are suitable role models for my baby as none of them are academic and are very different from my family. They get into trouble with poilce sometimes and are well quite rough :/ like a family out of Shameless, but they arent bad people so would it be unfair for them not see the baby? Im just worried they'll p on info about my baby to my ex or pester me to let him see him. I feel quite intimidated by his mum sometimes, i dont want them to know where i live, or for them to come to my mums house or for me to go to theirs just incase my ex is there or his horrible brother. Im so worried about this situtuation :( and do i tell them that my son has been born straight away or wait a few days or not tell them at all? :/ I really dont know what the right thing to do is. Should i say to his mum that she can see him along with her daughters as long as im there and that she understands i do not want my ex seeing the baby and i would appreciate it if she respected that? please give me some advice xxxx