Skinny men can build muscles?

I'm a skinny man who is 31 yrs old. I've been like that since my childhood, i've very thin bones, almost no shoulders. I mean my arms are still look like a 13 years old kid. Legs are also weak but i ride around 60 km bicycle on weekends for a few months. My question is how much i can build some muscles in my upper body? I don't want to go to the gym centers and embarrass myself. I just want to work at home with dumbbells or something.