My dog is home for seven hours, and I come home for a half an hour to walk her and play with her. She is doing great and has no problems with socialization. You are not cruel, and I think saying that is a little harsh. Do the best that you can for the pup.
And honestly, I think comparing a puppy to a toddler is a little strange. I love dogs, but that's pushing it.