I have a hot water heating system with radiators, and I want to replace the radiators in the upstairs bathroom and bedrooms (2) with hot water baseboards. Can radiators and baseboards be mixed and if so, how do I determine the size of the baseboards?

The quick answer is yes you can change them. The way that is mostly followed is approximately the length of the window, although that is a haphassard approach. The way that should be done is an accurate heat loss, followed by the output of the baseboard per foot.

I have a hot water heating system with radiators, and I want to replace the radiators in the upstairs bathroom and bedrooms (2) with hot water baseboards. Can radiators and baseboards be mixed and if so, how do I determine the size of the baseboards?
Just be aware that baseboard units have much less heat output. I've elected to go the other way. I salvaged old cast iron radiators and installed a few instead of the baseboards that went along the whole room and did not heat sufficiently. We are now warm and toasty.
You would be wise to get a pro to evaluate what you want to do.