I have Two Bathrooms upstairs, the venting is joining in the attic and going out one pipe. Could this cause contamination down the line and smell into the other bathroom?

Yes if there is no backflow damper in each fan duct.