The moral of the story is this: If you are carrying weed, don't ride with someone who isn't buckled up.
If they had reached their destination when the police stopped them, OP's lawyer might well argue that the police had no right to search him if he immediately left the car in order to go into the house. As I suggested earlier, I don't think they have a right to restrain the passenger from leaving the vehicle; thus they had no probable cause to question him; and therefore no right to make a (
Terry v. Ohio) search.