For the past day and a half, my 4 week old puppy has been whining and acting like he was going to throw up, but nothing comes up. Finally he threw up but it was clear. He still eats and drink well, but he will lick his mouth like it's hard for him to swallow. What do I do and what's wrong with him?

Is this an orphaned puppy that you are feeding or is mom nursing?
With a pup this young your best bet is your vet, he could have possibly aspirated or have an upper respiratory infection.
Little ones do not have much reserve if they get sick they go downhill very quickly, please get him checked by a vet.