On 12/31 the company signs a lease for new office space , and pays for the first three months rent (Jan-Mar) The amount is 4500.

Do I do a journal entry for that

Yes, you should since you have paid for something.

So it would be

Cr Cash 4500
Dr Lease/note payable 4500

Or because he pays for the year

Dr New Office 13500
Cr Cash 4500
Cr note payable 9000

The fact he signs the lease requires no JE.

The only JE would be the cash paid upfront, that is 3 months worth, a total of $4,500

Dr. Prepaid Rent 4,500
Cr. Cash 4,500

Then, after January, you will Debit Rent Expense for 1/3 of that amount and draw down your Prepaid Rent by the same amount.