I "bought" a financed car from this person with a bill of sale. I gave him a down payment of $4000. I still have a remaining balance to pay to the loan company. I have been making payments on time. But I am wondering, if the person is wanting the car back, even if it's still financed, can I get my down payment back?

You can ask for it. I doubt he'll return it and then you would take him to Small Claims Court. HOWEVER - and this is happening frequently in NY where I am - if you and the seller had a side deal without the consent of the bank or financing company the Court can and possibly look at this as an illegal transaction and refuse to take part in getting your deposit back.