Hello,
This special citizenship I have makes me French in France but the minute I am out of France I become American so do I have to fill a US IRS having only income in France ?

Yes, as a US citizen you are required to file a return no matter where you earn your income. However, you could use the foreign income exclusion (form 2555) or foreign tax credit/deduction (form 1116).

Even though I am not considered US citizen in France ?

Yes

TaxForAliens is correct. If you hold a dual U.S.-French citizenship, you are REQUIRED to file an annual U.S. income tax return.