My contractor is above to fill inside my detached garage with crushed asphalt. My question is " Is it safe to breath ? and then they will put 4" concrete on the top. When they build the foundation, they were a foot lower. They used the crushed asphalt to backfill instead of rocks. Does it create a health issue in the enclosed garage?

I would say he was trying to cut a few corners and get the job done quicker. Of course crushed ashpalt would be cheaper then rocks, which is what others use. Aside from the health issue, and concrete on top would take care of that, did he quote you a price where he stated he would use rocks?

I would start asking some questions about price difference.

3 things:
What does code allow in your town. More and more areas are allowing it, after charging a lot to dispose of it before.
Breathing isn't the problem, small amounts of lead possibly are (from the gasoline on roads), but very tiny amounts.
Asphalt has been shown to be structurally better than other aggregate for concrete base.

Crushed asphalt, also known as asphalt millings, make a good base for concrete. Also better for driveways than gravel.
Any lead contaminants would have washed away years ago, if they ever existed, would not be air borne and would be under 4" of concrete.