How do I figure the actual cost per unit for indirect materials?

it is usually done in an allocation process of determining the costs of indirect materials and dividing this by the number of units produced

Indirect materials are a part of overheads and the unit cost is always calculated for the overhead costs for the good production achieved.