Please define what you mean by the phrase "that they claim for me." If what you mean is a per diem that they bill a client on your behalf, then yes - in this case I assume you have an employment contract that stipulates how much of any client billing they keep.

He may also be talking about tax withholding on the per diem being paid.

That withholding is legitimate if the per diem is being paid for a period OVER one year in length.