[petroski]

I'd like to install a sub panel in my detached garage approximately 100 feet away.The previous owner ran 6/3 romex.Whats the largest amp sub panel I can use.He ran 6 gauge 3 wire with separate ground

[KISS]

35A, will give you a 2.8% drop at 120 V. Should be less than or = to 3%.
Since the garage is detached, it should have a ground rod.
Box can be rated higher and can optionally have a main breaker rated the same or higher (disconnect) as long as it's fed by a 35A breaker or less.
Grounds and neutrals should remain separate in the sub. Sometimes an accessory ground bar kit is required.

[stanfortyman]

I have a differing opinion on this as KISS knows. :) ;)
Where I differ is, at 100' voltage drop is not really an issue for a feeder such as this.
Most electricians don't use 120v in figuring VD for a 120/240v feeder, and less than 3% is suggested for branch circuits but 5% is fine for final numbers and feeders.

What I see wrong is you say "Romex". Is this typical indoor NM cable, or is it UF direct burial cable? Romex brand makes both kinds.

If it is UF cable then you can use a 50A breaker and have a 120/240v feeder.
If it is indoor NM cable you CANNOT use it at all.