I have two baby goats 6 weeks now, and I'm not sure how much milk I should be giving them.

Usually the bags of milk replacer will give you a feeding schedule. A pygmy goat will eat less than a standard size goat. A male goat will eat more than a female and he will weigh more when he is fully grown. If they have been on bottled milk since birth, at 6 weeks they should be drinking about a quart of milk a day, fed as 4 8 oz. bottles that are warmed up. You can feed cold milk in a pail as much as they want since they won't drink as much if it is cold. They should have good hay or grain available to get them started on solid food.