WAIT A MINUTE! Wasps do nest in the ground! If they are robust, mindless, hairy, fat little buggers they could be bumble bees, but if they are skinny, hairless, determined, and defensive then they are probably wasps.
Some of the most vicious wasps do nest in the ground! BE VERY CAREFUL! Without knowing the situation, I would highly recommend hiring out
this type of pest control. But if you do choose to do the work yourself, choose a "Wasp and Hornet Killer" type insecticide. These will contain fast-acting, knock down chemicals, that spray a stream, instead of a cone spray. This is required to make it harder for the wasps to find their way back to you. Wasps are diurnal. You should do this after sunset, or in the evening, when it is colder, to take advantage of the wasps more sluggish behavior at this time and to catch all the wasps at home. If there are numerous holes, you might have to cover the other holes so the rest of the colony does not come out to sting you. You should also be well dressed - don't leave any bare skin, don't wear wool. Have I scared you enough? Get this done professionally.
Note the below authoritative website indicates that for those that try to do the work themselves that "Stinging incidents are likely"
Consult this site for more information Yellowjackets and Other Social Wasp Management Guidelines--UC IPM