Ok we have low spot in the asphalt and its right when the fence gate is as you walk into the garden. My question is can I simple cut out a walkway from the asphalt and replace it with walking pavers. Would this let the rain water drain faster

You may try and add some asphalt to that area and float it out smooth, I'm not for sure but they may have a resurfacer for asphalt, id check Lowe's or home depot, or just call a asphalt company for options

Lowe's does carry asphalt repair products they would be worth checking out.