when I have a physician put me off like this... as in if it is apparent the practive is too busy to really care for me as a patient who deserves respectful, prompt care, I find another doctor. Sometimes practices just get too big and you need to move on to someone who still has the ability to better value and meet their patients needs.
we are at that stage with our general care provider. Been with her for probably 12+ years. The personal service is going down the toilet, mostly because they're just too busy these days.
you can decide whatever you want. Obviously having to go through the noise of seeing another doctor is another stressor... you can always ask the doctor, if the surgery isn't being scheduled to your satisfaction, if he would recommend another physician who might have more time for his patients.
yeah, it'll probably not thrill him, but you are the person giving him the business. You have the right to ask. And he might even be absolutely glad to refer you... perhaps another doctor in the same practice?
if trying to do all of this sounds like its more pain than you are already going through, then you maybe need to wait it out.