Last week, a friend told me that her mother had been fired as a patient by her primary care physician in a letter she received in the mail. “Our philosophies of care are too different,” was the explanation.

Last week, I fired my oncologist by email. “I have transferred my care to ___ ___.” I gave no explanation, although the note was polite and appreciative.

My friend and I spent some time talking about how this “doctor firing” upset us. Both of us (she as her mom’s caregiver, me as an active patient) initially felt solely responsible when we saw signs that the relationship was not working out. We worried that we hadn’t figured out how to communicate with the physician and staff, that we weren’t listening carefully enough, that we expected too much.

We both wrestled with accepting that, in fact, this relationship needed to end. She accepted it after the fact, recalling years of feeling that the doctor and her staff were unresponsive and uninterested in working with the complex demands of the long-distance caregiving that she and her mom had patched together. I agonized before the fact: maybe my doctor was non-responsive because he thought I was second-guessing him or because I should have called instead of e-mailed.  Maybe he sees my queries as a waste of time: he really doesn’t think there is any intervention that might have an impact on my current disease.

My friend and I looked back on our experiences and agreed that, short of firing one’s doctor by just never showing up again (not recommended), formally ending that relationship carries significant emotional freight, whether we originate the change or our physician does.

Part of this is due to the sheer hassle of finding a new physician. Depending on where we live  and what kind of clinician we are looking for, it can be a daunting challenge to find any replacement, much less one with whom we feel we can work well. Active illness or the need for constant monitoring means that we may face a big gap in care if we haven’t prepared for the change.  This was the case with my friend’s mother, since it is not easy to find a clinician a) about whom useful qualitative information is available; b) who is taking new patients; c) takes our insurance; and d) seems compatible on the basis of available information or a single visit.

What also adds to our discomfort is the asymmetry of expertise between us and our clinicians. We patients can talk all we want about establishing partnerships with our doctors and how they possess medical knowledge but we possess unique knowledge about our own bodies, etc., but when the rubber hits the road, it’s their training and expertise that positions them as the sole gatekeepers who can provide access to the tools of health care that might ease our suffering and save our lives. It can be hard to walk away from those promises, even if staying with a clinician we dislike means we must hide our opinions and preferences or subject ourselves to disrespectful treatment by him or her and staff.

Is this just another one of those problems of the privileged? Perhaps. Many people simply don’t have a choice of the clinician who cares for them: they seek help from the nearest or the cheapest doctor. They see the clinician they are assigned to at the clinic or by their health plan.

But I also know the remarkable difference my relationship with my clinician makes in my own willingness to take on all the tasks of caring for myself. If I feel she has really listened to what is going on with me and what it is going to take for me to do what she recommends … if we together make a plan about next steps … if I trust her expertise and experience, I am much more likely to try to do my part. On the other hand, if I feel like she sees me as just the next problematic body part to appear on a fast-moving assembly line, if she is talking at me, not with me, if she interrupts, corrects me or ignores me, all bets are off. I’ll take her recommendations under advisement but do what I think is best.

Again and again, evidence points to our relationship with our clinicians as key to our effective participation in our health care. People are given facile advice by advocates, the media and the government to fire their doctor if they are dissatisfied with their care or communication style. I absolutely believe that we should try to find clinicians to care for us who are compatible with our personal preferences. But I also know that change is hard for all of us, especially when we don’t feel well, and that that the effort required to find the right clinician to replace the wrong one and the anxiety that such a move generates often outweighs the impulse and intent to make this change.

Patient-physician incompatibility is a barrier to our participation in our health and health care that has no direct policy solutions. Sure, the greater availability of clinicians – primary care providers in particular – would help. So, of course, would more qualitative and quantitative information about individual physicians (including robust patient ratings). But the bulk of the change will probably have to originate with us.  And it is likely that we will only take this task on when we learn how much more effectively we are able to care for ourselves when our efforts are in harmony with clinicians we like and trust and whom we feel are working with us to live as long and as well as we can.

Jessie Gruman is the founder and president, Center for Advancing Health. She is the author of Aftershock: What to Do When You or Someone you Love is Diagnosed with a Devastating Diagnosis. She blogs regularly on the Prepared Patient blog.