It took me forever to retire—three years or so. It took so long that my nurses got tired of bringing me purple cakes decorated with whipped cream stethoscopes, but I did it. Now, a few years later, I can tell you that the things I worried about were not the ones that gave me trouble.

I had mostly agonized about money—had we saved enough to last us the rest of our natural lives? Could we afford to travel? Would we end up drinking boxed wine? What if the market crashes, forcing me back to work when I wouldn’t remember how to intubate trauma patients or run the latest sepsis protocol? Should we forgo travel to save money for the bills?

Five years later, I know that my worries were unfounded. We’ve been careful, of course, but we traveled to every continent, and we drank some fine whiskeys without working any shifts.

But if you think retirement was easy, think again. Like ships sailing uncharted waters, we avoided the dangers we’d anticipated but struggled with the unexpected ones. That’s why I thought I’d share with you.

Your money is finite. There are many better qualified to talk about retirement financing, but I can tell you that spending money from a finite pool feels different than telling yourself: I’ll just pick another shift to pay for those flights. There is no money besides what you saved, so every dollar comes from the same finite pool. Spend wisely.

The fun perks. Depending on where you work, there will be non-financial perks you take for granted: free turkeys at Thanksgiving, paid CME that lets you travel to meet old friends, the joy of working on a team, the sense of belonging to a community. That all goes away when you retire, and you might miss them more than you think. How will you fill that void?

Who are you? Unlike delivering pizza, grave digging, or fixing leaky plumbing, doctoring makes you a doctor 24/7. It’s not what you do but who you are. But who are you once you stop seeing patients? You might struggle to find your identity. That’s hard enough as a teenager and even harder in your golden years. Give yourself a little time.

Social status. As a doctor, you’re a minor celebrity. Patients recognize you in the store and let you cut in line. The police officer gives you a warning instead of a ticket when you speed. Whenever they find out you’re a doctor, people perk up and ask, “What’s your specialty?” They’re in awe if you’re a cardiologist or a brain surgeon, or even a dermatologist, but a retired doctor sparks no excitement. They come behind podiatrists and chiropractors.

Togetherness. I hope you’re happily married or live with someone you love. It’s lovely to come home and share snippets of your day, brag about a good catch, a grateful patient, or vent your frustrations. But being with your partner a few hours a day and on vacation is nothing like being together 24/7, so there’s no news to share and no privacy. If you like each other—and even more if you don’t—togetherness is something you should plan for. Plan your own space and your hobbies, or things might get too cozy.

Free time. What will you do with yourself? As a doctor, your life is structured to the minute. In retirement? Not so much. You may find the lack of routine disorienting. Plan your days with a sense of purpose. Whether it’s a morning walk, a volunteering gig, or a weekly trip to the library, a routine will keep you grounded.

The kids. You want to spend more time with your family. You’ll chill with the kids and spoil the grandkids, you say, but you might be surprised. Your kids are probably grown up, and they have jobs, friends and pets, and in-laws. They might have little time for you. The relationship with them is likely to shift. Who will pick up the dinner check? Will you babysit the grandkids? Will you dip into your savings to fix their dishwasher? I don’t know the answer, but you should.

Friends. Your old friends might still be working unless they moved to Florida or Mexico. You might downsize and move somewhere new. Making new friends at our age is not as easy as it was in kindergarten, when we spent every day together and didn’t care about politics. Be prepared to actively seek new friends, or you might find yourself feeling lonely.

Growing old is not for the weak. “I’ll hike and bike and travel and do all those things I never got to do,” you say. That sounds great if your knees still work and your bladder behaves. But as years sneak up on you, trails grow longer and backpacks heavier. It helps if you stay in shape.

Keeping sharp. Being a doctor keeps your brain in gear. Keeping up with the research, the new meds, the patient load, and even the cursed paperwork takes care of your self-actualization. But, once you retire, you’ll discover that watching TV, scrolling through Facebook, and doing crossword puzzles don’t stretch your brain one bit. To keep sharp, find a challenge you love, whether it’s learning a new language, playing chess, or taking up painting.

What’s next. For as long as I can remember, I lived in the future. I always looked forward to the next vacation, the next job, retirement. But after that, you run short of things to look forward to, unless you’re one of those who read the newspaper to find their friends’ obituaries. Setting new goals, whether it’s travel, carpentry, or learning how to play guitar, may ward off depression.

Stagnation. Just like trees, hippopotami, and the economy, humans need to grow for as long as they live—not physically—that’s easy—but intellectually and spiritually. When young, we have to—there’s school, then adulthood with its inherent responsibilities, then working, teaching, and being responsible for others. But not much keeps you growing in retirement. You need to keep learning, exploring, and challenging yourself to make progress.

Expect the unexpected. Life loves to throw you curveballs, even more so in retirement. Whether health issues, financial hits, or a family crisis, surprises are bound to happen. That’s why you’ll want to have an emergency fund, update your will and POA, and plan for the unexpected so you can handle whatever comes your way.

Retirement isn’t all sunshine and rainbows. It’s an emotional ride that will make you feel lonely, anxious, or even depressed. Going from a high-stakes, meaningful career to a retired life can be jarring, and the loss of identity and lack of social interaction may hurt.

“So, if it’s so terrible, should I keep working till I die?” you ask. Not at all, my friend. Just the opposite. Retire as soon as you’re ready, but plan for it. I suggest you plan your retirement with the same care and attention to detail you used when planning your career and marriage.

Rada Jones is an emergency physician.