Our reaction to COVID-19 brought us a very sudden socially distanced, remote-working, touchless economy, in which the majority of our daily interactions needed to be remote interactions. Some industries adapted easily (think tax preparation), and some, like health care where personal contact is often mandatory, struggled to find solutions. However, health care IT departments adjusted quickly and deployed a truly impressive array of low and no-contact services. The solutions ranged from Zoom-call appointments with patients to accelerating the deployment of take-home medical monitoring and testing devices that connect remotely to upload their data.
Today, even though society has started to open up again, not everything is going back to the way it was. For example, patients like not having to schedule an appointment at a clinic to have tests and measurements taken. As it turns out, they really like the take-home devices. Clinicians like getting results from the patient’s home environment as well. Results are both more plentiful and more contextually accurate. However, the demands that new services like these place on our server-side infrastructure are immense, and those demands aren’t going anywhere. This is the new normal.
IT heroics came at a cost
Many IT departments that deployed new solutions quickly did so without the chance to make foundational investments in their infrastructure first. Remote monitoring services were often built on top of the same server stack that had done little more than billing and scheduling in 2019. The intentions were noble: getting these new social-distancing services up and running as soon as possible could save lives. There would be plenty of time to fix infrastructure when things went back to normal. Except ‘back to normal’ didn’t mean what they thought it would mean, and the technical debt they had incurred when skipping the infrastructure investment wasn’t going to get paid anytime soon.
What does this technical debt look like?
Now that we’re beyond COVID lockdowns, we still have IT departments with monolithic server-side applications that support scheduling and billing, and now also support remote access, mobile apps, video appointments, all while trying to accommodate patient demands for mobile apps and updated UI, and the government’s demand that all the data be kept HIPAA information privacy compliant. Oh, and they also have to make sure this tower of babel doesn’t fall over twice a day. Maintaining this is a tall order that non-IT folks rarely appreciate. IT managers need to dedicate more staff to the maintenance and support of this stack than they had approved in their budgets, which reduces the staff they have on hand to re-architect those hastily deployed solutions. And as for implementing new initiatives? Only if you’re OK building those new services on top of the technical debt you’ve already incurred.
Where does that leave us post-COVID?
I haven’t spoken to one IT manager or CIO who would have chosen to build out the functionality the way they did if they had been given appropriate time and budget. The trade-offs they were required to make to address COVID have resulted in a server-side stack that is now much more fragile than it was just five years ago. Today, adding new functionality and new services to that stack often risks taking down the whole stack. And it’s not cheap to scale that old stack either. While you can usually add new virtual machines (VMs) in the cloud running copies of your server-side software, the costs add up quickly and you haven’t solved any of your foundational issues. You could say our server-side health is immuno-compromised. It wouldn’t take much to send many IT departments into critical care mode.
Moving forward
We will need some injections to get our server-side stack healthy again. Capital injections. It is likely that institutions that have incurred this kind of technical debt will need to invest in one of the following two solutions:
First, do no (additional) harm: The first solution focuses on containing the existing stack without making it worse when new services are required. New features will be added as simple ‘Does-one-thing’ applications called microservices that are deployed in stand-alone virtual machines or in self-sufficient environments called containers. These containers have everything needed to run the microservice. If, for some reason, the microservice fails, it only takes down its container. Other microservices in their containers are not affected, nor is the legacy server stack. This kind of architecture, where many smaller services run in their own self-contained environment, is very change-tolerant and much easier to troubleshoot and cheaper to scale than having one huge monolithic stack of services all running together on one platform. But getting to the point where you can deploy microservices will take some preparation work. To prepare your server-side architecture for microservices in containers, you will need to give these new services a way to access your data. While each new service could access the data directly in the database, that can create a great number of problems down the road. Access to your data is more typically granted by going into the legacy server stack and granting access to the data that new services can use in the future. (You’ll hear these access points referred to as APIs, and the place where they live is called the data access layer). With these access points in place, new services can interact with your existing data. If you don’t have a data access layer or business logic layer where you can add API access, a new data access layer and/or business logic layer may need to be created so that new services can be added without having to add them to the monolithic legacy stack. While this approach of leaving the legacy stack alone and deploying only new services as microservices doesn’t pay down any technical debt, it does prevent future borrowing.
Untie the knot: The second type of solution unties some or all of the legacy stack, converting parts of it into a microservice model. This is a good choice if you have stability or performance issues in your legacy stack that are causing you trouble on a regular basis. This approach focuses on paying down the technical debt as well as reducing the cost of future deployments. By taking an existing service (say, remote monitoring devices) out of your legacy stack and rebuilding that service in a microservice container-based architecture, you can lower the burden on the hardware, the server stack, and reduce the risk to the other services that share the legacy stack. Typically, the IT department will have a very good idea of which services put the most load on the legacy stack and can easily prioritize which services to re-architect to get to a sustainable legacy stack. In this engagement, as in the last, you’ll want to have access to a data layer that can live in its own VM or container and manage access for all current and future services. This approach pays the technical debt that is currently hurting you and draining resources while making future deployments much easier, cheaper, and safer.
Are we immune yet?
There is no such thing as immunity from change in IT, but change can be made less dramatic. Adding new services in a change-tolerant server-side architecture, as opposed to a less resilient architecture, can be like having an operation done as an outpatient procedure instead of a more invasive procedure requiring days in the hospital. Just as in surgery, the fewer systems you have to disturb, the faster the procedure can be, and the lower the risk of complications.
What of our post-COVID legacy in IT? We know COVID or something very much like it will happen again. And along with medical science, IT science will again need to respond quickly and with agility. The next time, though, IT will have change-tolerant microservices and container-based architecture, which will help critical new services come online faster, with less disruption, at a lower cost than we incurred the last time around. So, if that’s how COVID changes health care on the server-side, that’s not such a bad outcome.
Mike Hines is an evangelist, Resolute Software, Savannah, GA. He can be reached on Twitter @MikeFHines and Facebook. Mike comes to Resolute after four startups, 13 years in engineering at Microsoft, and ten years at Amazon. At Amazon, Mike helped launch the Amazon Appstore for Android apps. Mike has spoken at dozens of national and international conferences addressing both app development and user engagement. Mike has been granted five patents for his software, software/hardware integration, and augmented reality inventions. Mike wants to teach health care industry developers the best practices he has learned for creating responsive apps, engaging UI/UX, and fast-update, low-risk back-end architecture.