Cory O’Daniel is CEO and co-founder of Massdriver. A software and operations engineer operating in Los Angeles, he specializes in cloud solutions and app development.
I have this existential dread when I think about the future of software and the cloud. I want to take you there, but we will need to look at some Fortran code and analyze stats from the US Bureau of Labor Statistics. But first, let’s go for a hike.
A day without software
It’s hard to imagine going a whole day without interacting with a piece of software.
You wake up naturally, not to your phone or alarm clock. You walk out to the henhouse to gather eggs, not some store-bought eggs purchased through a point-of-sale system. You fry them up over an open fire, not using gas or electricity from the utility company. After using your manual toothbrush and boiling some water for a bath, you hop on your bicycle and ride out to the forest to spend the day in thought, blazing through red lights and walk signals, paying them no mind. If you live in a city, this may be a labyrinthine bike ride if you don’t know how to get to the woods without Google Maps.
Software’s insatiable hunger
We know that software is eating the world. We see it happening before our eyes as we upvote an article about an industry being disrupted by tech or as we swear at our thermometers that can’t take a temperature without being paired to our phones. But as an industry, we don’t discuss what the world being eaten by software means outside venture capital.
Today there are more than 27 million software engineers. Experts estimate an increase to 45 million worldwide by 2030. That’s a 67% increase in software engineers in the next seven years.
In the United States, we’ve gone from 612k software engineers in 2002 to more than 1.6m in 2022, a 161% increase in 20 years. The projected job growth by 2031 in the U.S. for software engineering is 25%. That is 5X the average growth rate for all occupations. By 2031 almost one out of every 100 adults in the U.S. will be a software engineer.
The number of software engineers in the U.S. will have outpaced more than 800 other careers. The 13 careers with higher job numbers, like retail workers and cashiers, may sharply decline.
More software engineers, less everything else.
Six decades of painting bike sheds
It’s incredible that in the last 60-ish years, we’ve gone from a handful of software engineers to more engineers than the population of Australia. At the same time, little has changed about how we write software. Sure, the tools we use to write software changed from punch cards to the vim/emacs wars to VS Code, but the software has remained aesthetically similar. A fresh boot camp grad can grok Fortran, and I’m 100% optimistic engineers from the 50s could comprehend Javascript (once you explain the event loop).
The past 60 years have been syntactic sugar over conditionals, loops, variables, and functions, with some high-brow arguments regarding the merits of functional programming versus object-oriented programming.
We’ve had consistency in programming languages for decades. The consistency makes it easy to educate and train the hordes of engineers that our swiftly digitizing world demands.
But how we operate, secure, and assess the health of our software is where my dread lives.
We’ve gone from mainframes to racks in data centers. From web hosting to virtual private servers. Today we find ourselves in the fog of the cloud, moving from virtual machines to containers to serverless to serverless containers.
Organizations are pushed from monoliths to microservices and event-driven architectures earlier and earlier based on the demand and scale of a global user base. The way we observe our software has gone from logs and basic metrics to distributed traces and service level indicators.
But no one is teaching our future professionals how to operate their software.
With the cloud churn that we have, I understand why. If you learn C++ in school, C++ will always be there. It looks like most other languages; the concepts translate. To teach the cloud is to teach vendor specifics that may be outdated by the time you are ready for the workforce.
Operations experience is production experience
The only way to get operations experience is on-the-job training. Keeping production up is arguably the most critical part of running a software company—which all companies are becoming. You can’t teach production experience in school, and the way we learn operations isn’t scalable today nor will it be tomorrow.
In the age of sprinting through delivering features, do we spend enough time communicating the value of our operations work to our business and product counterparts?
The 2022 StackOverflow survey found that 46% of engineers consider themselves full-stack, 43% backend, and 25% frontend. A measly 8% of engineers consider themselves to be cloud infrastructure engineers.
For an industry that praises the idea of “you build it, you run it,” it certainly looks like most engineers don’t know how to run it.
I couldn’t have said it better myself. The goal of the cloud isn’t to be simple, it’s to be capable. The market drives new capabilities and services. Unfortunately, those services and capabilities are designed based on feedback from the largest organizations in the world with the capital to hire the most experienced operations engineers.
That leaves the rest of us, from startups and MVPs to small businesses that may never be billion-dollar unicorns, to either engage in a bidding war over remaining talent or look to the simpler solution, a platform-as-a-service.
The era of Heroku was a decade ago, and the world has changed. Looking at PaaS solutions today, they are either stuck in time or stuck in opinion. Anyone making decisions about your architecture without understanding your business, software, and organizational structure leads you to the most brutal vendor lock-in.
A yawning chasm of options
It’s frustrating to see the lack of options in making the right choice for your business. As we move into the next few decades, software will pervasively infiltrate every aspect of our lives. It will be even more essential to ensure the uptime and security of critical systems like AI-enabled remote surgery and networks of self-driving vehicles.
The churn of how we operate software may become stable enough to teach operations in schools by then, but I doubt it. Today’s operational practices will not scale to the workforce size of tomorrow.
The cloud has done a great job of creating building blocks that can be pieced together to build anything. But the building blocks they provide are too small. The blocks aren’t easy to combine like Legos, and they leave teams trying to find ways to glue them all together with a concoction of IAM, IaC, CI/CD, and a host of other acronyms that shouldn’t be on your mind while you are trying to ship a product.
The cloud should be as easy to use as a PaaS. There should be low-risk future-proof paths to growing an organization’s architecture.
We need bigger building blocks. We need better glue.
We need to focus on developer intentions over cloud implementation details.
We need excellent self-service developer experiences that reduce learning curves while encouraging experimentation.
We need systems that are customizable but with governance and guardrails.
We need audit logs, security, and change history to satisfy compliance standards as a default, not an afterthought.
We need to codify the operational expertise of the few to enable engineers of all tenures to leverage the capability of the cloud without the hours and errors of effort.
Massdriver will be holding a webinar titled “Deploying 3-tier Web Applications on Azure Kubernetes Service (AKS)” at 4 pm EST on March 28, 2023. Learn to push docker images to the Azure Container Registry (ACR) and deploy applications to Azure Kubernetes Service (AKS) using Massdriver. Discuss the best practices, security, and compliance of various cloud resources deployed.
Massdriver is one of several trusted partner benefits available to members of Microsoft for Startups Founders Hub. Massdriver helps operations professionals focus on scalability and reliability without toiling over infrastructure configuration changes. Massdriver’s visual designer can reduce infrastructure development times by 90% while giving startups staging/production parity out of the box and total control of their infrastructure when they need it—all in their Azure account. Members receive $10,000 in credits for unlimited seats over the first 12 months.
For more resources for building your startup and access to the tools that can help you, sign up today for Microsoft for Startups Founders Hub.
