Maximizing Efficiency: The Impact of Layered Stacks in AWS Deployment
Written on
Chapter 1: Understanding Deployment Time
How much time can you save on deployments with layered stacks in AWS? Have you ever considered how long your developers wait for deployments to finalize? Is the delay significant?
Once, I was tasked with creating serverless applications for a company that imposed incredibly tight deadlines. They believed that four hours was sufficient to complete any task, regardless of its complexity. 😄 You had to be exceptionally quick to thrive in that environment!
When you're pressed for time, you become acutely aware of every moment of inactivity. For example, when waiting for your code to deploy to the testing environment, which could take around 5 minutes, you might normally fill the time with Slack chats or coffee breaks. However, under a strict four-hour deadline, every deployment could feel like a potential setback!
Although those frenetic days are behind me, I still find myself frustrated by idle moments at work. 😂 To combat this, I've adopted a strategy I refer to as the Layered Stack.
What is a Layered Stack?
We are all familiar with monolithic architectures, where everything is managed within a single stack. For instance, a monolithic stack might perform three functions: loading candle data, storing candle data, and providing technical indicators to users.
In contrast, a layered stack adheres to the Single Responsibility Principle, striving to separate concerns effectively. This allows us to divide the same application into three distinct Layered Stacks.
The goal is to decouple application-related stacks from the data layer. We can consolidate resources such as buckets, databases, glue tables, secrets, and lambda layers into a Common Data stack, then build independent application stacks on top of it.
Why Use Layered Stacks?
This raises a pertinent question! 🤣 My primary motivation was to minimize deployment time. I hypothesized that deploying layered stacks would take less time than deploying a monolithic stack, especially when we deploy a common stack first and then run the other stacks in parallel.
Over time, I found that this approach not only reduces deployment time but also facilitates the coupling of components that change together while allowing independent changes to other elements. For example, once the candle data model is stable, we can work on the Candle Loader and Indicator API simultaneously without them being interdependent.
So, does the layered stack strategy genuinely lead to significant reductions in deployment time? I've utilized layered stacks in my projects, completing twenty-three deployments within just two days. Some of these took longer than expected, leading me to ponder whether the layered stack architecture truly decreased my wait times.
The Experiment
I aimed to investigate the waiting time discrepancies between layered and monolithic stacks. To do this, I needed to measure two factors:
- The time spent waiting while using layered stacks.
- The time I would have waited had I used a monolithic stack.
I focused on three common types of changes:
- Change App Code: For instance, adding new lambda code to the Application Stack.
- Change Common Code: Such as integrating new code into the lambda layer in the Common Data Stack.
- Change Infrastructure: For example, introducing a new Application Stack.
From twenty-three deployments, I estimated that:
- One was for Change Infrastructure.
- Three were for Change Common Code.
- Nineteen were for Change App Code.
This breakdown allowed me to calculate the differences in wait times. I consolidated my layered stack setup (consisting of 1 Common Data Stack and 6 Application Stacks) into a single monolithic stack, which included 177 resources. Surprisingly, it took only 2 hours to transition from layered to monolithic—though I suspect the reverse would take considerably longer.
Now, let's delve into the results!
Results
I conducted three measurements for each type of change, and here’s what I discovered:
The total waiting time difference between the monolithic and layered approaches was 1157 seconds, or roughly 20 minutes. Over two days, this translates to about a 10-minute improvement, equating to a 2% increase in productive work time.
While my initial hypothesis was correct, the time savings weren’t as substantial as I had anticipated. Given that I invested four hours in setting up parallel deployments and an entire day on this experiment, the benefits of this strategy may take time to materialize. 🤣
A more significant concern is that longer wait times can lead to increased context switching. It’s not just about those 5 minutes of deployment; it’s what you do during that time—like taking a coffee break or engaging in side conversations—that can really slow you down.
This video provides a guided lab on automating infrastructure deployment using AWS CloudFormation, showcasing how to streamline your processes.
This video demonstrates an end-to-end project of deploying a three-tier architecture on AWS, illustrating practical applications of layered stacks.