Overview
We usually discuss configuration when designing new software or assembling a new system out of existing software components. How will we configure it? How will the configuration load? The software side includes how to structure the configuration, how to achieve the correct level of abstraction, and how to support diverging use cases seamlessly.
The problem most companies face: Shortcuts
Data assumes a common role in controlling software logic, in the form of application parameters or configuration data. Unfortunately, this carefully executed monitoring is sometimes avoided when there is a need for source code changes. Businesses are able to find ways to bypass the calculated regulations in an effort to keep up with changes within their company. Instead of following procedure, businesses will look for shortcuts that provide immediate results. This means that what was once set in stone is now completely reconfigured.
Even though it is considered bad practice to use shortcuts to change code on a whim, it is still shockingly easy for people to change configuration data this way.
The problem still exists with those that have the authority to change configuration data (i.e IT staff). Thus, restricting access to configuration data is only half of the solution. The other half is to introduce controls over configuration data, similar to the controls we have on source code (revisioning, delivery, etc).
How to curb risk, increase productivity
Our Problem
Our Solution
As new services were conceptualized to utilize Docker containers, it was decided to migrate existing services to use the same approach. This created a need for a new common configuration management system. There was a mismatch of practices across the board!
With the entire infrastructure being hosted on AWS, we chose to use AWS Parameter Store for storing the data.
Instead of integrating AWS Parameter Store SDK directly into 35+ micro-services across multiple languages, we chose an approach that would allow us to add and remove features without having to modify each micro-service.
To achieve this goal, we built a managed service that acts as a proxy caching system between micro-services and AWS Parameter Store. It lives within the same environment as the micro-services that use it. Each micro-service sends it requests. The proxy then parses the data, sends the request to AWS Parameter Store API, and upon response, applies any additional features we have in place. Then it caches the response for future requests for the same version.