Hello World. Let's talk infrastructure

I am currently pursuing a degree in Computer Science and plan to use this platform as a means to gain mental clarity on some of the topics I'll be researching in my capstone semester. I have worked as a DevOps Engineer for the past six years and am captivated by current technologies within the discipline of cloud engineering.

What I have learned so far from my brief time in the field is that the structure of an organization has a non-trivial business impact on its ability to deliver and maintain software. Separation of concerns and the single responsibility principle have long been recognized as best practices within the field of software development. The value of breaking out logical structures into discrete components provides the primary advantage of independent evolution. We can safely update or alter the inner-workings of component A without modifying components B and C assuming the interface of component A (i.e. the way other components communicate with it) has not changed.

Is it wise to use the same organizational principles for a collective human beings all working together to solve a specific problem?

The drawback of structuring an IT organization in such a manner is that the interface for communicating between teams is not standardized. One team prefers an email, another works only within the confines of a specific project management tool, another team wants you to send them a Slack or Teams message. The messages we send to our coworkers lack an interface definition language. Why are we sending them a message in the first place? Likely to communicate that they own one of our application's dependencies and we need to coordinate in some capacity to successfully introduce a change, or vice versa.

Communicating about change in software with the manner outlined above will inevitably lead to a failure. Oftentimes it's because you didn't even know about a particular dependency in the first place. Or perhaps the Network team failed to open a port, the SRE team failed to update an environment variable, the Infrastructure team didn't mount the filesystem at the correct path, or the Database Administrators didn't have the table created before the application was deployed. I have seen all these failures take place and they could 100% have been avoided if the work between groups was not siloed.

What is the common interface, or source of truth, we can all look to for reasoning about the state of an application or workload? The git repository, duh! I am not just advocating for adopting GitOps in your organization. I am pointing to the potential value of all engineers working in the same rhythm via the git repository. Having infrastructure, application code, configuration, and deployment pipelines all living in harmony in the same repository.

This ultimately introduces a new set of problems, but I am excited to spend the next few months exploring the solutions. The Cloud Development Kit (CDK) offered by AWS is my starting point. Until next time, stay soapy.