areinmeyer.dev

The Phoenix Project

November 11, 2019

Phoenix Project

The Phoenix Project is a novel that acts as a DevOps fable, showing the reader how embracing the new concepts introduced in the DevOps Handbook can save your sanity and work-life like Continuous Delivery/Continuous Integration.

While a novel, it’s a parable in so many ways. It reminds me of another sort of modern parable, Bo’s Café, written by pastors wanting to show Christians the benefit of small community and close friendships that can be leveraged to keep each other honest and following Christ. Bo’s Café is about building and relying on a community in the church, whereas Phoenix Project is about the community of work. There are clearly lessons to be learned and the characters are not quite fully fleshed out but enough for the reader to recognize themselves in each of them.

Characters

Bill is a likable character who at first is carried along in a tidal wave of promotion and unmasking the troubles that lurk beneath the surface of the company. He’s level-headed and smart but has been stuck in a middle-management position that doesn’t challenge him and runs fairly smoothly and stays isolated from the rest of the code and infrastructure at the company.

Wes is a character that I have seen a lot in my years of work. It’s that smart person who laments his lot in work-life but can’t rise above the whirlwind to take action and becomes the disgruntled fire-fighter. There’s always a reason that a process or improvement can “never happen” usually because the people like Wes create so much negativity that it’s doomed to failure. He doesn’t actively sabotage anything, insomuch as he also doesn’t support it.

Patty is another character I’ve seen. She believes that processes can save the company but isn’t able to holistically apply them to enhance work product and can end up causing more damage that way. Again, her intentions are well-meaning. She believes that change management has to, well, be managed. But by creating and supporting a cumbersome process for simple as well as complex changes she in effect, creates a culture of developers that want to skirt the process as much as possible.

You see some of Patty’s same flaw in John, the CISO, who wants to check off boxes on audits without comprehension for the bigger picture of how those changes affect everything. Every security flaw has equal merit and must be corrected immediately. He is Chicken Little of the tech world, always crying that the sky is falling but never understanding if the system itself is vulnerable.

It’s interesting to me as a developer, that the developers are only really represented by their leader, Chris. The mentality of the group is to largely throw code over to the Operations team and walk away. I have seen some development teams work this way, but I pray that this is more of an exaggeration for the novel’s sake than what most companies operate as.

Plot

Let’s be honest here, if you read this book and are surprised by the ending or annoyed at any spoilers I present below, you may not be grasping the point of the book. The situations go from bad to worse and seem hopeless. An enigmatic consultant, Erik, mentors Bill and teaches him the Way of Lean Ops and Agile Operations. What works well is that while many of the characters I mentioned above start as one-note examples, they grow and learn along with Bill and the reader. You can see how well-meaning actions are taken either in the heat of the moment to complete a task that can often have long-reaching unintended consequences and how building a structure of simple, but effective change management and rapid deployment allows departments and companies the flexibility to fail and recover quickly. Applying the principles Erik mysteriously explains provides wins for the IT group they need to keep their jobs and save being outsourced.

I suspect that any developer or operations staff at a medium to large company can relate to some of the problems presented in the book, if not all of them. Some of the solutions may not be new (automated deployments), but many involve a rather large mind shift by teams to think of every action as repeatable and documenting work and having it flow through an approval process.

Takeaways

There are quite a few valuable takeaways in the book. One of the biggest for me was how valuable it is to create repeatable steps not only for code and deploying that but also for the infrastructure that it has to run on. I have been seeing this first-hand recently in our DevOps work moving a 3rd party application to AWS. Forget to expose the right ports and your application won’t run at all. If you bake that into a startup script or configuration file, you never have to worry about that again.

The other big takeaway was the work of different people and how much effect they can have on everyone. Brent is a character that is super smart, but extremely over-tasked. Seeing how to manage someone like that and turn them from a bottleneck to an asset as tasks get migrated away was a great example of how to start learning how a team works together. I thought the resolution for Brent might have ended up with him fired for being a bottleneck, but Bill manages to get Brent to train and develop and shed tasks so that Brent can take on a larger role in the team.

This book also shows that undertaking these types of transformations are not easy and not without stumbling blocks. As the team starts to implement Agile/Lean practices, they stumble and fail occasionally. They realize though why they stumbled and instead of throwing their hands up and trying something new, they take the time to learn and adapt from the lessons learned. That is key for any organization, at any point in the journey, to understand and work into a routine. Agile has the retrospective that should occur after a sprint. Other methods may call it something different. But a regular cadence to develop, deploy, reflect seems critical for success.

Final thoughts

I will be tackling the DevOps Handbook shortly, which is the manual to this novel. But The Phoenix Project is a great way to be exposed to these ideas and see how they get implemented in a work environment. I imagine if you have worked even just near an IT department at some point in your career these characters or events are going to be relatable.


Allen Reinmeyer

Written by Allen Reinmeyer who lives and works in Perrysburg, OH You should follow him on Twitter