Software projects consistently remain over budget, late and most importantly, many times miss solving the very problem they set out to solve in the first place. A recent and poignant memory was the rollout of the Obamacare health insurance exchanges.
Why does this happen?
- As technology has improved and become more human centric (good), architectures to solve these problems have become more complex. For example, kubernetes is the latest and hottest virtualization technology promising efficient and reliable compute resources in cloud computing. However, using kubernetes in a secure, effective and efficient manner remains onerous at best, and tricky at the very least.
- Frameworks that promise productivity gains over the complex architecture keep on sprouting a dime a dozen, and hapless engineers and technologists spend a plethora of time trying to simlpify their own personal adoption of promising new technology by uptaking them. Many times, these frameworks do work well for the narrow usecases that they tout. However, extending and or customizing these frameworks is hard, and many times results in hacks put in by engineers who are eager to solve their problem and move on.
- As problems solved through software become ever more larger and interesting, the solutions often span across multiple software teams and organizations. In large corporations, this often results in a complex dependency graph of conflicting team priorities. Ultimately, this results in teams changing designs and architectures to gain better control of their shipping dates. This often results in having hidden hacks, security vulnerabilities, and maintenance problems in these solutions.
What can we do better?
Agile projects is an important advance that is typically used in different forms by large corporations. The agile project manifesto and values sound reasonable:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
The trick in applying these successfully lies in how we model a single software iteration.
Scope, Quality, Time, and Feedback
At its core, a single software iteration is a simultaneous changeset across a set of codebases that works in tandem to provide a set of features to its users.
In this blog post, we are assuming that the suggested outcome for the users is in fact, useful (a blog post for another day).
- Start by writing a well thought out design document. This document
functions as the anchoring foundation for the project.
- Collaborate amongst architects, users, and project stakeholders to deeply understand the pros and cons of suggested solutions in the design.
- Build milestones that make sense both to the users and stakeholders to realize incremental value of the ongoing project.
- Manage dependencies and compromise as needed. As discussed above,
this is harder than it first appears. Many times, engineers vie for
the best solution. Keeping this tendency in check is vital. Here’s
some do’s and don’ts:
- Do simplify bootstrapping, allowing the project to get off the ground without big optimizations.
- Do review feedback from customers, users, stake holders.
- Don’t compromize security. Come up with secure processes that can be audited well.
- Minimize change to the existing system for fitting your solution. Instead change your solution to fit the system in the real world.
- Iterate between defining scope, influencing quality, delivering
timely, and learning from feedback.
- Scope is the coordinated changeset identified by the design document
- Quality is the amount of risk in shipping and or delivering the identified scope to users and stakeholders
- Time is a reasonable milestone interval for the team(s) to provide incremental value to their users and stakeholders.
- Feedback is how engineering teams learn on their shipping process, and whether their incremental scope provided the value that they thought it would.