engineering

I have built and operated software development platforms in multiple industries for over two decades. A 1:1 discussion inspired this blog post. Software development platforms allow product development teams to deliver features in less time, with high security, reliability, performance, and built in compliance. Unfortunately, in the long haul, this turns out to be a two edged sword. As these platforms age, they tend to become monolithic (more code, less modular), and more complex (offering too many configuration options, feature flags)....

Silicon Valley is a hotbed of technology innovation, scrappy startups, and has seen the rise of a few lucky startups to be worldwide industry titans. Most innovations try to solve problems cheaper, faster and better. I immigrated to the US, and Silicon Valley at the end of the dotcom boom cycle (just before 2000). And shortly after that, America as a super power went through a generational transformation with the shocking terrorism attacks of 9/11....

I have set up engineering teams many times in my career. Each time is unique, and it is fascinating to observe a new team building its own micro culture and character. Cloud Application Management (CAM) is a brand new platform engineering team that provides services to manage platform extension applications. A customer or the application author decides: which configurations need to be changed for their solution what data sources are required what processing is required for the new data whether to list the application in application registry or keep it private An administrator installs, upgrades or removes an application on their cloud solution....

Earlier this year, I got an opportunity to work on an existing cloud provisioning automation effort that wasn’t going as well as leadership had hoped for. This was puzzling, as there is no shortage of scripting, APIs, and 3rd party tools available for automated cloud provisioning. Public cloud providers go out of their way in helping (for obvious reasons) corporations ramp up to their cloud offerings in an automated way. The platform teams that were tasked with this effort were also reasonably smart, motivated, and the goals they were shooting for felt reasonable and well thought out....

Our journey through life is a pathway created through the decisions that we make. Most non trivial decisions are costly and or impossible to revert. It is no wonder then that “undo” is a valued super power when working on complex software systems. The ability to reverse a decision completely (and without lingering side effects) in a given context is magical and godlike. Undo makes users of a software system more confident in exploring new features and options, as “undo” always provides a lifeline....

I previously discussed software feature scope in Quality, time, scope: pick any two. scope skōp noun: scope; plural noun: scopes the extent of the area or subject matter that something deals with or to which it is relevant. “we widened the scope of our investigation” Please keep in mind that we are addressing scope in a narrow sense of developing features for software distributed through the magical powers of the “zero additional” cost internet (after the recurring costs of connectivity, bandwidth, and access device)....

“Choose any two: quality, time, or features” is old age wisdom of software project planning. This rule of thumb makes for lively discussions, and used to advantage by many an engineering leader. Lets start with definitions first: Quality refers to known and unknown bugs that lie in the released features. Typically enterprises have SDETs (Software development engineer in test) who help craft a test plan, feature coverage, and regression scenarios to guard against unwanted side effects....

Calibrate (verb) mark (a gauge or instrument) with a standard scale of readings Managers across corporate america have a yearly ritual of bench marking their reports according to standards set by Human Resources (HR) for their assigned job level and role. This process results in rewarding (bonus, additional equity) and or promoting employees to higher levels and roles. A popular way of performing this calibration is by asking managers to place their assigned reports into a 9 box grid....

Bulk Execution Engine or BEE is a simple job runner. While writing distributed control plane APIs, there was a need for: waiting for distributed asynchronous transaction convergence retrying for intermittent errors providing status without holding up network connection After a brief survey of available solution landscape (and definitely not an intensive one), I found that popular products like Apache Airflow, AWS Step Functions could solve the problem, but at a high operational and cognitive cost....

Synthetics is a testing technique born in the crucible of cloud software engineering. In this blog post, we will review what role synthetics play, and how to exploit them to full advantage. Microservices modularize the functionality built by the organization. An API call on the cloud service typically results in multiple microservices being called to fulfill a given request. Microservice architecture also provides capability to change a single service in isolation with other services....