In the United States, we often find our motivation to achieve fueled by a culture that rewards success, while commonly denying the realities of failure. As a culture, we feel terrified to fail, despite the reality that true long-term success is often built upon the ability to recognize our failures and to learn from our mistakes.
Let’s say I am in a relationship and the relationship is not is going well, amongst all our problems, we can’t seem to agree on what to do or when. We could download the neatest, newest scheduling app and hope the issue will work out. Or we could go away for a weekend “communication” retreat, but most often these little efforts won’t resolve the underlying problem.
The same thing is true with software development and technology. We can decide to buy a great new technology to streamline a project and to solve an issue, and yet even if it doesn’t seem to be working, we will keep plugging along. We will push feelings of discomfort to the back of our mind and keep hoping the problem will resolve. Sadly, more time will most likely not fix the problem. And, if it does resolve, it is more likely to be the result of dumb luck than a specific tactic or more time.
Wait and see is not a strategy, it is a lack of strategy.
In general, neither time nor money will necessarily fix or change the problems in our relationships nor in our projects.
What or who can fix our problems?
We can. You can fix your problems and I can fix mine. Sometimes we need to admit that we have made a mistake and let a relationship go. Sometimes we need to admit that we brought on the wrong piece of technology and let it go. For better or worse, it is sometimes easier to admit relationship failures (or at least to terminate them), because we can blame the other person.
In software, we often think we only have two options: we can try to ignore the problem and hope it goes away or we can lose face and admit we made a mistake. I’d like to propose a third option, but first, let’s look at options one and two.
Ignoring the Problem
In a culture where people always want to be right, it is very difficult to make mistakes.
Admitting mistakes is terrifying, it makes us vulnerable. We hate to be seen as abandoning ship, giving up or point blank admitting failure. To make matters worse, it is often the ambitious people that are often in this position, the person with the most responsibility has the most to seemingly lose from admitting mistakes. Unfortunately, long term success (and happiness) has a lot to do with one’s ability to step-up and admit failure, and to be able to bounce back from that failure with panache.
Our culture continues to tells us that mistakes are wrong. But are they really?
Benjamin Franklin is quoted to have said that “Failing to prepare, is preparing to fail.” In fact, if we are living, playing, working, predicting, and aiming for success, at some point soon, we will inevitably make a mistake and or fail. But is this wrong? Should we be ashamed of our mistakes? Or should we see them as opportunities to learn? Should we prepare to fail?
When was the last time your manager (or you) made an admission like this:
“I made a decision and it sure as heck didn’t work out. I apologize that the biggest problem is that it took me so long to acknowledge this, but now that I have, we can take care of it and move on.”
As a manager, the above admission is terrifying to make, but once it’s been done, it is actually quite freeing. We know that mistakes happen and that we will have mistakes that lead to failures. So why don’t we set up projects to expect failure? What if we built in a project management failsafe that allowed us to externalize our mistakes?
Common and repeat mistakes lead to corporate insanity…
Anyone who has been in quality assurance for more than a few projects knows that projects fail, projects are late, and that broken software goes to market. And yet, we let the same mistakes happen again and again, often right under our noses.
We could call this bad luck, but it might more accurately be termed corporate insanity aka lessons we refuse to learn. There are many common mistakes in software development, but when we analyze the problems, we see a few common patterns emerge. In general, the biggest common mistakes are those where there is a gradual internal failure that lacks an external circuit breaker to administer a correction.
Why is it that mistakes are so hard to acknowledge?
Let’s say you are a project lead responsible for hiring a team and bringing on tools to get a job done. You make a hiring mistake and take on an engineer that can’t do what needs to be done. Outside of an Act of God, there is not an external solution to your problem that will correct your bad hiring decision. The only solution is to let the engineer go and get someone who fits your needs.
The same goes with a bad technology decision. If you have brought on a tool that doesn’t accomplish what you thought it would or should, denial won’t fix your problem. Both situations require at some level that you admit you made a mistake and that you correct the problem so that your project objectives can be met.
Mistakes in Monopoly are External.
Most Project Management Mistakes Are Internal.
Admitting mistakes has a number of profound psychological effects. In quality assurance, one of our worst enemies is often the timeframe. We don’t have a discreet end to most of our projects. Furthermore, project management is not like a game of Monopoly or Risk where you can make mistakes and an hour later they are resolved.
In a board game there are winners and losers, but when the game is over, life goes on. Mistakes in board game strategy create external loss which is not as emotionally stressful as decisions that involve us personally.
Unlike in a game of Monopoly, real life risk taking rarely results in a tidy “end.”
When we invest in the stock market, get involved in a romantic relationship or take on project management, we can easily fall victim to stressful internal loss. In the game of life, everyone hopes to win and personal mistakes and the resulting failures can result in much greater psychological challenges and fallout than a board game…
Internal Loss versus External Loss
Let’s look at a particularly compelling example of internal loss — losing money on the stock market. A significant financial loss is known to have life or death consequences for certain players. Why is this? It may have to do with the difference between plain luck and luck plus strategy. Particularly, the difference between the internal versus the external loss in a stock transaction given one scenario, but two different arrangements.
To start, everyone knows that investing in the stock market involves a tolerance for risk. Every once in awhile investors get lucky, but they can also get unlucky. To avoid stressful internal loss, there is the option to build external controls into one’s investments, but it is a voluntary control.
In 2001 we saw the effects of a powerful and gut-wrenching internal loss for folks that had ventured to put all their cash in Enron. What if all of these individuals had invested with a standing sell order set for a certain stock value? Would their losses have been as psychologically (and financially) painful?
Certainly, investing all of one’s cash in Enron at $100 a share, even with a standing sell order if the stock price falls to say $50, would still have been painful. However, the pain would have been minimized and externalized. The sell order creates an external arbiter that ends the game if and or when it turns south.
You still experience a loss, but there is never a moment in which you have to decide to pull the trigger (or not). There is not the internal stress of having to give up hope that it might still get better. The standing sell order does not prevent all loss, but it provides the peace of mind through a known endpoint that does not require any action on your part.
Why are Internal Decisions Painful?
Deciding if and when to call the end to a bad decision is a very high-stress activity. Not only does it indicate that we have given up hope, but we also open ourselves up to the vulnerable situation in which other people may see us as a failure. And, in a world that values success over failure, we personally invest too much in decisions, which then slows us down and in fact decreases our ability to act pragmatically.
Internalized Decisions Depend on Emotions and not on Logic
We worry about what people will think of us, our social and our professional reputations are at stake. We wonder if people will think we were wrong, stupid, arrogant or any number of negative things. We wonder how the decision will affect our future. We get caught up in the “what-ifs” and the “shame” of making a mistake.
This type of worry exists not only in investing and relationships, it is also a leading cause of failure in software projects. How many projects have failed not overnight, but over time, because the project manager was simply in a state of denial, hoping that “time” or “magic” or “luck” would resolve a problem?
Opportunities for Internal Loss in Software Development:
- New technology: As the lead developer you decide that in your next version release, you will bring on a new technology to make life better. Soon after implementation, you discover the technology does not live up to expectations. The pragmatic call is to acknowledge your mistake now, before too much time, money and resources have been invested and your smooth sailing project turns into a boat lost at sea. Unfortunately, it takes a lot of courage to fess up, and fess up early, it is much easier to pretend nothing is wrong and hope the problem resolves itself…
- New people: When we bring on consultants or permanent hires, we have high hopes as to what they can accomplish. New hires are a bit like having a child. Their success reflects back on us. What happens if and when they fall short? Maybe they lied on their resume. Maybe we asked the wrong questions. Maybe our business analyst underestimated the scope of the project. Whatever the cause, the pragmatic solution is to end the relationship. But when and how do we do this? Too often, it is too late and damage has already been done. Ironically, more damage is probably done by keeping on a bad team member than cutting them loose early on.
- Scope creep: This is the proverbial open tab the business is running up; at some point, someone needs to acknowledge that capacity has been exceeded or that people are losing focus, but instead management and staff simply increase stress levels and workloads and keep slogging along until something or someone actually breaks. Again, the pragmatic solution would have been to notify management early on that staff are stretched thin or that they lack the required skills and experience to take on project X.
Steve McConnell documents a large number of classic mistakes in his writings on “rapid development. ” Most of these mistakes follow the pattern of internal losses discussed above. These mistakes often are the result of decisions made over time that slowly refuse to resolve. Decisions that never directly fail on their own terms, but that still prevent success.
So what can we do?
How can we move from internal to external?
How can we fix our problems and facilitate success?
To start we can put in a circuit breaker, create our version of a standing sell order. When buying stocks, including a standing sell order is a voluntary action. We can ask our broker (or our financial management application) to take care of it for us, but we are not required to do so. Similarly, we can set-up an accountability check regarding our project management decisions.
Software projects are inherently more complex than stocks and yet no one would claim that investing is easy. In investing, a company is reduced to a single number, which is never true of the many working parts of a development project. At the same time, projects do have goals, objectives, milestones and schedules. Sometimes all we need to do is slow down and ask a few questions and determine at which points we need to check in and assess progress.
- Do we have the right tools on the project?
- Do we have the right people on the project?
- Do we have the best chain of command?
- Where are we getting caught up?
- What could be going better?
- What is this going to look like down the line?
The information to make decisions is available. Our problem is disinvesting the ego from the outcome. While also recognizing that the ego may actually achieve greater praise and success, by stopping a mildly bad decision before it devolves into a terrible situation. Unfortunately, humans tend not to be pragmatic when it comes to emotional decisions. Most of us have not spent 20 years or even 3 months in a monastery meditating, so our best option is to set-up a support structure that is invested in positive outcomes, but not in our personal decisions.
Mini Case Study: 7 Stages Too Late
We often hear of the 7 stages of grief. In his book, What I Learned Losing A Million Dollars, Bryan Moynihan notes that a similar process takes place when people are confronted with investment decisions that require accepting that we’ve made a mistake and that there is not any other person or event that can or will pull the plug on our bad decision. I find the same pattern haunts software.
Software development projects are often far from defined and transparent and pulling the plug on our own mistakes is often required. No one else will do it for us.
Let’s say a project manager at Company X writes up a proposal for a new piece of technology that will save her team time. Unfortunately, once the technology is in use, it becomes apparent that to get it to work with the existing system, more time will be spent on the integration than expected.
The project manager is naturally disinclined to admit she made a bad decision or maybe she doesn’t even know there is a problem because the developers don’t want to call attention the specific problem. Who wants to make their boss look bad?
Furthermore, it is much easier to overestimate the gains of the new technology down the road, than to admit right now that forcing it to work is a waste of time and resources. Company X’s team has just fallen into the trap of hoping that time will fix their problem or even make it go away. Eventually, they may have an even bigger problem to be solved.
Given the opaque nature of software, people rarely have a firm idea on how things are going and what the quality of the final result will be. By the time Company X’s project manager and her team finally get around to clearly identifying the problem, she and her team will probably have already wasted valuable time moving through the 7 stages of acceptance. And now, they will be forced into a reactive mode, to find a solution.
In an environment that claims to value pragmatic decision making and maximizing efficiencies, recognizing our problem after having already moved through the 7 stages, is 7 stages too late.
Imagine what could have happened if Company X had prepared to fail, by setting up a system to identify and act on problems as they arise. They could have avoided the 7 stages and eliminated much of the discomfort and the wasted resources involved in resolving an already established problem! No one would have had to worry about losing face or looking bad.
Be Prepared: The Argument for Setting up External Controls
Regardless of the size and structure of your organization, you participate in a decision-making chain of command. Imagine that you or a member of your team has just argued for and brought on a particular piece of technology. Next, imagine that your team, your manager or your fellow developers, in approving the technology request, require that the technology’s integration is assessed after a certain period of time? Say 30 days in?
In general, we don’t implement projects or technology hoping for failure, at the same time, we all know that stuff fails. Sometimes stuff fails because we made a mistake, sometimes because we miscalculated, and sometimes because of an external circumstance that we didn’t know about. Regardless of the cause of failure, the reality is that stuff fails. And, failures are much easier and less stressful to address when we are prepared.
When we set up a process, when we anticipate the possibility of failure, everyone feels comfortable being above board and emotionally safe in acknowledging the truth. Rather than fearing failure, we become empowered to be pragmatic and re-visit alternatives. Our peers or our manager can confidently and safely say, “let’s pull the technology and build what you need.”
Face is saved.
Time is saved.
Lessons are learned.
I cannot overemphasize the importance in making a plan and empowering people to safely make a decision to “flip the switch” early on. Early detection means that everyone is still fresh and able effectively explain what isn’t working and why. Not only are we in an even better position to recognize the actual tool needed to improve efficiency, we will be able to more clearly write up a quick “lesson’s learned” as part of the process to design and implement a better tool. Now, instead of cowering from a mistake, your team is learning, improving, and quite likely avoiding a similar situation in the future.
Preparing for Failure
No one wants to fail, but failure is an inherent challenge of software. A project design that allows us to assess our progress at regular checkpoints, implementing failure analysis along the way, and accepting that a few mistakes are inevitable, turns us into an organization that can learn from its mistakes. A learning organization is able to modify its behavior and improve over time, versus a reactive organization that is perpetually reacting to and scrambling to recover from mistakes.
Learning is Empowering whereas Reacting is Stressful
Preparing for failure also externalizes the emotional side of project management. It is easier and less stressful to share information, good and bad, and to learn from our mistakes when we have structures in place to do so. Being intentional with our development strategy and identifying tactics to externalize decisions makes it easier for us to draw on the help of others when it comes to putting a stop to our own mistakes. Planning for failure is empowering, whereas perpetual crisis management is stressful.
The key to success is preparing for failure before it happens…
I’ll say it again: Mistakes are inevitable. However, as I have shown, when properly managed, there is nothing to be ashamed of in making a mistake. Acknowledging the importance of dealing with mistakes and failures sets the stage to handle them effectively and resolve problems long before they spiral out of control.
Learning to put these structures in place is something does not happen overnight. Make a habit of listening to your gut. If something feels uncomfortable there is probably a reason. Maybe you need to fire your new hire the first time the possibility crosses your mind. And next time, perhaps you should build a 30-day review into new contracts.
Watch where you get stuck or your team gets stuck. Talk openly about the structures or the processes that could be implemented to avoid these issues in the future.
Find ways to avoid falling into the trap of internalizing failures.
Give up the habit of waiting and hoping for improvement.
Get used to the idea that admitting failure in a controlled setting may, in fact, be your key to success.
The question is not if you will fail because eventually, you will. The question is, how do you prepare to handle failure?
If you want to know more about learning from failure I recommend this series of Ted Talks on the benefits of failure. And, check back soon for more on the subject in regards to pragmatic software development. I have much more to say!