Finally, after all the meetings, late nights, early mornings, and long days the application is ready to go. The new software product is launched and everything comes to a grinding halt! The product release turned into a product escape. The help center queues are backed up for hours, angry customers are emailing in to express their dissatisfaction, and worse yet…they start telling other people about what a crappy product this has turned out to be.
You certainly didn’t account for any of this angst in your IT Project Plan. What went wrong? A whole lot of things. Whenever new code is introduced into a production environment there is always a collective holding of the breath until things begin to settle down. In theory, there is a pre-production environment that should precisely mirror production, but this is not always the case. Configurations may not be right, servers aren’t the same, and other applications may conflict with the new application. This causes hearts to race, tempers flare, and profitability to dissipate.
Fingers start to be pointed as to who is to blame. Is it the Quality Assurance team? Certainly they should have caught all these things in their testing. Is it the Product Configuration team? Maybe the deployed the wrong packages to the wrong environment or the right packages to the wrong environment. Is it the Project Manager? This person is responsible for the entire project, right?
Include Production Release Candidate Activity in Your IT Project PlanYou only have to live through a bad deployment once or twice and you will quickly learn that you want nothing to do with that ever again. You’ll start finding ways to prevent that from happening. You’ll look at your IT Project Plan and see what gaps need to be filled.
The following is a starting point if you have experienced the pain of a bad deployment. There’s no guarantee that this will help things go 100% smoothly, but it will certainly make the process smoother and remove the finger-pointing exercise that occurs at the end.
- Create a Form – Sure, a form is the solution for everything that ails a project manager. But, there’s something to be said for that. You want to create a Production Release Candidate form that includes the basics about what will be deployed. Include a brief description, the target date of when it will be deployed, what else will be deployed along with it, and room for LOTS of signatures.
- Talk About It Each Week – Talk about this production release candidate at each of your weekly status or PMO meetings. Start talking about it at least a month out so that people know it’s heading their way. It may be included in the IT Project Plan, but you can’t make assumptions that people will read the plan. IT needs to know it’s coming to make sure the right hardware is in place, QA needs to know it’s coming so they can thoroughly test it, the Documentation team needs to know it’s coming so they can prepare user guides and training manuals. There should not be one person in the company that is surprised that there is a release that will be happening in the next 3-4 weeks.
- Get Approval from All Who Have a Stake in the Project – The key managers from each department should be on that Production Release Candidate form. Once the application or product is in its finished state, you should go around to each manager and get their written approval. There should be no hesitation in signing this form if they feel everything is ready to go. Don’t settle for a verbal approval. Require that this person sign this piece of paper saying that they have done their due diligence in reviewing it from their department’s perspective and feel comfortable with it being deployed. Requiring their signature will make them look at things much closer and remove their ability to point fingers at other departments later. Who are these key managers? At a minimum you should have Engineering, QA, Client Support (Call Center), Marketing, and Training/Documentation. It will be different for every company, but the principle is that you want to make sure that everyone has had a chance to say their peace and feel comfortable with the deployment moving forward.
- Get Approval from One Last Person – You should have room on the bottom of the form from the one person these different departmental managers roll up to (depending upon the size of the company). It may be a VP, COO, or President of the company. This informs them what is going on and lets them know that the entire management staff feels comfortable with what is about to be deployed.You are now in a much more comfortable position to release the product or application into a production environment. You’ve accounted for approvals and sign-offs in the IT Project Plan and everyone’s signature is literally on the same page. There’s no guarantee that everything will go 100% right, but you can be assured that things will go much smoother with this high degree of scrutiny.
An Interesting Thought on Who Should Have the Final SayMost development shops are set up that once QA has signed off then everything should be just fine. They are usually one of the last phases in the IT Project Plan and have one of the toughest jobs. However, I’ve seen some development companies have the final call (if not joint final call with QA) be with the Support Center. This is the group that handles the customer calls and helps them with their issues. In talking to someone who runs a call center, his view was that “every call that comes in is an indictment against the software”. His goal was to mitigate and minimize as many of these calls as possible. This is the group that is on the front lines, hears the angry customers, and is responsible for helping them solve their problems. Unfortunately, these many times take the form of “workarounds” that may or may not ever be fixed the proper way.
Why not let this person be very involved, if not THE person that makes the call on whether a deployment is ready to be released, or if it’s just about to be another product escape? QA many times gets too sterile and removed from the true functionality of the software. Engineers are too close and can’t see the forest for the trees. A project manager just wants it out the door and the Business Analysts have forgotten about it at this point.
All of the above activity, scrutiny, and approvals take time. That’s why it is so important that you account for these in your IT Project Plan. Otherwise, people will be scrambling at the last minute and no good ever comes from last minute fire-drills.