Implementing Test Process Improvements

Several years ago I was engaged by a client to manage the testing on what was at the time, one of their major projects. When I was initially engaged the project had already delivered several releases, before subsequently being paused and restarted. I started my engagement with a directive to ensure the testing processes were streamlined, and the team were able to meet demanding timescales for a series of fast following releases. I thought I’d document how I approached the task.

Phase 1 – Understand the delivery team construct

Test team setups can be complex at the best of times, with any combination of offshore, onshore, nearshore as well as blends of supplier, permanent resource and contractors. In this particular engagement I quickly established I was working with an extremely diverse setup;

Client Team – Responsible for producing requirements, change requests, integration testing, co-ordination of UAT with business users, co-ordination of OAT and SMAT with appropriate stakeholders

Supplier 1 – Responsible for writing and system testing code, this was undertaken by an onshore architecture and development team, with a blend of nearshore and offshore test resources. There were also nearshore resources from Supplier 1 embedded within the Client Team to assist specifically with integration testing and defect triage.

Supplier 2 – Responsible for writing and system testing code, this was undertaken by an onshore architecture and development team, with a blend of nearshore and offshore test resources. There were also on-shore resources from Supplier 2 embedded within the Client Team to assist specifically with integration testing and defect triage.

Supplier 3 – Responsible for delivering infrastructure, test environments and code deployments. This work was all undertaken by a nearshore team.

Effectively my role was to define and implement the processes/procedures to be used across the Programme, whilst managing the testing undertaken by a blended team made up of; Client permanent staff, contingent labour, embedded resources from the two engaged suppliers, as well as embedded resources from the associated client business unit.

Phase 2 – Learn the system and know your team

Often people try to approach Test Management from a helicopter view, taking the approach that they can get by with just a basic high-level understanding of the system, as they aren’t writing or executing tests. In my experience this couldn’t be further from the truth, gaining sufficient knowledge of the system, the integration patterns and the technologies being used allows you to understand; where testing efficiencies can be made, where team members may need assistance/training, how to effectively assign/troubleshoot blocking defects etc.

For a system as complex as this one I knew I wouldn’t get the level of knowledge I needed overnight. There was also the additional challenge of joining an inflight project, which meant I was working in a fast-paced delivery focussed environment, so I wasn’t blessed with a period of downtime to familiarise myself with the system. Instead I initially spent time reading all the available documentation, before working with SME’s in my team to learn about each of the components, what they did, the technologies used and how the various components interfaced.

This approach had the added benefit of allowing me to question the team, which in turn helped me to refine my knowledge as well as assess their capabilities in terms of strengths, weaknesses, build relationships and get a feel for the individual personalities within the team (more on this to follow in a later blog on how to manage blended teams).

Phase 3 – Do Nothing! 

Now this may sound odd, but when starting a new role it is easy to think “This worked well for me in a similar role previously, I’ll implement the same processes/procedures here”

Often this approach results in implementing change for the sake of being seen to be doing something differently, and sometimes this can be counterproductive. I find the best way to establish how to improve is to understand exactly how the team is working now, what works well, what may benefit from ‘tweaking’ as well as what just doesn’t work and needs ripping up and starting again. Sometimes the ‘Do Nothing’ phase can be the most productive in identifying areas where improvements are needed!!

As I mentioned earlier, I joined the project at a busy time, and whilst this brought its own challenges in many respects it was beneficial. It allowed me to assess many of the existing processes and procedures being undertaken in a ‘real life’ scenario. I spent time with the testers during execution to understand how the team were testing, and where the hotspots were in terms of defects. I also sat in on defect triage sessions to see how they were being run, what sort of reporting was being produced and whether the defect management process was running efficiently.

As the newcomer to a project its often easier to ask questions that others may have thought of but held back on, most of my time was spent asking questions of both my team and the wider delivery team, I’d ask things like;

“So why do we do it like this?”

“What do you think we could do to make this more efficient?”

“Which processes do you find are the most inefficient/time consuming?”

“What do you think we do that works well?”

Often the people following the processes and procedures know there are improvements that could be made, but continue because “we’ve always done it like this” or “it’s been like this since I started” By ensuring they are actively engaged in the review and refinement of the processes and procedures you are more likely to gain their approval and acceptance – Implementing changes without full engagement of the team you need to deliver them is a sure-fire way to ensure that changes are resisted and therefore reduces their effectiveness.

Phase 4 – Establish Priorities 

Having spent time being ‘busy doing nothing’ my next steps were to look at the areas where I felt changes would make the team more efficient. I then split these improvements into various categories;

• Quick Wins – Easy to implement with immediate benefits. This category contained items that were low impact on the team in terms of productivity and low overhead to implement, for example buddying up Supplier SME with Internal Client Team resources to enhance knowledge sharing across the team.
• Small/Medium term changes – Minor changes to a processes or procedure, easy to define but needing engagement and buy in from all impacted parties. For example, updates to the defect management process, and ensuring business representation in defect triage sessions.
• Long Term changes – Changes which are likely to bring the greatest benefit but by nature can’t be made in a few days/weeks. These require engagement and buy in from all impacted parties, and possibly revisions to commercial agreements with Suppliers. This included items such as file sharing between suppliers for early integration testing to de-risk later test phases, or provision of additional test environments.

Within each category I then prioritised the proposed changes based on their value add, whether I could implement them without engagement outside the test team (internal process changes) and the impact/timeline to implement. This helped to inform the next phase of the process;

Phase 5 – Propose, Rollout and Refine

As mentioned in the Phase 3, one of the key elements when proposing and/or implementing change is to ensure that all stakeholders are engaged throughout the process. If you approach the owner of a defect management process, give them a document and simply state ‘you need to do it like this now’ you’ll be likely to get resistance!

With this in mind and having identified and prioritised the proposed changes, I then re-engaged with the senior Programme stakeholders and the impacted parties to walkthrough my proposals. This allowed them to understand not only what I was looking to do, but more importantly why, and what proposed benefits the changes would provide to both the organisation and the individuals.

As always, some proposals were accepted, agreed and rolled out, whilst others were deemed not to provide return on investment over the lifetime of the programme and were rightly rejected.

When taking the next steps and actually rolling out changes it is critical to appreciate that they will take time to bed in, and to use this time for review and reflection. It is particularly important to seek feedback from those people working with the processes who helped to identify, review and refine them in the first place;

• Is it working the way you expected?
• Are there issues you hadn’t anticipated, and if so what needs to be done to address them?
• Do we need to go back to the drawing board or can we fix forward?

Not everything works first time, and no matter how many review cycles you undertake there will always be the moments you think, ‘Why didn’t we think of that’ which is why I used the first 2 weeks after implementing a change as an opportunity to refine the processes, making minor tweaks and fine tuning them until we had something we could run with and assess over a longer timeframe.

Phase 6 – Repeat to fade

All too often processes and procedures once implemented are embedded, and become second nature to those working with them (see Phase 3 “we’ve always done it like this” or “it’s been like this since I started”) In reality it is critical that frequent reviews are held following the implementation of changes, to establish the effectiveness, have the proposed benefits been realised etc.

As well as a implementing a structured review and refinement model it is equally important to appreciate that new personnel joining a project, or new technologies becoming available/implemented within an organisation will bring new ideas or offer opportunities for improvement within your organisation. If you’re not moving forward you’re falling behind….

We can help you undertake either a full Quality Assurance review, or simply assist you in reviewing and enhancing your test processes and procedures. For more information on how, or if you would like to discuss the contents of this blog in more detail please do get in touch at info@seriosgroup.com.