Test Management

Testing is a complex activity. Testing is often a distinct sub-project within the larger softwaredevelopment, maintenance, or integration project. Testing usually accounts for a substantial proportion of the overall project budget. Therefore, its important to manage the testing that is done.

Independent and integrated testing

The approaches to organizing a test team vary, as do the places in the organization structure where the test team fits. Since testing is an assessment of quality, and since that assessment is not always positive, many organizations strive to create an organizational climate where testers can deliver an independent, objective assessment of quality.

When thinking about how independent the test team is, recognize that independence is not an either/orcondition, but a continuum. At one end of the continuum lies the absence of independence, where the programmer performs testing within the programming team.

Moving toward independence, you find an integrated tester or group of testers working alongside the programmers, but still within and reporting to the development manager. You might find a team of testers who are independent and outside the development team, but reporting to project management.


Near the other end of the continuum lies complete independence. You might see a separate test team reporting into the organization at a point equal to the development or project team.

You might find specialists in the business domain (such as users of the system), specialists in technology (such as data-base experts), and specialists in testing (such as security testers, certification testers, or test automation experts) in a separate test team, as part of a larger independent test team, or as part of a contract, outsourced test team.

An independent tester can often see more, other, and different defects than a tester working within a programming team or a tester who is by profession a programmer. 

While business analysts, marketing staff, designers, and programmers bring their own assumptions to the specification and implementation of the item under test, an independent tester brings a different set of assumptions to testing and to reviews, which often helps expose hidden defects and problems related to the group's way of thinking.

An independent tester brings a skeptical attitude of professional pessimism, a sense that, if there's any doubt about the observed behavior, they should ask: Is this a defect? 

At the team level, an independent test team reporting to a senior or executive manager may enjoy (once they earn it) more credibility in the organization than a test leader or tester who is part of the programming team.

An independent tester who reports to senior management can report his results honestly and without concern for reprisals that might result from pointing out problems in coworkers' or, worse yet, the manager's work. 

An independent test team often has a separate budget, which helps ensure the proper level of money is spent on tester training, testing tools, test equipment, and so forth. In addition, in some organizations, testers in an independent test team may find it easier to have a career path that leads up into more senior roles in testing.

Independent test teams are not risk-free. It's possible for the testers and the test team to become isolated. This can take the form of interpersonal isolation from the programmers, the designers, and the project team itself, or it can take the form of isolation from the broader view of quality and the business objec-tives (e.g., obsessive focus on defects, often accompanied by a refusal to accept business prioritization of defects).

This leads to communication problems, feelings of alienation and antipathy, a lack of identification with and support for the project goals, spontaneous blame festivals and political backstabbing. Even well-integrated test teams can suffer problems.

Other project stakeholders might come to see the independent test team - rightly or wrongly - as a bottleneck and a source of delay. Some programmers abdicate their responsibility for quality, saying, 'Well, we have this test team now, so why do I need to unit test my code?'

Due to a desire for the benefits of an independent test team, companies sometimes establish them, only to break them up again later. Why does that happen? A common cause is the failure of the test manager to effectively manage the risks of independence listed above. 

Some test teams succumb to the temptation to adopt a 'no can do' attitude, coming up with reasons why the project should bend to their needs rather than each side being flexible so as to enable project success.

Testers take to acting as enforcers of process or as auditors without a proper management mandate and support. Resentments and pressures build, until at last the organization decides that the independent test team causes more problems than it solves.

It's especially important for testers and test managers to understand the mission they serve and the reasons why the organization wants an independent test team. Often, the entire test team must realize that, whether they are part of the project team or independent, they exist to provide a service to the project team.

There is no one right approach to organizing testing. For each project, you must consider whether to use an independent test team based on

• the project
• the application domain and 
• the levels of risk, among other factors. 

As the size, complexity, and criticality of the project increases, it is important to have independence in later levels of testing (like integration test, system test and acceptance test), though some testing is often best done by other people such as project managers, quality managers, developers, business and domain experts or infrastructure or IT operations experts.

Working as a test leader

We have seen that the location of a test team within a project organization can vary widely. Similarly there is wide variation in the roles that people within the test team play. Some of these roles occur frequently, some infrequently. Two roles that are found within many test teams are those of the test leader and the tester, though the same people may play both roles at various points during the project.

Test leaders tend to be involved in the planning, monitoring, and control of the testing activities on the fundamental test process. At the outset of the project, test leaders, in

collaboration with the other stakeholders;

• devise the test objectives, organizational test policies , test strategies and test plans. 
• estimate the testing to be done and negotiate with management to acquire the necessary resources.
• recognize when test automation is appropriate and, if it is, they plan the effort, select the tools, and ensure training of the team.
• consult with other groups - programmers - to help them with their testing. 
• lead, guide and monitor the analysis, design, implementation and execution of the test cases, test procedures and test suites. 
• ensure proper configuration management of the testware produced and traceability of the tests to the test basis.

As test execution comes near,

• make sure the test environment is put into place before test execution and managed during test execution. 
• schedule the tests for execution and then they monitor, measure, control and report on the test progress, the product quality status and the test results, adapting the test plan and compensating as needed to adjust to evolving con-ditions.During test execution and as the project winds down,
• write summary reports on test status. Sometimes test leaders wear different titles, such as test manager or test coordinator. Alternatively, the test leader role may wind up assigned to a project manager, a development manager or a quality assurance manager. 

Working as a tester

As with test leaders, projects should include testers at the outset, though it is often the case that project doesn't need a full complement of testers until the test execution period. In the planning and preparation phases of the testing, testers should review and contribute to test plans, as well as analyzing, reviewing and assessing requirements and design specifications.

They may be involved in or even be the primary people identifying test conditions and creating test designs, test cases, test procedure specifications and test data, and may

automate or help to automate the tests. They often set up the test environments or assist system administration and network management staff in doing so.

• As test execution begins, the number of testers often increases, starting with the work required to implement tests in the test environment. 
• execute and log the tests, evaluate the results and document problems found.
• monitor the testing and the test environment, often using tools for this task, and often gather performance metrics. Throughout the testing life cycle, 

review each other's work, including test specifications, defect reports and test results.

Defining the skills test staff need

Doing testing properly requires more than defining the right positions and number of people for those positions. Good test teams have the right mix of skills based on the tasks and activities they need to carry out, and people outside the test team who are in charge of test tasks need the right skills, too.

 People involved in testing need basic professional and social qualifications such as literacy, the ability to prepare and deliver written and verbal reports, the ability to communicate effectively, and so on.

Going beyond that, when we think of the skills that testers need, three main areas come to mind:

• Application or business domain: A tester must understand the intended behavior, the problem the system will solve, the process it will automate and so forth, in order to spot improper behavior while testing and recognize the 'must work' functions and features.

• Technology: A tester must be aware of issues, limitations and capabilities of the chosen implementation technology, in order to effectively and efficiently locate problems and recognize the 'likely to fail' functions and features.

• Testing: A tester must have an understanding in testing in order to effectively and efficiently carry out the test tasks assigned.

The specific skills in each area and the level of skill required vary by project, organization, application, and the risks involved. The set of testing tasks and activities are many and varied, and so too are the skills required, so we often see specialization of skills and separation of roles.


For example, due to the special knowledge required in the areas of testing, technology and business domain, respectively, test tool experts may handle automating the regression tests, programmers may perform compo-nent and integration tests and users and operators may be involved in acceptance tests.