With the pressures of deadlines and budget constraints, IT managers find it hard to do anything but develop applications quickly and push them through to production. Although this initially makes business departments happy, the accolades can be short-lived. In the long run, software delivered without proper testing can lead to costly problems.

Cut the Costs

A conservative estimate is that the cost of finding a bug after an application has been deployed to production is ten times greater than if that same bug was found in the initial stages of development and 100 times greater than if it was found during application design. The only way to improve the reliability of delivered software is to make a commitment to creating a comprehensive testing culture in the organization.

Creating a testing culture and adopting a testing process that is well-defined, repeatable, and manageable will reduce the amount of time and money spent on maintenance and on post-production defects. Follow these five steps to get the IT department on its way to producing quality software.

Recommendations

1. Communicate the value of software testing. Not everyone will value testing, but perhaps they can place value on the cost of a major defect, in terms of lost productivity, revenue, and reputation. Work on changing the mindset within the organization to focus on application quality (of which testing is only a single component). This shift in focus will help everyone involved in the project realize that quality is a team effort and not the responsibility of the developer alone.

Focus on controlling the development process not the product. Incorporate testing into the development cycle by including testing in the project’s time and resource estimate. Once the entire enterprise accepts that time must be set aside for testing, pressure is relieved from IT managers who no longer have to fear the delivery of products with known defects.

2. Create a good testing team. No one likes checking their own work. Moreover, the author may glaze over mistakes that a fresh pair of eyes will catch. Do everyone a favor and remove the testing responsibility from the developer. Outside of Unit Testing, the developer that analyzes and designs the code should not be the person responsible for ensuring that the end product is thoroughly tested.

Designate people responsible for uncovering defects in the application. If hiring a testing team is not in the budget, make use of existing staff. Many successful testing installations rotate the developing and testing functions around their team. For example, one person may be working as a developer for systems A, B, and C, but working as a tester for systems X, Y, and Z. Develop a valuable testing team by using the Info-Tech Advisor template, “Test Team Experience Checklist.”

3. Develop a testing strategy. The purpose of testing is to find defects, so, any testing strategy should have the goal of finding the most defects with the least amount of resources. A test strategy should describe the objectives of the tests, the types of testing that will be carried out, the sort of risk that is acceptable for the software, and the amount of effort that will be spent on each objective.

Consider the following when devising a testing strategy:

• Testing and development should be done in tandem. It is not wise to leave testing to the end of the development cycle. For example, develop tests during design phases and include testing after each phase of development.
• Testing should be repeatable. There are plenty of automated tools that support building test case repositories and mechanical executions. For more information on automated testing tools, refer to the Info-Tech Advisor research note, “Get to Know Automated Testing Tools.”
• Regression testing should not be ignored. It is not sufficient only to test the new development while assuming that the existing components are not affected.

4. Create test plans. Once testing objectives are set, specific details about how the testing will be carried out need to be defined. Test plans ensure that there is no confusion by making the entire team aware of the procedures and protocols. For more information on creating test plans, refer to the Info-Tech Advisor In-Depth report, “Developing Application Test Plans.”

Include the following in the test plan:

• The scope of the testing, which defines functionality that will be tested.
• All testing tasks and who will complete them.
• The resources to be allocated or acquired, including test data.
• Test cases that focus on black box testing.
• A testing schedule detailing when and how the software will be tested.

Following the development of the strategy and plan, get commitment and sign-off from the project sponsor. This confirms that the sponsor is satisfied with the testing strategy and is willing to share the risk if the system fails for reasons beyond the scope of the plan.

5. Set up a proper testing environment. Testing should not occur in the production environment or on developer’s local machines. Instead, complete all testing in a dedicated testing environment that is configured with all the tools and resources required to complete the test plans.

The testing environment should be a reasonable simulation of the production environment in terms of server components, network connections, and application setup. Valid data is crucial to the test environment. Work with a recent copy of production data whenever possible; otherwise, create the test data when test cases are developed. For more information on creating a test environment, refer to the Info-Tech Advisor research note, “Test Labs: Nothing to Get Testy About.”

Bottom Line

Software testing is a constantly overlooked issue. Nevertheless, a software project cannot be delivered on time and on budget without the essential components of testing. Create a testing culture in the organization with these five steps to ensure software reliability.