Independent Verification and Validation (IV&V) Through the Eyes of DoD

IT departments at different agencies and organizations across the government each have ways of conducting their testing and evaluation activities. In the eyes of the U.S. Department of Defense, Independent Verification & Validation (IV&V) is an independent system assessment that analyzes and test the target system to 1) ensure that it performs its intended functions correctly, 2) ensure that it performs no unintended functions, and 3) measure its quality and reliability.

What is IV&V?

In the federal IT world it is often asked, “What is the difference between verification and validation?” Simply put, verification ensures the software product is built correctly while validation ensures the right software product is built. The intent of verification and validation is to improve the quality of the software during its life cycle, not afterwards, and must be performed as the software is being developed. Federal organizations requiring very high level of accuracy in the estimation, design, construction, execution, and management of their IT programs have long used some form of independent verification and validation to assure software quality. This process is sometimes used internally as a “sanity check.”

Independence

IV&V teams are independent of the development organization on a technical, managerial, financial, and contractual basis, but have well-established, working relationships with the development organization. Early this year, the U.S. Department of Education published an IV&V handbook that stated:

• Technical independence requires that IV&V personnel not be involved in any stage of the software requirements, design, or development process.
• Managerial independence requires that IV&V responsibility be vested in an organization that is separate from the development and program management organizations. The independent selection of the artifacts to be examined and tested, the techniques to be used, the issues to be chosen, and the reporting to be made further affirm this independence.
• Financial independence requires that the IV&V budget be vested in an organization independent from the development organization.
• Contractual independence requires that the IV&V contract be executed separately from the contract for development.

The IV&V team will generate the test plans, test designs, test cases, and test procedures in preparation for IV&V testing. This independent testing will complement rather than duplicate the development team’s testing.

Types of Testing

As a former Naval Sea System Command (NAVSEA) test engineer, the IV&V teams I had the pleasure of working alongside, 1 or 2 FTEs, conducting three primary test events when ensuring the software product was ready to move forward in the software acquisition life cycle. The team makeup is different depending on the software being developed, resource capacity, and organizational experience but throughout the entire DoD, it’s customary to conduct three unique tests before the software goes into production:

• Regression Testing
• Functional Testing
• Non-functional testing

Depending on the organization’s capacity, experience, and system being developed, some IV&V teams conduct Interface testing but this is usually done at system level integration.

REGRESSION TESTING

Regression testing is defined as any type of testing that seeks to uncover new bugs or defects in existing functional and non-functional areas of a system. Regression testing is typically conducted after changes such as enhancements, patches or configuration changes may have changed the behavior of the system.

FUNCTIONAL TESTING

Functional testing is a combination of quality assurance and software testing. Functional testing both “verifies a program by checking it against design document(s) and requirement(s)/specification(s)” (formally the quality assurance process), and by “validating the software against the published user or system requirements” (Software Testing).

NON-FUNCTIONAL TESTING

Non-Functional testing is the testing of non-functional requirements of a software application and includes a lot of mini-tests. Defense organizations typically focus on five non-functional tests: Stress, Endurance, Performance, Security, and Usability.

• Stress test is thorough testing used to determine the stability of a software product or system. IV&V teams deliberately try to break the system.
• Endurance test is testing software for significant periods of time to discover how the software behaves under sustained use.
• Performance test is testing to determine how the software performs in terms of responsiveness and stability under a particular workload. Performance testing is achieved throughout the testing cycle.
• Security test is a process to determine if the system is protecting its data and maintaining the functionality as intended.
• Usability test is a test used to evaluate a software application by testing it on users.

*System and Software are used interchangeably