21 Aug Intersect of (I)V&V

For about twenty years, I have been involved in Information Technology (IT) in both the public and private sectors. I have been a business analyst, a software tester, a test manager, and an independent verification and validation (IV&V) manager. I have taught software quality assurance at a Washington DC University. I co-authored the IEEE Std 829-2008, System and Software Test Documentation and I was involved in the development of the IEEE Std 1012-2004, Standard for Verification and Validation and I am currently part of the working group that is developing IEEE Std 1012-x to enhance best practices in system and hardware verification and validation in addition to the current software verification and validation.

Both in the classroom and on the client site I am asked to describe the difference between test and verification and validation. It is always difficult to explain that test performs a number of test tasks and that sometimes verification and validation performs duplicative test tasks.

Test is responsible for a wide-range of test tasks that occur throughout the system development life cycle. Early in the life cycle test will evaluate the requirements (system, software, interface) to insure that they can be validated. Test will become familiar with the concept documentation and the business needs. Later in the life cycle test will, based on the integrity level of the system, determine the necessary breadth and depth of testing. After preparing the required test documentation test will commence testing. This may result in testing each component, the integration of the components, system testing against the system requirements, and acceptance testing to validate that the system meets user needs. Or a determination may be made that test will do only acceptance testing while development is responsible for the earlier testing. After testing is completed test may prepare interim test

reports or merely a final test report. These and other test tasks are fully described in IEEE Std. 829-2008.

If the development effort has a verification and validation component then, based on the identified integrity level of the system, verification and validation will evaluate the adequacy of the test tasks and/or perform its own test tasks in addition to the ones performed by test. If the verification and validation evaluation tasks are performed by persons outside of the development structure, budget, and management then they are called “independent.”

The Verification and validation component will accomplish various tasks in addition to evaluating the adequacy of the test tasks. The verification and validation tasks entail assessing the adequacy of development tasks. This determination is based on the assessment of development artifacts against applicable voluntary consensus standards (IEEE, ISO, ANSI). An assessment of the system development process would use ISO 15288 while an assessment of a requirements document would use IEEE Std. 1233. In addition to assessing the artifact against the applicable standard Verification and Validation assesses the artifact in terms of good engineering practices. Thus an assessment of a design document would not only include the applicable standard but also would include the flow-down of the architecture document, the business needs of the organization, and the system or software requirements. This engineering perspective distinguishes Verification and Validation from Quality Assurance. Another musing will provide additional discussion on the intersect of Quality Assurance and (Independent)Verification and Validation.

Information on determining the integrity level is fully described in IEEE Std. 829-2008 and IEEE Std. 1012-2004. If merited by the identified integrity level, verification and validation will also engage in testing. Their testing may be in parallel with that being done by test or it may be after that done by test.

Just as test has tasks that are performed within the context of the development life cycle so does verification and validation. Only some of these tasks will intersect with the tasks being performed by test. And just as the test tasks will vary according to the identified integrity level of the system being developed so will the verification and validation tasks vary according to the identified integrity level. The table below shows the intersection of recommended tasks for a project with a level 3 criticality level. Projects with lower levels, indicating they are less critical, will have fewer tasks. The selected system development life cycle phases are only examples. Every project will have its own life cycle phases.

Phase: Management

Phase: Acquisition

Phase: Supply

Phase: Development/Concept

Phase: Development/Requirements

Phase: Development/Design

Phase: Development/Implementation

Phase: Development/Test

Phase: Development/Installation & Checkout

Phase: Operation

Phase: Maintenance

As can be seen above verification and validation is responsible for many of the same tasks as is test albeit from a different perspective. Both efforts have to be planned, managed, and executed. In both instances there will be interfaces with other organizational and supporting entities.

For systems with a lower integrity level verification and validation only may review the documentation created by test and witness test execution. For systems with a higher integrity level, verification and validation may review the documentation created by test and generate its own test

documentation. It may witness test execution and perform its own tests based on its own test documentation.

Conclusion:

Even though many organizations think of verification as being the same as test, I have shown that for high integrity systems both should be implemented. For low integrity systems that provide a low risk to the acquiring organization having only project level testing may be adequate.