02 Oct Guidance for the Verification and Validation of Neural Networks
Guidance for the Verification and Validation of Neural Networks
First published in Software Quality Professional, Sept. 2010
| Author | Laura Pullum, Brian J. Taylor, Marjorie A. Darrah |   |
| Publisher | John Wiley & Sons, Inc. | |
| Published | 2007 | |
| ISBN | -13: 978-0-470-08457-1-10: 047008457X | |
| # of Pages | 128 | |
| CQSE BOK | Standards and Models, Systems and Softwre Engineering Processes,Software Verification and Validation, Process and Product Measurement |
While traditional software performs its intended function without changing its behavior or its functionality during normal system usage, this is not the case for adaptive software. Adaptive systems, neural networks being only one example, can and do change their behavior, functionality, internal parameters or realization while in their normal operational state. Thus answering the question of how to perform verification and validation (V&V) on adaptive systems looms as an imperative. That, each adaptive system may have its own intrinsic V&V problems and some adaptive software algorithms are so new that they have never been applied to production or high-assurance projects creates a dilemma for the V&V practitioner.
Answering the question and the dilemma are the foundation for this book which definitely fulfills the need with its useful and practical guidelines accompanied by wonderful examples. This book provides a foundation for guidance to V&V practitioners, who may be confronted with the various types of adaptive systems whether these be neural networks, genetic algorithms, fuzzy logic, machine learning, expert systems, autonomous planning software, formal logic or some combination of these.
The guidance provided is based on lessons learned and applied research from the development and V&V of a safety- and mission-critical adaptive flight control system. In presenting this guidance, the authors rely on their V&V research, experience with adaptive systems and the lessons learned from practitioners at the NASA IV&V facility. They have bolstered their research and experience with pertinent information derived from basic and applied research and industry publications. Parts of the guidance are applicable to any form of adaptive system and other parts target neural networks, specifically. The book has been written to align with the IEEE Standard for Software Verification and Validation, IEEE Std 1012-1998, thus confirming the versatility of the standard. As the authors state just as IEEE Std 1012 is intended for software being developed, maintained, or reused, so is this guidance. The producer of adaptive software should consider V&V as part of the adaptive software life cycle process and not something done solely at the end of development.
This small and powerfully written book contains a scant four chapters. Chapter One provides an introduction and overview of the book. Chapter Two provides a summary of the areas of consideration when using adaptive systems/neural networks. Chapter Three provides the detailed, specific guidance for all life cycle processes, activities and tasks related to adaptive systems/neural networks. The organization of the chapter facilitates mapping (cross-walking) between the standard and the provided guidance. Chapter Four provides a high-level description of the updates in IEEE Std
1012 from the 1998 version to the 2004 version. It describes how these updates impact the V&V practitioner using the guidance provided in this book.
In Chapter Two, the authors identify numerous areas that need special consideration from the V&V practitioner. These include, but are not limited to: systems requiring special attention to prevent harm to humans or to the environment; system control and knowledge requirements; system’s training set analysis; and, stability analysis. Each of these areas is fully described and examples are provided. In addition the chapter provides, for each area, information on situations that are more or less appropriate for each identified adaptive system. The authors state that it is important, early in the project, to conduct an assessment to determine the appropriateness of using an adaptive system or neural network. They continue, “[neural networks]” are like multipurpose tools–generally used in statistical problem and different architectures are usable for different problems types.
While Chapter Two presents a mini-tutorial on adaptive systems especially neural networks, Chapter Three follows this up by presenting all of the activities and tasks required to evaluate these systems and places the activities and tasks into the framework of IEEE 1012-1998. For the V&V practitioner this chapter forms the heart of the book. For each task recommended by the standard, the authors describe a specific adaptive system perspective that emanates from the information provided in the prior chapter.
The chapter continues with examples of high-level goals and system requirements for neural networks related to Control and other system requirements related to Knowledge. This allows the reader to follow the required thought and relationship process. In the discussion related to the risk analysis and hazard analysis tasks, the authors include a synopsis of a case study in which nine (9) hazards associated with neural networks are identified. Each of these hazards is then evaluated for frequency of occurrence and severity of failure to plan for sufficient mitigation. The authors also identify a number of metrics that have proven useful in determining the pass/fail performance of the neural network as a whole. There is a definite relationship between these metrics and the included table identifying and describing the neural network quality characteristics. Later in the life cycle, the authors suggest test plans scope and test designs for the test levels (acceptance, system, integration, and component) identified in the standard.
Prior to reading this book, this reviewer had no specific knowledge either of adaptive systems or neural networks other than a lay person’a understanding
of swarming theory and adaptive technology as used in flight-systems, prototype automobiles, and science fiction books. However, after reading this book I had a bit more understanding of the importance and use of adaptive systems. As an independent verification and validation (IV&V) practitioner I had an extremely good general understanding of the need for and the complexity of performing V&V on adaptive systems, especially neural networks. This was best stated by the authors when they wrote traditional software development does not
adequately address the development of adaptive systems, specifically neural networks because one does not write code for these systems, rather they are formed as they are trained or in real-time as they adapt; and because the risks associated with neural networks are heavily tied to the development activity.
This book was not an easy read because of the complexity of the subject matter. However the authors did a masterful job of explaining and describing each step along the way. Unlike many authors they did not skip any steps in explaining the concepts and constructs. Because I wished that they had included some practical examples of adaptive software systems I contacted the leading author and requested that such information be provided for this review. I had expected some esoteric or futuristic applications. Instead
I learned that there are actual working and practical examples of adaptive software systems.
At the time this book was written, there was no similar book available. To this reviewer’s knowledge, there is still no similar book available. Whether readers want to have a general understanding of adaptive software systems or whether they are required to verify and validate any type of adaptive software
system, this is a book that must be read and kept close at hand for easy reference.
