Title

Experiences and Results from Initiating Field Defect Prediction and Product Test Prioritization Efforts at ABB Inc.

Date of Original Version

5-2006

Type

Conference Proceeding

Rights Management

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ICSE'06, May 20–28, 2006, Shanghai, China. Copyright 2006 ACM 1-59593-085-X/06/0005...$5.00. Can archive pre-print and post-print or publisher's version/PDF

Abstract or Description

Quantitatively-based risk management can reduce the risks associated with field defects for both software producers and software consumers. In this paper, we report experiences and results from initiating risk-management activities at a large systems development organization. The initiated activities aim to improve product testing (system/integration testing), to improve maintenance resource allocation, and to plan for future process improvements. The experiences we report address practical issues not commonly addressed in research studies: how to select an appropriate modeling method for product testing prioritization and process improvement planning, how to evaluate accuracy of predictions across multiple releases in time, and how to conduct analysis with incomplete information. In addition, we report initial empirical results for two systems with 13 and 15 releases. We present prioritization of configurations to guide product testing, field defect predictions within the first year of deployment to aid maintenance resource allocation, and important predictors across both systems to guide process improvement planning. Our results and experiences are steps towards quantitatively-based risk management.

Comments

This research was supported by the National Science Foundation under grants ITR-0086003 and CCF-0438929, and by the Carnegie Mellon Sloan Software Center. We would like to thank Anne Poorman, Janet Kaufman, Patrick Weckerly, and Rob Davenport for their input and expert knowledge.