Organizer:

James E. Gentle, jgentle@gmu.edu

Speakers

10:15 a.m.
AETGWeb - A Combinatorial Approach to Designing Test Cases for Testing Software
Sid R. Dalal, A. Jain, G. Patton and M. Rathi, Telcordia Technologies (Formerly Bellcore)

Software testing is a critical component of software development and can consume as much as 30% of total resources. A significant part of testing is generation of efficient test cases. The combinatorial design method is a relatively new approach to the test generation. It uses a new set of combinatorial design algorithms to reduce the number of tests. This paradigm has been implemented in AETGWeb system and is being used by several groups in Telcordia and elsewhere for unit and system testing. In this talk we compare this approach to the standard experimental design approaches and report on some experiences and experiments. We also describe a new web based service and a high level language for specifying requirements so that test cases can be generated automatically.

Our experience is that the combinatorial design method can be used effectively by practicing engineers to test products.

10:45 a.m.
Statistical Reference Datasets (StRD) for Assessing the Numerical Accuracy of Statistical Software
W.F. Guthrie, NIST (Gaithersburg)
J.E. Rogers, NIST (Boulder)
J.J. Filliben, NIST (Gaithersburg)
L.M. Gill, NIST (Gaithersburg)
E. Lagergren, NIST (Gaithersburg)
M.G. Vangel, NIST (Gaithersburg)

With the widespread availability of statistical software, concerns about its numerical accuracy are now greater than ever. Inevitably, numerical accuracy problems will affect some of this software despite extensive testing. In response to concerns of both the statistical community and industrial users, the statistics and mathematics groups at NIST have developed a web site that provides reference data sets for checking the numerical accuracy of computationsfor a variety of statistical methods. This service is called Statistical Reference Datasets (StRD). The StRD web pages are intended to complement the testing of statistical software by providing data sets and corresponding certified values of commonly-computed statistics for comparison with output from a user's software.

This talk will describe

1. sources of numerical inaccuracy in mathematical and statistical software,
2. selection of reference data sets for different types of statistical procedures,
3. computation of error-free statistical results for each data set, and
4. use of the StRD web site.

The types of statistical procedures discussed will include univariate summary statistics, linear and nonlinear regression, and analysis of variance (ANOVA).

11:15 a.m.
FDA and Software Validation
Ghanshyam Gupta, U.S. Food and Drug Administration
Peter A. Lachenbruch, U.S. Food and Drug Administration

We discuss things to validate (e.g., programs, macros, and data translations), who is responsible for validation, and what the FDA needs to see in order to have assurance that the software is functioning as purported. The major software vendors have well-developed processes for validation and can usually supply the needed documentation. The FDA does not usually want to see 20,000 pages of computer output documenting accuracy under a variety of extreme conditions. The validation process for user-developed macros (e.g., from statlib) is also needed since these usually do not guarantee accuracy. The ability to trace subsetting and output 'prettyfiers' is important. Data translation from one program to another needs to be checked for accuracy, missing values, etc.

11:45 a.m.
Discussant: Bruce McCullough, Federal Communications Commission