Tukey-Kramer Multiple Comparison Pruning of Neural Networks

Keywords: Multiple Comparisons; Network; Studentized range; Subset selection

Abstract: Reducing a neural network's complexity improves the ability of the network to generalize future examples. Like an overfitted regression function, neural networks may miss their target because of the excessive degrees of freedom stored up in unnecessary parameters. Over the past decade, the subject of pruning networks produced non-statistical algorithms like Skeletonization, Optimal Brain Damage, and Optimal Brain Surgery as methods to remove connections with the least salience. Current algorithms perform in an iterative fashion. There are conflicting views as to whether more than one parameter can be removed at a time. The method proposed uses statistical multiple comparison procedures to remove multiple parameters in the model when no significant difference exists. While computationally intensive, this method compares well with Optimal Brain Surgery in pruning and network performance.