Scorpion is a research project at OCLC Online Computer Library Center, Inc. building tools for automatic subject assignment by combining library science and information retrieval techniques. While Scorpion already produces good subject assignments for documents, it often returns some subjects that a person would not expect or find useful while searching for that document. We are continually seeking ways to remove such unwanted results from the Scorpion output. This automatic clean up or filtering of Scorpion result sets has become a major focus for our research group. To determine if a filter is good, we must be able to determine if the filtered results are better than the unfiltered results. Unfortunately, it is difficult to evaluate large numbers of Scorpion results objectively or to determine when one Scorpion result set is better than another. This paper describes recent research to objectively evaluate the performance of Scorpion and work to compare Scorpion results to the human-assigned classifications in NetFirst.