The report reflects both the authors' efforts to be thorough and their reservations about the limitations of the study. Unfortunately, however, the validity of the report was doomed from the start -- they had no way to properly apply the scientific method, for they had no way of creating a valid control group. They eliminated the plus-tick requirement on about a third of the stocks in the Russell 3000, starting in May 2005, and compared the price action of those stocks with the price action of the rest of the stocks in the same index, which remained subject to the rule, as did all others that were not in the index. Within this context, the statisticians were of course careful in their selection and overall methodology, but remember the three people stuck on a desert island with a case of canned beans and no way open them? The economist says, "Assume we have a can opener..."
There was simply no way to study the effects on those individual securities of a market in which all of the other securities traded were not themselves subject to the rule. In other words, the rule was designed to address negative emotional behavior, and emotional behavior might be reasonably considered to be affected by the price behavior of the majority of stocks in the market that were not included in the study, of course. For the study group to effectively test for the outcome they sought would be so cumbersome, if even possible, that they really had no choice but to assume it away. Their only other choice was not to do the study at all, and that may not have been their best option under the circumstances.
How the Change Went Through
It is not really fair to lay all of the blame at the door of those running the SEC, however. They did solicit opinions from all of us at the time. Although they might have stepped back and decided at any moment that the combination of faulty methodology and the possibility of bias in the motivations of those arguing the loudest for the rule's elimination made it a risk not worth taking, the blame also lies with traders, especially professionals. Many of us who did not comment might have thought how obtuse the whole notion was of doing away with the rule really, yet we did little to argue the point with the people who would ultimately make the decision. I know I did, and didn't.