Appendix
Remarkability,* or R, refers in a very technical sense to the "weight of evidence," derived from specified data and favoring a most plausible alternative explanatory hypothesis over a null hypothesis. R is most conveniently quantified as bits of information. R maintains two important properties: the value of R expected from an isolated analysis of random data is zero, and more generally, 2R is an estimate of the number of independent analyses of random data that must be performed in order to have a fifty-fifty chance of observing a new R greater than the given R. Thus, the implications of a large positive R are similar to the implications of conventional "statistical significance," but with the advantage that R may be legitimately computed in a variety of situations that do not permit significance tests.
A great advantage of the informational metric used for remarkability is that it is relatively easy, even at the end of the process, to identify and isolate the contribution of the data, independent of any subjective coloration. There is, of course, a subjective component in any complete description of a process for scientific use of data. The unavoidable subjective ingredient in the remarkability approach lies in the choice of a minimum value for "large, positive R." Investigators are free to disagree in this choice, but when they do so, the origin and impact of the disagreement will be immediately clear to all.
The informational transform of p = .05 is 4.25 bits, and the transform of p = .001 is 9.96 bits. Thresholds for minimally impressive R in the range from about 5 to about 15 bits are roughly consistent with typical decision rules used in significance testing.
Refer to the Meaning of Data by D.R. Saunders (privately published) for a complete discussion of these points.
* This explanation has been adapted from a more complete description of remarkability in the paper, "Implication of the Personality Assessment System for Marital Counseling: A Pilot Study," by D.R. Saunders, S.J. Kaplan, and W.G. Rodd, in Psychological Reports, 1980, 46, pp. 15455.