First page
Back Continue Last page Summary Graphics
It is assumed that the majority of the unique noise has been removed, therefore the three GOC curves are tending towards the sample ROC curve. Thus the differences between them can be attributed to common noise sampling variability.
An important point is shown here: as you can see the GOC curves are nowhere near the theoretical ROC curve. Because all three GOC curves are in the same vicinity it can be assumed that the results are not due to an extreme sample ROC curve and that instead, this observer cannot produce ideal performance.
If you remember, one of the strategies for choosing the number of stimuli was to base it on the theory you were testing.
The trouble is, the number of stimuli need to be selected before the experiment. Thus the number of stimuli can only be an educated guesstimate if the true theoretical performance is unknown. Once again, I would suggest doing a pilot study or estimating the sample size based on an area of 0.5.
I don’t really want to comment on the implications of sampling variability outside of psychophysics - for instance - hypothesis testing, which has its own way of dealing with it. However, anytime psychophysical measures are used e.g., the area, log d, d’, P(C) etc. then common noise sampling variability is an issue. All of these measures assume underlying distributions and in all likelihood, experimenters will only be using samples. What sample sizes are common in the literature?