Seminar - Out of Bounds? Distribution assumptions in validated psychometric instruments

Validated psychometric scales such as the System Usability Scale (SUS), NASA-TLX, and UEQ are staples of empirical HCI research. However, in practice these instruments frequently produce heavily skewed score distributions. SUS scores, for example, routinely cluster near the top of the range. This raises important questions: do the original validation claims — reliability, factor structure, discriminative sensitivity — still hold under pronounced ceiling or floor effects? And how should researchers handle the resulting violations of statistical assumptions?

In this seminar, you will  conduct a literature review mapping what existing work has to say about these questions. What does the psychometric and methodological literature recommend when validated scales exhibit ceiling or floor effects? When are parametric tests still defensible, and when should researchers turn to transformations, non-parametric alternatives, or robust methods? Under what conditions does skew undermine a scale's discriminative power or inflate error rates?