Master - Lab - Usability evaluation of a tutorial for power analysis
Supervisor: Anna-Marie (ortloff@cs.uni-bonn.de)
You can do this lab as a group.
In Usable Security and Privacy, a priori power analysis is only rarely used to calculate necessary sample sizes prior to conducting a study [1]. Reasons for this could be that the procedure of power analysis and necessary values (effectsizes) vary depending on the planned statistical test [2,3] and the terminology which is used may also differ between different tools and publications [2,3]. In our work in [1], we created tutorials for power analysis and published them on a companion website (https://powerdb.info/) - first as a PDF, but they will be integrated into the website in the future.
Your task:
- Familiarize yourself with the basic concept of power analysis
- Usability evalution (e.g. heuristic evaluation, cognitive walkthrough) of the tutorials [4]
- Development of improvement for the identified usability problems
Literature to start with:
-
[1] Ortloff et al. (2023). SoK: I Have the (Developer) Power! Sample Size Estimation for Fisher’s Exact, Chi-Squared, McNemar’s, Wilcoxon Rank-Sum, Wilcoxon Signed-Rank and t-tests in Developer-Centered Usable Security, available online her: https://amortloff.github.io/papers/Ortloff2023_SOUPS_DeveloperPower.pdf , companion website: https://powerdb.info/
-
[2] Faul et al (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences, tool available online here: https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower
-
[3] Ellis (2010). The Essential Guide to Effect Sizes. Statistical Power, Meta-Analysis, and the Interpretation of Research Results. (Book can be lent from the work group if interested)
-
[4] Usability Methoden: Kapitel 14 aus Baxter et al. (2015). Understanding your users: a practical guide to user research methods. (PDF can be provided if interested)