Which statement about post-hoc power analyses is true?

• A. They provide a precise forecast of the study's future success.
• B. Post-hoc power analyses help refine hypotheses only.
• C. They determine the study's sample size after data collection.
• D. Post-hoc power analyses are controversial and often misleading; power is determined by effect size and sample size; better to report observed effect sizes and CIs.

Answer: (D)

Post-hoc power analyses try to assign a probability of detecting an effect after you’ve collected the data. This approach is problematic because the calculated power basically mirrors the observed effect size and the actual sample size, making it circular and unstable. In other words, once you’ve seen the results, the computed power doesn’t give a reliable forecast for a future study; it can be distorted by the randomness of the current sample and the particular observed effect. Because of these issues, post-hoc power analyses are controversial and often misleading. The clearer, more informative practice is to report the observed effect size with a confidence interval, which communicates magnitude and precision, and to use that information (along with planned power analyses for future studies) to decide on the needed sample size for replication or new research.