Dear Editor
Aronson rightly acknowledged that observational studies are only generating conjectures and warned that residual confounding can be quite difficult to fix (1). However, the issue is deeper.
Indeed, the case of the Finnish study reporting elevated suicide rates during a full-moon only among premenopausal women and only in winter that was debunked by Plöderl and colleagues questions the integrity and the professionalism of our system.(2) Indeed, hypotheses were not pre-specified and results only identified in subgroups, not to mention the lack of rationale.
Why is not pre-registration–e.g. using Open Science Framework (3) of the hypotheses, designs and analytical plan–yet the mandatory norm in observational research? The Original Sin of observational studies is that they are subject to vibration of effect due to the large number of researcher degrees of freedom inherent with such analysis.(4) Such flexibility is a gateway for p-hacking and/or HARking which stands for “Hypotheses After the Results are Known”, as in the Texas sharpshooter fallacy, the shooter painting the target centered on his hit.(5)
Additionally, editors should require the use of STROBE guidelines by researchers for the reporting of observational research. STROBE stands for an international, collaborative initiative of epidemiologists, methodologists, statisticians, researchers and journal editors involved in the conduct and dissemination of observational studies, with the common aim of STrengthening the Reporting of OBservational studies in Epidemiology (6). STROBE is to observational studies as CONSORT is to randomized clinical trials.
Better attention to those two quite simple practices may surely improve the overall credibility of observational research findings.
References
1. Aronson JK. When I use a word . . . . Too much healthcare-observational studies. BMJ. 2022;379:o2734. doi:10.1136/bmj.o2734
2. Plöderl M, Westerlund J, Hökby S, Hadlaczky G, Hengartner MP. Increased suicide risk among younger women in winter during full moon in northern Europe. An artifact or a novel finding? Mol Psychiatry. 2022. Online Oct 20. doi:10.1038/s41380-022-01823-0
3. OFS support. Create a Preregistration. Available at https://help.osf.io/hc/en-us/articles/360019738834-Create-a-Preregistration Accessed 14 November 2022.
4. Patel CJ, Burford B, Ioannidis JP. Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. J Clin Epidemiol. 2015 Sep;68(9):1046-58. doi: 10.1016/j.jclinepi.2015.05.029.
5. Braillon A. Vitamin D and breast cancer: Stop torturing the data!. Cancer. 2022;128(15):2999. doi:10.1002/cncr.34272
6. STROBE. Available at https://www.strobe-statement.org/ Accessed 14 November 2022
STROBE and pre-registration of observational studies vs p-hacking and p-HARKing