Study the Studies
“Facts are stubborn, but statistics are more pliable.”
Mark Twain
A new study has been published by the American Journal of Obstetrics and Gynecology. Predictably, as the prominent organizations of OB/GYN’s have denounced home birth for years, the study determined that home birth posed a significant risk of death for the newborns. However, having familiarized myself in only a peripheral way in statistical analysis, it was quickly apparent to me that this study was unreliable at best, just from reading a synopsis of it. How could I know that so quickly?
The first step in evaluating a study is determining how the data was gathered. The gold standard of scientific studies is the double-blind prospective study. This means that participants are randomly assigned to comparative groups, with neither the participant, care-giver nor author knowing which group is which, at the onset of care and evaluating the results going forward. Of course, this sort of study would be impossible since parents themselves choose place of birth and care giver.
But, the most unreliable method of studying scientific data is the retrospective study, particularly a meta-analysis. A retrospective study looks at historical data, data acquired any time in the past, whether recently or decades ago. A meta-analysis is a whole group of studies which were designed to evaluate different things, but have some factors in common. Combining these two types of studies consists of compiling all sorts of data from studies large or small, over sometimes a significant period of time. The design of this type of study is usually intended as a first step in designing a prospective study, culling data to lead investigators to factors worth studying. These studies should not be used as a means to draw conclusions on their own. That is contrary to their very design!
In my email, I was forwarded a response to the study in question that has been submitted to the journal which published it, and has been accepted for publication. This Letter to the Editor details specific areas in which the study is flawed.
Maternal and newborn outcomes in planned home birth vs planned hospital births: a metaanalysis •
02 July 2010
Joseph R. Wax, F. Lee Lucas, Maryanne Lamont, Michael G. Pinette, Angelina Cartin, Jacquelyn Blackstone
American Journal of Obstetrics & Gynecology
This study is so deeply flawed that it cannot be seen as anything other than politically motivated. Below find the letter that Dr. Patti Janssen and Dr Michael Klein sent to the Am J of Obs and Gyn and which has been accepted for publication.
Michael
To the Editors:
The recent paper comparing maternal and newborn morbidity among births at
home, hospital and in birth centers by Wax et al, reported that babies born
at home more frequently experienced 5 minute Apgar scores below 7.1 The
methodology employed brings into question the validity of this conclusion.
This retrospective study utilized 2006 US Standard Certificates of Live
Birth, used by 19 states in the US. To establish a low obstetrical risk
population, multiple exclusions were applied to the data with the result
that only 36.0% (745, 690/2,073,368) of women in participating states were
included. Inclusion of only slightly more than one third of the potentially
eligible population raises questions about the ability of birth certificates
to identify women at low risk and consequently the generalizability of study
findings.
Secondly, ascertainment of the type of birth attendant is missing for 4801
women or 0.6% of the sample. It is possible that at least some of these
births were unattended. If this indeed the case, then these births, which
would be expected to have high rates of suboptimal outcomes, might be
over-represented in the home birth group, where the attendants are less
likely to arrive on time for a precipitous birth. In addition, some women
may have deliberately chosen to have an unattended birth and these would of
course take place outside of a hospital or birth centre.
Since only 75% of the births studies were recorded as attended by a
physician or midwife, fully one quarter may have been unplanned home births.
Unplanned home births are well known to be at higher risk for adverse
outcomes.
Lastly, the authors acknowledge that births for which complications
necessitated transfer to hospital are attributed to hospital rather than to
home or birth centre births. In contrast to the above biases, this bias
would favor home births. They also acknowledge that perinatal mortality is
not measured, which eliminates deaths occurring during labour.
In view of these serous flaws, the statement that this study provides a
“robust evaluation of maternal and newborn outcomes that is generalizable
and reflects actual practice” cannot be supported. Without internal
validity, placed in question by missing data and the inability to attribute
births to planned place of birth, the issue of external validity or
generalizability is irrelevant. Recent studies in Canada2, 3 and the
Netherlands4 have used population-based perinatal databases with mandated
participation by midwives and documentation of intended place of birth and
attendant, as well as relevant outcomes including intrapartum fetal death.
American studies of place of birth must meet this standard in order to draw
valid conclusions and allow international comparisons.
1. Wax J, Pinette M, Cartin A, Blackstone D. Maternal and
newborn morbidity by birth facility among selected United States 2006
low-risk births. Am J Obstet Gynecol. 2009;202(2): 152e151-e155.
2. Janssen P, Saxell L, Page L, Klein M, LIston R, Lee S.
Outcomes of planned home birth with registered midwife versus planned
hospital birth with midwife or physician. Can Med Assoc J. 2009;181:277- 383.
3. Hutton K, Reitsma A, Kaufman K. Outcomes associated with
planned home and planned hospital births in low-risk women attended by
midwives in Ontario, Canada, 2003-2006: A retrospective cohort study. BIRTH.
2009;36(3):180- 189.
4. de Jonge A, van der Goes B, Ravelli A, et al. Perinatal
mortality and morbidity in a nationalwide cohort of 529, 688 low-risk
planned home and hospital births. BJOG. 2009;116:1177- 1184.
