Figures
Abstract
Background
There is a need for studies evaluating maternal health interventions in low-income countries. This paper evaluates one such intervention designed to promote maternal health among rural women in Nepal.
Methods and Results
This was a five-year controlled, non-randomised, repeated cross-sectional study (2007, 2010, 2012) of a participatory community-based maternal health promotion intervention focusing on women’s groups to improve maternal health services uptake. In total 1,236 women of childbearing age, who had their last child ≤ two years ago, were interviewed. Difference-in-Difference estimation assessed the effects of the intervention on selected outcome variables while controlling for a constructed wealth index and women’s characteristics. In the first three years (from 2007 to the 2010), the intervention increased women’s likelihood of attending for antenatal care at least once during pregnancy by seven times [OR = 7.0, 95%CI (2.3; 21.4)], of taking iron and folic acid by three times [OR = 3.0, 95%CI (1.2; 7.8)], and of seeking four or more antenatal care visits of two times, although not significantly [OR = 2.2, 95%CI (1.0; 4.7)]. Over five years, women were more likely to seek antenatal care at least once [OR = 3.0, 95%CI (1.5; 5.2)], to take iron/folic acid [OR = 1.9, [95% CI (1.1; 3.2)], and to attend postnatal care [OR = 1.5, [95% CI (1.1; 2.2)]. No improvement was found on attending antenatal care in the first trimester, birthing at an institution or with a skilled birth attendant.
Conclusion
Community-based health promotion has a much stronger effect on the uptake of antenatal care and less on delivery care. Other factors not easily resolved through health promotion interventions may influence these outcomes, such as costs or geographical constraints. The evaluation has implications for policy and practice in public health, especially maternal health promotion.
Citation: Sharma S, Teijlingen Ev, Belizán JM, Hundley V, Simkhada P, Sicuri E (2016) Measuring What Works: An Impact Evaluation of Women’s Groups on Maternal Health Uptake in Rural Nepal. PLoS ONE 11(5): e0155144. https://doi.org/10.1371/journal.pone.0155144
Editor: C. Mary Schooling, Hunter College, UNITED STATES
Received: March 2, 2015; Accepted: April 25, 2016; Published: May 23, 2016
Copyright: © 2016 Sharma et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data are third-party data. The data sets belong to Green Tara Nepal who gave us permission to use the data for this paper. As a charity working in the development field Green Tara Nepal is the ethical and legal guardian of the data which they are willing to provide to researchers on request. We confirm that other interested researchers can obtain the data the same way the authors obtained the data by contacting by email the Programme Manager: greentaranepal@gmail.com.
Funding: This work was supported by Green Tara Trust (UK) and Bournemouth University (UK). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Health and development interventions in low-income countries (LICs) need high-quality evaluation [1–3]. Although evidence-based practice is making inroads into public health, there are still too many programmes that lack impact evaluation [4–6].
The main question of impact evaluation is one of attribution: isolating the effect of the programme from other intervening factors and potential selection bias of participants [7–10]. Impact evaluation assesses the medium- to longer-term changes of selected outcomes that can be attributed to a particular intervention, such as a project, programme or policy. These may be both the intended and, unintended outcomes [9,11].
In 2007, Green Tara Nepal (GTN), a non-governmental organisation (NGO), designed and implemented a five-year intervention to improve maternal health uptake in rural Nepal. The intervention included participatory action approaches, health promotion and small incentives. In Nepal, there is inequitable access to maternal health services due to factors such as geography, caste, ethnicity, religion and corruption [12–15].
The GTN intervention was designed to fit national policies and incorporated maternal health promotion focusing on women’s groups [16–18]. Such health promotion empowers women to seek care when it is needed, which is particularly important in low-income settings [4,19].
This paper reports on the impact evaluation of GTN, selecting as surrogate maternal mortality outcomes factors such as antenatal care (ANC), skilled birth attendance (SBA) and institutional delivery. These outcomes were chosen because they are indirectly correlated with maternal mortality [20–22]. Difference-in-Difference (DiD), analysis was used to identify and measure the effect of the intervention on the selected outcomes [23–28].
Methods
Study area and population
The programme selected two districts (one intervention and one control) with similar socio-economic characteristics. The districts were close to Kathmandu, each with a total population of just under 9,000. Baseline population characteristics (wealth, age, and education) and health system characteristics were not statistically different between intervention and control districts [29–30]. For instance, the intervention area has a community hospital providing basic emergency obstetric care and two health posts. The control area has two health posts and a primary health care centre nearby (similar to the community hospital in intervention area).
The intervention villages were chosen from a few pre-selected districts not far from Kathmandu that were (a) safe to work at the time of Maoist rebellion (1996–2006), which was still on-going at the time the intervention was designed (2005/2006); (b) with the local maternal health needs identified by the community and (c) with political commitment towards change [31].
Intervention design
Two health promoters, an auxiliary nurse midwife and a community medical assistant, established and supported women’s groups with enrolment running between 2006 and 2012. Groups comprised (a) women age 15 to 49 with at least one child under the age of two; (b) mothers-in-law; and (c) husbands, as the latter two are particularly influential in terms of women’s ability to access health services [19, 29, 32–33]. Family decision-making affects the first of the “3 delays” in the maternal morbidity and mortality conceptual framework, namely the delay in making the decision to seek care [34–35].
The intervention took advantage of existing regular monthly women’s group meetings, which were originally savings or literacy-based. The rationale was that women’s groups have the potential for scaling-up within existing social systems, where community and health institutions are working in synergy to improve access to quality maternal and newborn health services [36]. Once the GTN intervention was implemented, its groups met regularly (monthly) for health promotion activities. The evidence-based maternal health promotion was designed by GTN and maternal health researchers and delivered by GTN health promoters. Participatory activities with visual cards and role-playing were conducted on for example ANC, iron/folic supplementation, danger signs of pregnancy, safe delivery and postnatal care (PNC). The implementation consisted of 24 sessions of health promotion, each one lasting one hour. In 2010, there were 37 groups (of 12 daughters-in-law, two men and 23 mixed mothers-in-law/daughters-in-law) reaching over 1,100 people.
By 2012, there were 46 groups (11 daughter-in-law groups, two male only groups, and 33 mother-in-law and daughter-in-law groups) reaching over 733 people.
GTN’s prerogative was to include all castes and empower lower castes to attend. The groups were generally mixed-caste, however, in certain areas, (where not all castes are represented) groups consisted of only one or two castes (S2 Fig).
The GTN health promoters supported the existing health system of sub-health posts by health communication training with mother-child health workers, traditional healers and hospital staff (e.g. neonatal training); and mobile clinics visits to outlying areas.
The intervention included small individual incentives consisting of a baby blanket awarded on completion of four ANC visits, safe delivery kits available at subsidised prices and other goods of the value of less than ten US cents. These incentives aimed to encourage group attendance and health- care seeking behaviour.
Finally, the intervention was flexible; at each stage, barriers were identified; solutions incorporated and the intervention was reapplied to meet the local needs [31].
Evaluation design
This was a five-year controlled non-randomised intervention. Surveys for its evaluation took place at three points in time (repeated cross-sections): before the intervention started in 2007 (baseline), after 2½ years, in 2010, and after five years in 2012. Trained fieldworkers conducted survey interviews. Surveys were conducted in four villages, two in the intervention and two in control districts. The two villages of the intervention district were the ones where the intervention took place; the two villages of the control area were very similar to the intervention villages in terms of health system characteristics and socio-economic conditions. Within the district, the intervention was carried out in two villages selected based on (a) the presence of an adequate health system infrastructure, specifically a community hospital basic emergency obstetric care and two health posts; (b) having obtained approval from the village head; (c) the willingness to collaborate of the local health staff; (d) being neither the richest nor poorest in the district. The list of all women in the four villages (N = 474, 484 and 463, in the first, second and third survey, respectively) meeting inclusion criteria (having at least one child under the age of two years at the moment of survey) was compiled for both the intervention and the control areas. Hence all women meeting the inclusion criteria in the area determined the sample size, and the sampling frame covered the total sample. Women were then approached and interviewed at home. If women were not available on the first home visit, interviewers returned on several occasions, and after the third negative attempt women were dropped from the list. Health and socio-economic data were collected on individual, household and village level predicators/indicators using a structured questionnaire.
All women who met the inclusion criteria, not only those who participated in the groups, were eligible for this evaluation: the evaluation aimed to capture positive ‘spillover’ effects as any community-based programme can have an impact not only on the women directly receiving the intervention but also on the overall community [37,38].
The Nepal Health Research Council (NHRC) granted ethical approval for the study. The study aims were verbally explained to participants, as well as the consequences of their involvement and the right to withdraw at any point. After verbal consent all data were recorded anonymously.
Data analysis
Descriptive analysis included demographic, cultural and socio-economic characteristics as well as maternal outcomes data and decision-making. DiD assessed the individual probability of engaging in each of the outcome variables and measured the difference in each outcome between intervention and control groups and before and after treatment while controlling for potential explanatory variables.
Logistic multivariate regressions were applied. Control variables, in addition to the ones representing the impact of the intervention, were chosen based on the literature and from previously published Demographic Health Studies data based studies [12,39].
With the aim of evaluating the intervention at two points in time, two different types of regressions were estimated: (a) regressions on the sample of women interviewed at baseline (year 2007) and at midline (year 2010); (b) regressions on the sample of all women in the study, including baseline, midline and final evaluation (years 2007, 2010 and 2012). The former (a) permitted the research team to assess the effects of the intervention after 2 years and the latter (b) to evaluate the overall impact of the intervention after 5 years from start.
Specifically, the regressions from baseline to midline were:
Eq [1]
The regressions from baseline to final evaluation were:
Eq [2]
Where i = 1,…,N is indicator of each women participating in surveys. Yn are the binary response variables and n indicates:
- (n = 1) ANC attendance at least once during whole pregnancy;
- (n = 2) ANC attendance at least once during first trimester;
- (n = 3) at least four ANC visits. The WHO recommends a minimum of four ANC visits and that the first one should be within the first trimester of pregnancy [40];
- (n = 4) presence of a SBA during delivery; SBA in Nepal was defined as nurse-midwives, auxiliary nurse-midwives and obstetricians. The following groups were, excluded: traditional birth attendants, health attendants, medical students as they are not classified by WHO as SBA [41];
- (n = 5) institutional delivery (ID, including at hospital, primary health centre, private hospital or clinic. This was chosen as an outcome because it is a recognised strategy to improve maternal child health [42–44];
- (n = 6) attending PNC defined as mother and newborn being seen 24 hours after birth, as 60% of maternal deaths in the low and middle-income countries occur postpartum [45–47]; and
- (n = 7) taking iron and folic acid; in Nepal, supplementation is provided at government health facilities [39] to prevent anaemia and neural tube defects [48].
The control variables were:- Intervention (treat) {Eqs [1] and [2]} denotes the observations of the two groups: intervention and control;
- After {Eq [1]} denotes time, before (baseline) and after intervention started;
- After*intervention (treat-after) {Eq [1]} identifies the group of observations belonging to the intervention group after the intervention started as compared to the remaining observations (namely all observations belonging to control group and intervention group before the programme started), its estimated coefficient,
3 represents the impact of the intervention [49].
- Afterafter {Eq [2]} identifies the observations collected both in the midline and in the final evaluation as compared to baseline. As for the variable after in Eq [1], this variable also represent time;
- Afterafter*intervention (treat-afterafter) {Eq [2]} identifies observations belonging to the intervention group at the midline and at the final evaluation as opposed to baseline and to all the observations in the control group at any time. Its estimated coefficient,
11 represents the impact of the overall intervention [49].
- Age = {Eqs [1] and [2]} represents the age of the individual.
- Wealth index {Eqs [1] and [2]} is extracted from a series of assets owned (see details below). This variable tests the hypothesis of maternal attendance/compliance depending on women’s socio-economic status [50–52]. This variable was included instead of caste due to the statistically significant relationship between the two variables both in the midline and in the overall evaluation (Respectively: Pearson chi2(14) = 326.14; Pr = 0.00 and Pearson chi2(14) = 424.75 Pr = 0.00).
- Education = {Eqs [1] and [2]} indicates women’s level of education.
- Parity = {Eqs [1] and [2]}.
S1 Table further describes control variables.
Wealth index construction
The wealth index was constructed using Principal Component Analysis (PCA) among a number of assets owned by women’s families [53–54]. PCA is commonly used when household expenditure or income data are not available [55]. Assets included in the PCA were: owning a bicycle, motorcycle, goat or car, type of access to hygienic facilities (source of drinking water, type of toilet), number of rooms in house, and materials used in the dwelling (materials used for flooring, walls, roofing). A description of variables included in the PCA is provided in S1 Table. In S1 Fig we present the distribution of wealth according to high, middle and low castes, according to the published definition of caste [39,56]. High caste (= 1) included: Brahmin, Chhetri, Newar; middle caste (= 2) included: Tamang middle, or low (= 3): Newar Dalit, Balami, Dalit and others (Christian or Muslim), S1. In S1 Fig the zero represents the median of the wealth index ranking. The distribution is skewed to the left due to the poverty level of this population.
Data were analysed with STATA™ version 11.0 (Stata/SE 11.0 Stata corporation, College Station, TX, USA).
Results
A total of 1,236 women (611 in the control and 625 in intervention area) completed the surveys, with an overall average response rate of 87%. Specifically, N = 412 women participated in the first, N = 421 in the second and N = 403 in the third survey (S2 Fig). Overall, the mean age of respondents was 25.4 ± 5.1 years, the mean age of marriage 19.6 ± 3.3 years and the mean age at their first pregnancy 20.9 ± 3.2 years. The main occupation of respondents was either housewife or farmer (89.5%). Most women were multigravida (56.1%) and 43.9% were primigravida (Table 1).
In the five-year period, the characteristics of the sample changed, the number of Buddhists decreased, while Hindus remained the majority. At baseline, most women were married before the age of 20; this proportion fell to 40.4% in the control group and to 48.5% in the intervention group in last survey. Literacy rates increased steadily (Table 1).
In terms of decision-making, over time a larger proportion of women reported to have planned pregnancy and to have more autonomy over maternity care, for instance ANC and place of delivery self-decisions increased over the duration of the study (Table 2). These changes were statistically significant in the intervention area.
At baseline 80.4% of women in the control area attended ANC compared to 84.6% in the intervention area. In the intervention area, from baseline to final evaluation, the proportion of women who sought ANC at least once significantly increased from 84.6% to 98.0%. The proportion seeking ANC in the first trimester significantly increased from 47.7% to 62.4%; those seeking ANC four or more times significantly increased from 67.3% to 81.0%. In addition, a greater proportion of women reported taking iron/folic acid (from 86.5% to 96.0%) and seeing a SBA (from 60.6% to 82.0%). Significant increases were also seen in seeking an institutional delivery (from 60.6% to 76.0%) and PNC (from 52.2% to 85.9%). Use of safe delivery kit significantly increases from 5.0% to 34.3%. Improvements were also registered in the control group but not all were significant (Table 3).
Impact of the intervention
Tables 4 and 5 show the estimated odds ratios (OR) for the mid and overall evaluations, respectively. The effect of the intervention is represented by the estimated values of the OR of the variables treat-after (midline evaluation) and treat-afterafter (final evaluation). From baseline to the midline there was an increase in women’s likelihood of attending ANC at least once during whole pregnancy by 7.0 times [OR = 7.0, 95%CI (2.3; 21.4)]. A significant increase was also seen in the probability of taking iron/folic acid [OR = 3.0, 95%CI (1.2; 7.9)]. The probability of seeking four or more ANC check-ups had doubled, but this increase was not statistically significant [OR = 2.2, 95%CI (1.0; 4.7)] (Table 4). Over the five years (from baseline to final term), women were three times more likely of seeking ANC at least once [OR = 3.0, 95%CI (1.5; 5.8)]. Women were nearly twice as likely [OR = 1.9, [95% CI (1.1; 3.2)] to take iron/folic acid, and once and a half times as likely to attend PNC [OR = 1.5, [95% CI (1.1; 2.2)].
No effect was seen in the midline, or in the overall evaluation, on attending ANC in the first trimester, seeking an institutional delivery and having a SBA (Tables 4 and 5). A high OR was found for four ANC visits (Table 4), but not in the overall evaluation (Table 5).
Results on remaining covariates
Wealth was a significant factor explaining a high proportion of the variation in all the outcomes both in the midline and in the overall evaluation. In particular, being richer (3rd tertile) compared to being poorer (1st tertile) increased substantially the probability of having a SBA at birth by the midline [(OR = 11.0, 95% CI (6.3; 19.4)] and by the overall evaluation [OR = 9.3, 95% CI (5.9; 14.7)].
Relative to the construction of the wealth index, the first component extracted explained 20% of total variability in the population. The scores based on the first component were grouped into tertiles in S1 Fig, with the lowest (group 1) representing the poorest and the higher (group 3) representing the richest women. Age was a significant factor in determining whether women sought one antenatal visit and took iron/folic acid both at the midline and final evaluation. In both cases, being older lowered the probability of a positive outcome. Having higher education compared to no education increased the probability of all the outcomes considered. In particular, having secondary school or higher-level education increased the probability of attending ANC at least once in the midline [OR = 9.3, 95% CI (3.1; 28.0)] and overall evaluation [OR = 11.0, 95% CI (4.2; 29.0)].
In the intervention area (variable treat), women were more predisposed to seek an institutional delivery [OR = 1.5, 95% CI (1.0; 2.3)] and PNC [OR = 1.5, 95% CI (1.5; 2.3)] at any time. Women in the intervention area were 2.3 [OR = 2.3, 95% CI (1.2; 4.1) times more likely at midline to take iron/folic and 2.4 [OR = 2.4, 95% CI (1.4; 4.2)] by year 5. Over time (variable afterafter), women become increasingly more likely to have a SBA at birth [OR = 1.3, 95% CI (1.1; 1.7)], institutional delivery [OR = 1.3, 95% CI (1.0; 1.6)] and PNC [OR = 1.8, 95%CI (1.4; 2.2)], reflecting background changes. With increasing parity, the ORs for all outcomes remain significantly below 1, indicating a negative relationship between having more than one child and the attendance to any type of maternal health service (either ANC or PNC) or the adherence to any type of pregnancy intervention, such as folic acid and iron intake.
Discussion
This evaluation showed that the health promotion intervention had a positive effect on the uptake of ANC (attending at least once), iron/folic acid intake and PNC, but not on institutional delivery. While there was a positive effect on ANC attendance at least once during pregnancy, no effect was seen on ANC attendance in the first trimester. This may be because women become aware of the pregnancy status later or there are cultural reasons for the pregnancy to be kept a ‘secret’ [33–34, 42, 57]. Often first-time mothers need to ask permission and money from her family to attend ANC [58].
ANC attendance reduced from the midline to the overall evaluation, suggesting that the intervention may have diminishing returns: it could be argued that the effect was stronger when moving from low coverage to a medium level beyond which marginal improvements started decreasing [59]. The same trend as for ANC at least once was seen for iron/folic acid supplementation [60].
The barriers to accessing institutional delivery may be due to distance and socio-cultural factors (e.g. not part of a family’s birth preparedness plans) that cannot be overcome by a community-based intervention. Other studies in Nepal have suggested socio-economic, financial and geographical obstacles to seeking delivery care [45,61]. Studies with a focus in similar interventions have found that women’s groups in LICs increase the knowledge of obstetric ‘danger’ signs but have little impact on SBA due to other barriers, such as the cost of reaching a facility [62–64].
Although ANC attendance and provision of iron/folic acid during pregnancy are targets likely to be achieved in a community-based intervention such interventions without additional resources (such as ambulances) may not be influential enough to increase SBA or institutional delivery. Whilst PNC uptake is generally low for similar reasons to ANC non-attendance [65–66]. According to descriptive statistics, coverage of PNC greatly increased over the five years, yet the intervention only had a positive and significant effect at the very end of the intervention.
However, during 2007–2010, improvements were not associated with the GTN intervention but were due to other determinants, as witnessed by the significance of the variable “time” in both midline and overall regressions. The progressive improvement in women’s level of education and empowerment within the household may have played a role. Empowerment increased for women in particular with regard to decision-making power for ANC and delivery care. However, empowerment was not included in the estimated models as the trend was captured both by time and education (education, age and parity were strongly correlated). Interestingly, the increased women’s empowerment could be attributed to the health promotion intervention itself. Complex relationships are likely to exist among education, empowerment, maternal outcomes and the health promotion intervention [67,68].
In the overall evaluation, the factor “time” (represented by variable “afterafter”) was significant for PNC, delivery care (SBA and institutional delivery) outcomes, highlighting that other factors, not the GTN intervention, played a role. For instance, women were more likely to attend care if they have high household wealth, higher levels of education and lower parity [57,69]. A wealth index was preferred to caste as a more refined measurement for its greater inclusivity as a socio-economic indicator.
These results suggest that health promotion groups improve access to maternal health when individual, socio-economic and environment (health system) conditions are addressed [70]. Moreover, for this rural LICs setting, a health promotion intervention facilitating behavioural change may be more suitable as opposed to a cash transfer scheme alone. Cash transfer schemes, such as the ‘Safe Mother Program' (Aama-Suraksha-Karyakram) maternity incentive scheme, are often not financially sustainable [71–72].
Measuring the effect of a community-based intervention is not straightforward because of confounding environmental factors. Therefore, a (quasi-) experimental design is needed to ascertain whether the changes or improvements are due to the intervention or to external factors [5,28,73]. Previous studies have evaluated community-based interventions through randomised controlled trials (RCTs), the gold standard methodology for measuring effectiveness [74]. However, RCTs are difficult to organise, expensive and often setting-inappropriate for up-scaling or replicating and difficult to set up and run in LICs [25,75].
There is a need for community-based interventions to be accompanied by proper evaluations. Such evaluations can take the guesswork out of policy-making by knowing what works, what does not work and why [76–79]. Given the quasi-experimental study design, DiD permitted an approximation as close to an assessment of the effect of the programme on the outcomes of interest. Previous studies have assessed the effectiveness of programmes whereby a DiD approach was applied to community-based interventions [24–25]. In terms of data analysis, the criteria for inclusion of variables in a multivariate model vary between problems and disciplines. We followed methodologists suggesting the inclusion of all relevant variables in the model regardless of their significance in order to control for as many as possible covariates [80]. This approach, however, led in some cases to wide standard errors in estimated parameters, but as control variables were based on the literature, the impact of irrelevant variables was limited.
For this evaluation, maternal health behaviour before and after the intervention was not independently measured but based on self-reporting, which may have led to issues of validity and to recall bias. However, it is unlikely that the quality of self-reporting was different between intervention and control groups.
A weakness of this study is that the data collected did not identify any social barriers (e.g. gender of health worker), financial or geographical barriers (e.g. travel distance) to healthcare uptake that might account for the changes found, or lack thereof, between the data collection points. Moreover, for any intervention to improve maternal care and leverage commitment that maternal health should be a human right, it is important to know whether it is sustainable, scalable and cost-effective [81].
Conclusion
This impact evaluation suggested that the community-based health promotion intervention had a greater effect on the uptake of ANC than on delivery care. Other factors, not easily resolved through health promotion interventions, may influence the latter outcomes, such as costs or geographical constraints. However, all the selected maternal outcomes improved in time while, in parallel, women’s level of education and empowerment also increased, which may either have facilitated or be the consequence of the intervention. Interventions should prioritise impact evaluations to inform future health policies.
Supporting Information
S1 Fig. Wealth index distribution of participants.
https://doi.org/10.1371/journal.pone.0155144.s001
(DOCX)
S2 Fig. GTN health promotion intervention in Nepal and its evaluation.
https://doi.org/10.1371/journal.pone.0155144.s002
(DOCX)
S1 Table. Variables description and codification, overall evaluation.
https://doi.org/10.1371/journal.pone.0155144.s003
(DOCX)
Acknowledgments
We thank all research participants, also the staff of Green Tara Nepal, the health posts and the Female Community Health Volunteers. We also thank the anonymous reviewers for PLOS One for their helpful comments. We are grateful to the rural communities in Nepal who participated in the interventions and gave us advice.
Author Contributions
Conceived and designed the experiments: SS EvT PS. Analyzed the data: SS ES. Wrote the paper: SS EvT JMB VH PS ES.
References
- 1. Bhutta ZA, Darmstadt GL, Hasan BS, Haws RA. Community-based interventions for improving perinatal and neonatal health outcomes in developing countries: a review of the evidence. Pediatrics. 2005;115(Suppl 2):519–617.
- 2. Godin G, Gagnon H, Alary M, Levy J, Otis J. The degree of planning: an indicator of the potential success of health education programs. Promot Educ. 2007;14(3):138–42. pmid:18154222
- 3. Adam T, Hsu J, de Savigny D, Lavis JN, Røttingen J-A, Bennett S. Evaluating health systems strengthening interventions in low-income and middle-income countries: are we asking the right questions? Health Policy Plan. 2012;27(Suppl 4): iv9–19. pmid:23014156
- 4.
World Health Organization (WHO) Regional Publications. European Series. Evaluation in health promotion: principles and perspectives. 92 ed. Rootman I, Goodstadt M, Hyndman B, McQueen D, Potvin L, Springett J, et al., editors. Geneva: WHO; 2001. Available from: http://www.euro.who.int/__data/assets/pdf_file/0007/108934/E73455.pdf
- 5. Subba N, Subba S. Modification of delivery practice in Rajbanshi mothers of Nepal. J Nobel Med Coll. 2014;3(1):10–5.
- 6. Malenga G, Molyneux M. The Millennium Villages project. Lancet. 2012;379(9832):2131–33. pmid:22572601
- 7.
Christiansen G. Evaluation as a quality assurance tool in health promotion. Research and practice for health promotion. Research and Practice of Health Promotion. Cologne, Germany: Federal Centre for Health Education; 1999.
- 8.
Fitz-Gibbon C, Morris L. How to design a program evaluation. Newbury Park, CA: Sage Publications; 1987.
- 9.
Khandker SR, Koolwal GB, Samad HA. Handbook on impact evaluation: quantitative methods and practices. Washington, D.C.: The World Bank; 2010. Available from: http://books.google.es/books?id=vVOQUUZmNqMC
- 10.
J-PAL. Impact Evaluation. 2015 [cited 2015 Feb]. Available from: http://www.povertyactionlab.org/methodology/what-evaluation/impact-evaluation
- 11.
The World Bank. Impact evaluation—The experience of the independent evaluation group of the World bank. Washington, D.C.: The independent evaluation group of the World Bank. 2006 [cited 2016 March]. Available from: http://lnweb90.worldbank.org/oed/oeddoclib.nsf/DocUNIDViewForJavaSearch/35BC420995BF58F8852571E00068C6BD/$file/impact_evaluation.pdf
- 12.
Ministry of Health and Population (MOHP) New Era and ICF International Inc. Nepal Demographic and Health Survey 2006. Kathmandu, Nepal: Ministry of Health and Population (MOHP) New Era and ICF International Inc. 2007.
- 13.
Transparency International. Transparency International: the global coalition against corruption. Nepal. 2014. Available from: http://www.transparency.org/country#NPL
- 14.
Oxford Poverty and Human Development Initiative. “Nepal Country Briefing”, Multidimensional Poverty Index Data Bank. Oxford: Oxford University: 2013. Available from: www.ophi.org.uk/multidimensional-poverty-index/mpi-country-briefings/
- 15. Muldoon KA, Galway LP, Nakajima M, Kanters S, Hogg RS, Bendavid E, et al. Health system determinants of infant, child and maternal mortality: A cross-sectional study of UN member countries. Global Health. 2011;7(1):42.
- 16. Rosato M, Malamba F, Kunyenge B, Phiri T, Mwansambo C, Kazembe P, et al. Strategies developed and implemented by women’s groups to improve mother and infant health and reduce mortality in rural Malawi. Int Heal. 2012;4 (3): 176–84.
- 17. Manandhar DS, Osrin D, Shrestha BP, Mesko N, Morrison J, Tumbahangphe KM, et al. Effect of a participatory intervention with women’s groups on birth outcomes in Nepal: cluster-randomised controlled trial. Lancet. 2004;364(9438):970–79. pmid:15364188
- 18. Shrestha B, Bhandari B, Manandhar D, Osrin D, Costello A, Saville N. Community interventions to reduce child mortality in Dhanusha, Nepal: Study protocol for a cluster randomized controlled trial. Trials. 2011;12(1):136.
- 19. Simkhada B, van Teijlingen E, Porter M, Simkhada P. Major problems and key issues in maternal health in Nepal. Kathmandu Univ Med J. 2006;4(2):258–63.
- 20. Say L, Raine R. A systematic review of inequalities in the use of maternal health care in developing countries: examining the scale of the problem and the importance of context. Bull World Health Organ. 2007;85(10):812–9. pmid:18038064
- 21. Filippi V, Ronsmans C, Campbell OMR, Graham WJ, Mills A, Borghi J, et al. Maternal health in poor countries: the broader context and a call for action. Lancet. 2006;368(9546):1535–41. pmid:17071287
- 22. Donnay F. Maternal survival in developing countries: what has been done, what can be achieved in the next decade. Int J Gynecol Obstet. 2000;70(1):89–97.
- 23. Bonell CP, Hargreaves J, Cousens S, Ross D, Hayes R, Petticrew M, et al. Alternatives to randomisation in the evaluation of public health interventions: Design challenges and solutions. J Epidemiol Community Health. 2011;65 (7): 582–7. pmid:19213758
- 24. Liu X, Yan H, Wang D. The evaluation of “Safe Motherhood” program on maternal care utilization in rural western China: a difference in difference approach. BMC Public Health. 2010;10(1):566.
- 25. Alderman H, Ndiaye B, Linnemayr S, Ka A, Rokx C, Dieng K, et al. Effectiveness of a community-based intervention to improve nutrition in young children in Senegal: A difference in difference analysis. Public Health Nutr. 2009;12(05):667–73.
- 26.
Waldinger F. Difference-in-Differences. Lecture 3 [Internet]. 2010. Available from: http://www2.warwick.ac.uk/fac/soc/economics/staff/academic/waldinger/teaching/ec9a8/slides/lecture_3_-_did.pdf
- 27. Reynolds HW, Wong EL, Tucker H. Adolescents use of maternal and child health services in developing countries. Int Fam Plan Perspect. 2006;32(1):6–16. pmid:16723297
- 28. Rauniar G, Das B, Manandhar T, Bhattacharya S. Effectiveness of an educational feedback intervention on drug prescribing in dental practice. Kathmandu Univ Med J. 2012;10(40):30–5.
- 29. Simkhada P, van Teijlingen E, Kadel S, Stephens J, Sharma S, Sharma M. Reliability of national data sets: evidence from a detailed small area study in rural Kathmandu Valley, Nepal. Asian J Epidemiol. 2009;2:44–8.
- 30.
Central Bureau of Statistics. Statistical yearbook of Nepal. Kathmandu: Central Bureau of Statistics; 2001. Available from: http://cbs.gov.np/?page_id=1233
- 31. van Teijlingen E, Simkhada P, Stephens J, Simkhada B, Rogers S, Sharma S. Making the best use of all resources: Developing a health promotion intervention in rural Nepal. Health Renaissance. 2012;10(3):229–35.
- 32. Milne L, van Teijlingen E, Hundley V, Simkhada P, Ireland J. Staff perspectives of barriers to women accessing birthing services in Nepal: A qualitative study. BMC Pregnancy Childbirth. 2015; 15: 142 pmid:26133977
- 33. Simkhada B, Porter M, van Teijlingen E. The role of mothers-in-law in antenatal care decision-making in Nepal: A qualitative study. BMC Pregnancy Childbirth. 2010;10(1):34. Available from: http://www.biomedcentral.com/1471-2393/10/34
- 34. Thaddeus S, Maine D. Too far to walk: Maternal mortality in context. Soc Sci Med. 1994;38(8):1091–110. pmid:8042057
- 35. Nour NM. An introduction to maternal mortality. Rev Obstet Gynecol. 2008;1(2):77–81. pmid:18769668
- 36.
World Health Organization (WHO). Technical Brief No.3 Scaling Up Health Services: Challenges and Choices [Internet]. 2008. Available from: http://www.who.int/healthsystems/topics/delivery/technical_brief_scale-up_june12.pdf
- 37. de Heer HD, Koehly L, Pederson R, Morera O. Effectiveness and spillover of an after-school health promotion program for Hispanic elementary school children. Am J Public Health. 2011;101: 1907–13. pmid:21852659
- 38. Vanderweele TJ, Hong G, Jones SM, Brown JL. Mediation and spillover effects in group-randomized trials: A case study of the 4Rs educational intervention. J Am Stat Assoc. 2013;108: 469–482. pmid:23997375
- 39.
Ministry of Health and Population. Nepal annual report 2009/2010 [Internet]. Kathmandu: Ministry of Health and Population, Nepal; 2012. Available from: http://dohs.gov.np/wp-content/uploads/2014/04/Annual_Report_2066_67.pdf
- 40.
World Health Organization (WHO), United Nations Children’s Fund (UNICEF). Antenatal Care in Developing Countries: Promises, Achievements and Missed Opportunities: An Analysis of Trends, Levels, and Differentials: 1990–2001., Geneva: WHO; 2003. Available from: http://www.childinfo.org/files/antenatal_care.pdf
- 41.
World Health Organization (WHO). Making pregnancy safer: the critical role of the skilled attendant: A joint statement by WHO, ICM and FIGO [Internet]. 2004. Available from: http://www.who.int/maternal_child_adolescent/documents/9241591692/en/
- 42. Asefa A, Teshome W, Mesele T, Letamo Y. Use of institutional delivery services in the Southern Nations, Nationalities, and People’s Region, Ethiopia: A cross-sectional comparative mixed methods study. Lancet. 2013;382:9. Available from: http://www.sciencedirect.com/science/article/pii/S0140673613621704
- 43. Kesterton A, Cleland J, Sloggett A, Ronsmans C. Institutional delivery in rural India: the relative importance of accessibility and economic status. BMC Pregnancy Childbirth. 2010;10(1):30.
- 44. Kestler E, Walker D, Bonvecchio A, de Tejada SS, Donner A. A matched pair cluster randomized implementation trail to measure the effectiveness of an intervention package aiming to decrease perinatal mortality and increase institution-based obstetric care among indigenous women in Guatemala: Study protocol. BMC Pregnancy Childbirth. 2013;13(1):73.
- 45.
World Health Organization (WHO). WHO recommendations on postnatal care of the mother and newborn. Geneva: WHO; 2013. Available from: http://apps.who.int/iris/bitstream/10665/97603/1/9789241506649_eng.pdf
- 46.
Middleberg MI. Promoting reproductive security in developing countries. New York: Kluwer Academic/Plenum Publishers; 2003.
- 47. Li XF, Fortney JA, Kotelchuck M, Glover LH. The postpartum period: The key to maternal mortality. Int J Gynecol Obstet. 2014;54(1):1–10.
- 48.
World Health Organization (WHO). Daily iron and folic acid supplementation in pregnant women: guidelines. Geneva: WHO; 2012. Available from: http://apps.who.int/iris/bitstream/10665/77770/1/9789241501996_eng.pdf
- 49.
Schlotter M, Schwerdt G, Woessmann L, Econometric Methods for Causal Evaluation of Education Policies and Practices: A Non-Technical Guide. Ifo Institute, University of Munich and IZA Discussion Paper No. 4725, 2010. Available from: http://repec.iza.org/dp4725.pdf
- 50. Ahmed S, Creanga AA, Gillespie DG, Tsui AO. Economic status, education and empowerment: Implications for maternal health service utilization in developing countries. PLoS One. 2010;5(6):e11190. Available from: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0011190 pmid:20585646
- 51. Simkhada B, van Teijlingen ER, Porter M, Simkhada P. Factors affecting the utilization of antenatal care in developing countries: Systematic review of the literature. J Adv Nurs. 2008;61(3):244–60. pmid:18197860
- 52. Tuntiseranee P, Olsen J, Chongsuvivatwong V, Limbutara S. Socioeconomic and work related determinants of pregnancy outcome in southern Thailand. J Epidemiol Community Health; 1999;53(10):624–9. pmid:10616674
- 53. Howe L, Hargreaves J, Huttly S. Issues in the construction of wealth indices for the measurement of socio-economic position in low-income countries. Emerg Themes Epidemiol. 2008;5(1):1–14. http://dx.doi.org/10.1186/1742-7622-5-3
- 54. Vyas S, Kumaranayake L. Constructing socio-economic status indices: how to use principal components analysis. Heal Policy Plan. 2006;21 (6): 459–68.
- 55. Filmer D, Pritchett L. Estimating wealth effects without expenditure data—or tears: an application to educational enrollments in states of India. Demography. 2001;38(1):115–32. pmid:11227840
- 56.
Bennett L, Dahal DR, Govindasamy PCaste, Ethnic and Regional Identity in Nepal: Further Analysis of the 2006 Nepal Demographic and Health Survey. Calverton, Maryland, USA: Macro International Inc., 2008. Available at: http://pdf.usaid.gov/pdf_docs/Pnadm638.pdf
- 57. Finlayson K, Downe S. Why do women not use antenatal services in low- and middle-income countries? A meta-synthesis of qualitative studies. PLoS Med. 2013;10(1):e1001373. http://dx.doi.org/10.1371/journal.pmed.1001373 pmid:23349622
- 58. Puri M, Tamang J, Shah I. Suffering in silence: consequences of sexual violence within marriage among young women in Nepal. BMC Public Health. 2011;11(1):29. Available from: http://www.biomedcentral.com/1471-2458/11/29
- 59.
Davies M, Macdowall W. Health promotion theory. Maidenhead: Open University Press; 2005.
- 60. Khanal V, Zhao Y, Sauer K. Role of antenatal care and iron supplementation during pregnancy in preventing low birth weight in Nepal: comparison of national surveys 2006 and 2011. Arch Public Heal. 2014;72(1):4.
- 61. McPherson RA, Khadka N, Moore JM, Sharma M. Are birth-preparedness programmes effective? Results from a field trial in Siraha district, Nepal. J Health Popul Nutr. 2006; 24(4):479–88. pmid:17591345
- 62. Borghi J, Ensor T, Neupane BD, Tiwari S. Implicaciones financieras de la atención cualificada en el momento del parto en Nepal. Trop Med Int Health. 2006;11(2):228–37. http://dx.doi.org/10.1111/j.1365-3156.2005.01546.x pmid:16451348
- 63. Choulagai B, Onta S, Subedi N, Mehata S, Bhandari G, Poudyal A, et al. Barriers to using skilled birth attendants’ services in mid- and far-western Nepal: a cross-sectional study. BMC Int Health Hum Rights. 2013;13(1):49.
- 64. Furuta M, Salway S. Women’s position within the household as a determinant of maternal health care use in Nepal. Int Fam Plan Perspect. 2006;32(1):17–27. pmid:16723298
- 65.
Warren C, Daly P, Toure L, Mongi P. Postnatal care. In: Lawn J and Kerber K, ed. Opportunities for Africa’s newborns: Practical data, policy and programmatic support for newborn care in Africa. Cape Town, South Africa: WHO on behalf of The Partnership for Maternal Newborn and Child Health; 2006.
- 66. Khanal V, Adhikari M, Karkee R, Gavidia T. Factors associated with the utilisation of postnatal care services among the mothers of Nepal: Analysis of Nepal Demographic and Health Survey 2011. BMC Womens Health. 2014;14(1):19.
- 67.
United Nations Population Fund (UNFPA). International Conference on Population and Development. Gender equality, equity and empowerment of women. Cairo, Egypt, 1995.
- 68.
National Institute for Health and Care Excellence (NICE). Behaviour change: the principles for effective interventions. NICE public health guidance 6. NICE; 2007. Available from: http://www.nice.org.uk/guidance/ph6/resources/guidance-behaviour-change-the-principles-for-effective-interventions-pdf
- 69. Lee RB, Abellera M, Triunfante CC, Mirandilla MEL. Utilization of maternal health care services among low income Filipino women. Asia-Pacific E-Journal Heal Soc Sci. 2012;1(1).
- 70. Irwin A, Valentine N, Brown C, Loewenson R, Solar O, Brown H, et al. The commission on social determinants of health: Tackling the social roots of health inequities. PLoS Med 2006; 3(6): e106. pmid:16681414
- 71. Witter S, Khadka S, Nath H, Tiwari S. The national free delivery policy in Nepal: early evidence of its effects on health facilities. Heal Policy Plan. 2011;26 (suppl 2): ii84–91.
- 72. Ensor T, Clapham S, Prasai DP. What drives health policy formulation: Insights from the Nepal maternity incentive scheme? Health Pol. 2014;90(2):247–53.
- 73. Coast E, Jones E, Portela A, Lattof SR. Maternity care services and culture: a systematic global mapping of interventions. PLoS One. 2014;9(9):e108130. pmid:25268940
- 74. Osrin D, Azad K, Fernandez A, Manandhar DS, Mwansambo CW, Tripathy P, et al. Ethical challenges in cluster randomized controlled trials: Experiences from public health interventions in Africa & Asia. Bull World Health Organ. 2009;87(10):772–9. pmid:19876544
- 75. Bhutta ZA, Ali S, Cousens S, Ali TM, Haider BA, Rizvi A, et al. Interventions to address maternal, newborn, and child survival: What difference can integrated primary health care strategies make? Lancet. 2008;372(9642):972–89. pmid:18790320
- 76. Datta J, Petticrew M. Challenges to evaluating complex interventions: A content analysis of published papers. BMC Public Health. 2013;13(1):568.
- 77. Ensor T, Green C, Quigley P, Badru AR, Kaluba D, Kureya T. Mobilizing communities to improve maternal health: Results of an intervention in rural Zambia. Bull World Health Organ. 2014;92(1):51–9. pmid:24391300
- 78. Judd J, Frankish CJ, Moulton G. Setting standards in the evaluation of community-based health promotion programmes—a unifying approach. Heal Promot Int. 2001;16 (4):367–80.
- 79. Duflo E. Rigorous evaluation of human behavior. Science. 2012;336(6087):1398. Available from: http://www.sciencemag.org/content/336/6087/1398.2.short
- 80. Bursac Z, Gauss CH, Williams DK, Hosmer DW. Purposeful selection of variables in logistic regression. Source Code Biol Med. 2008;3:17. pmid:19087314
- 81. Prost A, Colbourn T, Seward N, Azad K, Coomarasamy A, Copas A, et al. Women’s groups practising participatory learning and action to improve maternal and newborn health in low-resource settings: a systematic review and meta-analysis. Lancet. 2013;381(9879):1736–46. pmid:23683640