Monitoring and Evaluation

Perinatal Care Project has developed a monitoring system to record all births and their outcomes within the 18 control and intervention unions for the duration of the project. The intention was to create a monitoring system that can be easily replicated, at low cost, elsewhere. The monitoring system comprises of two stages:

I. The Identification Stage
The system needs to be able to identify all women in our study area who have:

Two different approaches will be tested for identifying women fitting one of the above criteria (Figure 1). The first will utilise government outreach workers, called Family Welfare Assistants (FWAs), whose current role is to visit households in the community and systematically record births and outcomes.

Figure 1: The Monitoring System


The Monitors meet with FWAs once a month to collect the list of women.

For the purposes of monitoring the interventions in this project a reliable monitoring system needs to be in place, therefore a secondary system has been developed to ensure maximum identification. The second approach for identifying women will expand the role of TBAs. TBAs are responsible for conducting the majority of deliveries and are therefore in a good position to identify births.

Base line information was collected on all TBAs in our study area. We have selected 3 TBAs per ward, covering a population of 600 households in both the control and intervention wards. A selection committee comprising of the following members was involved in the selection process:

• Thana Health and Family Planning Officer
  
• Thana Family Planning Officer
  
• 1 Family Welfare Visitor (FWV) per Union
  
• 3 female member of the Union Council per Union
  
• Project Area Manager
  
• Field Supervisor
  

The committee considered the following criteria when making their selection:

• Location of household (each TBA needs to cover a separate area of 200 households, therefore it is best if they are evenly distributed throughout the ward)
  
• Number of deliveries they currently attend (more deliveries better)
  
• Mobility (they need to cover an area of 200 households so we need TBAs who are mobile)
  
• Acceptability within the community
  
• Well known with in the community
  
• Good health
  
• Ability to keep simple records, i.e. name and address of mother, date of delivery - Willingness to identify deliveries and meet our staff once a month
  
• Willingness to be trained
  
• Willingness to learn better practices
  

Each of the selected TBA is responsible for covering a logical geographic area of approximately 200 households to identify all eligible women every month within this area irrespective of whether they attended the birth. Since the monitoring is outside the traditional role of the TBA incentives are necessary for them to perform this additional role and TBAs are paid for all accurate identifications. The Monitors meet with TBAs once a month to collect the list of women.

2. The Interview Stage
If both approaches are successful at identifying all eligible women there will be a 100% duplication of women identified. Therefore these duplications need to be removed before interviewing stage. Women will be interviewed 6 weeks after delivery. They will be asked to complete a questionnaire covering background characteristics, the antenatal period, delivery, and post-partum. In the event of a stillbirth or neonatal death a verbal autopsy will be also completed. Where a maternal death is identified up to two months post-partum a verbal autopsy will be conducted. Verbal autopsies will be conducted with the mother herself (or close relative / friend in the case of a maternal death) as well as with the provider (if one was present). All eligible women identified will be snowballed - to see if they can identify any other women.

Monitoring and evaluation activities are carried out with structured Monitoring and Evaluation questionnaire. Monitors are primarily responsible for collection of data under the direct supervision of Monitoring Coordinator. Information about delivery outcome is taken from the TBA in the community and FWAs in the facilities. After identification the information is noted down by the UPMs in their register. In the next month UPM visits the household where the maternal or newborn event took place and carry out interview by using specific sections of the Monitoring and Evaluation Questionnaire.

After completing the interview, the Monitor immediately checks for any error and submit the completed questionnaire to the Field Supervisor at the monthly meeting. Monitoring Coordinator also checks the questionnaire for error and then sends all the completed questionnaires to PCP Head Office at Dhaka. Here Field Surveillance Manager also checks the questionnaires then data entry operators will enter the data into the computer.


Handling of temporary residents: It is customary for women to go to their parents house for delivery, and to return to their husband’s house after delivery. The UPM register includes information on the union of identification and whether the resident was a permanent or temporary resident in the village in which the identification was made.

The following residence categories are used:

Type 1: permanent resident, i.e. it is the village where there respondent normally lives with her husband

Type 2: Temporary 2: temporary resident, permanent residence is in PCP study area.

Type 3: Temporary 3: temporary and permanent residence is outside PCP study area

In practice:

• Type 1 (permanent) residence includes women who are permanent residents to the project area, who temporarily moved outside of the project area to give birth, and return after birth to the project area. The key identifier will have identified the pregnant permanent resident, and will identify her delivery, up to 1 year after returning from outside of the project area.
  
• Temporary 2 includes residents that both move within the same union (but outside of the village of their husband), and that move temporarily to another union. For these, we don’t know the union of permanent residence.

All people, temporary and permanent, are interviewed if available 6 weeks after delivery. For women who deliver in the project area but whose permanent home is outside the project area, only survival status information is obtained; full interview is not done.

In the M&E dataset, the union of interview is identified. This is not necessarily the same as the union in which the birth is identified (as the woman may have moved back to the husband’s home after delivery and delivery identification). From the Version 1 M&E questionnaire we are not able to tell if the woman was a permanent resident in the place where she was interviewed. On the cover of the M&E questionnaire Version 1, the monitors stamp the residence type (1,2,3). NB: this is residence type according to the village in which the identification took place, not the residence type according to the place of interview.

Quality Control :

Supervision structure:

• 500 Key identifiers are supervised by 37 monitors
  
• 37 full-time salaried Monitors are coordinated and supervised by 6 Monitoring coordinators
  
• 6 full-time salaried Monitoring coordinators are supervised by 3 full-time salaried District Managers (1 per District)
  
• Alls surveillance activities are supervised by the Surveillance Manager in Dhaka office

Quality control process

• Verification by monitor of births and deaths reported by key identifier
  
• Monitoring coordinators check all completed questionnaires before they are sent to the Dhaka office. They clarify with the respondent if they find any consistencies or have any queries.
  
• Monitoring coordinators cross check at least 20% of questions of 20% of interviews. From 1 Jan 2011 this will be reduced to at least 20% of questions 10% of questionnaires. If there are any discrepancies, the reply given to the monitoring coordinator will be assumed to be correct. Monitors use different pen colour from the coordinators and the manager. The new database it is entered whether the answers given to coordinators are the same or not as given to the monitors (if there is a difference, only the answer given to the coordinators is retained), to check test-retest reliability.
  
• Monthly meeting in district office with monitoring team and Sanjit to check the questionnaires.
  
• 3 days refresher training on monitoring system, every 6 months in the district office for district managers, monitors and coordinators.
  
• Under-5 surveillance: 20% of under-10 deaths are cross-checked by monitoring coordinators, district managers and field surveillance manager.
  
• All questionnaires are checked by surveillance managers and data imputers in the Dhaka office, and returned to the field in case of any questions or clarified through phone conversation with monitors (all monitors have their private mobile phones).

Systematic data cleaning, using STATA do-files, has been set up.