Baseline study & mobile follow-ups: a first overview


This post will give you a first overview of the project. Specifically, I will provide some information about the baseline survey (including sampling technicalities) and the technologies we will use for the mobile phone follow-up.

While too young to be included in our sample, these youngsters in the ward of Keko were very interested in our doings (and above all very much amused by my lack of pigmentation - Mzungu! Mzungu!)

We are charting new territory here, so we expect to learn a lot from both our mistakes and your comments.  (The only project that I know of, which is to some extent comparable to what we will do in terms of approach and scope, is Brian Dillon’s work in rural Tanzania)

The aim of this project is to build up a mobile phone based panel in Dar es Salaam, allowing for high frequency and low cost gathering of up-to-date survey data. In order to get a good sense of our sample respondents and to understand and track individual changes, first a representative face-to-face baseline survey is conducted. The focus of the baseline questionnaire is on the quality of public services such as water supply, waste collection, hospitals and education, but also includes items on household characteristics, wealth, access to information, security, citizen participation, elections & travel time to work (the complete questionnaire in English will be available soon). We draw a stratified sample on different administrative levels (see box below), interviewing a total of 550 citizens.

In the field: our team of researchers and interviewers at the office of one of the balozi's (leader of the ten cell), retrieving the list of ten-cells and households.

Some technical stuff on sampling
The sample is drawn from all three districts of Dar es Salaam (Ilala, Kinondoni & Temeke), and clustered on three levels: the ward, the street (mtaa), the “ten cell” (group of about ten households) and the household. First, from each district 10 wards are randomly selected. From the selected wards, 37 mtaa’s are randomly chosen per district. Then, for each mtaa two ten cells are randomly chosen to then select two or three households within each ten cell (thus resulting in five households per mtaa). In each household, one individual is randomly selected from all adult household members (18 years or older). This then results in a nonproportional clustered sample, requiring for weighting in the actual analysis. In case you’re interested which wards & mtaa’s were selected, have a look at the sample frame / sampling list (or click on the map below to see the selected wards in Google Maps).

Map showing the wards included in our sample. Per selected ward, 15-30 interviews are conducted (click to load in Google Maps)

At the end of the face-to-face interview (which typically lasts for about 1,5 – 2 hours), the respondent receives 3000 Tsh (about $2) in mobile phone credit and is asked to participate in the weekly follow-up survey, for which he/she will receive a credit top-up of between 300 and 500 Tsh ($0.2 / $0.3) per wave.

In principle, four different technologies will be used to gather the weekly panel data. I will probably elaborate a bit more on the different technologies and their implications soon, but for now, these are the options we will be using:

  • Interactive Voice Response (IVR)
    The respondent receives a text message each Friday morning, asking him/her to “beep” a number (i.e. call and hang up) before Sunday evening. This will trigger the IVR system to call back and a pre-recoded voice-based menu guides the respondent through the questionnaire.
  • Wireless Application Protocol (WAP)

    our current survey WAP application in action

    Users that have WAP capabilities and the knowledge to use it, will be asked to download a survey application that was developed by DataVision to gather survey data. Just like with IVR, the respondent receives a text message on Friday morning asking to fill in the questionnaire before Sunday evening.

  • Unstructured Supplementary Service Data (USSD)
    A simple form of communication between the mobile phone user and the provider’s server, supported by all GSM phones. It is widely used in Tanzania and other African countries to transfer money and pay bills (see http://en.wikipedia.org/wiki/M-Pesa), offering an alternative to bank transfers – as many people don’t have bank accounts (as far as I know, in the US and most European countries it is mainly only used to request information such as prepaid credit, by typing *somecode#). Again, the respondent receives a text message on Friday morning to start the questionnaire. Unfortunately, due to technical difficulties, it is not yet clear at what point during the survey we will be able to use USSD.
  • Call Center
    Possibly the most convenient (for the respondent) and accurate mode of data collection – and also the most expensive one. During the baseline interview, the respondents gets to pick a day & time (during the weekend) when to receive the weekly call from the call center agent.

The reason for us to use this rather broad range of technologies is that we want to look into the implications of the modes of data collection for attrition (panel mortality) and data quality. Furthermore, the process of getting all these channels up and running will teach us a lot about obstacles and advantages that come with each technology (much more so than if we would only focus on for example IVR). In order to be able to estimate these effect, we assign respondents to the different groups, as described by the graph below.

procedure used to assign respondents to the different technologies of data collection

Similarly, we are interested in the effects of the amount of incentive, i.e. the phone credit the respondent receives for completing the weekly mobile phone surveys. We therefore vary this variable in such a manner that the households of each mtaa (street) receive either 300, 400 or 500 Tsh (assigned randomly per mtaa). Therefore, all respondents within the same mtaa receive the same amount as to avoid upsetting respondents who obviously should not be aware that others are getting more / less. While we had some difficulties convincing the team of enumerators of the necessity to do this (they pointed out how unfair this was – and indeed it is), we are quite sure that this will give us very useful information on if and how incentives matter.

As you probably already noticed, the graph on the right always tells you whereabout in the process we are. At the time of writing, our interviewers have started conducting baseline interviews in the district of Temeke which means that we should soon be getting our first data on panel respondent characteristics.

This was it for now. More will follow soon.

This entry was posted in call center, dar es salaam, incentives, IVR, sampling, survey methodology, USSD, WAP. Bookmark the permalink.

3 Responses to Baseline study & mobile follow-ups: a first overview

  1. julia hoffmann says:

    This is a very interesting project! Glad you are writing a blog about it as well, so yet more knowledge gets ‘liberated’ and others may be able to set up similar projects in the future. Looking forward to seeing the questionnaire and to reading about your experiences from the first analyses!

  2. Steve says:

    This sounds like a very interesting project and I’m glad to have discovered (via a World Bank blog) that you exist. Just one technical point: you write

    “The sample is drawn from all three districts of Dar es Salaam (Ilala, Kinondoni & Temeke), and stratified on three levels: the ward, the street (mtaa), the “ten cell” (group of about ten households) and the household.”

    but I think you meant to write “clustered” and not “stratified”.

    Good luck!
    Steve

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s