- @Darmobilesurvey results are closer to the mainland results than Zanzibaris results @CenterEconProsp @RaiaMwemaTz http://t.co/y9HLFRcHs0 - 6 years ago
- Zaidi ya nusu ya wakaazi wa Dar Wapinga kubadili ufaulu Sekondari via @RaiaMwema #Education #NECTA @CenterEconProsp http://t.co/FrE4OoaOsG - 6 years ago
- Zaidi ya nusu ya wakaazi wa Dar Wapinga kubadili ufaulu Sekondari via @RaiaMwema #Education #NECTA @CenterEconProsp http://t.co/LOGzVUVlwx - 6 years ago
- byebye and thanks for all the fish
- dressing up nicely to move into the next phase
- moving into round 10
- findings on reliability of power grid picked by The Guardian (TZ)
- getting the word out (and the numbers)
- monitoring public service in the city
- some USSD testing and some delays
- so here we are
- and…we’re back
- Breaking the silence
Two years ago, I set up this blog to report on our efforts to gather reliable and timely data on citizens’ lives in Dar es Salaam through mobile phones. By now, our project (now called Listening to Dar) has developed into something much more than what we could have imaged when we started this process. Since the World Bank started funding the data gathering, we published 16 data reports on Public Services in Dar es Salaam and our findings generate considerable media interest in Tanzania. Our data have also been used in this year’s World Bank Global Montoring Report (see www.listeningtodar.org).
At the same time, very little has happened on this blog during the previous months. Not because we have not been doing exciting things (we have), but because by now, we reached a certain level of routine and moved beyond the phase of experimentation, of trial-and-error. At this point, we invest most of our efforts in gathering relevant data on different aspects of people’s living conditions, publishing thought-provoking and accessible reports, and making sure they enter and inform public discourse. Much of this we achieve through the official project’s website, our twitter account, and our bi-weekly press conferences held in Dar es Salaam.
For these reasons, I will stop posting here for now. However, I will keep this blog online as an archive of our mistakes and experiences, and most of all as a potential learning resource for those who plan to set up a similar infrastructure elsewhere.
To celebrate the fact that our project is about to move into its next phase, with biweekly panel interviews to be conducted until June, we gave our website a new face, a new feeling and a brand new URL: www.listeningtodar.org.
On our new site, you will find previous reports on Public Services in Dar es Salaam, access all of our data and see how our findings have been used in the media and in other reports (e.g. in the recent World Bank Global Monitoring Report, p. 33). Also, you can tell us what you think we should focus on in future survey rounds. So have a look, browse around, and do let us know what you think!
Since the World Bank started funding this project some five months ago, and we began using it to monitor the quality of public service provision in Dar es Salaam, we have successfully ran ten survey rounds, distributed our data reports to various stakeholders, and saw the mass media picking up our findings.
We covered a wide array of topics, including citizens’ food consumption habits, their experiences with the health care system in Dar es Salaam, their satisfaction with the city’s water and electricity infrastructures and the hardship caused by the devastating 2011 December flood. We are currently finalizing our newest report on the widespread problem of teacher absenteeism in both primary and secondary schools, for which we drew from our 550 panel households to interview more than 300 school children from all three districts of the city. That report will soon be available on the project’s website – where we not only publish all of our reports, but also the full data sets to encourage the statistically inclined to run their own analyses. To whet your appetite, see below for some visualizations of the type of findings we are getting from all of this.
After the Citizen and the Sunday Citizen have started using our findings during the last couple of weeks (see previous post), The Guardian now also has published an article on the poor state of Dar es Salaam’s power infrastructure that draws on our recent report on electricity in the city (both the English and Swahili report will be published soon on the project’s website).
At a time where the Tanzanian mainstream news media have started picking up our reports (see here and here), the blogging community has also started to pay attention to our data and the broader stories they tell.
In the meanwhile, in order to increase our reach and help our findings to travel back into the public sphere, we are also posting some of our most interesting bits of data on our twitter account @Darmobilesurvey
A lot has happened.
So it’s high time for a brief update.
As you will know if you have followed this blog, my colleagues at Twaweza and DataVision and me started off this projected in 2010 with an extensive face-to-face interviews of a stratified sample of 550 household in Dar es Salaam.
Those respondents who had access to a mobile phone (about 83%) were asked to participate in our mobile phone follow up survey. As an incentive to regularly answer questions through the phone, credit top-ups were offered as incentives to respond to the weekly set of questions.
To reduce selection bias in our mobile phone panel waves, much effort was invested in trying to get those people into the panel that originally did not own a phone. At this point, 20 phones have been handed out and a team of researchers is currently going to back to baseline respondents to distribute more phones.
By now, the mobile panel of around 350 active respondents is part of the World Bank-funded Urban Public Service Monitoring Project. Every two weeks, participants are approached with a short set of questions on their daily lives and their experiences with public services in the city.
For each survey round, I write up a report that is then translated into Swahili, published online and distributed to different stakeholders and journalists in the city. The first round focused on water infrastructure, followed by a survey on education, the health system, urban governance, electricity and consumption (see graphs for some ideas about the type of questions that are being addressed with this exciting new data gathering tool).
For the full reports and data files, visit the projects new website at http://monitor.public-transparency.org/
I ended my last post with the promise to share our experiences with the distribution of mobile phones and also to talk about the testing of new technologies for data gathering (particularly USSD). Unfortunately, as there have been some delays in setting a date for resuming the panel (which is currently not active), also the distribution of phones has been stalled. As things stand now, the 50 phones will most likely be distributed during the course of next week, which means that I will not be involved in the exercise personally as I will be leaving for Amsterdam in a couple of days.
When it comes to the testing of USSD, however, there is more to say. About three weeks ago, we conducted a pilot test of USSD for administering a two-question survey to 50 respondents selected from our panel. As these respondents had so far always been called for the interviews and never used any other channel for submitting their responses, we first sent a text message explaining the procedure and asking them to initiate the questionnaire by punching in a specific number code. However, it soon became clear that a simple text message is not sufficient to make respondents comfortable with switching technologies: quite a few respondents sent their responses as a text message and others called up the call centre number to ask what they were supposed to do. In the end, only one person answered both questions using USSD.
While this seems like a highly disappointing result at first, the main lesson that I would take away from this, is not that USSD cannot be used for conducting surveys. I still believe that the technology has great potential (see my earlier posts on this) for quickly gathering simple data from large samples. The main lesson, rather, seems to be that personal on-the-spot training of respondents is crucial when anything else than voice is being used. Even more so, as many respondents probably have used USSD for other purposes in the past (e.g. for transferring money). While this is admittedly not based on solid evidence, I believe that if respondents had been trained during the face-to-face to use USSD as a surveying tool, response rates would have been much larger. But this is something that remains to be seen, as Data Vision now has a automated system in place capable of further testing the technology for surveying purposes.
As I have mentioned above- and much to my frustration – I will not be present for the actual distribution of the mobile phones. As this is such an important step in not only increasing panel size (which is currently at about 310 active respondents) before the new round of surveying will start, but foremost of rendering the sample more representative, I will make sure to soon post some of the results and field experiences of my colleagues at Uwazi/Twaweza and Data Vision.
I am back in Dar es Salaam, back at Twaweza, and back working on mobile data gathering. Before giving you an overview about the things that will be happening in the weeks to come, I will first use this post to recapitulate where we are coming from.
During August/September last year, we went into the field to draw a stratified sample of 550 respondents in Dar es Salaam, conducting face-to-face household interviews. Our extensive survey covered quite an impressively broad range of topics such as family composition, work situation, political orientations, infrastructure, health issues and children’s education (actually, given that the survey took about 1,5h to complete, I was somewhat surprised to see that almost all people took the time to answer all of our questions and that non-response rates were minimal). After completion of the questionnaire, respondents were asked if they owned or had access to a mobile phone. If so, we invited them to participate in follow-up mobile surveys. Out of those respondents that we interviewed, 76% had their own mobile phone and another 12,5% had access to the phone of another household member.
In the weeks after the baseline survey was finished, an infrastructure was set up by DataVision to call respondents for short surveys (10-15 questions) and transfer small credit top-ups after the interview as incentives. Initially, we also had planned to use USSD and IVR (and even a web application), but the technical difficulties in putting these in place seemed more serious than expected, and we resorted to simply calling respondents during the weekends.
The mobile survey system was up and running by January 2011, and in the following 18 weeks, respondents were called weekly and a rich body of real-time data were gathered, mainly about the quality of public services such as education, health, garbage collection, water and electricity. After some substantial drop-out in the first weeks, the panel size quickly stabilized around 330. Each weekend, respondents were called by the same interviewers, contributing to sense of commitment so crucial for preventing attrition in panels.
About three weeks ago, the data gathering was temporarily paused as the whole project was transferred from Twaweza to the World Bank. Interested in exploring the potential of mobile surveys for independent third party monitoring, the World Bank will be running the panel for six months, conducting biweekly surveys on topics related to its activities in Dar es Salaam.
So this is where we are.
But there is more to come:
To correct the bias in our mobile sample created by the lack of phone access in baseline respondents, we will be heading back into the field shortly to hand out 50 mobile phones. Also, we now seem to be ready to throw in two new technologies as data gathering channels: USSD and IVR. More on all this very shortly.
After many months of silence on this blog, things are about to change.
Since its launch in August 2010, this platform has worked very successful in drawing attention to what we were doing and in establishing new connections with interested and interesting people. At the same time, this blog had unfortunately somewhat dried up after I had returned to my job at the University of Rotterdam, in October last year – also because I had not been able to keep involved close enough to report back on our progress.
But indeed, things are very much about to change.
In just a couple of weeks, I will pick up my work at Twaweza on the mobile survey project again. And I can’t wait to get back to that strange city of Dar es Salaam, talk to those people who have worked hard to keep this thing up and running, get their first-hand experiences with the day-to-day handling of the mobile panel waves (the first wave was launched after I had already left), to dive into the wealth of data produced, analyze dropout patterns, look into technology issues to be solved, and all of the other things that make up this exciting project.
So don’t despair, stick with me, great things are bound to happen .
Not much has happened on this blog during the last two weeks. First, because I spent some of that time travelling through the north of this amazing country. Second, and more importantly, because the kick-off of the mobile survey phase had to be postponed. Some of the things we are doing – in terms of technology – have not been done before, which is what makes this whole beast an exciting one but also sometimes tricky to predict. Especially the technical implementation of some of the data gathering modes has proven to be more challenging than expected. By now, however, all of the more serious issues have been resolved and we will be able to start re-approaching respondents very soon. So for now, some more patience is required.