As development professionals and data analysts, we are challenged to learn – and think differently –every day. This is all the more critical and often necessary in places like South Sudan, Afghanistan and other conflict-affected settings, where data collection takes place in challenging security and political environments.

Recently, during the Education in Crisis & Conflict Network (ECCN) Rapid Education and Risk Analysis (RERA) Training workshop, I realized that my personal experiences conducting a RERA in 2017 in South Sudan reinforced how essential this “different” thinking is.

MSI conducted the RERA with a clear study objective: to rapidly derive findings from the most recent evidence, and to help USAID make better and more targeted education-related programmatic decisions in the short- and medium-term.

A RERA is “a good enough situation analysis of educational institutions, learners, and their communities as a dynamic system of relationships involving assets and multiple contextual risks.”¹  But what makes for good enough?

Considerations for a good enough study must surely take place at each phase of the study: design, planning, implementation, analysis, reporting and dissemination. However, in crisis and conflict-affected settings, the challenges are particularly acute in the collection of primary data. Primary data collection is necessary not only to have timely and up-to-date information to assess and act in response to crisis and conflict, but also because there is usually a lack of information from other sources (such as administrative data or past assessments).

My experiences conducting a RERA in the complex, conflict-affected environment of South Sudan taught me a lot about critical considerations that need to be made in order to achieve good enough data in constrained contexts. In thinking about how to balance a rigorous data collection methodology while also being responsive to the challenges that these contexts impose, below are my recommendations to guide data collection with a good enough consideration in mind.

#1: Delineate the study’s objective and set limits to your research questions

Usually, RERA studies use a mix of qualitative and quantitative approaches and analyze both primary and secondary sources of data (including the limited amount of records on the number of schools, teachers, etc.) in conflict and crisis-affected settings. Before kicking off the planning of our RERA data collection activities, we conducted a thorough desk review, which summarized the latest information available about the armed conflict in South Sudan and its consequences. Our research helped us understand the sources of conflict and its dynamics better, but also helped us realize that to understand the interaction between conflict and education, we needed a community-level lens to answer our research questions.

Before setting our research questions, we identified the gaps and opportunities in the data available and made sure to narrow down the key aspects we needed to study, and which of those would require the collection of primary data. When facing acute contexts of deprivation and risk, it was tempting to broaden the scope of our data collection. However, we constantly reminded ourselves not to step outside of our scope and collect only the data we needed to thoroughly answer our research questions, as it was vital to ensure efficiency and manage security risks in our data collection approach within our set timeframe.

#2: Rely on a local, multi-disciplinary team of experts

There is no question that a rich study represents the voices of those affected by conflict and crisis. In our study, we made sure to consult with local experts at every phase. A multi-disciplinary approach is key, since risks are not unidimensional and different types of risk are in constant interaction in crisis and conflict settings.

Once we completed our desk review, we consulted national and international experts in an array of relevant areas. Though the objective of our study was to gain a deeper understanding of the relationship between the conflict and the educational system, we made sure to consult experts that shed light on the multiple array of contextual factors that were also affecting teachers, students, and the education system as a whole, such as displacement, famine, drought, road safety and the national budget.

Our collaborative approach helped us not only to refine our research questions but also to plan our data collection efforts in terms of data collection sites, timing and sample. We continued a collaborative approach at each phase of the study, capturing a rich diversity of voices heard during the analyses and beyond.

After the analyses, we conducted validation workshops in Juba, South Sudan (with our local data collection teams to provide richer insights on some of our findings) and Washington D.C. We co-created conclusions and recommendations with a range of stakeholders, including representatives from UNICEF, the Education and Children Protection Clusters, the Ministry of Education, USAID and the RERA Advisory Group.

#3 Understand (exactly) what you are asking

It is essential for a good enough approach to recognize and prevent the unintended negative consequences that data collection endeavors can have in the local context. RERA draws from a combination of conflict, disaster risk reduction, and resilience frameworks. All of these are framed under the principle of ‘do no harm’, that is, to consider both the intended and unintended effects of aid and/or interventions.

A key aspect is to carefully delineate the questions asked of participants and consider conflict and crisis-related sensitivities. For our study, we relied on local expertise to design the focus groups’ content and questions for each of our respondents: students, teachers, community leaders, etc. Each data collection team had a research supervisor who kept in close communication with the research team during data collection, so we were able to change and adapt questions as the data collection efforts developed.

#4 Consider who is asking and who is responding

The ‘do no harm’ approach is vital in the planning and composition of the research team. We need to consider the complex power and identity dynamics in play at each level of interaction in the study: the interaction between the objectives of the research and data collection teams, the data collection team and the study participants, and the data collection team and the local community.

Language, for example, is a critical identity feature central to the data collection process. We should try to mitigate not only language barriers between the data collection team(s) and study participants, but also avoid any transgression to the social and cultural scripts in the use of language in our day-to-day work.

In our RERA study, for example, we ensured that team members had the appropriate local language skills according to the regions visited. We also ensured gender diversity in our data collection teams, and each team had one female researcher who led the focus group discussions with female students, mothers and community members.

#5 To be safe, be flexible

Routinely assessing the safety and risks is paramount when collecting data in conflict and crisis contexts. A good enough approach avoids any activity that can compromise the safety of both participants and the research team. Adaptive management practices are equally important and should be inherent in the RERA approach. This means creating contingency plans and risk mitigation strategies each step of the way.

During our work, the data collection team experienced two security issues that affected their ability to conduct data collection. In one instance, we modified data collection timelines and refrained from site visits until the threat of violence against Equatorians working with international NGOs was contained. A second threat of violence due to escalating tensions between ethnic groups necessitated that we postponed data collection until we eventually agreed that additional data from the site was not essential and canceled data collection activities.

#6 Secure your data and think ahead

The limited availability of data in conflict-affected settings makes any primary data collected a great resource and the result of hard-earned team efforts. However, it can also turn into a risk depending on the context and type of data collected. Because of the context in general, and the qualitative nature of the RERA in particular, data collection usually includes personal and community-level information that, if disclosed, could easily lead to the identification of individuals and/or groups already at risk.

It is critical to take the necessary actions for our data to be safe during and after data collection. For example, we should define clear protocols for respondent sampling, ensure confidentiality of respondents, establish how data will be collected and safely stored, among other techniques. In our work, we made sure to follow security best practices and guidelines, de-identifying records as soon as they were saved in our data systems, as well as maintaining the anonymity of respondents throughout the reporting and dissemination of results.

Some great resources that cover this topic are the Office for the Coordination of Human Affairs’ Data Responsibility Guidelines Working Draft (here) and the Handbook on Data Protection in Humanitarian Action (here).

Conclusion

Our RERA exercise highlighted the critical role of data collection in contexts with limited reliable data and challenged us to think differently about how to achieve a critical balance during rigorous data collection in constrained settings.

We recently published findings of our RERA study in the Network for international policies and cooperation in education and training (NORRAG) Special Issue on Data Collection and Evidence Building to Support Education in Emergencies (you can review it here). Dr. Nitika Tolani, team leader for the RERA in South Sudan, will also soon release an article she co-authored with Dr. Leben Moro, national expert on conflict in South Sudan and part of the RERA team, to extend the findings of the RERA to present-day issues around education quality and policy in South Sudan.

The data emerging from RERA studies has been a game-changer for USAID. In South Sudan, the results of the RERA study have tangibly influenced USAID-funded programming in areas such as access to education, provision of quality education services, scale of service delivery and national/sub-national policy.