The numbers tell the tale

A few weeks ago, on a Tuesday evening, I found myself sitting in a primary school listening to two kindergarten teachers, explaining me how my son is doing in school. The base for this talk was a grading system compiled out of five pages of indicators with different phases. They had observed him, and scored him through these 30+ indicators over the last few months.

I admit, I am easily thrown off by numbers, so the ten minute timeframe we had was almost entirely taken up by me trying to understand the system, the indicators, and what these meant. While all I want to know is whether he is feeling ok, and doing all right. I mean, he’s four years old.

I find it interesting (in the sarcastic British meaning of the word) that such an elaborate evaluation system is used, which takes up huge amounts of time for the teachers to fill out for the 20+ individual toddlers, and that it doesn’t match with the kind of information I would want to receive as a parent.

In the complexity and challenges of using a system measuring the progress of a toddler and its diffuse outcome, I see a parallel with the field of qualifying progress in security & justice. The fields of security and the Rule of Law are notoriously difficult to measure. Their highly politicized nature and cross-cutting impact alone make effective impartial measurement of progress or decline linger between flawed and partial. For instance, the selection of indicators, the ability to handle large amounts of data and the chosen timeframe of collection all color the tale the numbers tell in the end. A partial view is all we get. And we should realize that.

The difficulties in measuring the effects of programs and policies on security and rule of law in the development sector makes it an immense, (time) costly task, met with severe challenges in relation to funding, reach, access and analysis. 

When we realize that worldwide insecurity is the rule, and security the exception for the majority of all human beings, and agree that real progress and change in a country takes roughly 30-40 years, then why are we only spending 0,13% of ODA funding on data collection that can strengthen the efficiency and effectivity of foreign policies and development programs? I think the answer is simple, because data is not sexy.

However, when we translate this back to the field of security and the rule of law, look at the amounts of money that is poured into these field, the ineffectiveness of many programs and policies due to the lack of a sound base in evidence in relation to the importance for the world as a whole, then I don’t get it. And yes, there are a lot of sensitivities when it comes to the collection of data, but we’re not talking about all the buts and ifs, we’re talking about trying to get a better and realistic picture then the old Polaroid’s with the nice white frame we liked so much.

Why wouldn’t we be more interested in breaking new ground when it comes to improving the collecting of information in security and justice, and subsequently improve our policies and programming in this field? The more our work is based in evidence, the more effective our work can be and perhaps the more progress we can make.

When we connect innovative data specialists with index producers, policy makers, and practitioners, we can set the first steps in acknowledging the blind spots we are currently working so hard around. Such cooperation could have several outcomes, for instance: allow for more time and funding to do good evidence based conflict analysis and M&E method in calls from national and international donors; understand the importance of data collection and subsequent analysis and implementation as it allows to get more grip on these highly versatile topics. And again, yes, we will be met with major challenges in funds, ways of collection and so on, but that is the name of the game when you are working in these fields. And perhaps even more so in the field of security than any other field we should challenge ourselves constantly as stakeholders and actors to turn towards the difficulties and not away from them. No, data is not sexy, but the numbers tell the tale and it is essential if we want to improve our work in this field, which is a responsibility of all actors active in this field. We need people and organizations that have the guts to break new ground and who dare to fail, as it is failure that teaches us often more than success.

In short, despite the popular rhetoric of evidence-based approaches to policy making, there is insufficient funding for data collection, which complicates security and justice programming and hampers its efficiency and effectivity. Apart from that we need to be gutsy and try out different ways of collecting and diversifying data. One of the most important in this regard is how we can include the local voice and (political) dynamics that is now often lacking in our indexes and oversights. Ignoring that is simply a disregard for reality. And, in my opinion, in the state of the world anno 2016 there is no room for policy short-cuts, far-fetched fairy tales or unrealistic views in security & justice.

This blog was written in response to our past event entitled "Measuring Peace".

See additional resources below:

News item "Measuring Peace".

Policy paper by OECD on the vital information that is not captured through indexes, namely local, political dynamics in security and justice programming on the ground.