Effectiveness and efficiency are two words often incorrectly used synonymously. In reality they mean very different things. ‘Effectiveness’ looks to results and if you have achieved change. ‘Efficiency’ is all about how much that change costs and if the cost is worth the result. To achieve long-lasting change and bring the world out of poverty, one needs to develop programs that are both effective and efficient.
One of the widest programs in the world is Development Aid – child sponsorships promoted by many aid agencies and very popular in the West. Give a few dollars a month to rescue a child – or so the advertising goes. Does it work?
Since 1953 the top eight child sponsorship providers have seen nearly 3.5 million children pass through a sponsorship scheme. Surely one would expect to see a great deal of academic study and research on both the effectiveness and efficiency of these child sponsorship programs? If you don’t have a measure, how do you know if you are making a difference? How do you improve?
Bruce Wydick conducted one of the few independent international studies. Over the last 20 years a combined US $30 billion has been spent through child sponsorship programs. Wydick asked has it made a difference?
Observers of the aid world usually concentrate more on the motivation of an organisation trying to bring about change, rather than the large-scale change itself. They pick individual stories and market them letting you assume they are representative of the work of the entire agency. All that’s needed to gain a continuity of funding for an organisation or program is to create the perception that the organisation is well-meaning and trying to make a difference. For many years the aid world said it was simply too hard to measure the impact of a program so analysis was usually done on process. By this most agencies talk about how much money is spent. That is the process. Rarely do agencies promote the results.
My experience has been similar. For years the aid world said that it is too hard to measure outcomes, so reports measure process. If you pick up almost any United Nations agency report it will say something along the lines of “we were given $100 million to spend on a child literacy program. We spent $97 million. That is 97 per cent implemented and therefore the program is successful.” Rarely will the report go on to say “with that $97 million we saw an increase in child literacy from 13 per cent of the population to 27 per cent of the population, with that 14 per cent change being more efficient than a similar program in country X, but less efficient than in country Y. The lessons that we take away to improve efficiency are…”
Failing to properly evaluate the program fails to measure the impact on the people that the program is intended to serve. Bruce Wydick’s study did find improvements from child sponsorships, but the increase in school attendance as a result of the study was only two years. But is the measurement of time in school the right indicator of success?
It would make a more interesting study to determine the long-term impact of attending school on the future employability and economic well-being of the sponsored child and his or her later family. Surely if the objective of a sponsorship is to break the cycle of poverty, then the only true measure is to ask if the child is or is not now out of poverty? How many children who have passed through child sponsorships are now doctors? How many are teachers? How many are government ministers or business leaders?
More damningly, how many recipients of current child sponsorships are the children of former recipients? We should ask if pumping funds into sponsorship alleviates a problem for only a short time but makes no real difference in the long term. The truth is this is rarely, if ever, measured.
The big question around child sponsorships should be ‘have you lifted a child out of poverty?’
In over 50 years of aid has there been a macro change, or are the cynics right when they say ‘the definition of aid is poor people in rich countries giving money to rich people in poor countries’?
Wydick’s is the most optimistic study of effectiveness, but even his study finds that “we make no claims in this research that child sponsorship represents the most cost-effective path”. Regarding efficiency, his study went on to say that: “…cost of these services (sponsorships) was about $28 per month to the sponsor, and the mean number of years of sponsorship in our sample was 11.33 years. Thus the total cost of sponsoring a single child to the average sponsor was approximately $3,806. Using our more conservative impact estimate of 2.88 additional years of schooling from child sponsorship, this puts the average cost of an additional year of schooling from the program at $1,321 per year, per sponsored child.”
I raise an obvious question: Could you spend $3,806 (this is roughly 10 years average income in the DRC) in a way that achieves a better result than two additional years in school? Is this the optimum use of those funds?
Wydick goes on: “When addressing issues of cost efficiency it is important to understand that the development of international child sponsorship programs fundamentally arose from their usefulness as a marketing tool for mobilizing resources in rich countries to fight poverty in poor countries”.
In academic language Wydick has hit on the elephant in the aid room: Child sponsorships are about marketing. The child is used as a marketing tool to tug at the heart strings of people in rich countries.
There is, in my mind, an inherent dishonesty in this. If an organisation intends to use the money for broader programs, then it should market programs as such.
Perhaps criticising either the effectiveness or efficiency of child sponsorships is a risky business. There is such a feeling of goodwill around child sponsorships that an assumption of a mean, Scrooge-like person is often thrown at anyone who dares raise questions. Hence criticism is rare.
I have another view. If child sponsorships are actually succeeding in bringing families out of poverty then it is a good thing to evaluate and show the success. But is it legitimate to show the opposite? It is legitimate to question if $1,321 is best spent to gain one additional year of schooling, or if that funding should be spent more effectively in another way to break the cycle of poverty?
One must question the sponsorship effectiveness because children are in need. If the child sponsorship doesn’t break the cycle of poverty, then the sponsorship system is in reality a betrayal of those people to not question the system.
This is why it is best not to ask an agency ‘what it is they will do for this generation’. It is better to ask ‘what is it that you have done for previous generations?’
Unfortunately, the mentality of analysing effectiveness and efficiency does not exist within many organisations in the aid world. If one questions an aid agency about their effectiveness or efficiency from inside the system it can be career limiting. If criticising from outside the system one tends to be labelled ‘right wing’ and ‘focused on capitalism and money’.
Until recently, no one has ever held the aid industry to account for its results. Only recently have reports, such as the Australian Government Aid Effectiveness Review, begun to ask ‘what is the result for all our spending?’.