DEC publishes review of member agency assurance mechanisms
The Disasters Emergency Committee (DEC) has published today an independent report which shows how its member agencies provide assurance that they are following agreed ways of working when they respond to emergencies.
The report “DEC Accountability Self-Assessment Validation 2013-14” http://www.dec.org.uk/annual-assessments was prepared for the DEC by consultants from One World Trust who validated members’ self-assessed performance against 21 ‘Ways of Working’ http://www.dec.org.uk/sites/default/files/files/DEC%20Accountability%20Framework%20-%20Explained.pdf [PDF] which DEC members are committed to following.
The report shows that DEC member agencies continue to report performance improvements and that the large majority of these self-assessments were likely to be accurate. Importantly however it also drew the attention of a minority of members to areas where their self-assessments were insufficiently supported by evidence.
DEC Chief Executive Saleh Saeed said:
“The DEC Accountability Framework shows that our members do not simply sign up to commitments to observe best practice, they also put in place systems to ensure they live up to these commitments.”
“The process members go through to demonstrate they are following the agreed DEC ways of working can seem very abstract because it is about assurance mechanisms that provide evidence that policy and procedures are appropriate and systematically implemented. In reality, this approach is at the heart of the way the DEC works because it helps ensure our members are consistently working to the highest possible standards and continually seeking systematic improvements in their work.
“What it comes down to in the end is that getting aid work right isn’t just about good people who know what they doing delivering good work. If you want to ensure you are delivering consistently excellent work you also need good processes.”
Amber or indeed Red self-assessed rankings against particular ways of working, or re-grading of members’ self assessments, are not necessarily evidence of any shortcoming in the way agencies have delivered aid in emergencies - this is tested by a separate process of independent reviews and evaluations as part of each DEC appeal http://www.dec.org.uk/appeals. What these amber or red rankings show is that there is insufficient evidence that a policy or procedure is being systematically followed or, in a very small number of cases, it highlights that the relevant policies and procedures are inadequate.
Self assessments showed that in 2013-14 members collectively rated themselves as ‘green’ against nearly 51 % of the Ways of Work. This means that they felt they could provide strong evidence to show they were systematically following robust policies and procedures. They ranked themselves as ‘amber’ against over 48% which means they had robust policies and procedures and could provide some evidence they were being followed. Less than 1% of ratings were red which suggests that relevant policies and procedures are either inadequate or non-existent.
Although this process can seem very abstract and distant from front line aid work, the DEC and its members believe it is extremely important because it helps ensure that the lessons we learn from experience and evaluation get locked into the way we work in future. This is important because it means agencies do not rely exclusively on the personal experience of staff to ensure we learn from the past and it allows for continuous improvement.
This year each member agency had its self assessments for five ways of working closely scrutinised to verify the accuracy of member agency self assessment procedures. The five ways of working checked for each agency were selected from a pool of eight criteria:
Documented processes are in place at the appropriate level governing the use and management of funds Programme design and procurement processes maximise value for money – balancing quality, cost and timeliness at each phase of the response Programmes respond to clearly defined needs and are adjusted as needs change Staff and partners understand and integrate agreed standards into their programmes A defined and documented accountability framework is in place governing accountability to disaster affected populations Information on agency background, programme timelines, beneficiary entitlements and selection criteria is communicated to disaster affected populations Disaster affected populations participate in programme assessments, design, implementation and evaluation Key learning (including from evaluations) is incorporated into processes and programmes in a systematic and timely manner.
Of the 13 DEC member agencies nine were found to have correctly assessed their performance against all criteria. CARE, Concern and Save the Children each had one of their assessments re-rated but the evaluators concluded that the majority of their remaining assessments were likely to be accurate.
As two of ActionAid’s self-assessments needed re-rating the consultants concluded that they could not initially validate the reminder of their self-assessments as likely to be accurate. ActionAid responded by commissioning One World Trust to check all 21 of its self-assessments. Following this process a total of eight ratings were changed by ActionAid from green to amber so their final ratings were eight green, 13 amber and no red. The revised ratings do not suggest that ActionAid’s assurance processes are less strong than those of other agencies. Nor do the ratings of assurance systems reflect directly the quality or impact of any agency’s humanitarian programme work.
There will be no external validation next year as DEC Trustees have agreed this should become a biennial process. Agencies will however undertake a self-assessment and report on the improvements they have made over the year. The framework is being updated to reflect the outcomes of the Core Humanitarian Standards process. An earlier review implemented in 2011-12 led to a raising of the bar, strengthening and widening the expectations within the ways of working and the current developments are expected to do the same.
Further details about how the DEC and its member agencies work to continually improve standards in aid delivery can be found here http://www.dec.org.uk/about-dec/how-we-ensure-quality .
DEC Head of Communications Brendan Paddy has written a personal account of how the process works and why it matters http://www.dec.org.uk/blog/process-vs-outcomes
Notes to editors
-
The DEC brings 13 leading UK aid charities together in times of crisis: ActionAid UK, Age International, British Red Cross, CAFOD, CARE International, Christian Aid, Concern Worldwide, Islamic Relief, Oxfam, Plan UK, Save the Children, Tearfund and World Vision; all collectively raising money to reach those in need quickly.
-
To make a donation to the DEC Ebola Crisis Appeal visit www.dec.org.uk or call the 24 hour hotline on 0370 60 60 900.