Afghanistan + 2 more

Real-time evaluation of UNHCR's response to the Afghanistan emergency: Bulletin No. 1

Evaluation and Lessons Learned
Originally published
UNHCR is committed to undertaking real-time evaluations of its role in emergency operations, thereby enabling the organization to examine and assess its emergency preparedness and response activities. A first real-time evaluation was undertaken in 2000, in relation to the Sudan/ Eritrea emergency. UNHCR's Evaluation and Policy Analysis Unit (EPAU) has now launched a real-time evaluation of the organization's response to the Afghanistan emergency.

This first bulletin of the real-time evaluation is based on EPAU's participation in UNHCR's Headquarters Afghanistan Task Force, as well as interviews in Geneva with staff members and external stakeholders, including some who have recently returned from the region. UNHCR management have exercised no editorial control over the content of this bulletin.

It should be stressed that the findings presented in this bulletin are provisional and qualitative in nature. It will hopefully be complemented by future bulletins, based on missions to the field, by additional interviews in Geneva and other locations, and ideally by some quantitative analysis.

EPAU welcomes all responses to the contents and format of this document (to be sent to <hgep00(>) and will strive to incorporate any comments received in subsequent editions of the bulletin.

General assessment

At an early stage of the Afghan crisis, UNHCR recognized the importance of ensuring an effective and timely response to what was evidently going to be a large-scale and high-profile humanitarian emergency. Learning from its experience in the 1999 Kosovo emergency, and strengthened by the emergency preparedness and response plan of action which has been implemented during the past two years, UNHCR has effectively mobilized itself in the Afghan context, moving quickly to appeal for funds, to procure and stockpile supplies, to deploy additional staff in the field and to engage with the authorities in countries that are most likely to be affected by a refugee exodus.

At the same time, it must be recognized that the expected exodus has not yet materialized. This is partly because countries in the region have closed their borders to refugees (which they did well before the current crisis started), but also because many Afghans are too poor or weak to travel long distances, and because male Afghans have been prevented from leaving the country.

As a result, UNHCR's emergency preparedness and response arrangements have not been seriously tested. Most staff members engaged in the operation acknowledge that a refugee movement of the scale and speed that occurred in Iraq (1991), Rwanda (1994) or Kosovo (1999) would have overwhelmed UNHCR's capacity.

In this respect, several staff members observed that the organization continues to lack 'fast-track' budgetary and administrative procedures that can be triggered when an emergency occurs. The absence of comprehensive tracking procedures, providing constantly updated information on the deployment and location of staff, as well as the delivery and distribution of relief supplies and the movement of refugees, is another issue of concern.

To conclude this general assessment, some reference must be made to the complexity of the operational environment in which UNHCR is engaged. The Afghan crisis is evidently a highly politicized one, with UNHCR's major donor, the United States, a primary protagonist in the conflict. Many countries in the region have closed their borders, raising important issues in relation to UNHCR's protection mandate and its potential role inside Afghanistan.

Staff security has been a major concern since the beginning of the emergency, and has prevented staff in the border regions of Pakistan from undertaking important operational activities. It remains to be seen if UNHCR personnel from western countries will be able to continue in the operation, particularly if and when the organization begins large-scale activities inside Afghanistan.

The complexity of the operational environment also derives from the fact that the number of 'old caseload' Afghan refugees in Pakistan and Iran continues to be much greater than those who have entered both countries since the events of September 11. An estimated 200,000 Afghans entered Pakistan between September 2000 and January 2001, and many of them require urgent assistance.

According to a number of UNHCR staff, there is a distinct danger that in mobilizing for the current emergency, the organization may neglect the situation of these people. Similarly, while UNHCR's role in the eventual return and reintegration of exiled and displaced Afghans might not yet be clear, planning for such a development should proceed, even as the emergency unfolds. A holistic approach is evidently required, dealing with the region as a whole, and with the many different groups of Afghans and host communities concerned.

Finally, the uncertain trajectory of events in the region of Afghanistan have made scenario planning and contingency planning difficult for UNHCR and its partners. With the crisis in its second month, it is not easy to predict the exact nature, scale or location of the emergency. Even so, data is available on the problems and needs that exist, both in Afghanistan and in the neighbouring states, enabling the planning (and in some areas the implementation) of programmes to proceed.

Policy and protection

The primary development in UNHCR policy towards Afghanistan (a development that predates the current crisis) has been a growing recognition of the fact that the borders of Pakistan and Iran are unlikely to be opened to large numbers of refugees, and that the organization must consequently envisage a significant role in meeting the needs of displaced and conflict-affected populations within Afghanistan. It has also been balanced by a continued emphasis on the need for neighbouring states to grant temporary protection to those who need it.

UNHCR's pragmatic, country-of-origin focused approach (which has a clear precedent in the Northern Iraq operation of 1991) has been welcomed by the majority of staff members and external stakeholders. Indeed, a number of individuals interviewed by EPAU stated that UNHCR's initial insistence on open borders, and its public criticism of the policies pursued by potential host countries, had the effect of antagonizing some important governmental interlocutors.

In broader perspective, UNHCR's policy may have some negative consequences. First, the organizations acceptance of the need to work with displaced people inside Afghanistan might be seized upon by asylum countries elsewhere in the world as a justification for the closure of their borders. Similarly, UNHCR's acknowledgement of the fact that Pakistan wishes to keep any new arrivals as close to the Afghan frontier as possible may undermine the organizations efforts to promote the location of camps away from borders in other parts of the world.

Second, there are some evident risks in the establishment of IDP camps inside Afghanistan: security risks to UNHCR and other humanitarian personnel; a risk that assistance provided inside the country may not be used for humanitarian purposes; and a risk that the establishment of such camps (especially if they are close to the border) will prompt the involuntary return of refugees from neighbouring states.

Third, according to some members of the donor community, UNHCR's Afghanistan policy raises further questions about the 'Action 1' prioritization process of early 2001, which was interpreted as an indication that UNHCR would in future be less inclined to become engaged with IDPs(1). Third, at a time when the Global Consultations on International Protection are seeking to confirm state support for the 1951 Refugee Convention, events in South-West Asia would seem to suggest that a simple insistence on the right of asylum and the location of camps away from borders is not always a viable policy option for UNHCR.

On a more detailed note, while UNHCR has promoted the notion of temporary protection from an early point in the emergency, some concern has been expressed that the UNHCR paper articulating this position was not issued for almost a month, According to some staff members, there has also been a lack of guidance to personnel in the field with regard to the interpretation and implementation of the policy. With the focus now shifting to Afghanistan, a major policy and protection priority will be for UNHCR to determine the conditions under which it will and will not work in the country, and how it will exercise its mandate for protection and solutions in a volatile and highly politicized IDP context.

Human resource management

It has become a cliche in UNHCR to state that the key to an effective emergency response is to have the right people in the right place at the right time. Like many cliches it contains more than a grain of truth. Indeed, it was UNHCR's perceived weakness in the area of human resource management and emergency deployment that prompted so much international criticism of the agency's performance in the 1999 Kosovo crisis.

UNHCR's initial response to the Afghanistan emergency suggests that the Emergency Preparedness Plan of Action implemented over the past two years has had a positive impact in the area of human resources. Despite the constraints imposed by the organizations recent budget-cutting exercise, which has led to the reduction of several hundred posts worldwide, UNHCR was largely able to meet its immediate human resource needs. This was achieved through the early deployment of emergency response teams, the use of UNHCR's emergency staff roster, and the activation of standby agreements with partner organizations.

Staff members interviewed in the course of this review highlighted a number of commendable initiatives in the human resource sector: first, a decision not to create new posts for the emergency, thereby reducing the time spent on selection procedures and averting the need for a submission to the Operations Review Board; second, the early identification and deployment of a technical coordinator in Pakistan, a specialist in reproductive health and gender issues; third, the early dissemination of a policy document, setting out the conditions of service for the Afghanistan emergency; and fourth, the swift despatch of security advisors and staff support kits to the region, coupled with a decision allowing UNHCR officers to engage additional personal security guards.

Such positive initiatives should not be taken to suggest that the human resources area has been entirely problem-free.

While the Afghan Desk at Headquarters has been strengthened with additional personnel, this action does not seem to have been taken quickly enough and did not meet the expressed needs of the incumbent team.

In the field, and as indicated above, UNHCR's response capacity in Pakistan has been limited by the recent departure of many staff, experienced national officers who could have played an important role in the emergency if they had remained with the organization. As usual in an emergency, the regional bureau affected by the crisis has been obliged to negotiate with other bureaux for the release of staff, a problem compounded by the reluctance of some personnel to serve in an area where the security risks are so evident.

Delays in issuing visas by countries in the region of Afghanistan, coupled with a reluctance to admit UNHCR staff members of certain nationalities, have also proven to be a frustrating constraint on the organizations response. In this respect, more attention could perhaps be given to the hiring of local emergency personnel, especially in a country such as Pakistan, which has a substantial body of people with relevant qualifications and experience.

While UNHCR has been relatively effective in deploying personnel to the field, some concern has been expressed in relation to the organizational structures within which those people must function. In this respect, more work is required in terms of developing model emergency staffing organigrams, prioritizing the staffing profiles required in different emergency scenarios and developing standard terms of reference for emergency deployments.

Finally, and as discussed elsewhere in this bulletin, UNHCR must ensure that it incorporates a longer-term focus in its emergency response procedures, so as to be prepared for the time when the crisis comes to an end and displaced populations are able to return to their homes. This task will be facilitated by the fact that linkages between humanitarian and development actors have already been established in relation to Afghanistan, and by the fact that many agency personnel have been evacuated from Afghanistan and could now initiate longer-term planning activities.

In other situations, UNHCR should consider including a 'post-conflict' planning specialist in any emergency team, responsible for establishing early linkages with potential partners in the process of reintegration and rehabilitation. Such an approach would avert the more usual scenario, in which UNHCR establishes a reintegration and rehabilitation programme, and then tries - often unsuccessfully - to effect a 'hand-over to longer-term development actors.

Procurement and logistics

UNHCR's performance in an emergency tends to be judged by the simple measure of how many tents, blankets and other basic relief items the organization is able to purchase and deliver in the early days of a crisis. In the first two weeks of the Afghanistan operation, the organization struggled to meet the anticipated needs, and was spared by the fact that relatively few refugees were able to escape the country. Indeed, in the absence of any large-scale influxes, a decision was taken not to airlift supplies from UNHCR's Copenhagen stockpile.

The problems encountered by UNHCR in the domain of procurement and logistics are largely perennial ones. Canvas tents cannot be stockpiled because they rot. UNHCR has established frame agreements with a number of suppliers (mainly in Pakistan), but still has to wait for the tents to be manufactured once such agreements are activated. In the Afghan emergency, as in other recent crises, the agency has also had to compete for supplies and transport capacity with other organizations, and has been obstructed by the need to observe time-consuming customs and import procedures in destination countries.

These familiar difficulties have been compounded by the fact that UNHCR continues to lack fast-track procedures and standardized tools to track the purchase, transport, delivery and distribution of relief items. It also lacks a standardized list of the non-food items that each beneficiary should receive in an emergency - something that became clear when the Task Force realized that UNHCR was planning its procurement on the basis of one blanket for a family of five people! The introduction of such procedures, tools and lists should evidently be a high priority for the organization.

In addition, UNHCR should quickly conclude the tests which it is currently conducting in relation to the use of lightweight and stockable tents. It should also consider the further development of a stockpile and logistics capacity in Dubai, a location which is secure, which has excellent international transport links, and which is conveniently located with respect to three emergency-prone regions: South Asia, South-West Asia and the Horn of Africa.

Internal coordination

Emergencies have an inevitable tendency to create tensions and conflict between the different actors and entities involved, whether at Headquarters, in the field, and between Headquarters and the field. The Afghanistan operation has been no exception to this rule. Nevertheless, it is of some significance that in the course of this review, staff on the Afghan Desk declared themselves to be very satisfied with the support they had received from other parts of the Headquarters structure.

A key entity in the operation is the Geneva-based Task Force, which is coordinated by the regional bureau covering South-West Asia, and which brings together representatives of entities dealing with functions such as emergency response, protection, procurement and logistics, budget, donor relations, media relations, telecommunications and evaluation. In the early days of the emergency, the Task Force met on a daily basis, including weekends. It currently meet on a 'as needs' basis, usually two or three times a week.

Participants in the Task Force generally feel that it has acted as a valuable forum for the exchange of information, enabling them to keep abreast of the action taken by other entities. This includes the Regional Coordinator in Islamabad, who has a regular telephone conversation with the Task Force chairman immediately before each meeting. There is also an appreciation amongst Task Force members that the meetings have become briefer and more focused as the operation has progressed, with sub-groups being established to deal with detailed issues that are not of direct concern to the Task Force as a whole.

According to participants, other aspects of the Task Force require further consideration. These include: the absence of formal terms of reference for the Task Force, and consequently a lack of clarity in relation to its role and decision-making authority; the seniority of staff participating in the Task Force; the limited information received by the Task Force on decisions made by the senior management policy group dealing with the Afghan crisis; and the limited information received by the Task Force on the activities of other humanitarian organizations involved in the emergency.

At this stage of the review, the evaluation team is not in a position to comment on field-level coordination issues within the UNHCR operational structure. This topic will hopefully be addressed in a future bulletin, once missions to the region have taken place.

Systems and procedures

There is no doubt that UNHCR's emergency practices, procedures and tools have been enhanced in recent years, especially since the Kosovo emergency and the publication of an independent evaluation on the organizations performance in that crisis. In terms of issues such as emergency training, teams, rosters, handbooks, stockpiles and stand-by agreements, major improvements can be reported.

At the same time, UNHCR staff continue to suggest that the organizations systems are not geared for rapid action, and that fast-track procedures, which can be triggered when an emergency is declared by the High Commissioner, have not yet been established.

A case often cited is the Emergency Letter of Instruction (ELOI), which provides an authorization to disburse funds. While intended to move at a faster pace than a regular Letter of Instruction, ELOI's nevertheless have to be cleared at a number of different levels, and do not appear to provide UNHCR with the ability to respond with the necessary speed in an emergency.

Similarly, staff members complain that the budgets and project proposals they are required to establish are unnecessarily detailed in an emergency context and could readily be simplified. It is recommended that such systemic and procedural constraints be examined, with a view to the establishment of fast-track procedures, as proposed by other emergency evaluations.

Information management

UNHCR currently lacks a comprehensive tracking procedure, capable of providing constantly updated information on the deployment and location of staff, as well as the procurement, delivery and distribution of relief supplies. Such a system would be invaluable for planning, programming, donor and media relations purposes, and should be established without delay.

The real-time evaluation of the UNHCR operation in Sudan and Eritrea pointed to the need for an integrated information management system, providing all individuals and entities concerned with access to the same data and documents. Such action was not taken immediately in the context of Afghanistan; in the early days of the operation, information exchange was ensured by e-mail and undertaken by the secretary to the Task Force.

Within a short space of time, however, the Records and Archives Section introduced a web-based information management system, titled 'Livelink', to manage and structure information flows at Headquarters and from the field for current use and future reference. Given the current sense amongst Task Force members that Livelink is not being extensively used, it is recommended that its value is assessed, with a view to making any necessary improvements to the way it functions.

Media relations

UNHCR has pursued an aggressive and competitive public information strategy in the Afghan crisis, which has proved highly successful in terms of visibility and international media coverage. Unlike the early days of the Kosovo crisis, UNHCR's presence in the region of Afghanistan and its efforts to prepare for a humanitarian emergency have been effectively communicated to the public and to governments. More than in any other recent emergency, UNHCR has also linked its media relations strategy to its fund-raising efforts, especially those relating to the private sector and general public.

The implications of this approach warrant some reflection. First, according to some staff members, UNHCR's bullish media relations campaign in the early days of the emergency may have raised expectations that the organization could not have met if a large-scale refugee movement had taken place. In this respect, it is interesting to note that UNHCR quickly changed its message and began to make extensive use of the media to publicize the operational constraints confronting the organization.

Second, while UNHCR has performed as least as well as its competitors in the humanitarian media marketplace, some interviewees have suggested that this competition may lead to inter-agency friction and that it is contrary to the notion of a united United Nations. According to one donor, it would be refreshing to see a UN spokesperson on TV, representing all of the multilateral humanitarian organizations involved in the Afghan emergency.

Donor relations

Donors seem generally satisfied with UNHCR's early performance in the Afghan crisis, although they are fully aware that the organizations response capacity has not yet been put to the test. They are appreciative of the quantity and quality of the information they receive from UNHCR, but, like a number of UNHCR staff members, express some surprise and concern about the size of the appeal issued at the beginning of the emergency: $268 million, almost a third of the organizations regular annual budget. One donor pointed to the more modest approach to fund-raising pursued by the ICRC, and recommended it to UNHCR.

As far as the organization is concerned, UNHCR is generally content with the initial donor state response to the emergency, but expresses some frustration in relation to the time-lag that can occur between the receipt of a donor state pledge and the actual receipt of the promised funds.

Finally, some mention should be made of the political and strategic interests of UNHCR's major donors, which are generally supportive of the US in its response the events of September 11.

Public support for the military campaign in Afghanistan may well be undermined if it is perceived to be the cause of a major humanitarian disaster, involving the suffering of innocent Afghans. The donors therefore have a very direct interest in averting such a scenario, and are looking to UNHCR and other humanitarian agencies to assist them in this respect. They are also seriously concerned about the potential for destabilization in Pakistan, and are unlikely to question that countrys decision to close its border to refugees. In this highly charged international context, UNHCR must evidently be careful to safeguard its "entirely non-political character" and its mandate for refugee protection.

Elca Stigter and Jeff Crisp EPAU



(1) One donor state representative expressed the opinion that between the Action 1 initiative and the October meeting of the Executive Committee, UNHCR had "done a 180 degree turn" on the question of its involvement with IDPs.