User Guide To Poverty & Social Impact Analysis

A User’s Guide to Poverty and Social Impact Analysis World Bank Poverty Reduction Group (PRMPR) and Social Development Department (SDV)

WORK IN PROGRESS DRAFT FOR COMMENT APRIL 19, 2002 VERSION

Table of Contents I.

Introduction........................................................................................................................... 2

II. Background and Challenges ................................................................................................ 2 III. Implications: Principles for Operationalizing Poverty and Social Impact Analysis ...... 4 IV. A Conceptual Framework for Understanding Poverty and Social Impacts ................... 5 V.

Elements for Good Poverty and Social Impact Analysis (PSIA) .................................... 11 1. Asking the Right Question 12 2. Identifying Stakeholders 12 3. Understanding Transmission Channels 14 4. Assessing Institutions 15 5. Gathering Data and Information 18 6. Analyzing Impacts 23 7. Contemplating Enhancement and Compensation Measures 36 8. Assessing Risks to PSIA 38 9. Monitoring, Social Accountability, and Evaluation of PSIA 39 10. Feedback of PSIA into Country Policy Choice 46

VI. Possible Summary Matrix .................................................................................................. 47 VII. Conclusions .......................................................................................................................... 51 Appendix 1 Typology of Tools for Poverty and Social Impact Analysis ............................... 52 Appendix 2 Poverty and Social Impact Analysis – Reform-by-Reform Application of Key Tools ........................................................................................................................ 56 References .................................................................................................................................. 58

i

Annex: Economic and Social Tools for Poverty and Social Impact Analysis I. Identifying Stakeholders ..........................................................................................................66 Stakeholder Analysis ..............................................................................................66 II. Assessing Institutions ..............................................................................................................67 Institutional analysis ..............................................................................................67 III. Analyzing Impacts – Social Tools ........................................................................................68 Social Impact Assessment (SIA) ...........................................................................68 Participatory Poverty Assessments (PPA) .............................................................71 Social Capital Assessment Tool (SOCAT) ............................................................72 IV. Analyzing Impacts – Economics Tools ................................................................................73 1. Direct Impact Analysis – Incidence Tools .....................................................................73 Simple Benefit Incidence Analysis ........................................................................73 Poverty Mapping....................................................................................................75 2. Behavioral Models. ........................................................................................................76 Behavioral incidence analysis ................................................................................76 Demand analysis ....................................................................................................77 Supply analysis ......................................................................................................78 Household models..................................................................................................80 3. Partial Equilibrium Models ............................................................................................81 Multimarket models ...............................................................................................81 Reduced- form estimation.......................................................................................82 4. General Equilibrium Models..........................................................................................83 Social Accounting Matrices and Input-Output models..........................................83 Computable General Equilibrium (CGE) Models .................................................83 5. Microeconomic Simulations Linked to Macroeconomic/Sectoral Models ..................86 V. Assessing Risks ........................................................................................................................89 Social Risk Assessment .........................................................................................89 Scenario Analysis...................................................................................................91

ii

Preface This draft User’s Guide lays out an approach to poverty and social impact analysis (PSIA). This draft Guide provides a menu of possible tools and techniques for such analysis; it does not set out minimum standards. PSIA is not a discrete product within the Bank’s Economic and Sector Work (ESW) “product line” but rather an approach that may be used as part of Bank ESW or by other donors or developing countries. This draft Guide does not constitute Bank operational policy for poverty and social impact analysis. Specific guidelines for Bank staff on poverty and social impact analysis are being developed separately. The draft Guide is a work in progress. It is being circulated in draft form for external comment, and we would very much welcome reactions on its usefulness from policy makers and analysts in developing countries, as well as from representatives of donor agencies and civil society organizations. Feedback and comments can be sent to [email protected]. At the same time, a process of consultation with national teams and Bank operational staff is underway. We plan to update the draft Guide over time to reflect further country experience and good practice with PSIA as well comments on the approach. In this regard, we would be particularly grateful for country examples of good practice. The draft Guide is one element of a much larger work program by the Bank on poverty and social impact analysis. Key elements of this work program include: • PSIA Toolkit: this will provide more detailed technical guidance on the tools highlighted in the draft User’s Guide, and will be available later this year. • Reform-by-Reform Supplement to the draft User’s Guide: this will discuss tools for PSIA on a reform-by-reform basis. This will also be available later this year. • Learning Program: this is underway and is being rolled out gradually. It will focus in the first instance on Bank and Fund staff, and over time be extended to country counterparts. In addition to formal training, the Learning Program includes action learning focusing on teams applying PSIA to operational work and on learning through cross country experience. • Country Cases: PSIA of specific reforms is being carried out in a set of low income countries in order to derive lessons for wider application. This work is being done in collaboration with DFID who are supporting similar case studies.

1

I.

Introduction

1. Increased attention to poverty reduction as the central goal of development has highlighted the need for an improved understanding of how policies affect the poor. With the advent of the Poverty Reduction Strategy Paper (PRSP) process, many low income countries are struggling over the choice of public actions that will have the greatest impact on poverty. The Bank has a major role to play in helping countries improve the analysis on which such choices are made, and the international community has high expectations of our delivery in this area. 2. This draft User’s Guide is intended for practitioners undertaking PSIA in developing countries.. The draft Guide provides a menu of possible tools and techniques for such analysis; it does not set out minimum standards. The main section defines PSIA and outlines the various tools that can be used to carry it out. The Annex provides more detail on individual tools. The draft User’s Guide, in advocating a multi-disciplinary approach to PSIA, presents both economic and social tools and quantitative and qualitative methods. 1 3. Given the broad scope of policy issues and methods, this draft User’s Guide cannot be comprehensive in its treatment. Instead, it attempts to provide some general guidance on how to approach the analysis of poverty and social impacts of policy reform, and it provides a typology of tools, illustrated by country examples. To supplement the draft User’s Guide, DEC/PREM/WBI plan to issue a toolkit on PSIA which will provide more in-depth guidance on the economic tools discussed in this paper. The Social Development Department is organizing an intensive learning program to provide further guidance on the social development tools for PSIA. In addition, the draft User’s Guide will itself be appended to include more guidance on the application of tools and approaches to address specific reform issues confronting low income countries. 2 4. The draft User’s Guide – and the other guidance material as it becomes available forms the basis of a new learning program for Bank staff, beginning in April 2002.

II. Background and Challenges 5. “Poverty and social impact analysis” (PSIA) is used in this draft User’s Guide to mean analysis of the distributional impact of policy reforms on the well-being or welfare of different stakeholder groups, with particular focus on the poor and vulnerable 3 . In so doing, it also addresses issues of sustainability and risks to policy reform that come with social impacts of policy changes. Section IV provides a more detailed conceptual framework for PSIA. 6. Analysis of poverty and social impacts of public action is not new. There is a long history of work in this area, initially in the context of projects. Many of the methods now considered standard for project analysis were developed at the Bank in the 1960s and 1970s 1

This work has been the product of a joint PRMPR-SDV team. The draft User’s Guide is envisioned as an evolving product that will incorporate good practice and comment as it becomes available. 3 This paper uses the concepts of ‘well-being’ and ‘welfare’ synonymously. 2

2

(Gittinger, 1985; Timmer, Falcon and Parson, 1983; Squire and van der Tak, 1975). The impact of government policies on poverty and distribution received more attention following the adjustment reforms of the 1980s. Similarly, social impact analysis has emerged in the anthropological and sociological literature initially for project analysis (Goldman, 2000; Becker, 1997; Finsterbusch, 1994) with increasing recognition of its relevance to policy (Rickson, 1990; Kudat, 2000). Partly in response to a critical report on the social costs of adjustment by Cornia et al (1987), the analysis of poverty and social impacts of policy became intensified in the 1980s and 1990s. 4 7. While PSIA is not new, this draft User’s Guide is written in recognition of the fact that there have been a number of weaknesses in its application to the design of government policy. In particular: • • • •

PSIA is often not undertaken early enough to inform the design of policy. When there is analysis, the assumptions behind it are frequently not set out in public documents, which reduces transparency. The risks to policy implementation are generally not well addressed. Policy- makers often fail to explicitly recognize potential losers from reform and/or consider mitigating measures.

8. These gaps can weaken the likelihood that reforms will succeed in meeting their poverty and distributional objectives. The fact that PSIA has not been undertaken systematically to inform major policy decisions reflects the challenges involved: •

Data constraints. In many instances, the data required to do a comprehensive analysis are not readily available. Household survey data, which are particularly relevant to undertaking distributional analysis on a national level, sometimes do not exist or are dated. Equally common, the survey data that do exist do not address questions relevant to the reform at hand.

•

Analytical constraints. ⇒ The impact of macroeconomic and structural reforms cannot easily be analyzed at the microeconomic or household level. Policies have many direct and indirect effects at the microeconomic level, mediated through local institutions and behavior. Fully capturing the complexity of reality ex-ante in a model is difficult. The analyst has to walk a difficult line between simplifying reality to explain impacts and capturing context- specific institutions and behavior. ⇒ Impacts may differ over time. For instance, a policy may result in short-term losses and gains among different groups, even when longer-term effects are expected to be positive. Capturing these inter-temporal dimensions within distributional analysis is a complex undertaking.

4

Notably such analysis was done under the auspices of the Social Dimensions of Adjustment program within the Bank, and the USAID funded Cornell University Food and Nutrition Policy Program. Related work on this issue includes Demery, Ferroni, Grootaert, 1993.

3

⇒ Rigorous analysis requires a comparison to be made between impacts with and without reform, (the status quo itself being an alternative policy choice or counterfactual). This is hard enough to do on an ex-ante basis. It is also challenging in ex-post analysis as many factors will have changed, masking reform-specific effects. ⇒ Addressing these analytical challenges requires the right economic and social tools. Many useful tools exist, and this draft User’s Guide will highlight some of the main ones. But, more work is needed to develop analytical methods which are better equipped to meet the gaps and to develop more rigorous indicative survey tools and analysis thereof, where good data is otherwise lacking. •

Capacity constraints. The analytical challenges are complicated by capacity constraints. In poor countries capacity to analyze policy is weak among government agencies, academia and civil society organizations. So while rigorous analysis might call for complex tools and methods, local capacity may be suited to more basic approaches. Over time, however, the capacity of in-country development agencies also requires strengthening both in terms of core analytical expertise and resources allocated to PSIA.

•

Time constraints. While the analyst may face difficult data and analytical challenges, the policy- maker is often under pressure to make fast policy decisions which cannot wait for a rigorous PSIA to be completed, (such as in an economic crisis). In these circumstances, arguments for postponing policy decisions until there is adequate analysis, debate and consensus will need to be weighed against the case for acting expediently to address a crisis, or at a time which is more in tune with the policy or political cycle.

III.

Implications: Principles for Operationalizing Poverty and Social Impact Analysis

9. The challenges outlined above have often deterred policy analysts and decision makers from undertaking ex-ante assessments of the poverty and social impacts of reform. While some have argued that “no analysis is better than bad analysis,” it is important to consider what analysis is feasible, even where data and capacity are limited. The question, then, is how is to approach poverty and social impact analysis in the face of the various constraints. 10.

Good practice suggests that the following principles are important: •

Country ownership. If PSIA is to be an effective tool for policy, it needs to be countryowned. Countries are responsible for the choice of reforms and for the analysis. In undertaking the analysis, they can seek external assistance from partners including the World Bank, UN and bilateral donors.

•

Increased attention to ex-ante analysis. It is important that ex-ante analysis of expected poverty and social impacts underpin the design and choice of policies, particularly those that are expected to have the greatest impacts in the short to medium term. This will help ensure that policies are conceived, designed, and implemented with a view to enhancing poverty reduction and social objectives. In doing so, it is important to recognize that

4

there is also a need to build on analysis of earlier reforms in order to inform ex ante analysis. •

Monitoring and evaluation t o validate ex ante analysis. Ex ante analysis cannot fully capture policy impacts. It is therefore important to track actual results through monitoring and, where possible, ex-post evaluation. That way, mid-course corrections can be made to reforms that are not having their intended poverty or social impact.

•

Flexibility on tools and methods. It is important to tailor approaches to country capacity, reform issues, data availability and time pressures. In some circumstances, qualitative analysis based on economic intuition may be appropriate, while in others complex econometric modeling may be the most useful method. Understanding of impacts is enhanced when results from different analytical techniques reinforce each other or highlight different aspects of impacts.

•

More transparency in the links between policy and poverty. There is much to be gained from laying out for public scrutiny the logic behind a policy choice – including expected losers and winners from reform, key assumptions and transmission mechanisms. It can help promote national debate and acceptance of reform, and serve as a baseline against which to monitor progress. Moreover, it can highlight potential trade-offs between the long run benefits of reform in terms of higher growth and poverty reduction, versus possible short-run worsening of welfare.

•

More explicit consideration of measures to enhance gains, minimize losses – especially among the poor (such as alternative policy choices, complementary or compensatory policies). This will strengthen the pro-poor impact of policies, and improve their acceptability and sustainability.

•

Building national capacity. Building national capacity is key to improving analytical rigor over time, in tandem with strengthened country ownership. Many low income countries have limited capacity and experience in areas of critical importance to PSIA. These areas range from data collection systems, analytical capacity, monitoring and evaluation systems, the capacity to translate data and analysis into policy, and the capacity for debate on such policy issues in the public domain. Building national capacity in these areas thus must be a fundamental cross-cutting aspect of PSIA. Development partners, including the Bank, have an important role in strengthe ning national capacity and in filling in analytical gaps. PSIA approaches that foster “learning by doing” would be a key tenet to development partners’ assistance to countries.

IV.

A Conceptual Framework for Understanding Poverty and Social Impacts

11. This section presents the main concepts underlying poverty and social impact analysis. It addresses seven key areas: (i) What is being analyzed? (ii) What is the welfare measure being assessed? (iii) Who is being analyzed? (iv) How are impacts channeled? (v) How do institutions

5

affect outcomes? (vi) When do impacts materialize? (vii) What are the risks of an unexpected outcome? Impact of what: what is being analyzed? 12. Poverty and Social Impact Analysis is focused on the impact of policy change. To a large extent, developing countries have gone beyond the first generation of adjustment reforms common in the 1980s, namely with a primary focus on macroeconomic stabilization and adjustment and “getting prices right”. In most low income countries economic stabilization, including exchange rate adjustment and large scale cuts in inflation, have been achieved. Consequently, the policy debate in many countries is now increasingly focused on specific structural and public expenditure reforms. A survey undertaken of national poverty reduction strategies 5 , albeit a small sample, shows that public actions that constitute such poverty reduction strategies most commonly focus on enhanced expenditure programs (especially in health, education, water and sanitation, and roads and infrastructure); institutional reforms to improve governance (e.g. decentralization, civil service reform, tax reform); and structural reforms (e.g. trade reform, privatization, financial sector reform, and agriculture sector reform). 6 Very few strategies to date advocate major macroeconomic policy changes other than adjustments in the fiscal target. In light of this reality, tools for PSIA need to be able to address not just major macroeconomic reforms, but also the key structural and sectoral policy changes with which countries are currently contending7 . 13. This shift from broad-based “stabilization and adjustment” suggests that PSIA be undertaken on a reform-specific basis. Such an approach also makes the task of analyzing the impact of several reforms more tractable. While conceptually preferable, few tools are able to assess the combined effect of a series of policy changes in a single analytical framework – and these tend to be complex and data intensive. Therefore, it is often more practical to disaggregate expected overall impacts to individual reforms, and consider sequencing on a reform-specific basis. Consideration of the impacts of a “package” of reforms is still pertinent, however. Where they cannot be analyzed in a single analytical framework, their combined effects on various groups such as the poor may be most practically considered by independently assessing the impact of each reform set on each group. However, such an approach will tend to lose interaction effects.

5

The sample consists of 9 full PRSPs completed by end March 2002. They include the PRSPs for Uganda, Burkina Faso, Tanzania, Mauritania, Bolivia, Mozambique, Honduras, Nicaragua, and Niger. 6 For example, 44% of the 9 full PRSPs to date call for changes in tariff rates; 44% plan for the privatization of utilities; 33% plan for pension reforms of the public sector; 44% call for civil service reform; 89% have fiscal decentralization on the agenda; 56% call for reform of agricultural legislation; 33% plan on raising VAT and other consumption taxes; 22% plan on liberalizing interests rates; and 44% call for changing fiscal deficit targets (vis -àvis the previous IMF program or PFP). 7 Of course, structural changes could have macroeconomic effects. For instance, trade liberalization could have serious consequences for the fiscal deficit, the current account deficit and macroeconomic stability. Understanding how these impacts affect the poor is critical to PSIA.

6

Impact on what: what is the welfare measure? 14. PSIA is focused on assessing distributional impacts on welfare, or well-being. In recent years, increased attention has been given to the non- income as well as income dimension of welfare. Poverty is now recognized as being multi-dimensional (World Bank, 2000a), and development efforts are now being targeted to address both income and non- income measures of welfare and poverty, as recently captured in part by the Millennium Development Goals. It is important, therefore, that the analysis of poverty and social impacts of policy cover both income and non- income dimensions of welfare. To date, the income dimension of welfare has been the typical focus of poverty and distributional analysis, and economic tools ha ve been most often applied in analyzing the money- metric welfare measure 8 . Non- income dimensions of welfare and poverty – such as human development indicators and indicators of risk, vulnerability, and social capital – are now being given closer consideration. In undertaking poverty and social impact analysis, the analyst will need to choose appropriate indicators of welfare and poverty based on country and policy context. Impact on whom: who’s welfare is being analyzed? 15. PSIA is concerned with the distributional impacts of policy change on various social groups, with a particular focus on the welfare impacts among the poor and those vulnerable to impoverishment. Depending on country circumstance, social groups may be defined in terms of income classes, gender, ethnicity, age, geographic location, livelihoods, etc. 16. PSIA is concerned about distributional impacts for two reasons. First, as already discussed, the welfare impact of policy on target groups such as the poor, or women, is the ultimate objective of policy change. Understanding the impacts of policy change on these groups thus motivates the policy change and the analysis of impacts. Second, understanding the distributional impacts of policy, even among non-target groups is important for the effectiveness of a policy and its ultimate sustainability. Policy changes — even if welfare improving — almost invariably result in losers, at least in the short-run. While losers may not necessarily be poor, reduction in their welfare may not be acceptable for social welfare or political economy reasons and may significantly affect the implementation and sustainability of reform. For instance, trade liberalization can be derailed by business and union interests who fear the impact of competition on protected labor and commodity markets. Similarly, reforms can also be derailed by interests within the public sector. PSIA thus ought to identify and analyze the impact of policy on other stakeholders, beyond the poor, who are affected by or can influence reforms.

8

This draft Guide lays out existing economic and social tools and approaches for poverty and distributional analysis in order to give a broader picture of poverty to policy analysts and decision makers. Insofar as the economic tools draw on existing examples of such analysis, applications focus mainly on income/expenditure measures of welfare. Increased attention to assessing the impacts of policy on non-income measures of welfare is an important priority for future work. The social development tools described in this draft User’s Guide are more focused on non-income dimensions of poverty, such as social capital and vulnerability.

7

Impact how? How are impacts channeled? 17. Policy reform can be expected to have an impact on poor households and other stakeholders through five main transmission channels: employment and wages (labor market); prices (markets for goods and services); access (all markets); assets; and transfers and taxes. 18. Employment. The principal source of income for most households is through employment. To the extent that a policy change affects the structure of the labor market or the demand for labor, particularly in sectors which employ the poor (e.g. unskilled labor; off- farm and agricultural rural labor), the welfare of low income households will be affected. There may be direct transmissions through this channel in the case of certain policies (e.g. retrenchment of workers in restructuring of a state enterprise) or indirect transmissions in the case of other policies (e.g. macro policies leading to faster growth may lead to increased employment among the poor; an exchange rate depreciation or trade liberalization could lead to contractions and layoffs in the non-tradeable sector). Alternatively some policies will have different impacts on formal labor markets and informal labor markets which employ many of the poor. For example, expenditure reduction and expenditure switching may increase formal sector employment at the expense of the informal sector employment due to labor market segmentation (Agenor and Aizenman 1999). 19. Prices – production, consumption, and wages. Prices determine real household income. Prices in the markets for goods and services differentially affect real income of households to the extent they are consumers or producers of these products. How policy affects prices will have an important bearing on income and (directly or indirectly) on non- income measures of welfare. For all households, but especially for small farmers and the self-employed price changes will affect both consumption and resource allocation decisions. On the consumption side, policies that cause an increase in the prices of goods consumed by the poor will have a direct negative effect on household welfare. These can include import tariffs on traded staples, or increased utility tariff rates. Consumer prices may be indirectly affected, as well, e.g. through expansionary monetary policy that leads to general price inflation. Producers will also be affected by policies that cause relative price changes – particularly changes to the price of their outputs or their inputs. Producer incomes are further affected by the difference between farmgate and market prices, often conditioned by transport costs and the degree to which private markets are efficient and competitive, rather than monopsonistic. Wage changes will affect net buyers and sellers of labor differently; and policies that change relative prices will induce shifts in both demand and supply. 20. Access. Well-being will be affected by the access of households to goods and services, be it through physical or effective access to markets, or through access to publicly or privately provided services. Policy can affect access directly by enhancing the provision of infrastructure or services in question, or indirectly by removing constraints to particular households’ or groups’ access. For example, improved road infrastructure could dramatically enhance the access of groups in certain geographic areas to markets. A policy that expands connections to an electricity grid, particularly among the poor, can also represent a welfare gain. 9 In this regard, 9

To the extent that increasing access is viewed as a reduction in transport and transaction costs it is effectively reducing the “price” of the good or service in question.

8

privatization of service provision could either increase or decrease access relative to public sector provision10 . Policies that address issues of social exclusion are also relevant here. 21. Assets. Changes in the value of households’ assets will affect income and non- income dimensions of welfare. Changes in asset values can be due to changes in their levels or their returns. Assets themselves can be categorized into five classes, all of relevance to poor households: physical (e.g. housing); natural (e.g. land, water), human (e.g. education, skills); financial (e.g. savings account); and social (e.g. membership in social networks that increase access to information or resources). Policy changes can have a direct or indirect impact on these assets and their returns. For example, land reform may directly result in an increase or decrease in land assets to the poor. Policy changes may also impact assets through indirect channels. For example, inflationary policies will have a negative wealth effect on those with monetary savings; participatory budgeting or community programs may increase social capital; pricing or trade changes could affect the natural resource assets of households or groups (e.g. increasing/decreasing deforestation or desertification) or even their human capital (e.g. by causing a deterioration in health conditions due to increased indoor air pollution as a result of energy price changes). In many cases, certain assets may also be prerequisites to benefit from a reform. For example, if a farme r cannot reach a market due to lack of physical assets, the benefit of price liberalization may be internalized by middlemen and traders. 22. Transfers and taxes. Household welfare, finally, is affected by the extent of transfers to and from the household. These transfers can take the form of private flows (e.g. gifts and remittances) or that of public flows (e.g. subsidies and taxes). Public finance has a direct impact on welfare of specific groups through transfers and tax policy. Public expenditure programs may directly focus on granting additional resources to particular groups through transfer policies. These may be in the form of direct targeted income transfer programs, or subsidies. Social protection programs may be useful in protecting the poor against risk and vulnerability, (depending on their targeting). Tax policy has direct distributional effects to the extent that the resources or income of a household are taxed. Regressive tax regimes disproportionately burden less well off households. Subsidies may be captured by the non-poor or may simply be badly targeted. There may also be a conflict between strict progressivity and the political feasibility of policies (see Gelbach and Pritchett, 2000). Poorer households may also be hurt in the lo ng run, if the funds for public expenditure are borrowed and must be repaid; they will suffer either from any attempt to “inflate away” the debt or from increased future taxes needed for repayment. Impact how: how do institutions affect outcomes? 23. The impacts of policy reform on economic agents are mediated through institutions. Institutions are the formal and informal rules of the game in society. They are the shared

10

Sometimes an increase in access may come at the cost of a higher price, or at a “high” price whereas previously there was no access at all. In urban Peru, liberalization of telephone services led to greater access for the poor as well as lower prices. On the other hand, liberalization of electricity has led to greater access and reliability, but higher prices and lower overall consumption (Torero and Pascó-Font, 2001).

9

understandings that allow organizational entities to interact. 11 Policy reform is affected by public, private and civil society institutions whose rules mediate economic activity in a society (Rutherford, 1994). These include markets, legal systems and the formal rules and informal behavior among implementing agencies (including governme nt). Policy reform can affect institutions by changing organizational structures, roles and responsibilities, or rules and incentives, as well as by altering market incentives, e.g. by removing price distortions or encouraging competition. These in turn affect the behavior of economic agents and interest groups, and thereby economic outcomes, including distribution and poverty reduction. 24. Many reforms depend for their implementation on institutional change. This may involve creating new organizations or changing rules and incentives to achieve new objectives in existing organizations (e.g., improved cooperation among government agencies). Creation or modification of organizational structures does not per se guarantee the institutional changes necessary for the reform to succeed. 12 Changes in the formal rules of the game often need changes in the incentives in order to alter the behavior of agents. Moreover, it is often assumed that institutions (including markets) function smoothly and according to formal rules. In practice, transaction costs, ineffective enforcement, or lack of competition or accountability can lead to sub-optimal performance of government, market or civil institutions. Perverse outcomes can also arise when institutional change accompanying policy reform is not internalized by key implementing agents. 25. Understanding the distributional impact of policy requires an appreciation of the organizational structures and the institutional rules governing them on which the poor and other stakeho lders rely. Careful organizational and institutional analysis – including explicitly setting out critical assumptions about institutional rules and the behavior of key stakeholders that affect reform outcomes – is, therefore, central to PSIA. Impact when: when do impacts materialize? 26. A major challenge to PSIA is understanding that policy impacts can have substantially different impacts on different groups over time. This is in large part because the economic and behavioral responses to a policy change take time. What is fixed in the short-term may be variable in the longer term. For instance, an exchange rate depreciation may lead to employment losses in the non-tradeable sector in the short-term. Increased efficiency may well result in net growth and expansion of the economy. Some workers may eventually find jobs in the expanding tradeable sector; others may fall back on the informal sector. Moreover, to the extent that consumers switch to cheaper non-traded goods, consumption effects would be mitigated. With adjustment leading to relative price and income changes, the combined effect will determine the net impact on different groups over the longer term. The speed with which the economy adjusts, 11

Organizations are purposive entities (e.g., public agencies or firms) which have a formal structure and seek to achieve certain objectives within the opportunities and constraints afforded by the institutional framework of society (North, 1990). 12 Formal changes in organizational structure are relatively easy to make but may take much longer to be institutionalized. In such cases, it is important to pay attention to the capacity and accountability of the concerned agencies as well as the power relations within them. Understanding these issues allows for the mobilization of existing capacity and for the tailoring of interventions to the institutional and organizational contexts in which they will be implemented.

10

however, will depend on many things, including the options for production and consumption substitution. 27. A mirror example is also useful to illustrate the importance of understanding ex ante the potential for different short and long-run impacts. Maintaining an unsustainably appreciated exchange rate and consequent fiscal stance can be extremely beneficial to some groups in the population, including the poor, over the short-term. However these unsustainable policies, often reflected in a ballooning public debt, may result in massive default and protracted economic crisis, possibly at a great cost – and perhaps particularly to the poor – over the longer term. Another example is that of targeted social assistance programs. While possibly an effective means of support to the poor in the short term, they can prove fiscally unsustainable and thus harm the poor and other beneficiaries in the longer-run if they lead to fiscal collapse. Understanding and explaining how short-run losses (gains) may result in longer-term gains (losses) for given groups is one of the challenges inherent to PSIA. Impact if: what are the risks of an unexpected outcome? 28. A crucial element of PSIA is understanding and (publicly) articulating ex ante the key assumptions for the success of the policy reform. 13 Assumptions need to be made explicit as to how economic agents and institutions are expected to act (e.g. the sign and magnitude of an elasticity) and how policy impacts would then transmit to households. A second set of assumptions concerns conditions exogenous to the policy that need to be in place for the reform to achieve its intended impacts. Clearly identifying and articulating critical assumptions will serve to sharpen the rigor of the analysis; increase its transparency; facilitate its validation (and if necessary, correction) by knowledgeable stakeholders; and permit the monitoring and hence improved understanding of transmission channels and impacts with possible adjustments to the reform program over time.

V. Elements for Good Poverty and Social Impact Analysis (PSIA) 29. Although there is no template methodology to analyzing the poverty and social impacts of policy, there are several elements that make for good PSIA. Many practitioners, in attempting to undertake or advise on the analysis of the poverty and social impacts of policy, will be confronted with whether and how to address each of these elements. This section outlines these different elements, providing a road- map on what to consider in undertaking good practice PSIA. These are: (i) asking the right question; (ii) identifying stakeholders; (iii) understanding transmission channels; (iv) assessing institutions; (v) gathering data and information; (vi) analyzing impacts; (vii) contemplating design and compensatory schemes; (viii) assessing risks; (ix) setting- up monitoring and evaluation systems; and (x) fostering policy debate and feeding back for policy adjustment. While there is a logical sequence to addressing these elements, this does not imply that they need to be undertaken in a strict chronological order or that all the steps 13

Forecasting or simulating likely impacts of policy by definition pre-supposes a view of likely causality and behavior. Depending on the analyst’s information base these can be empirically “estimated” based on the past, derived on the basis of theory, or assessed on the basis of knowledge of the country context and discussions with key stakeholders and experts.

11

will be feasible in all country circumstances. This section, moreover, provides a broad overview of specific tools that can be used to address each of these elements, pointing to the Annex for more details and references on covered tools. (The convention used in this paper is that specific methods or tools discussed in further detail in the annex are presented in bold in the text.) The elements presented in this section and their different components represent a menu of tools to choose from depending on country circumstance rather than any minimum requirement or standard for PSIA. Building capacity has been presented in Section III as an overarching ‘principle for operationalizing PSIA’ rather than as a discrete element of PSIA below. It is important that PSIA be undertaken in a fashion that strengthens capacity at every step. 1. Asking the Right Question 30. The first step in analyzing policy impacts is to identify and prioritize development objectives. While poverty reduction may be an overarching objective, poor countries may well have multiple priorities in reaching that objective, and measures to address all of them are not necessarily compatible. For example, a country’s priorities may include fiscal discipline, full employment and greater efficiency of firms. Tightening budgets may improve macroeconomic stability while slowing growth and reducing employment in the short term. While inflation hurts the poor, so too does joblessness. In order to undertake country-level PSIA, it is important to identify the priority reforms that merit further analysis. This selection process is a matter of judgment at the country- level, and will depend on factors such as the likely size and uncertainty of the poverty and social impacts (e.g. with regard to the number of people affected and the magnitude of impact); the degree of controversy of the reform issue; and the time sensitivity of the reform. 31. Policy reforms are often implemented to remove constraints which stand in the way of achieving certain development goals. For instance, a country may be unable to balance its budget because of unsustainable losses by state owned enterprises. The problem in this case will be to improve the overall fiscal balance as well as the performance of individual agencies. A problem diagnosis helps trace a chain of cause-effect relationships from policy objectives, to constraints, to choices, to impacts. For some objectives, there may be multiple constraints, some being more important than others. In such cases, it may be necessary to pursue more than one policy reform, but also to be on the alert for interactive effects that those reforms might have on each other. 32. Identifying policy constraints is the first stage in the analytical process and can often prevent subsequent missteps. For example, a policy- maker faced with inadequate public revenues may decide to raise taxes. However, this will not be the appropriate response if the real problem is that expenditures are too high, rather than that revenues are too low. In order to avoid inappropriate or mismatched policies, it is important that the constraints on development objectives be made explicit – rather than assumed – at the beginning of the PSIA process. 2. Identifying Stakeholders 33. Upon asking the right question and identifying the problem that requires solution, an early identification of relevant stakeholders is important. Policy choices can affect diverse

12

stakeholders or economic agents in different ways. Furthermore, these stakeholders can also influence whether policy is adopted and how it is implemented. 34. Stakeholder analysis identifies people, groups and organizations that are important to take into account when conducting PSIA. 14 It addresses those people who are affected by policy, as well as those who affect policy. Identifying and disaggregating the first type -- intended beneficiaries and those who suffer adverse impacts - is central to the analysis of poverty and social impact of policy. For modeling work, stakeholder analysis can serve as an input into determining how best to disaggregate representative household groups. The second type -organized groups such as unions, business associations or NGOs -- may become sources of support or opposition to policies. Box 1 illustrates the use of stakeholder analysis to address the impact of mine closures in Russia. Box 1: Analyzing the impact of mine closure in Russia: Stakeholder analysis Ghani1 carried out a stakeholder analysis using structured interviews in Moscow, mine visits and discussions with union leaders allowed the team to group the stakeholders into several camps. Stakeholder analysis was designed to: clarify the nature of the problem; identify the interests of various actors; and develop a solution for effective fund transfer based on existing actors. Government ministries were not seen as neutral agents, and their interests were explicitly identified. Similarly, the options of mine employees were differentiated by their previous employment. Workers on the mine face, analytic and administrative support workers, and workers in the schools and hospitals previously funded by mine revenue would be impacted differently as mines closed. The interests of municipal and oblast-level governments were based partially on the revenues that each could muster in the event of mine closure. The difference between stakeholder groups lay largely in their analysis of the core problem. • • •

The Ministry of Energy, regional governments, the labor union and the mine face workers advocated a narrow solution focused on preserving the mining industry in some form. Municipal governments, social service workers employed by the mines and local businesses focused on the need to find new drivers of growth in mono-industry towns as well as sources of funding for services previously supplied by the mines. Municipal governments did not have the revenue base to support the schools and other services formerly provided by the mines, and were hard -hit by changes.

As a result of the analysis, an Interagency Coal Commission was created with representatives from municipalities, ministries and government agencies that helped discuss and plan reforms. The Ministry of the Treasury was identified as a transparent way to get social protection funds directly to the workers rather than physically moving it from the MoE through regional governments. Last, stakeholder interests were used to create a system of checks, balances and independent assessments to ensure that all actors followed the rules laid out in mine closure plans. 1

As described in Lockhart and Ghani, 2001.

35. A distinction should be made between stakeholders that readily identify themselves as a cohesive group (e.g., unions) versus analytical categories that do not (e.g. “the fourth income quintile,” or even “the poor”). Stakeholder analysis can also describe the stated or unstated interests of actors vis-à-vis the policy, as well as the nature and degree of their organization or 14

To the extent that stakeholder analysis helps focus subsequent research on specific sets of actors, it increases the relevance of more complex analysis of poverty and social impacts while reducing time and cost.

13

ability to mobilize behind a common purpose, (see Box 2). To the extent that groups of the first type are atomized or unorganized (e,g., landless peasants, non- unionized workers, small businesses), they are less likely to play a significant role in terms of support or opposition to a policy. 15 While secondary resources (e.g. social science research, news media, and advocacy literature) can help identify broad political economy issues and social tensions, key informant interviews may be critical to analyze the interests of stakeholders whose support is critical to reform implementation, especially those within government agencies, or interest groups able to influence reform. Box 2: Collective Action and Political Pressure Estimating the influence of a particular group over decisions is as much art as science. However, there are some useful criteria for predicting the propensity of a group to lobby the government. The logic of collective action suggests that interes ts will exert more pressure on policy-makers or elected leaders when: (1) the number of group members is small; (2) the benefits or rents that accrue to each member are easy to perceive; and (3) the benefits or rents that accrue to each member are significant for each member. The behavioral premise is simple: people fight harder when they know a lot is at stake. This explains why the interests of unorganized groups such as consumers are typically not influential. Many development interventions are designed to reduce or eliminate rents among a small group of privileged interests and increase the overall welfare of the public. However, these are precisely the policies that are most likely to be fought, making either tough political decisions or a concerted communications strategy paramount.

36. Stakeholder analysis does more than describe the main actors. It contributes to an assessment of the extent of ownership to understand how different interests are likely to influence government in general, and the policy process in particular. Ownership assessment helps reveal sources of potential resistance to or distortion of policy choices. It examines government’s willingness to undertake and stick with reform over time. Weak ownership can lead governments to abandon reforms, mid-term or produce distorted policies. For example, some countries pursue bank deregulation and privatization, but refuse to remove barriers to entry, resulting in an oligopolistic sector that charges high interest rates and provides poor service. 37. The basic output of an ownership assessment is an estimate of the location and extent of pressure that government will experience in adopting a policy reform. Factors which most typically affect ownership can be analyzed by looking at the political economy and social diversity (e.g.. ethnic, religious, linguistic, gender, and age) of a country. The former identifies affected groups, and assesses their influence over government decision- makers. The latter recognizes that reforms may in the short term polarize existing social tensions based on ethnic, religious or other lines, even while raising growth in the longer run.

3. Understanding Transmission Channe ls

15

The identification process disaggregates these actors in terms of social characteristics – such as cultural, structural, economic, political, or governmental.

14

38. Having identified potential stakeholders, laying out a priori the channels by which the analyst expects a particular policy change to impact various stakeholder groups is an important early step to the PSIA process. 16 The transmission channels that are going to dominate will be different, depending on the reform in question. Also, the importance and role of indirect “feedback” effects (perhaps through secondary transmission channels) might be particularly important in the case of some reforms. Also, given that feedback effects may be time differentiated, so will net impacts on various stakeholders. An important example is that of policies aimed at improving the efficiency of the economy, which by virtue of changing the status quo may have certain dis tributional effects in the short-term, but, by aiming to foster increases in productivity and growth may be expected to have a positive net welfare improvement for most people over the longer term. 39. The expected impacts of a policy change on the welfare of target groups and other key stakeholders manifests itself through various transmission channels and over time. It is important to make explicit hypotheses and assumptions that can then be further analyzed empirically and/or validated through interviews, focus group techniques, and other qualitative techniques. It is also important that the analyst consider the impact of policy on the poor and other groups through the five transmission channels introduced above. • • • • •

Employment Prices – production, consumption, and wages Access to goods and services Assets Transfers and taxes

4. Assessing Institutions 40. As discussed above, institutions affect the impact that policy has on poverty and welfare among different groups. First, institutions are important in that they mediate the transmission of certain policy impacts to households or groups. Important in this regard are markets as institutions. Understanding context- specific market structure, therefore, will be critical to understanding how a given policy change (such as deregulation, privatization, or removal of an export tax) will affect impacts. Second, institutions are often the object of many types of policy reform. Indeed privatization, civil service reform, decentralization, and expenditure management reform are examples of institutional reform which change the incentives and rules that govern public and private organizations. Third, many types of policies or policy changes call for a central (or in some instances an accompanying) role of particular organizations in their implementation. The incentives, performance, and capacity of these organizations will also be critical to the actual implementation of policy and thus its impact. Two key areas of focus when assessing institutions are (i) the analysis of market structure and (ii) the analysis of institutions. 41. Analysis of market structure. Surveys among consumers and producers of goods and services can be useful approaches to enhancing understanding of context specific market 16

Doing this early on provides basis for early validation of hypotheses, subsequent identification of data and information needs, and more rigorous analysis of hypotheses in subsequent steps of PSIA.

15

structure. Identifying the nature of the market (e.g. monopoly, monopsony, oligopoly, perfectly competitive, etc.) and what determines this market structure (e.g. natural monopoly, restrictions to entry, collusion, etc.) is a crucial first step to understand the enabling conditions that would need to be created for market reform to lead to improvements in performance and better outcomes for the poor. 42. Enterprise (or trader) surveys can be useful for understanding the nature of the market, the number and types of economic agents, and market constraints as well as de jure and de facto barriers to entry and transactions costs. In the case of privatization or liberalization, where an assumption is that market entry will lead to competition and price reduction, it might also be useful to concurrently undertake an analysis of the constraints to private sector entry and participation. Quantitative or qualitative household surveys can also get at who and where services are bought, and at what price. Quantitative Service Delivery Surveys (QSDS) and Citizens’ Report Cards can be applied to the analysis of the effectiveness of state marketing agencies. Price analysis is always useful in ascertaining the competitiveness of a market and of market structure. 43. Analysis of institutions. It is also important to understand the functioning of institutions that matter for the effective implementation of a policy. In judging the likely poverty impacts of reforms that involve a change in government responsibility, or cooperation among government agencies, the flow of decision-making, information and resources within and among public sector organizations needs to be considered (See Box 3). Box 3 Decentralization in Indonesia: Institutional analysis and social accountability Guggenheim, et al. (2000) 2 carried out an institutional analysis of governmental structures at the village level and traditional village decision-making bodies in Indonesia as part of a decentralization program designed to address the issues of corruption and top-down decision making. The Kecamatan 1 Development Program was committed to using local capacity rather than developing a separate Project Implementation Unit. The institutional analysis addressed: the relative strength and capacity of existing systems, the flow of money and information, and the location of decision-making in the chain. Institutional analysis, conducted through focus groups and interviews with government officials and villagers throughout Indonesia, helped the project to identify existing structures and understand how they made decisions. The project changed the role and authority of those structures, changing the locus of power within the system. Through the interview process, the team identified Village Infrastructure Project (VIP) as a field-tested means to get money directly from central accounts to the village level. An existing government agency, the Department of Community Development (BPM), acted as a partner and enforcing agency. The KDP used transparency and social accountability to make the new institutional structure work. Existing kecamatan-level village councils, which were formal organizations that had met once a year to feed into the government’s planning process, became the primary decision-making body. Decisions on proposals from villages were made in public meetings of the council, procurement forms were limited to one page, expenditure information was kept on cash ledgers and information about the program was disseminated through posters, flyers and radio broadcasts. Further, the KDP worked with the Association of Independent Journalists to ensure media coverage and gave small grants to reporters to build capacity for independent reporting. 1 2

Kecamatan is the sub-district level in Indonesia National Management Consultants. Second Annual Report of the Kecamatan Development Program. Sep 2000

44. Two options for collecting this kind of information are organizational mapping and the institutional assessment tool:

16

•

Organizational mapping is a method enhances understanding of the internal behavior of organizations by creating an inventory of the actors carrying out reforms and explicitly revealing relationship among them. Organizational mapping has two components: (i) static mapping and (ii) process mapping. Static mapping identifies ex ante the specific public actions associated with a policy reform, and the organizations (which may be outside government) responsible for implementing these. It maps out the relations among the implementing agencies and identifies those expected to support or obstruct the reform. The exercise is informed by earlier stakeholder analysis (see Section V.2) of government and other organized actors. Process mapping, which draws on work carried out to improve efficiency in the public and private sector in industrialized countries (Hunt, 1996), identifies current practices and norms in relevant organizations that cannot be easily gleaned from documents or diagrams. It does so by tracing flows of critical resources, decision- making authority and information in the current system. This helps create an understanding of the rules and incentives that affect internal behavior and the extent to which organizations pursue development objectives. Process mapping can help identify constraints to effective policy implementation at three levels: in organizational procedures, the relationship between organizations, the relationship with the authorizing environment (e.g. Ministry of Finance). Addressing them may require fine tuning procedures, recasting fundamental rules of operation, or even replacing entire organizations. Process maps are constructed through in-depth, semi-structured interviews with staff at all levels of the organization, focusing particularly on those at the ‘front- line’ of delivering services. The main advantage of organizational mapping is its ability to “drill down” into a problem area that may not be readily visible by relying directly on stakeholders to describe their interests and constraints, (see Box 3). A drawback is that it is more time-consuming, costly and technically demanding than guided questionnaires. Good process mapping needs to be used iteratively to test assumptions by monitoring institutional performance over time.

•

The institutional assessment tool was designed to permit an institutional analysis of various components of a project. The tool consists of questions that help the analyst structure thinking about the complex relationships and processes within organizations upon which reforms depend. 17 The questions are used to evaluate the effectiveness of institutions, from performance incentives to their capacity to implement policy. They address key issues of relevant organizations, including: (i) roles; (ii) knowledge and access to information; (iii) incentive structures; (iv) receptivity to policy change; (v) capacity; (vi) resources or financial clout; (vii) scope to adapt to the new reform agenda. The advantage of the institutional assessment tool is that it can enable more systematic analysis of issues ranging from political incentives to administrative capacity at low cost. The disadvantage is that the tool relies on a desk assessment, and lacks the interactive dimension of interviews with staff of the organizations that are being reformed. The tool is currently better suited for the analysis of institutions with respect to investment

17

Some questions address issues of ownership and commitment discussed in the previous section. In situations in which informant interviews are not feasible or where findings are not considered reliable, the institutional assessment tool can be used to conduct or complement stakeholder analysis.

17

operations but could be adopted to assess institutions in the context of the implementation of policy reform. 18 5. Gathering Data and Information 45. Assessing current data to determine the type of analysis and planning the phasing of future data collection efforts are an important part of PSIA. Identifying data needs will benefit from the prior identification of policy issues, stakeholders, and likely transmission channels, outlined above. Four discrete steps are suggested: mapping out desirable data for PSIA; taking stock of available data and analysis; coping with PSIA data limitations ex-ante; and addressing PSIA data limitations today so they are not limitations for PSIA in the future. A. Mapping desirable data for PSIA 46. The analysis of the poverty and social impacts of policy can be extremely data- intensive. Specific data requirement s will, of course, depend on the nature of the reform issue being analyzed and the analytical model being employed. In approaching data and methods, it is useful to distinguish among data collection instruments (close-ended or open-ended); data type (numeric or non-numeric); and associated methods of data analysis (quantitative or qualitative). Traditionally analytical approaches have been either quantitative in nature, based on numeric data collected using close-ended data collection methods; or been qua litative in nature, based on non-numeric data collected using open-ended data collection methods. In addition, it is increasingly being recognized that “mixed methods” can be extremely useful. •

Quantitative analysis; numeric data; close-ended data collection. Analyzing the poverty and distributional impacts of policy on welfare indicators will require linking data at the macro or sectoral level (generally corresponding to the level of policy intervention) to disaggregated household level data which captures the welfare measure of interest (most generally an income/expenditure aggregate, but possibly other welfare measures such as literacy or infant mortality) and other behavioral variables (such as access). Close-ended surveys have generally been used to collect such data. For analysis to be generalizable, data ought to be derived from a random sample. Sometimes, purposive sampling may make sense (for example when the interest is in collecting information from laid-off mine workers). Numeric data can be used to undertake statistical and multivariate analysis to test hypotheses and determine relationships. (See Table 1).

•

Qualitative analysis; non-numeric data; open-ended data collection. A variety of openended data collection methods can also be used to collect non-numeric data relevant to the analysis of the poverty and social impacts of policy. Non-numeric and contextual data can be collected through participatory appraisals, asset mapping, structured interviewing of individuals, communities, or focus groups. These data can be used to undertake stakeholder analysis (discussed above), participatory poverty assessment,

18

A dedicated website for the electronic tool is being prepared by the Public Sector Group and should be available during the spring.

18

beneficiary assessment, institutional analysis, and risk analysis (discussed below). Openended data collection methods such as those described in Table 1 permit an interactive analytical process – one in which research questions can be formulated, answered, and analyzed iteratively in the field. The open-ended approach allows subjects to articulate the research problem and question. This interactive analytical process could enable quicker turn-around and shorter lapse in time between questionnaire design and analysis than close-ended data collection methods and associated statistical analyses19 . Openended data collection methods may also be undertaken using a random sample or a purposive sample. 20 Aspect Data collection instrument

Analytic method

Table 1: Data Collection Methods Close-ended Open-ended • Structured, formal, pre• In-depth, open-ended or semi -structured interviews, designed questionnaires, such such as key informant interviews and case histories, as LSMS, social impact focus group interviews, community interviews, miniassessment (SIA) survey, surveys client satisfaction survey, or • Ethnographic observation citizen report card. • Systematic (or directed) consultation, such as beneficiary assessment • Participatory data collection methods, such as participatory action research , participatory rural appraisal , participatory public expenditure review • Focus group discussion • Community and institutional surveys • Written documents (for example, program records, process documentation, media reports)

• Predominantly statistical analysis • Deductive reasoning.

Advantages

Disadvantages

• Inductive reasoning. • Interactive analytical process: research questions formulated, answered, and analyzed iteratively, e.g. in stakeholder analysis, participatory poverty assessment, scenario analysis.

• Findings can be generalized;

• Ability to analyze behavioral responses, explore new

can quantitatively estimate size and distribution of impacts

hypothesis or recognize previously undiscovered phenomena. Allows subjects to articulate their own views. Can be rapid.

• Results not available for long

• Findings difficult to generalize, and difficult to

period of time; limited types of information can be gathered; expensive and time-consuming

aggregate and make systematic comparisons. Needs further validation. Fieldwork requires greater research skills than for quantitative enumeration.

Source: Adapted from Carvalho and White, 1997; Baker, 2000; and World Bank, 2002a. Note: this table is intended to provide an indicative distinction between these methods and not a comprehensive description of individual techniques.

•

Mixed methods. In undertaking poverty and social impact analysis there is much benefit to mixing and, where possible, matching among the above approaches. Much is to be

19

This statement requires qualifiers. First, faster methods of close-ended data collection and analysis are being developed (e.g. CWIQ). Second, reliable open-ended analysis requires time and care if quality is to be ensured. 20 These data collection instruments have often been employed using non-random samples, for example in ethnographic analysis. However there is no reason that they cannot be used on random samples to generate statistically representative data. Likewise, non-numeric data could be coded into numeric data.

19

gained by not restraining data collection and analytical methods along the silos outlined above. 21 Close-ended and/or open-ended data collection techniques can be used to generate numeric and/or non-numeric data, for analysis using quantitative and/or qualitative analytical techniques and approaches. Moreover, analytical methods can be mixed sequentially (e.g. with qualitative analysis informing the design of a close-ended data collection instrument or the specification of an econometric model) or in parallel over time. Mixed methods can leverage the benefits of quantitative and qualitative analysis: qualitative social analysis can inform the design of close-ended questionnaires and generate hypotheses to be tested further through quantitative research; hypotheses generated by qualitative analyses can be tested for generalizability by quantitative approaches; quantitative results can be examined using open-ended data collection methods to develop a richer understanding of the impacts of policy on different subsets of the population; a successful mixture can elucidate history, context, process, and identification of transmission channels and differential impacts. While mixed methods can involve higher costs, skills and coordination with multidisciplinary teams, the benefits can also outweigh the costs. B. Taking stock of available data and analysis 47. The first element of the stocktaking is to ascertain the existence of key data. This will allow identification of data gaps that need to be filled or taken into account when choosing an analytical approach. Household survey data is generally pivotal to undertaking quantitative poverty and distributional analysis 22 . An important consideration for poverty and social impact analysis is whether, in addition to a welfare (e.g. income/expenditure) aggregate, there is information in the survey that provides the relevant variable (or the computatio n of such a variable) related to the policy lever in question (e.g. household expenses on transport, or specifically public bus transport, if bus tariffs are to be increased; or purchases of maize at subsidized prices, if the subsidy is to be removed). In analyzing policy reform, it is useful, where possible, to match data from different sources. For example, in Armenia household survey data was matched, through utility account numbers, to administrative billing information for consumers. This permitted an analysis of utility consumption behavior by income group. 48. Second, beyond identifying the availability of relevant primary data, ascertaining the existence of analysis and secondary data on the policy issue at hand is an obvious next step. Project and program documentation, as well as data and analyses from other development agencies are invaluable. Academic research and theses can also yield in depth insights not normally available in official reports. In many instances, burning policy issues have been the attention of analysis and debate in the past; it is useful to draw on whatever analysis already exists; and to whatever public debate has already occurred.

21

Social impact assessment adopts a more eclectic approach to data collection, choosing among open-ended, semi structured and close-ended instruments to fill information gaps for mixed method analysis. 22 Most countries have now undertaken at least one national household survey, although at times the vintage and quality of data is sometimes an issue. Intra-household data, when it ideally exists, can permit distributional analysis at the level of the individual household members, a particular concern in considering the welfare of women or other individuals who may be less powerful or privileged within the household.

20

49. Third, it is useful to ascertain the capacity of local data collection agencies (e.g. national statistical offices, ministries, universities, think-tanks, consulting firms, etc.) to collect data and possibly to analyze them. C. Coping with PSIA data limitations ex-ante 50. In many countries there are severe data limitations to conducting poverty and social impact analysis. Some or many of the desired data outlined above may simply not be available. In this case, policy- makers and analysts approaching the challenge of impact analysis will need to bear in mind several factors, outlined below. •

Adapt analytical approach to data currently available. If the urgency for policy action severely limits the time to gather further data, expeditious analysis focused on using whatever data is available may be required. Some tools and approaches to poverty and social impact analysis are far less demanding on data than are others. Adapting the analytical approach to the available data, such as using time use data or focus group data to construct a simple household model, might be the necessary course of actio n. While any analysis entails making assumptions, taking short-cuts generally means making more assumptions in order to proceed. The analysis should be honest and transparent in stating these assumptions. Qualitative techniques, such as individual, community, or focus group interviews can be used to validate assumptions and inform the design of quantitative surveys.

•

Collect more data. If there are identified and critical data gaps, it may be useful to gather the data – whether it be administrative or survey data. In the interest of building national capacity and enhancing ownership of the data and its analysis, where possible these data collection efforts should be undertaken through national institutions, such as the statistical agency, ministries, or universities. Undertaking a national household survey is a large undertaking; it can take months to plan, implement, and analyze the data from larger surveys. ⇒ Where possible, it is useful to identify planned household surveys that are to be fielded imminently and to add key questions relevant to the policy issue at hand. These questions can leverage a wealth of analytical possibilities in the context of a full- fledged household survey. ⇒ Alternatively, there are now several “off-the-shelf” survey instruments that can be used to quickly collect, enter, and analyze data (e.g. the CWIQ survey). ⇒ Social impact assessment surveys, based on purposive sampling, can often be turnedaround in a shorter time than a representative national household survey. ⇒ Likewise, depending on the reform issue at hand, quantitative surveys can be employed using a purposive sample (e.g. among workers of a firm that is to be downsized). 23 When possible, mixed methods, combining qualitative and quantitative

23

In examining an economy -wide reform, such as a rice tariff increase, it would obviously be preferable to adopt a representative sample for any new survey, or to adopt the same sample (or select a panel) from a household survey for which there is already data. Where the reform is location specific, or affects a specified population – e.g. with the

21

analytical approaches to triangulate results increases the confidence of findings. The use of data from a non-representative sample to estimate parameters may sometimes be required – as may be the “borrowing” of parameters from other countries. Again, clearly stating assumptions (e.g. that these elasticities apply to the population at hand) will be important in these instances. Care should be taken when generalizing from such a purposive sample. 24 •

Rethink the policy decision or the sequencing and pace of reform. One option is to postpone the policy decision until adequate data can be collected and appropriate analysis conducted. If this course is taken, the costs of delaying reform (a policy decision in itself) would need to be considered. Other possibilities are to pilot or phase the reform, so that progress can be monitored before a final decision is taken to implement a national program.

51. In the end, a tactical judgment will have to be made as to how to proceed based on these considerations. This judgment will be influenced by the time and resources at one’s disposal, which in turn will depend critically on political and economic pressure for action. In most instances, decision makers may not want to embark on a major policy change without a sound understanding of the poverty and social implications of policy action, particularly if such action is aimed at reducing poverty. In some instances, however, political or economic imperatives (e.g. in a crisis situation) may lead policy- makers to take quick action. Where this happens, it will be important to undertake PSIA as soon as is feasible and to consider measures to protect the poor from adverse impacts and vulnerability to significant risks (see Section V.8). D. Addressing PSIA data limitations today so that they do not limit future PSIA 52. When circumstances dictate that a policy-decision needs to be taken without adequate data, it is important that steps be put into place to improve the information set over time. Since PSIA is necessarily a dynamic process of formulating and adjusting policy based on increased knowledge, it would also be important to put into place a strategy to gather the necessary data to enhance the basis for further and future (ex ante and ex post) analysis of the poverty and social impacts of policy. Such a strategy can be designed in a manner that builds national capacity for data collection and analysis. Where possible, a strategy for data collection should be linked to the timetable for policy formulation, or for policy review and reformulation. In other words, a strategy for future data collection is not motivated solely on ex-post monitoring and evaluation of current policy decision (see Section V.9), but also to permit future ex-ante analysis. Developing such a strategy is an integral part of the PSIA effort.

shutting down or privatization of a state mining company – a purposive sample of those expected to be directly affected would be appropriate. 24 Using a non-representative sample to extrapolate differentiated impacts of policies among groups nation-wide assumes that national distributional characteristics are identical to those of the non-representative sample, a nontrivial assumption.

22

6. Analyzing Impacts 53. Analysis of the likely poverty and social impacts of policy reforms is central to PSIA. This section begins with general considerations in choosing impact analysis approaches, and then provides an overview of several broad classes of methods for estimating these impacts. These classes are categorized by the extent to which they capture “feedback” effects in the economy resulting from a policy change. Approaches include: direct impact analysis and behavioral analysis (“low” in capturing feedback effects); as well as partial equilibrium analysis; general equilibrium analysis; and micro-distributional analysis linked to macro- models (“high” in capturing feedback effects). Under each class of methods, the discussion presents an overview of more specific tools (referred to in bold text). These are summarized in Appendix 1 and are discussed in greater detail in the Annex (including data requirements and particular advantages and shortcomings). An ongoing collaborative effort between DEC/PREM/WBI is developing more detailed modules for each economic tool. In parallel, the Social Development Department is engaged in a similar effort for its social tools for PSIA. These products will be the basis for the development of an expanded PSIA learning program. A. Considerations in Choosing Impact Analysis Approaches 54. In general, there are five factors that will condition the choice of approach or tool to be used in analyzing the poverty and distributional consequences of a given reform: the type of reform; the importance of feedback effects in determining the final net impact; data availability; time availability; and capacity. For purposes of presenting a simple typology, these five factors can effectively be collapsed into two dimens ions. •

Importance of feedback effects. Depending on the reform in question and the structure of the economy, the impact of a policy change may be direct and short-term or may have a high degree of feedback effect on the economy. “Feedback effect” is used here to indicate multiple-round effects that are transmitted through other markets or with a time lag. Effectively, the type of reform and the importance of feedback effects can be collapsed into one dimension for the simple typology presented here. “High” feedback effects would represent policy reforms, where the net effect is transmitted through several channels and markets, leads to behavioral changes at the household level, and has multiple round effects which will take time to work themselves through the economy. An example could be a massive devaluation which immediately results in changes in relative prices, consumption, and power structures, but over time might be expected to lead to shifts in the structure of employment and the economy, changes in productivity, improved governance (by removing rent seeking) and possibly growth. “Low” feedback effects would represent reforms where impacts are short-term, direct and transmitted via few channels. Small-scale civil service lays-offs might be one example.

•

Data/time/local capacity availability. As discussed in Section V.5, data availability and domestic capacity (for data collection and analysis) will necessarily constrain the type of approach adopted. The simple typology presented here collapses data/time/capacity into a single dimension. Where local capacity is less extensive than in the Bank or in other

23

donor agencies, this may be the binding constraint on the choice of approach. Over time, an objective of PSIA ought to be to improve the capacity of local practitioners and users. Wherever possible, it is important that local partners – in the government or outside organizations, as appropriate – are involved both in selecting tools for analysis and in applying them. This engagement can be the basis for strengthening domestic skills, so that over time a larger share of the analysis is conducted by local people rather than development agencies. 55. Table 2 presents an indicative typology of how an analyst may want to select an approach. It lays out a choice of tools based on the importance of feedback effects (as defined above) for the reform in question, taking into consideration constraints of data, time, and capacity25 . Table 2. Considerations in Choosing Impact Analysis Approaches

Low

Data/Time/Local Capacity Availability Medium

• Social impact assessment (Qual)

• Social impact assessment (Qual and Quant) • Participatory poverty assessments • Benefit incidence analysis • Social capital assessment tool (SOCAT) • Demand/Supply analysis • Household models

•

• Social impact assessment (Qual) • Collect more data • Use tools in adjacent cells in conjunction with assumptions

• •

• Social Accounting Matrices – Input/Output • Computable general equilibrium • Macro-model + microsimulation

Low

Feedback Effects

High

High

Multi-market analysis Reduced form

Poverty mapping

NOTE: The tools presented along the dimension of “Data/Time/Capacity Availability” are additive across rows. That is to say, any tool that can be used in the context of lower data/time/capacity can also be used with increased data/time/capacity. For instance, social impact assessment can be used across the entire row.

25

For the purpose of presenting this simple table, the indicative classifications of “high”, “medium”, and ‘low” are used, whereas clearly this is a continuum in practice. While recognizing that time, data and local capacity are not perfectly correlated, they are deemed a close enough match to collapse into a single dimension. Using data as a proxy for all three factors, the following criteria were used for the classification along the data/time/capacity dimension: low (no nationally representative household survey data); medium (nationally representative household survey data exists); high (need to use nationally representative household survey data in conjunction with other data – e.g. census data for poverty mapping; national accounts and other data for computable general equilibrium models, etc.)

24

56. In contemplating the choice of tools, most analysts will have a particular reform in mind. A helpful first step, therefore, is to consider whether the reform in question is more likely to have low or high feedback effects. The answer will depend partly on the scale of the reform and its importance to the economy, as well as the time horizon. With regard to time horizon, short-run elasticities are lower than in the long-run. So a tax reform may have low feedback effects in the first year of implementation, but much larger impacts in subsequent years as agents adjust to the new tax rates. For instance, the feedback effects of utility reform could be very low – e.g., in the case of changes in tariffs paid only by a handful of rich consumers – or very significant – as with the wholesale restructuring of the electricity sector in an industrialized country. Similarly, the feedback effects of privatization will vary depending on the number and importance of the state owned enterprises that are divested. Moreover, the individual feedback effect of a series of reforms may be low, but taken as a package the combined impact could be high. Nevertheless, while country circumstances will ultimately determine the strength of feedback effects, it is possible to classify in broad terms specific reforms between lower and higher feedbacks, based on the scale on which they are being undertaken in most low income countries. Box 4 provides an indicative breakdown. Appendix 2 sets out in more detail types of tools suitable for each reform, and possible transmissions mechanisms. Box 4: Indicative Categorization of Reforms According to Scale of Feedback Effects This categorization is indicative only: actual feedback effects of a given reform will ultimately be driven by country circumstances, including the scale and complexity of the policy adjustment. A. Higher Feedback Reforms • Macroeconomic and fiscal Reform: Monetary policy reforms, affecting inflation and interest rates; broad external policy, affecting balance of payments and reserves; and broad fiscal policy, affecting fiscal deficits. • Trade and exchange rate reform: Reform of tariff and non-tariff barriers, and exchange rate adjustments. • Agricultural reform: Elimination of administered prices, changes in domestic subsidies and taxes, abolition of marketing boards. • Financial sector reform: Liberalization of interest rates, allocation of credit; lowering barriers to entry; and regulatory reform. B. Lower Feedback reforms • Public finance reform: Changes in allocation and level of public expenditures; changes in level and composition of revenues; improvements in tax administration; cost recovery. • Land reform: Distribution to landless; changes in legal rights to own, exchange, inherit land. • Utility Reform: Restructuring of state owned utilities; increased private participation; full divestiture. • Financial sector reform: Privatization/closure of state banks; promotion of financial institutions serving the poor. • Privatization: Lease of assets; private management contracts; full divestiture. • Labor market reform: Changes in minimum wages/job security regulation; active labor market measures for laid off workers. • Civil service retrenchment: Lay-offs, reductions in the wage-bill. • Decentralization of public services. • Social safety nets: Changes in targeted cash/in-kind transfers, benefits to needy groups (eg AIDs orphans)·and social insurance benefits. • Pensions: Scaling back pay-as-you-go public schemes; increased private provision; introduction of social pensions (cash assistance for poorest pensioners).

25

57. Having determined the relevance of feedback effects, the next consideration will be the availability of data, time, and capacity. Where these are in short supply, simpler approaches may be all that is feasible. In such cases, limitations of the approach should be explicitly stated, and an action plan to strengthen data and capacity should be put in place for more robust analysis in the future. This way countries in the “low” data and capacity situation could aim to improve their information base so they have the option of adopting methods in the “medium” and “high” columns, as appropriate, (see Appendix 1 for data, time and skill requirements for each tool). 58. The rest of this section briefly lays out the different social and economic tools for PSIA, and the reforms to which they are best applied. For ease of exposition, each approach is discussed in turn. Where feasible it is advisable to integrate economic and social analyses in order to deepen the analysis. For instance, social impact assessments can be used to help define the parameters and explanatory variables used in econometric modeling. B. Methods: Reforms with “Low” feedback effects 59. There are two broad approaches to analyzing reforms with low feedback effects. The first – direct impact analysis - is a simple assessment of who is directly affected by the policy change, and by how much. It assumes no behavioral response from affected households or groups i.e., if prices change, quantities do not adjust. Effectively all elasticities are assumed to be zero, including own-price elasticities. This assumption is appropriate for assessing short-term reform impacts, before economic agents have time to make adjustments. It otherwise represents a limitation of the approach. In particular it will tend to overstate the impact on household welfare. The approach can be used to analyze any type of policy change – for example, a change in prices (e.g., commodity price, tariff, wage, or exchange rate) or a change in public finance policy (e.g., expenditure program subsidy, tax, civil service or state owned enterprise retrenchment). But it is best suited to reforms whose impacts dominate in the short term e.g., the removal of a subsidy, a small-scale privatization, a single price change in a relatively isolated market. 60. Three tools fall within this approach: social impact assessment, simple incidence analysis, and poverty mapping. These range in terms of data/time/capacity requirements from “low” to “high” in Table 2, poverty mapping being by far the most demanding. • Social impact assessment (SIA) is used to assess how the costs and benefits of reforms are distributed among different stakeholders and over time. It is particularly useful in understanding how assets (physical, financial), capabilities (human, organizational), economic and social relations (e.g. gender, exclusion) of stakeholders, and the institutional mechanisms through which policy actions are transmitted affect policy outcomes. Stakeholder analysis is a prerequisite for SIA. When reasonable national survey exists, SIA uses a range of qualitative data collection tools (focus groups, semi-structured key informant interviews, ethnographic field research, stakeholder workshops) to determine impacts, stakeholder preferences and priorities, and constraints on implementation. In the absence of adequate quantitative data, SIA supplements qualitative, sociological impact analysis with purposive surveys that capture direct impacts and behavioral responses to reform, or specific dimensions (e.g. time-use patterns) that affect reform outcomes; (the “low- low” cell in Table

26

2). SIA is useful to examine the impacts of structural reforms such as privatization of state owned enterprises, agricultural reform, reform of basic services, utility reform, civil service reform and fiscal policy. It is particularly relevant for understanding the quality of impact on different social groups, and examining how the poor cope with reforms and access market opportunities. Given the overlap of research methods, SIA is more cost effective when undertaken simultaneously with institutional analysis and social risk assessment. • Simple incidence analysis estimates the distributional incidence of a component of income or expenditure at the household level. The analysis is an appropriate starting point where quantitative data are available; (the “low- medium” cell in Table 2). A useful first step is to examine key descriptive statistics for the country to see which households are “exposed” to the policy change. The most common application is to tax the expenditure reform, and the technique has been used for instance to estimate the incidence of education expenditure in Malawi. It can also be used for reforms which affect prices and consequently household incomes, such as utility or agricultural reform. Applications of this type include access to utility services in Guatemala (see Annex Box 3). The approach has some drawbacks, including the fact that it measures average not marginal benefits i.e., it says nothing about the next unit of expenditure 26 . Marginal incidence analysis addresses this shortcoming, (see Box 5).Poverty maps are geographical profiles that show the spatial distribution of poverty within a country, and where policies might have the greatest impact on poverty reduction. For instance, a poverty map can be combined with maps that show the placement of primary health care facilities to understand the access to health services by the poor. The technique is particularly suited to reforms with regionally-differentiated impacts such as decentralization and agricultural reform. Applications include planning of public investments in education, health, and transport and targeting of direct social assistance and food aid to vulnerable populations. The method is most useful when constructed at a fine level of disaggregation, but this requires very large datasets.

26

Not only does simple incidence analysis focus on average, not marginal, benefits, its other drawbacks. First, it does not explain why the things are the way they are. Second, whereas incidence may use public expenditure as the measure of the service’s benefit to the recipient, there may be no correlation between expenditure and received (or perceived) value, or outcomes. Third, as with many interpersonal welfare comparisons, the results of the analysis may vary depending on the method and the dimension used to rank households. See Demery (1997) and van de Walle (1998).

27

Box 5: Impact of Social Expenditures in Indonesia: Average versus Marginal Benefit Incidence Average and marginal benefit incidence has been examined by Lanjouw et al. (2001) to assess how education and health expenditures affect different income groups in Indonesia. Static benefit incidence analysis entailed dividing groups by expenditure quintiles and computing utilization rates of the facilities for each group. For primary education total government outlays in 1998 amounted to nearly 8000 billion Rupiah (covering both routine and development expenditures). In that year there were just over 25 million students enrolled in public primary schools. Assuming uniform transfers, the government thus transferred some 307,000 Rupiah per student per year in public primary school. The Table below gives the incidence of Government primary education spending per expenditure quintile, and illustrates that because the poorest quintile includes nearly twice as many students as the richest (6.2 million versus 3.3) the amount received per person in the botton quintile (47,898 Rupiah) is also nearly twice that received in the top quintile (25,270). The incidence of public spending on primary education thus clearly favors the relatively poor. Benefit incidence of primary education spending by expenditure quintile Population age 7-12 Percent of total Public school students Per capita transfer Percent of total

1 6.8 mn. 25.4 6.2 mn. 47,898 24.0

2 6.2 mn. 22.9 5.9 mn. 45,324 23.5

3 5.4 mn. 20.1 5.2 mn. 40,004 20.7

4 4.8 mn. 17.6 4.5 mn. 34,375 17.8

5 3.8 mn. 14.0 3.3 mn. 25,270 13.1

Total 27 mn. 100.0 25.2 mn. 38,574 100.0

A similar exercise was carried out for junior and senior secondary education and indicated that benefits of public spending for higher education levels becomes increasingly regressive. In health, per capita transfers on primary health care were found to be rather evenly distributed across quintiles, while government spending on hospitals was highly regressive. Lanjouw et al (2001) also considered the marginal benefit incidence of public social expenditures. In other words, they asked how a change in government spending would be felt across expenditure groups. First the incidence of changes in education and health provisioning across two periods of approximately a decade each was analyzed. Second, the quintile-specific “marginal odds ratio” of participation -defined as the incremental increase in the quintile-specific participation rate associated with an aggregate change in the program participation rate - was estimated on the basis of survey data. This was compared with the “average odds ratio” - the quintile specific participation rate in a given year relative to the participation rate for the population. On the basis of the historical analysis as well as the estimation results, the evidence suggests that changes in public spending on primary education would be even more strongly felt among the bottom two quintiles than what static analysis would suggest. Source: Lanjouw, et al, (2001).

61. The second approach goes beyond direct impact analysis to allow for some behavioral responses among households and economic agents. This class of behavioral analysis includes methods which permit non- zero own-price and cross-price elasticities. In other words, with a price or other policy change, households may switch to consuming or producing other goods and services and move along their respective demand or supply curves. The approach is, however, limited to a purely “micro” focus. Namely, supply is not equated to demand in a market; markets do not clear; and prices are therefore not endogenous. Rather, households simply react to an exogenous policy shock based on behavioral specifications and assumptions. If data, time, and capacity permit, behavioral analysis ought always to supplement simpler incidence analysis to more fully understand household responses to policy change. 62. The tools of behavioral analysis are social impact assessments (discussed above), participatory poverty assessments, behavioral incidence analysis, the social capital assessment

28

tool (SOCAT), demand/supply analyses and household models. With the exception of SIA which is in the “low- low” category in Table 2, all tools in this class are “medium- low” in that they require greater data and in some cases econometric skills. •

Participatory Poverty Assessments (PPAs) have been more commonly used for poverty diagnostics but have demonstrated good potential to include poor people’s views to understand poverty impacts and formulate pro-poor public policy (Norton et. al. 2001) through a series of rapid assessment tools and structured task based analytical exercises. PPAs are particularly useful in understanding non- income dimensions of poverty and in understanding the processes through which reform actions filter down to the poor. They are more relevant to broad-based fiscal/expenditure and sectoral reforms with potential impacts on livelihoods and vulnerability (Dulamdary et. al. 2001). The participatory nature of PPAs strengthens poor people’s influence over decisions that affect their lives. Beneficiary assessments are a subset of SIA that captures poverty impacts primarily through unstructured or semi-structured beneficiary interviews.

•

Behavioral incidence analysis, combines simple incidence analysis with econometric estimates of household behavior. It can be used to explain distributional changes arising from a policy change, (and thereby addresses one of the shortcomings of simple incidence analysis). Applications include analysis of the role of government policy (vis-àvis the private sector) in expanding access to education in Malaysia (Hammer et al, 1995), examination of the disincentive effects of food stamps on labor supply in Sri Lanka (Sahn and Alderman, 1995), and study of the crowding out of private transfers by public funds in the Philippines and South Africa, (Cox and Jimenez, 1995) and Jensen (1996) respectively.

•

Social Capital Assessment (SOCAT) measures social capital (institutions and networks, and their underlying norms and values) at the level of households, communities, and key organizations. It allows analysts to identify how these social assets affect productive behavior, (e.g., income generation and risk management) and how this in turn responds to policy reform. For instance, well functioning networks with high levels of trust such as among parent-teachers’ associations or farmers’ associations may facilitate policy changes which call for collective action or cooperation. Alternatively, SOCAT data make it possible to assess whether certain policies strengthen or undermine social assets. The tool can be tailored to specific policies or used to give depth to other methods of data collection and analysis. A tailored version of the SOCAT survey was administered in Bosnia and Herzegovina, where measurement of the level of social capital led to recommendations for reform to the social welfare system, and improvements in service provision and the integration of returning refugees (World Bank, 2002c).

•

Demand/supply analyses estimate the response of consumers/producers to price changes. Demand analysis can assess how willing consumers at different income levels are to pay for public services like water and electricity. It has been used to assess the impact of higher ele ctricity tariff rates in Armenia (Box 6), and is being applied to the same issue in Kyrgyz Republic. Supply analysis is most suited to analyzing agricultural

29

reforms which affect the poor in their role as producers, and has been used to examine the impact on poor farmers of agricultural liberalization in Mexico (see Annex Box 4). •

Household models are somewhat more involved in that they analyze impacts taking account of households as both consumers and producers. They are particularly suited to addressing agricultural reforms, but have been used in relation to large set of reforms, including taxation.

Box 6: Impact of Utility Pricing on the Poor in Armenia – Demand Analysis A recent World Bank study uses multivariate welfare analysis to assess the poverty impact of raising tariffs in the electricity and water sectors in Armenia. It looks ex post at the impact of higher electricity prices (and an accompanying expansion in social safety net provision) and ex ante at increased water tariffs. The study estimates a demand function to examine consumers’ responses to changes in prices, including through substitution from electricity to other forms of fuel. Possible supply side adjustments (to the cost and structure of production) are not taken into account. The analysis draws on two specially commissioned surveys, undertaken over the course of the electricity reform - a quantitative household survey of water and electricity consumption patterns (as well as of standard information on income and demographics) and a qualitative consumer satisfaction survey based on focus group research concerned with attitudes towards provision. For electricity, the data are matched with administrative statistics on payment and consumption. The electricity study examines changes in consumption and payment behavior (pattern of arrears etc) of poor and non-poor households following reform. The water analysis considers (i) how much extra poor and non-poor households would be willing to pay for an improved service, and (ii) the policy trade-off between raising tariffs by enough to cover costs vis -à-vis the threat of reduced household consumption. In both cases results from survey data are corroborated against the predictions from multivariate models of household expenditure per head. The models include as explanatory variables demography, asset holding and regional location; each is estimated separately for rural and urban households. The electricity study finds that households cut their consumption and switched to wood and natural gas alternatives as a result of the rate increase. This effect was particularly marked for poor households. As a result, the reform has produced only a modest improvement in revenue. One policy implication is that future tariff rises are more closely aligned with likely consumer responses. Another is the need for action to mitigate poverty and environmental impacts. The results of the water analysis suggest that consumers are reluctant to pay significantly more for a service they deem unreliable. The authors suggest that reform should, therefore, proceed in two stages – first enforcing payment from households with reliable service, and then raising tariffs incrementally to balance cost recovery with the need to maintain access of poor users. Source: Julian Lampietti et al, (2001)

30

C. Methods: Reforms with High Feedback Effects27 63. There are four classes of methods able to analyze with high feedback effects such as macroeconomic and exchange rate adjustments: partial equilibrium analysis, general equilibrium techniques, and macroeconomic frameworks linked to microeconomic models. All of them are taxing in terms of data/time/capacity, the latter two particularly so, (and range from the “medium- high” to “high- high” cells in Table 2). Social impact assessments ma y also be used, and is less demanding. As discussed in Section V.5., there is much to gained from mixing analytical methods, if possible. Qualitative analyses can inform and enrich the analysis derived from quantitative modeling methods described in this section when undertaken in conjunction with such methods. (i) Partial Equilibrium Analysis 64. Partial equilibrium analysis is a step beyond behavioral impact analysis in that it equates supply and demand in one or more markets so that prices clear at their equilibrium level. 28 Thus prices are now endogenous. Partial equilibrium analysis is distinguished from general equilibrium analysis (below) in that it does not include all production and consumption accounts in an economy, and does not attempt to capture all markets and prices in an economy. Thus, while partial equilibrium approaches (which include elasticities on both the demand and supply side) will allow for indirect feedback effects from the impact of changes in one market on other markets, they will not capture all such changes to the extent that they exclude any key markets.29 This is its biggest drawback relative to general equilibrium approaches, (see below). For this reason, partial equilibrium analysis is better suited to analyzing sectoral reforms (such as agricultural marketing and pricing and utility pricing reforms) that are less likely to have large impacts on macro aggregates. Partial equilibriums techniques fall within the “high- medium” category of Table 2 in that they at least require household survey data. 65. Tools for partial equilibrium analysis are multi-market models and reduced form techniques. The former permits the combined estimation of systems of supply and demand relationships, so that the impact of policies in one sector can be seen on other related sectors. They represent a simpler alternative to computable general equilibrium models, and have been used in a number of contexts to examine: the welfare impact of technical change in agriculture, increased exports, and input subsidies in India (Binswanger and Quizon (1984, 1986); and agricultural subsidies and tariffs in Turkey, (see Box 7). The latter technique can be used to simulate the impact of different policy variables on poverty and social outcomes. It has many applications, including assessing the impact of public services or policy reforms on poverty or other welfare measure. The approach is less data intensive than multi- market modeling. For

27

It is particularly important that reforms with high feedback effects are subjected to good stakeholder analysis to isolate impacts on different groups, and to careful risk analysis to assess what might undermine the reform. 28

Behavioral impact analysis, in focusing specifically on demand analysis separately and on supply analysis separately, can also be argued as a “partial” equilibrium analysis. The distinction drawn here is that since market demand and supply are not equated and do not clear, it is not technically “equilibrium” analysis. 29 In general equilibrium terms, it also effectively assumes a closure.

31

Box 7: Impact of agricultural subsidies and tariffs in Turkey: Multi-market modeling Hammer and Tan (1989) construct a multimarket model of the agricultural sector in Turkey. Their model contains eight separate agricultural markets, all of which are potential substitutes for each other. Some of these are traded internationally. Incomes in the rural areas are derived from agricultural profits. The model also includes an explicit government account, which taxes, provides subsidies, and intervenes directly in the markets for selected outputs. Elasticities for supply and demand were taken from published sources, and modified to satisfy theoretical restrictions and to conform with base data. Sensitivity analysis confirmed that the model was robust to large changes in these and other assumptions. The model simulates the impact of changes in government policy concerning direct intervention (subsidies and support prices), and tariffs. The results indicate that reducing export taxes leads to a broad-based increase in supply and exports; and that the incidence of subsidies to fertilizer and feedgrains is sufficiently skewed that they could be cut without damage to farm incomes or export earnings. Also, import duties on milk products are regressive. Imposing border prices (removing import tariffs and restrictions) leads to improved government finance and foreign exchange earnings. It also improves the incomes of middle and wealthy farm households, but at the risk of harming consumers – especially the poor – through higher prices. Source: Hammer and Tan, 1989.

instance, reduced form techniques were used to study rural poverty in Zambia taking advantage of household budget data, time use information, and other sociological and anthropological data. (ii) General Equilibrium Analysis 66. General equilibrium analysis goes beyond partial equilibrium analysis in that it models all economic accounts in the economy and thus aims to present a comprehensive picture of the economy. What they have in common is a complete specification of the economy, in varying degrees of aggregation. In theory, a well-specified general equilibrium model can capture indirect feedback effects of policy generated from all other markets. However, in practice, as with any economic estimation, it captures feedback effects only from those markets that are included in the model, and results depend on the model specification and parameters. 30 While general equilibrium analysis can be used to analyze most types of policy reform, it is most relevant to reforms with multiple and significant feedback effects on the economy through a number of transmission channels. An exchange rate devaluation or alternative aggregate fiscal policies would be best assessed with a general equilibrium approach, data and capacity permitting. General equilibrium analysis, in capturing accounts from the entire economy, require not only household survey data but also comprehensive and consistent national aggregate data. The computational and capacity requirements are also generally high. Other drawbacks are that the technique can be difficult to explain to policy makers, and results are sensitive to the assumptions on which a particular model is based. The approach is hence presented in the “highhigh” cell in Table 2. Specific tools for general equilibrium analysis are Social Accounting Matrices and Input-Output Models (SAMs), and computable general equilibrium models (CGEs): •

SAMs can be used for simple policy simulations (by selecting some accounts as exogenous, and leaving the others endogenous). For instance, in a SAM containing

30

The standard caution and caveat with respect to economic modeling thus applies: great care should be taken in specifying the model and its parameters to country context and great care should be taken in making explicit the specific assumptions and limitations of simulations derived from such models.

32

agricultural production and transportation accounts, the impact of an exogenous change to agriculture can be simulated (leaving transport fixed) or the other way round. 31 SAMs have some serious limitations, including the fact that prices do not adjust to reflect changes in real activity, and that results are highly sensitive to which accounts are assumed endogenous and which exogenous. •

CGEs are completely-specified models of an economy, (or a region). They vary in their complexity from the basic 1-2-3 model, (one country, two activities, and three goods) to versions with several activities and actors and hundreds of parameters. CGEss can be used in a number of policy contexts, including public finance reform and macroeconomic stabilization. 32 Box 8 illustrates the use of a CGE model to calculate the impact of fiscal incidence in the Philippines. As well as being data intensive, CGEs – even simple ones – can be difficult to build and understand.

Box 8 : Net Fiscal Incidence in the Philippines Ideally one should be able to analyze the incidence of tax and expenditure policies simultaneously, i.e., conduct a net fiscal incidence analysis. In practice, this type of analysis is difficult to undertake because the data requirements are extensive. One of the few examples of this type of analysis was done in the Phillipines by Hossain and Devarajan (1998). The net incidence of fiscal policy (indirect taxes, direct taxes, and expenditures) was estimated using a variety of data sources and tools. For both direct and indirect taxes, the authors calculate the effective tax rate for each income decile defined as the change in purchasing power by each income class. For direct taxes, the authors calculated the effective tax rate using actual tax collection rates broken down by gross income. The family income and expenditure survey was used to map income classes into deciles. For indirect taxes, a multisector CGE model was used to calculate the incidence of taxes. The effective tax rate for each type of tax (e.g. VAT, import tariffs, excise taxes) was calculated individually. This was done by simulating the removal of each type of tax with the CGE model. The incidence reflects both actual tax collections and the increased costs associated with each tax. The effective rates for indirect and direct taxes were aggregated to get overall tax burden. For expenditures, the authors focused on health, education, and infrastructure spending. Nation-wide incidence patterns were derived from regional pattern of expenditures along with information on income distribution. To derive benefit incidence, the authors inferred the implicit subsidy on health, education, and infrastructure for each income decile. Overall incidence of public expenditures in health, education, and infrastructure was calculated as the weighted average of the regional incidence, with the weights being the regional allocations of these expenditures. Total incidence of public expenditures was calculated as benefits as a share of gross income. The results indicate that tax incidence is fairly neutral. Expenditure incidence is strongly progressive as is the combined incidence. Source: Devarajan and Hossain (1998)

31

Supply is either perfectly elastic (if chosen to be endogenous) and entirely demand driven, or perfectly inelastic – that is, supply is constant. SAM-IO simulations also vary greatly depending on the assumptions made about which accounts are exogenous and which endogenous. 32 Dervis, de Melo, and Robinson (1982) and Shoven and Whalley (1992) provide good summaries.

33

(iii) Micro-level Simulation linked to Macro/Sectoral Models (or Assumptions) 67. This class of analysis links microeconomic behavior and/or distribution with a consistent macroeconomic framework or model. In contrast to CGE models described above, the detailed and disaggregated household behavioral responses are not modeled simultaneously in the computation of general equilibrium outcomes. Rather, distributional and poverty outcomes are arrived at iteratively – and outside the macro modeling exercise. In its simplest form, the macro or sectoral model is solved to derive the main equilibrium parameters (e.g. prices, wages, fiscal deficit, etc.); these are then fed into a micro model. In principle, any variety of macro or sectoral models (such as a CGE, multimarket, or macro-consistency framework, for example) could be used to derive the equilibrium economic parameters. Likewise, any number of micro approaches can be used to derive poverty and distributional outcomes based on the parameters derived from the macro or sectoral model. 33 Hence, an advantage of the approach is the flexibility it affords the analyst. The approach can be applied to a wide variety of reforms. However, it is data and skills intensive, and is located in the “high- high” cell in Table 2. 34 Specific techniques include: •

Linking Macro-Framework to a Reduced Form Estimation. This is a minimalist approach which simulates poverty impacts on the basis of various macroeconomic variables 35 . Tools have also been developed to examine how changes in certain macrovariables – most particularly growth rates - affect poverty, based on a country-specific distribution. SimSIP is a tool of this type, (see Annex). 36

•

Linking Macro-Framework to Behavioral Analysis Estimated for Representative Households. This has been done in the 1-2-3 PRSP model (Devarajan et al, 2001), which links the 123 model to a behavioral analysis of representative households, and PAMS, which joins a labor-poverty module to a macro-consistency model (such as the Bank’s RMSM-X). 37 The technique can be used to simulate a wide range of policies, from labor and wage policies to taxation, prices, and the allocation and levels of government spending. Applications include the linking of a simple CGE model with a demand system for food to examine the impact of macroeconomic policy changes on food consumption and nutritional status in the Philippines (Orbeta and Alba, 1998)38 .

33

It is also possible to run the micro-simulation exercise not on the basis of parameters derived from a consistent macro-model, but on the basis of exogenously assumed changes in parameters. Such an approach would not be so different from the simplest form of “direct impact analysis” described earlier. 34 This is also an area where work is still ongoing and new tools and applications continue to be developed. 35 This has been done, for examp le, by Agenor (2002) who has estimated such an equation – including the relevant elasticities – on the basis of a cross-country regression tailored to take as inputs the outputs of the RMSM-X model. One limitation to this approach relates to the robustness of cross-country estimates of these elasticities when applied to a national context. 36 SimSIP has a module that looks at growth impacts, is being expanded to include a module that will accept as inputs the key aggregate wage and consumption variables generated from the 1-2-3 model. 37 The separate labor and poverty (LP) module can simulate the impact of policies on the labor market, income and expenditures, and related social welfare indicators. It permits the reallocation of labor in response to changes in prices and wages. 38 A similar approach was undertaken by Ianchovichina, Nicita, and Soloaga (2001) to examine the impact of NAFTA on household welfare in Mexico.

34

•

Linking Macro-Framework to Micro-simulation. A more disaggregated variant of the representative household method above is to simulate behavior at the level of the individual household. Robillard et al. (2001) use this approach to analyze the poverty impact of the Indonesian financial crisis, (see Box 9). Their household model is linked to a CGE through wages, and the sectoral allocation of employment and prices. It is constrained to be consistent with the output of the CGE.

Box 9: Impact of the Indonesian financial crisis on the poor: partial equilibrium modeling and CGE modeling with micro-simulation General equilibrium models permit the analyst to examine explicitly the indirect and second-round consequences of policy changes. These indirect consequences are often larger than the direct, immediate impact, and may have different distributional implications. General equilibrium models may thus reach significantly different conclusions from partial equilibrium models. These differences are illustrated by comparing the conclusions reached by examining the same event using different methods. Levinsohn et al (1999) and Robillard et al (2001) both look at the impact of the Indonesian financial crisis on the poor; the former using partial equilibrium methods, the latter using a CGE model. Levinsohn et al (1999). use consumption data for nearly 60,000 households from the 1993 SUSENAS survey, together with detailed information on price changes over the 1997/98 crisis period, to compute householdspecific cost-of-living changes. They find that the poorest urban households were hit hardest by the shock, experiencing a 10-30 percent increase in the cost of living (depending on the method used to calculate the change). Rural households and wealthy urban households actually saw the cost of living fall. These results suggest that the poor are just as integrated into the economy as other classes, but that they have fewer opportunities to smooth consumption during a crisis. However, the methods used have at least three serious drawbacks. First, the consumption parameters are fixed – no substitution is permitted between more expensive and less expensive consumption items. Second, the results are exclusively nominal, in that the welfare changes are due entirely to changes in the price of consumption, and do not account for any concommitant change in income. Third, this analysis can’t control for other exogenous events, such as the El Niño drought and resulting widespread forest fires that occurred at the same time. Robillard et al, (2001). use a CGE model, connected to a microsimulation model. The results are obtained in two steps. First, the CGE is run to derive a set of parameters for prices, wages, and labor demand. These results are fed into a microsimulation model to estimate the effects on each of 10,000 households in the 1996 SUSENAS survey. In the microsimulation model, workers are divided into groups according to sex, residence, and skill. Individuals earn factor income from wage labor and enterprise profits, and households accrue profits and income to factors in proportion to their endowments. Labor supply is endogenous. The microsimulation model is constrained to conform to the aggregate levels provided by the CGE model. Robillard et al (2001). find that poverty did increase during the crisis, although not as severly as the previous results suggest. Also, the increase in poverty was due in equal part to the crisis and to the El Niño drought. Comparing their microsimulation results to those produced by the CGE alone, they find that the representative household model is likely to underestimate the impact of shocks on poverty. On the other hand, ignoring both substitution and income effects, as Levinsohn et al. (1999) do, is likely to lead to overestimate the increase in poverty, since it does not permit the household to reallocate resources in response to the shock. Source: Levinsohn et al, 1999 and Robillard et al, 2001.

35

7. Contemplating Enhancement and Compensation Measures 68. Poverty and social impact analysis is undertaken to maximize the welfare gains among the poor and other targeted groups. To the extent that there are losers from the reform, PSIA can inform policy design that minimizes the number of losers or the extent of losses. Where appropriate it also informs compensation mechanisms. In short, if in the first instance ex ante poverty and social impact analysis (above) shows that a proposed reform will have short term adverse impacts on the living standards or the poor or other groups, it is critical that the analyst address the following considerations. A. Consider alternative design. 69. The design of reform may be improved by including enhancement or mitigation measures, or different sequencing of public actions. •

First, one may opt to proceed with the implementation of a reform as planned, but with a subsidization arrangement to protect the poor or others adversely affected by the policy. For example, a water tariff increase associated with utility reform may be designed to protect those who consume relatively small quantities of water by incorporating lifeline tariffs. Alternatively, analysis of an electrical utility reform may determine access to be the main constraint for the poor, resulting in the design of subsidized grid connection fees for targeted poor communities. 39 In fiscal reform, key staple goods that comprise the bulk of consumption for the poor may be exempted from taxation. 40

•

Second, the policy set may need to be expanded beyond the core policy measures (driven by the problem diagnosis) to include complementary measures. For example, if “behind the border” bottlenecks (such as barriers to entry in the domestic transport sector) reduce the benefits of trade liberalization accruing to intended beneficiaries, taking measures to address those constraints will be critical to achieving expected welfare gains. Similarly, understanding and addressing the factors that constrain the poor or other target groups from benefiting from market reforms – e.g. due to lack of assets (land, credit, electricity grid connection) or capabilities (price information, market access) will be essential. Microeconometric analysis as well as qualitative analysis can assist in identifying what type of complementary measures might be necessary.

•

Third, it is important to carefully consider sequencing. For example, shutting down a commodity board can eliminate monopsony and subsidized inputs at the same time. If critical inputs are likely to be unavailable or prohibitively expensive for vulnerable farmers in certain locations, PSIA might suggest that government first take action to drop barriers to entry or encourage private merchants to pursue untapped markets before

39

Subsidization choice would depend, at least in part, on institutional capacity and transactions costs of delivering the subsidy. 40 It is worth noting that such exemptions may introduce undesirable distortions into the tax and incentive scheme, not only from an efficiency standpoint. To the extent that it allows non-poor producers to avoid taxes legally or facilitates tax evasion, exemptions that appear patently progressive can limit progressive budgets that address social program for the poor.

36

dismantling the commodity board. Also, sustainability of the reform process can be enhanced with quick wins among key stakeho lders to build support for reform. For example, new resources for mining safety in Russia were used to persuade the unions of reform. B. Consider direct compensatory mechanisms 70. When adverse impacts of reform are unavoidable, considerations driving the decision to compensate losers may be based on: (i) poverty grounds (especially if some of the poor lose in the short run and the objective of the policy is poverty reduction); (ii) on equity grounds (especially if groups that have traditionally been the poorest and most vulnerable lose ground to those with greater economic security); or (iii) on political economy grounds (especially if the losers have the capacity to organize and threaten either the sustainability of reform or survival of the government). 71. Careful consideration is required in the design of compensatory schemes – to ensure appropriate targeting of intended beneficiaries and cost effectiveness, and to avoid perverse or distortionary incentive schemes that might compromise implementation of the intended policy, (see Box 10). It is also important to calculate the cost of compensation, and consider it relative to the expected benefits of reform. In terms of costs, the compensation scheme itself (e.g., a large scale retrenchment or social program) will have fiscal costs that, depending on magnitude, can have feedback effects on fiscal stability, prices, and the economy. Related, there is an opportunity cost to any compensation scheme, which will use resources that would otherwise have been spent elsewhere. 41 Box 10: Labor Downsizing and the Design of Compensation Packages in Vietnam The issue of labor downsizing and the design of compensation packages has been analyzed ex ante in the context of Vietnam by Martin Rama (2001). Proposed reforms included a major downsizing operation involving the liquidation, divestiture, or restructuring of approximately 6000 state-owned enterprises (SOEs), resulting in unemployment of roughly 5% of the Vietnamese labor force or 450,000 workers. In anticipation of the massive layoffs a special compensation package was developed which amounted to two months of salary per year of service plus a substantial cash training allowance. This package was a result of policy debates around simulations generated by Rama using DOSE (Downsizing Options Simulation Exercise). The simulation computed ‘acceptance rates’ for alternative severance packages, based on the characteristics of individual workers. The acceptance rate is defined as the fraction of the SOE workers for whom the separation package would exceed the present value of the estimated loss from job separation. Rama found that a formula based solely on earnings history had a consistently higher acceptance rate for men; while women found a uniform lump sum compensation more attractive. Based on these simulations, the government of Vietnam picked a separation package which involved a sizeable lump -sum component under the form of the training allowance in order to ensure that female workers are not unduly penalized by the layoffs. Source: Rama, 2001.

41

The opportunity cost calculation is complicated to the extent that the reform package as a whole might be conditional on the compensation mechanism.

37

C. Consider delay or suspension 72. If the findings of PSIA suggest that the short-to- long term benefit of the best designed policy intervention does not exceed the short-term (or longer-term) costs of mitigating or compensating the poor, or that other important groups might suffer irreversible losses, then consideration could be given to delaying the reform (i.e., re-sequencing) or abandoning or suspending implementation of the policy. 8. Assessing Risks to PSIA 42 73. Upon laying out the broad parameters of possible reform alternatives, it is important to consider the risk that some of the assumptions underlying the analysis are not realized. This process may provide further insight into policy choice and design, including sequencing. Risk analysis addresses the issue of what may go wrong in terms of a policy reform delivering intended poverty or social impacts. By addressing these questions explicitly, adjustments can be made to mitigate the risks (e.g. modifying reform, or introducing complementary measures). 74. Risk analysis can therefore help governments to anticipate – and avoid – major unintended consequences. Part of the challenge is to identify explicitly in the analysis the assumptions that must be valid for policy to have its intended impact. This is a difficult task and underscores the need to make operating assumptions explicit in monitoring the evolution of the policy reform and its evolving impacts (see Section V.9). 75.

There are four main types of risk in PSIA: •

• • •

Institutional risks. These include risks that assumptions made regarding institutional performance were incorrect. This could be due, for example, to market or institutional failures in existence where none were assumed (e.g. asymmetric information or missing markets) or to the fact that key organizations involved perform in unexpected ways. Political economy risks. This includes the risk that powerful interest groups may undermine reform objectives by blocking implementation, capturing benefits, or reversing reform actions. Exogenous risks. These include risk of shocks to the external environment such as a natural disaster or regional economic crisis that might have a bearing on the vulnerability of the poor. Country risks. These include the threat of an increase political instability or social tensions that undermine effective implementation.

76. There are three main methods available to conduct a risk analysis: social risk assessment, sensitivity analysis, and scenario analysis. The first and third are discussed in more detail in the Annex.

42

This discussion relates to risk analysis only as it relates to PSIA. It is not intended as a comprehensive treatment of the issue.

38

•

Social Risk Assessment. This is an approach for systematically identifying risks, and their importance to the reform at hand. It is based on the premise that risks become reality when assumptions turn out to be wrong. The likelihood of assumption being invalid is, therefore, another way of judging the extent of risk. The first step is to identify the assumptions – implicit or explicit – about what should (or should not) happen in order for a policy to achieve its goals. The next step is to make a judgment as to the likelihood that the assumption will hold, and its importance to policy. The more likely it is that an important assumption will be invalid, the greater will be the need to alter the policy. If assumptions are considered important but more likely to be valid, there may be a need for a contingency plan.

•

Sensitivity Analysis. This is usually applied in the context of quantitative economic models, and entails varying the magnitude of certain key parameters to judge their sensitivity to the model’s outcomes. Sensitivity analysis is especially important for parameters which are particularly uncertain (as may be the case where these are based on estimates from other countries) or where risks are known (e.g. droughts in the Sahel). One practical limitation of the approach is that it is more often used to test sensitivity within a given model rather than to assess alternate scenarios using different models, (not always feasible). 43

•

Scenario Analysis.44 This is a tool for helping decision- makers consider how policy impacts might vary in different plausible scenarios. Scenarios are based on a range of social, economic, political or technological outcomes that drive change in the country. In this way, unexpected risks can be highlighted, and contingencies made.

9. Monitoring, Social Accountability, and Evaluation of PSIA45 77. Upon identifying and designing reform based on ex-ante PSIA, it is important to consider setting up at an early stage systems for ex-post monitoring, social accountability and evaluation of the impacts. In doing so, there are some specific concerns to bear in mind in the context of reform-specific PSIA. This section outlines these issues. 78. As noted in Section III, good PSIA calls for monitoring and evaluation, both to validate ex ante analyses and to influence the reformulation of policy. Effective PSIA therefore implies a heavy demand on data and information bases. In considering the information needs of PSIA, it is essential to build where possible on existing systems of M&E. This should be done with a view to developing a coherent national poverty monitoring system, which brings together information bases, indicators, mechanisms for linking M&E and policy decision making etc. This is another area where capacity building is an embedded part of PSIA: the development or refinement of

43

This is being done, for example, in Madagascar, where three different modeling approaches are being used to assess the impact of a rice tariff on distribution. 44 For an operational discussion with examples, see Maack, 2001. For in-depth case studies of applied scenario analysis, see: www.gbn.org/public/gbnstory/downloads/gbn_mont_fleur.pdf (South Africa) 45 This discussion relates to monitoring, social accountability and evaluation only as they relate to PSIA. It is not intended as a comprehensive treatment of the issue.

39

systems for monitoring, social accountability, and evaluation is most effective where it strengthens in-country capacity. 46 79. Monitoring involves tracking the progress of processes and implementation (as measured by indicators on inputs, outputs, and outcomes) associated with an intervention. This is done to ensure agreed targets are met and the policy is on track. Evaluation explains or analyzes how and why observed changes in indicators have occurred. Impact evaluation assesses the extent to which a past intervention has contributed to changes in outcomes or impacts for individuals, groups, households, and institutions. An M&E system includes the institutional structure and processes by which indicators are identified and tracked, as well as an explanation of how and why changes have occurred. A. Particular characteristics of M&E in the context of PSIA. 80. M&E in the context of PSIA may be seen as a sub-set of a national poverty monitoring system, and as having several characteristics. First, it is focused on monitoring impacts of specific policy reforms with a view to validating policy analysis or informing policy adjustment during the course of implementation. This ideally requires information on key indicators before (i.e. baseline data), during, and after the reform. Second, the evaluation problem is particularly challenging in the case of economy-wide policy reforms. As these reforms often apply to whole sectors or economies, unlike projects, which are restricted to a group or specific region, it is difficult to establish the counterfactual. Use of control groups is possible only if the policy has been initially designed as a pilot or phased in so that those who do not initially experience the reforms are used as controls. The particular challenges of ex-post evaluation for certain kinds of economy- wide reform require particular foresight in setting up an evaluation framework ex-ante. Third, in part given the challenges of ex-post evaluation and in part driven by the need for more rapid feedback on the evolution and impact of policy, PSIA implies a special role for monitoring for purely practical purposes. Although monitoring cannot attribute causality, it can say something about whether, for whatever reasons, assumptions are holding and expected impacts are materializing. Related, monitoring can inform where “things are going wrong”, as well as where supplementary interventions or changes in policy may be needed to ensure that the desired impacts materialize. B. Choosing indicators for PSIA. 81. Several key criteria may be used to choose relevant indicators to monitor for PSIA. First, if impacts are transmitted through specific channels (e.g. changes in producer prices, increases in sectoral employment, etc. – see Section V.3), these are obvious indicators to track. Second, if the conceptual framework underpinning the analysis hinges on specific assumptions (e.g. traders or firms will enter with liberalization, consumers or producers will substitute, or even that certain elasticities will be of certain magnitude), the validity of these assumptions holding over time can also be monitored. As discussed above, tracing through transmission channels and making all 46

Building capacity in this context includes not only the development of technical skills, but also changes in incentives and demands for such information among country stakeholders (including government) as well as improved understanding of what constitutes a good information base and how that information base can be used for more creative analysis and for immediate policy decision making.

40

assumptions explicit in undertaking PSIA, increases understanding of the theoretical premises on which the program is based. In the context of M&E, the process of tracing through the theorybased transmission channels also enables one to identify potential intermediate and process indicators that can be used to monitor the implementation and outcomes of reform. Third, given the importance of monitoring for adjusting policy in “real time”, some indicators for PSIA (e.g. prices) ought to be chosen so that they can be tracked over a short time period (e.g. 6 months). The implication is to identify proxy or intermediate indicators for outcome or impact indicators that take longer periods to change. One way to do this is to trace through the critical assumptions or ‘theory’ through which it is believed the reform will influence outcomes. Fourth, it is important to establish indicators to monitor key risks to reform (see Section V.8). These might cover reform– specific risks (e.g., regarding transmission mechanisms or institutions) or broader threats (e.g., to the political economy). 82. In addition indicators should satisfy a simple set of basic technical criteria true for all monitoring indicators. The ideal indicator will be: • • • • • •

highly and unambiguously correlated with the objective variable of interest (e.g., test scores accurately reflect literacy); sensitive to changes in the outcome or impact of interest; timely, in that it can be collected in time to feedback into policy adjustment; relatively insensitive to other unrelated changes in the sector; relatively difficult to manipulate, either by target groups or by policymakers; not too costly to monitor.

C. Effective monitoring facilitates good evaluation. 83. Understanding gained during the process of ex-ante analysis in the course of PSIA and the identification of indicators helps in designing a good evaluation. Process evaluation are important to understanding the “hows” and “whys”, and the associated indicators are usually timely and not costly to collect. Tracing transmission mechanisms prior to the reform helps in thinking through implicit assumptions and highlights where potential constraints or risks may arise. The process also helps to evaluate whether the expected impact of the reform is borne out in practice. Where it is not, more in-depth analysis to explain divergence can be conducted. When results confirm the assumptions, documenting the lessons learned can help design similar reforms elsewhere. 84. The approach used in identifying indicators will ideally encompass both open-ended and close-ended methods, and as far as possible incorporate participatory methods. Open-ended participatory methods examine the how and why of policy reform, and in the case of participatory methods, promote ownership, accountability and transparency. Close-ended methods, on the other hand, only touch on questions of how and why of changes, and are primarily designed to answer the question of how much.

41

D. M&E to Promote Accountability and Transparency. 85. Monitoring and evaluation can also be implemented to promote social accountability during the process of reform, thereby leading to increased ownership and sustainability. There are various M&E tools available that if appropriately, used can help to promote social accountability. These include Public Expenditure Tracking Surveys (PETS), Quantitative Service Delivery Surveys (QSDS), Citizen Report Cards, and Participatory Public Expenditure reviews, all illustrated in Box 11. Another useful tool is perception, which provide another means of Box 11. M&E Tools for Promoting Accountability and Transparency during Policy Reform Public Expenditure Tracking Surveys (PETS), Quantitative Service Delivery Surveys (QSDS), Participatory Public Expenditure Reviews and Citizens’ Report Cards are useful tools for tracking public expenditure and monitoring the effectiveness of reforms, as it pertains to the expected outcomes, processes and impacts that will occur as a result of policy reform. PETS and QSDS collect data through structured interviews and documentation from service providers. While PETS trace money through an organization, QSDS provide a more robust analysis by pinpointing organizational weaknesses that can be addressed through reform. One output of these survey instruments is a case-specific diagnosis of public service delivery, helping to pinpoint weaknesses in implementation capacity and suggesting where reform efforts should be concentrated. Data from PETS and QSDS can help provide answers to several kinds of questions, such as: • • •

How to strengthen the “voice” of service users? What kind of accountability mechanisms between different levels of government can improve service delivery? How to regulate private providers?

Drawing on a number of successful cases and tested models from around the globe, the World Bank has developed a framework for Participatory Public Expenditure System, in which civic groups influence stages of the budget process in a cyclic and iterative manner. The framework can also be applied to the participatory monitoring and evaluation of policy reforms covering all levels of indicators—input, output, outcome, impact—in a participatory manner. The system spans four key stages: • • • •

formulation: how expenditure proposals are made – to which sectors and in which amount. analysis: review of the impact and implication of alternative policy proposals and allocations expenditure tracking: identification of elusive bureaucratic channels through which funds flow, bottlenecks in the flow of resource, and other deficiencies of delivery systems performance evaluation: direct feedback from citizens on the quality of, access to, and satisfaction with public services (e.g., report cards).

One-off engagement at any stage of the cycle can be useful, but participatory public expenditure systems only deliver when the feedback loop is institutionalized and space is given to external voice at each stage. Achieving that level of institutionalization requires the commitment of significant resources over the long term.

pinpointing problems within service provider organizations. Ideally, quantitative and perception surveys can be used in tandem to provide critical information on the issues surrounding design and access to policy reform.

42

86.

There are key principles to bear in mind in establishing the M&E system: •

• •

•

Promoting ownership of reform can be facilitated through the use of participatory monitoring and evaluation. This can be used to identify output, outcome, process and impact indicators that are meaningful to stakeholders. Reaching agreement on key performance indicators can be challenging, and is much better dealt with prior to the reform. Agreement on standards to be achieved is valuable both for policy managers as well as for affected parties who are then more likely to accept the results of monitoring reports and use this to improve policy. In addition, follow- up public disclosure of information strengthens commitment to the reform. Promoting accountability can be achieved by employing specific data collection tools designed to allow beneficiaries to monitor inputs and outputs of the reform, while also soliciting their views on effects of policy outcomes on their well-being Selectivity in choosing whether to conduct an impact evaluation or not is important because impact evaluation is data and time intensive relative to other forms of evaluation, and often can only be implemented some time after the reform is already in place. Therefore, the decision to do impact evaluations should be based on a need to fill knowledge gaps, or to apply lessons learned in expanding reforms. Employing local counterparts in the implementation and setting up of M&E system –i.e. relevant ministry, statistical office, planning office, private research agencies, universities, NGOs— not only promotes ownership, but helps to build capacity in poverty analysis.

E. Planning and implementing M&E: M&E activities related to each stage of the PSIA. 87. Prior to the reform, while still grappling with the key questions and objectives of the PSIA, preliminary list of indicators, and required tasks for the M&E system can be identified, timeline and TORs defined, and consultants hired. In particular, it will be important to ascertain the existing information base and gaps, including the availability of relevant baseline data with regard to key indicators and welfare measures and possible need for collecting baseline data (See Table 3). 88. Once some ex ante analysis is completed, and there is an improved understanding of how reform will operate, the preliminary list of indicators (which may include views perceptions of those to be affected), particularly intermediate and proxy indicators can be refined, and instrument developed to be used in measuring indicators. It is important that improved understanding of the program, and indicators feed into the design of the quantitative evaluation. At this point in time, once indicators have been identified, plans can be made to collect any missing baseline data, ideally prior to implementation of the reform. 89. During the reform or implementation period, there could be a periodic collection of indicators, (proxy/intermediate every, 2-4 or 3-6 months; some indicators, such as prices, every month; outcome/impact indicators on a 6 month plus cycle, depending on the reform). Soon after implementation begins, perhaps 3-6 months, preliminary monitoring and evaluation of processes can be conducted to see whether the theory of how reform would work is proceeding as

43

planned—i.e. do specified inputs/outputs appear to be leading to outcomes/impact, in the manner expected? If not, why not? At this time mid-stream adjustments are made, as required, to ensure the reform is on track. 90. In the post reform or completion stage, roughly 3-6 months after reform is in place, there could be – as a matter of good practice - a follow- up qualitative assessment and an incidence analysis of basic outcome indicators to identify early ‘losers’ and ‘winners’ and reasons for observed patterns. This analysis, along with a more rigorous evaluation can be repeated once a year or as requir ed to fill knowledge gaps in key policy areas, or to inform plans to further deepen or expand reforms, or scale up pilots.

44

Table 3: Planning M&E as part of Poverty and Social Impact Analysis: Guidance for Countries

Reform Timeline STAGE 1: Prior to reform

PSIA Timeline Identify key reform issues, questions, outcomes and risks for investigation Trace out ‘theory’ of how reform will lead to the desired results on the ground.

M&E Processes Timeline Identify input, output, intermediate, outcome and impact indicators Identify existing information sources and gaps; identify availability of baseline data Specify required tasks/needs for covering gaps TORs for M&E

Preliminary field visit for ex-ante analysis Design ex-ante analysis

Identify specific institution(s) to be responsible for M&E and/or hire consultant(s); begin to define process for M&E –i.e. periodicity for data collection; storage; maintenance etc. Plan collection of baseline data, if it does not exist.

Conduct ex-ante analysis

Refine preliminary indicators Collect baseline information Design instrument to be used in measuring indicators Stakeholder analysis Institutional analysis

STAGE 2: During implementation of reforms

3-6 months after initial implementation (and periodically up until completion of reforms): follow-up analysis

Process evaluation Social impact assessment Preliminary incidence analysis Institutional analysis

STAGE 3: Completion/ Post implementation of reform

3-6 months to 1 year after completion of reforms (depending on outcomes of interest)

Process evaluation Social impact assessment Incidence analysis

45

10. Feedback of PSIA into Country Policy Choice 47 91. Fostering and drawing upon public discussion of policy can be useful at various points of the PSIA process – for example to help identify stakeholders and their positions, to understand transmission channels, to validate technical impact analysis, or to leverage social accountability. Ensuring that the lessons learned from impact analysis, monitoring and evaluation, social accountability, and public policy debate, furthermore, is critical for PSIA to “close the loop” and actually inform and affect policy. A. Fostering policy debate. 92. There are situations when the process of policy debate, including among stakeholders, is just as important as analysis. First, policy debate among stakeholders can be useful in developing consensus and building ownership. Numerous studies have concluded that policy is most likely to be effective only where there is broad ownership. Second, a policy forum can produce invaluable information. Insights gained through dialogue may be technical (e.g. academic research) or social (e.g. the perspectives and concerns of social groups that typically do not participate in the deliberate process). For low-income countries, PSIA has been conceptualized as an integral part of the PRSP process and as an element of the dialogue on the country’s poverty reduction strategy. These insights can either validate or revise previous hypotheses or analysis, including critical assumptions. It can also enhance the understanding of the logic behind a given policy reform among stakeholders. Such initiatives would be particularly relevant in the context of widespread uncertainty, suspicio n, and ignorance – or in countries in which poor or marginalized groups have no political voice. Third, and related, establishing systems and fora for policy debate is important not only with respect to its value as part of ex-ante PSIA, but also for its contribution with regard to monitoring and social accountability during implementation of a reform and ex-post, as discussed above. 93. Convening such policy fora among stakeholders, however, is not without its own risks. One is the risk that implicit conflict between major interests becomes open hostility. Indeed there may be compelling political reasons to avoid such a fora. 48 There may also be good reasons for a government to take a policy forum seriously. Elected leaders who rely on democratic legitimacy to bolster their popularity may find such a forum attractive, as may do policy-makers who are genuinely uncertain about which policy reform path to take. From a leadership perspective, it may be sensible and more sustainable to pursue a policy that rests on a social coalition or bargain than one that theory may dictate as first-best.

47

This discussion relates to policy feedback only as it relates to PSIA. It is not intended as a comprehensive treatment of the issue. 48 Examples of situations in which a policy dialogue may be unadvisable are: (a) government commitment to a policy is irreversible regardless of public reaction; (b) an intransigent opposition party or social movement is expected to use forum simply as a vehicle to embarrass the government; (c) representatives of marginalized people are lacking, meaning that the only organized interests likely to have a seat at the table are privileged social groups; (d) open violence between participants is a serious possibility; (e) the government is an authoritarian regime that prohibits autonomous organizations or public expression of citizens’ opinions.

46

94. Managing the process of policy debate and discussion itself requires some planning, particularly to manage risks. In particular, once the decision is made to convene a forum, three concrete issues must be addressed: who to invite; what to discuss; and how to format the dialogue. Moreover, in the context of social accountability discussed above, the government may wish to consider setting up longer-term structures of policy debate – such as regular consultations, “national workshops”, or “town hall meetings”. 49 B. Feeding back for policy adjustment. 95. Ensuring that the lessons learned from the continual monitoring and analysis of policy implementation feed back to the redesign and adjustment of policy is a major objective of PSIA. Sound ex-ante PSIA, as discussed, ought to lead to an explicit articulation of expected impacts, transmission mechanisms, and assumptions, and the establishment of a monitoring system for key indicators tracking the evolution of the reform program. Necessarily, ex-ante PSIA will not get everything right. Rather, monitoring and evaluation, during and after policy implementation, is a critical part of PSIA with the objective of (i) correcting flawed policie s, (ii) making adjustments to improve correct policy choices, and/or (iii) identifying constraints and opportunities for further public action to maximize poverty reducing impacts. 96. The feedback of lessons from the monitoring during implementation and subsequent evaluation of poverty and social impacts of policy choice to the adjustment of policy is thus a critical step in the PSIA loop. Fundamental here are institutional setups. A common pitfall is that units or systems charged with M&E are not properly linked with the decision- making bodies responsible for policy formulation. Ensuring that the key decision- making body with regard to a particular policy reform is accountable for and charged with the reporting of related M&E and the periodic reassessment of policy is the crucial final link to an effective PSIA process. Here again, building institutional capacity, by creating such linkages where they may not previously exist, is an important part of the PSIA agenda.

VI.

Possible Summary Matrix

97. Section V above has presented a series of elements for good PSIA. Pulling these elements together in a coherent, strategic, and integrative fashion is what makes for good poverty and social impact analysis. Invariably, as discussed throughout this paper, a sensible approach to PSIA is going to be country and context specific, dependant upon available data and capacity as well as the reform issue in question. Box 12 provides an example of the PSIA approach currently being undertaken in Chad to address an ongoing reform issue in that country. 98. A summary matrix may be a useful tool to aid analysts in capturing and integrating the various elements of good PSIA outlined in the previous section. Table 4 presents such a summary matrix. In addition to providing the analyst with a framework for considering and

49

This is another area where good PSIA should consider capacity building as part of the agenda. At the institutional level, capacity is required to organize such fora and to open up the space for policy discussion. At the individual level, capacity is often required for informed and effective participation and thus for an informed debate.

47

Box 12. Poverty and Social Impact Analysis of Cotton Reform in Chad Cotton is a key crop in Chad, both for revenue generation and poverty reduction. Cotton accounted for 24% of total government revenues in 1997 and is the most important cash-earner for about 300,000 rural farm families. Weak organization and knowledge among farmers’ groups coupled with structural inefficiencies in the sector have resulted in low yields and low farmer revenues. To address these inefficiencies, the government of Chad has decided to privatize Cotontchad, the parastatal that currently manages national cotton production, and strengthen farmers’ groups. A key objective of the cotton sector reforms is to improve farmer incomes. Several factors underscore the government’s decision to proceed carefully in designing and implementing the reforms: the possibility that yields will fall further if reform prompts farmers to return to subsistence agriculture, the limited availability of information on rural poverty, and cotton farmers’ perceptions of the risks involved with the reform. For these reasons, the government is carrying out a Poverty and Social Impact Analysis to guide the reform. In order to analyze the likely poverty and social impacts of reform e x-ante, the PSIA needs to do a problem analysis and clarify the assumptions on which the program is based. The PSIA team, in consultation with the government and local counterparts, have identified pathways through which they expect the reform to improve performance. By explaining the causal links that tie program inputs to expected outputs, outcomes and the ultimate goal of improving farmer income, the team has explicitly outlined the assumptions for each transmission channel of the reform so that they can be verified. The PSIA that grew out of these discussions has three components: (i) an economic scenario study of different options for privatization; (ii) ex-ante qualitative analysis, and baseline quantitative survey; (iii) ex-post analysis that will include both qualitative and quantitative methods. The aim of the scenario study is to identify and evaluate the technical and economic efficiency of alternative scenarios for privatizing Cotontchad. The study examines options for privatization (e.g. continued vertical integration, separate private ginneries, etc) and assesses the risk posed by each. In parallel with this study, the PSIA of the reforms assesses the impacts on the welfare of farmers in the sector. The ex-ante qualitative component identifies relevant stakeholders (e.g. farmers, Cotontchad employees, micro -entrepreneurs), barriers faced by stakeholders under different reform scenarios, the strength of current institutional structures, and the social risks of reform. The quantitative and qualitative analyses look at the compensation and enhancement measures necessary for reform success and highlights: farmer capacity, access to credit, input use, and transport. Further work involves a “quasi-comparison group” for different types of farmers—i.e. those that produce cotton and those that do not or have abandoned cotton—in order to analyze the likely impact of the reforms on different groups and to get a sense of the welfare impact on farmers who abandon cotton production. The different scenarios for partial and complete privatization and the ex-ante qualitative and quantitative work will be discussed during a stakeholder forum. This public discussion is meant to increase the transparency of the reform and build ownership by fostering policy debate. In addition, there will be an ex-post impact evaluation of the reform. The ex-ante analysis will define baseline indicators to be monitored for policy feedback in ex-post analysis. The ex-post analysis will employ quantitative methods of impact evaluation, which attempt as far as possible to assess impact based on what would have happened in the absence of reforms. This ex-post quantitative analysis will be applied to a panel dataset, to estimate the impact on producer welfare.

articulating key aspects of PSIA for a given reform, it gives a template for making some of the results and assumptions underlying such analysis explicit.

48

99. The matrix calls for the analyst to set out ex ante the anticipated net effect of the policy reform on the poor (or conceivably another target stakeholder group), as transmitted through each of the five channels discussed in Section IV (i.e., labor market, prices, access, assets, transfers), in the short-term (through direct and indirect effects) and in the medium-term. 100. Furthermore, it allows the analyst to explicitly recognize the stakeholders who are likely to gain and those who are likely to lose from the policy change. It also provides space for an explicit cons ideration of enhancement or compensation measures. Depending on the country concerned, conclusions on likely policy impacts will draw on differing information bases and tools. For example, in one country context, the matrix may be filled out using informed reasoning based on secondary data and focus group interviews; in another context, the conclusions may be based on empirically simulated effects derived from modeling techniques and using recent household survey data. In either instance, the matrix calls for a description of the nature of the information base and analytical methodology. It also calls for the analyst to specify critical assumptions and key risks, as well as the key indicators for monitoring implementation. 101. The matrix can itself serve as a useful tool during the PSIA process. For instance, an analyst may wish to sketch out the priors in each of the ten elements of good PSIA before even undertaking an analysis, and then return to the matrix to validate or correct these hypotheses. The matrix can be particularly useful in articulating impacts (Section V.6); identifying assumptions and risks (Section V.8); identifying key indicators for monitoring (Section V.9); and informing the policy debate, which may be instrumental for reformulating policy if necessary (Section V.10).

49

Table 4. A Summary Matrix for Poverty and Social Impact Analysis of Policy Change Reform: Objective: Stakeholders50 Channel General

Specific

Potential winners

Potential losers

Effect on the poor (or target stakeholder group) Indirect shortDirect short-run run

Medium-run

Labor Market

formal informal output demand output supply input demand Prices input supply other private goods and services Access public goods and services physical financial Assets human social natural Transfers and private transfers public transfers and taxes taxes Net Impact Other generally relevant assumptions (e.g., economic growth, political stability, external environment): Key risks: Information base and analytical methodology: Mitigation or enhance ment measures: Summary recommendations: Adopted from Bolt and Fujimura (2002)

50

Stakeholders include those who influence policy and those who are influenced by policy.

50

Critical assumptions (including counter-factual)

Institutional Changes

Indicators

VII. Conclusions 102. This draft User’s Guide to PSIA has sought to provide an initial overview of the key considerations for practitioners contemplating the poverty and social impacts of policy options with a view to informing policy choice and design. It takes the view that undertaking ex-ante analysis of the likely poverty and social impact of specific reform can be undertaken more systematically. It also takes the practical vie w that, to do so, approaches and methods will need to be adapted to fit the context and circumstances at hand and that the limits to what is possible through ex-ante analysis will need to be addressed by continual monitoring, analysis, and reevaluation of policy over time. 103. This draft User’s Guide has laid out ten key elements to be considered by the analyst and policy maker in approaching PSIA. Furthermore, it has given a brief overview of some of the tools and methods that might be used in undertaking analysis of poverty and social concerns associated with policy change. In so doing, it has attempted to draw from tools used by economists and social scientists and to present them in an integrated fashion. These tools have seldom been used in tandem in the design of policy change (Box 10 provides an example of an attempt to do so). Applying these tools to the operational context using this multidisciplinary approach will lead to a richer, more integrated understanding of policy impacts. Moreover, due to the very different types of reform issues, transmission channels, and available data, the choice of tools and methods used for PSIA will vary substantially by type of reform. Appendix 2 represents work in progress to develop pointers on issues, challenges, and tools that may be of specific relevance in tackling reform in specific areas.

51

Appendix 1 Typology of Tools for Poverty and Social Impact Analysis Tools

What it does

A. Measuring Impact – Social Tools *Qualitative SIA is usually a 1. Social Impact Assessment SIA complement to other analytical using qualitative work; it identifies perceptions, techniques only preferences, coping behavior, and attributes that determine impacts, and pre-conditions for affected persons to benefit from policy reforms

2.

3.

SIA with mixed methods

* Assesses how individuals and diverse social groups are impacted differentially by reforms and are likely to respond to opportunities and constraints based on their assets and capabilities.

Participatory Poverty Assessment

*PPAs and beneficiary assessments (BA) use participatory techniques to include poor people’s views in the analysis of poverty and the formulation of policies.

Social Capital Assessment Tool

*Estimates likely changes in productive behavior after policy change based on an assessment of levels of social capital

B. Measuring Direct Impact – Incidence Tools *Measures direct benefit 4. Average incidence on households at Benefit different income levels Incidence Analysis

Advantages

Drawbacks

Reforms

*Can be done relatively quickly and examines impacts in an open-ended manner *Extremely effective in capturing peoples’ perceptions of reform *Can be used to generate or refine hypotheses *Can be applied to countries where quantitative data are lacking or where it does not permit disaggregation into social categories relevant to the reform.

*Results are subjective and difficult to aggregate without contextual analysis. Results need verification by triangulation and running several iterations.

*Any type of policy reform

* Focus groups, key informant interviews, semi-structured interviews, on-site field assessment and stakeholder workshop.

*Time: 2-4 months for data collection and analysis. *Skills: field researcher with ethnog/sociological and facilitation skills is required.

* Provides an assessment of reform outcomes, not marginal impacts. Sample may not be nationally representative.

* Any type of policy change; quantitative SIA has comparative advantage for reforms with low feedback effects.

* As above. Also: purposive household surveys, time-use surveys.

* Very effective in capturing multiple dimensions of poverty *PPA uses structured analytical exercises to engage the poor in policy analysis *BA can be done relatively quickly

*Data derived from perceptions and personal experiences of respondents; can be quantified but care needed in generalization; more useful in understanding depth rather than incidence of poverty

*More useful for simultaneous, multiple reforms or for reforms with high feedback effects. *BA is more useful for examining direct impacts of reforms with low feedback effects

*Participatory action research using a series of rapid appraisal techniques, unstructured and semi-structured interviews *BA: Qualitative data from focus groups, openended interviews, and participant observation with a purposive sample.

*Time: 6-8 mos for data collection and analysis. *Skills: Field researcher with survey design, ethnog/sociological and facilitation skills is required. *National-level PPAs take 5-9 months, 1-2 missions. Skills: social researcher plus 4-8 teams of local researchers. Facilitation and rapid appraisal skills are required. Gender balance is essential.

*Assesses the strength and nature of underlying social networks in order to predict impacts of reform *Can be carried out at the nat ional level (potentially as module of LSMS)

*Can be costly and timeintensive; standard survey questionnaire may need to be tailored to reform and country conditions

* Particularly relevant to agricultural, land, pension and labor market reforms. Can also be used for safety net, decentralization and utility reforms

* Household survey, community questionnaire, and an organizational interview.

*Time: 6-8 months with at least one mission. *Skills: Field researcher with ethnog/sociological, facilitation, and statistical analysis skills.

*Simple *Assesses distributional incidence

*Outcomes may not correlate with incidence; hence, this method does not quite assess impacts. *Measures average not marginal impacts; (the two may differ significantly)

*Any type of policy change or public finance reform. Most often used for tax and expenditure policy, but also applicable to reforms affecting prices which change household income or expenditure, e.g., trade or agricultural reform

*Household/individuallevel data on welfare indicator and consumption of public service being analyzed. Could also use aggregated data, e.g., decile-level. *Govt unit cost of provision

*Time: 2-4 weeks for analysis. *Skills: desk economist and RA. Prior experience not absolutely necessary.

*Does not explain pattern of incidence

52

Data

Completion Time /Skills Required

*BA: 3-4 months, 2 missions. Skills: anthropologist plus local field researchers.

Tools 5.

6.

What it does

Drawbacks

Poverty Mapping

*Computes locality-specific benefit incidence by combining household-level data with census data.

*Provides a higher level of regional disaggregation than is possible by using household-data alone.

*Data intensive; relies on complex statistical techniques *Impacts poorly estimated for provinces underrepresented in survey data.

*Particularly useful for reforms with regionally differentiated impacts, e.g., decentralization, regionally targeted expenditures, etc.

* Can be used to explain distributional impacts

* More complex – requires econometric modeling

*Models including prices of subsitutes/complements (incl. fully specified demand systems) are data intensive and complex. * Demand models do not capture supply–side impacts, i.e., on production, wages, profits, and income. Hence, may yield misleading estimates of the total impact of a price change on welfare. *Prices are exogenous *Supply model do not capture demand–related impacts *Prices are exogenous

As for simple incidence analysis; in addition, data must be appropriate for analyzing behavioral responses, e.g., changes in labor supply. *Household-level consumption and income data, with sufficient variation in prices (geographically, over time). Household and community characteristics also required. *Willingness to pay analysis requires specific survey data on consumpion of public services in question.

*Time 4-6 weeks for analysis. *Skills: desk economist plus RA with some econometric skills.

*Single good demand models (that do not account for cross-price elasticities) can be used to assess willingness to pay of households for goods with no close substitutes (e.g., water, electricity)

As for simple incidence analysis. In addition, can be used to explain contribution of government policy (vis-avis private sector) to observed changes. *Useful for reforms of goods without close substitutes, e.g., water and electricity; can also be used to study impact of reforms of goods with subsitutes, e.g. agricultural subsidies, trade liberalization. *Reforms with limited supply responses.

*As above

*Household or firm-level production data depending on unit of analysis *Typically, time series data (panel or crosssectional) has been used. *Complete household survey with information on both the demand and supply side.

*As for demand analysis.

*Suffers from same weaknesses as simple incidence analysis except measurement of average effects

9.

Supply analysis

Measures impact of price changes on household/labor, firm supply, and aggregate supply.

* Useful in analyzing supply responses, either for one or several commodities

10.

Household models

Measures distributional impacts in a framework in which households treated as both consumers and producers

*Takes both the household supply and demand sides into account *Wide range of applications

*Data intensive; can be technically complex. * Prices are exogenous

53

*Wide range of microeconomic reforms: price and marketing changes, market failures, taxation; often used to analyze agricultural reform

*As above for simple incidence analysis

Completion Time /Skills Required *Time: 4-6 weeks for analysis.

* Analyses marginal incidence

Measures impact of prices and income on household consumption

*As above for simple benefit incidence analysis.

Data

*Measures marginal benefit incidence on households at different income levels

Demand analysis

*More complex and data intensive

Reforms

Marginal Benefit Incidence Analysis

C. Measuring Behavioral Impact – Microsimulation Tools Combines econometric 7. Behavioral estimation of behavioral Incidence impacts with benefit incidence Analysis analysis (simple or marginal).

8.

Advantages

*Panel data is ideal; however, there are methods for exploiting more commonly available crosssectional data *Household survey data. *Census data

*Skills: desk economist and RA. Prior experience not absolutely necessary.

*Time: 1 year (minimum) to build map. * Skills: desk economist plus specialist(s) with prior knowledge of technique.

* Also, ideally one would want data on allocation of time within the household (e.g., on childcare)

*Time: 2-3 weeks to build model; 2 weeks for estimation/simulation. *Desk economist plus RA with some modelling/econometric skills.

*Time: 1-2 months to build model; 2 weeks for estimation/simulation. *Skills: Desk economist plus RA with modelling/econometric skills.

Tools

What it does

D. Measuring Partial Equilibrium Effects *Measure distributional impact 11 Multi-market of policy changes on prices and Models outputs in one or selected sectors *Combined estimation of systems of supply and demand relationships. Models are solved to derive output supply, inout demand, prices, profits, and incomes. 12.

Reduced Form Models

*Systems of supply and demand equations are solved to yield reduced form equations which are then estimated. *Used to estimate the impact of past policy changes on welfare *Often used to examine the determinants of poverty/income; can also be used to examine income growth rates.

E. Measuring Economy Wider Impacts – General Equilibrium Measures distributional 13. Social impacts using policy Accounting simulations Matrices and Input-Output Models (SAMIO)

14.

Computable General Equilibrium (CGE) Models (e.g., 1-2-3 model, IMMPA, etc)

*Measures distributional impact of policy reforms in a completely specified model of economy *Can be as simple as the 1-2-3 model (one country, 2 activities, 3 goods) *A more complex example is the IMMPA model which captures features “typical” of a developing country, such as labor market segmentation, informal employment, and financial accounts.

Advantages

Drawbacks

Reforms

Data

Completion Time /Skills Required

*Simpler alternative to computable general equilibrium (CGE) models, particularly where indirect effects are limited (see below) – do not require macro balances or complete specification of all markets. *Prices are endogenous, i.e., markets can clear in these models *Provide good snapshots of impact of reforms on many different welfare indicators.

*These models may potentially omit important characteristics of an economy; missing key impacts along the way. *Data intensive and technically complex.

*Sectoral reforms; most often applied to agricultural reform

More complex variants (with modelling of more sectors) are more data intensive. In general, will require complete household survey plus sectoral accounting data.

*Time: 2-4 months to build model; 1 month for estimation/simulation. *Desk economist plus RA with considerable modelling/econometric skills.

*Exclusively ex-post analysis, but results can be used to simulate ex-ante impacts of further changes in policy/price variables. This is done through computing elasticities of outcome variables with respect to the policy variables.

* Any policy change; e.g. used to analyze civil service reforms, general structural adjustment, and shocks.

*Time series data preferable – before and after reform – for welfare measure and policy variables. *Can estimate with crosssectional data if there is sufficient variation across the sample.

*As for demand analysis.

*Complete specification of the economy; however, this is simpler than a full-blown CGE (see below).

*Prices are fixed and exogenous to the model *Results vary greatly depending on assumptions about the exogeneity/ endogeneity of various accounts

*Sectoral reforms

*National acounts

*Completely specified; endogenous prices.

*Models vary in their complexity, but all require a lot of time (possibly more than a year) to construct and have extensive data needs. *A single model may not be amenable to analyze different types of reforms. *Results heavily dependent on priors and may be unintuitive to policy makers.

*Wide range of reforms, e.g., prices, taxes, subsidies, public expenditure, stabilization and adjustment, trade.

*Time: 6 months to build SAM; 1 month for simulation. *Skills required are that of a desk conomist plus RA preferably with previous experience in building SAMs. *1-2-3 CGE model requires about 2 months to build; and about 1 week to run the simulations. Skills: desk economist plus research assistant with previous CGE experience. *The IMMPA model takes about a year to build; simulations require 1-2 months. Skills: desk economist plus a research team experienced in CGE modelling.

*Flexible – can be as simple as the 1-2-3 model, or enormously complex. *Measure indirect effects of policy changes as well as direct effects (which is missed by partial equilibrium approaches). *Disciplines analyst to set out prior assumptions on distributional and other channels.

54

*Household survey

*National accounts (required to build the social accounting matrix) *Household survey

Tools

What it does

Advantages

F. Microeconomic Simulations Linked to Macroeconomic/Sectoral Models *Can be adapted to use 15. Linking macro- *Simulates (using a reduced form estimation) the impact on outputs from Bank’s f/work to poverty reduction of various RMSM-X reduced form macro variables *Sim-SIP is relatively userestimation of friendly. poverty *Sim-SIP is a specific tool that examines how changes in macro variables (e.g., growth rates, sectoral growth rates) may affect income distribution and poverty. Uses the countryspecific distribution as the starting point. 16. Linking macro- *Simulates the impact of macro *Behavioral changes can be variables on household welfare, simulated. f/work to allowing behavioral changes. behavioral Micro-level parameter changes analysis are estimated for representative estimated for household groups. representative households (e.g. *1-2-3 PRSP (a modification of 1-2-3 PRSP, the simple 1-2-3) is used to PAMS) derive macro parameters; these parameters are then used to simulate the impact of policy changes on representative households. *PAMS links a labor-poverty module to any macroconsistency model (e.g. RMSM-X or the Bank’s Financial Planning Model. *Simulates distributional *Arguably provides more 17. Linking macro impacts on welfare at the level accurate estimate of impacts f-work to of individual households. as it takes greater account of microhousehold inequality simulation

Drawbacks *Generally, data intensive and time consuming

Reforms

Data

Completion Time /Skills Required

*Wide range of reforms, e.g., labor and wage policies, taxation, prices, changes in public spending, trade, macro stabilization.

*National accounts, sectoral accounts

As above

As above

*National accounts, household survey/aggregated data.

*1-2-3 PRSP: 6 mos–1 year to build model; 4 weeks to complete simulations. *PAMS: 6 mos-1 year to build model; 6 weeks to complete simulation. *Skills: both need desk economist plus RA with prior modelling experience.

*Very data intensive, time consuming, and still under development; therefore, currently of limited applicability in low income countries

As above

*National accounts *Household –level data

*Time: 1 year to build model; 1-2 months to complete simulation *Skills: desk economist plus experienced specialist(s) required.

55

*Household survey data.. Note that aggregated data can also be used.

*Time: Sim-SIP requires a week to master; and 1 week to complete simulations. *Skills: desk economist plus research assistant. Prior experience not necessary.

Appendix 2 Poverty and Social Impact Analysis – Reform-by-Reform Application of Key Tools [Note: PSIA for each reform needs to undertake a Stakeholder Analysis, some form of Impact Analysis, Institutional Analysis, and Risk Analysis. It is also important that it include some form of monitoring. This table provides additional guidance on which tool would be most appropriate for Impact, Institutional, or Risk Analysis for individual reforms]

1

Reform Macroeconomic and Fiscal Reform Monetary policy reforms – i.e., reforms influencing inflation and interest rates.

Broad external policy reforms, i.e., balance of trade and foreign exchange reserves

2

3

Broad fiscal policy reforms, i.e., reforms related to the fiscal deficit. (Note: typically adjustment involves reducing expenditures and increasing taxes). Public Finance Expenditure reform, e.g., changes in levels/allocation of sectoral spending

Revenue policies – levels, composition, improvements in tax administration, cost recovery in public services Trade and Exchange Rate Reform Reforms of tariff and non-tariff barriers Exchange rate reforms

4

Agricultural Reform Eliminating administered prices (i.e., price bands, floor and ceiling prices); ending buffer stock programs (used to maintain prices). Changing domestic subsidies and taxes

Eliminating marketing boards

5

Land Reform , i.e., distribution to the landless or passing of laws governing the right to own, exchange, or inherit land

6

Financial Sector Reform Financial liberalization (interest rates, allocation of credit)

Selected Possible Transmissions Channels

Most Useful Tools for PSIA

Price effect – from changes in inflation and interest rates. Income effect – e.g., low and stable inflation will encourage foreign investment, leading to growth and higher incomes. Assets – e.g., low inflation rates will protect the value of assets. Access effect on credit – can be negative following contractions in money supply (loanable funds decline and interest rates increase). Price effect – improving the balance of trade and foreign exchange reserves will lead to increasing the price of tradeables relative to non-tradeables. This will change consumption and production. Income effect – incomes of those producing tradeables will increase. Access to services – may be affected if public spending is adjusted downward. Income effect – there can be a short-run negative impact on growth due to adjustment.

PPA; demand and and supply analysis; reduced form models.

Income effect – can be positive or negative depending on the beneficiary group in question and the direction of the changes. Access to public services – can expand through increases in spending.

Social impact assessment; PPA; benefit incidence analysis (simple and behavioral); demand/supply analysis (including willingness to pay); reduced form models; institutional assessment tool (IAT) As above, except for PPA.

Income effect on taxpayers – will be positive (negative) with decreased (increased) taxation. Price effect – from changes in indirect taxes

Demand and supply analysis; reduced form models

As above; also: Social impact assessment; PPA; marginal incidence analysis; scenario analysis.

Price effect – lower prices will result from removal of barriers. Access – removal of barriers should expand access to goods. Employment and wages – there will be a negative impact on previously protected sectors. Price effect – terms of trade will change affecting both consumer and producer prices. Income effect – liberalization may increase volatility and risk in the short run. In the long run, liberalized markets should lead to growth.

Social impact assessment; demand/supply analysis; multimarket models; reduced form models; CGE

Price effect – will directly affect price of liberalized good and thereby production and consumption behavior. Employment – changing relative prices will have a direct impact on the allocation of labor within agriculture and across sectors Price effect – will directly affect price of liberalized good and thereby production and consumption behavior. Employment – changing relative prices will have a direct impact on the allocation of labor within agriculture and across sectors Access to services – will be affected by changes in budget balance. Price effect – will directly affect agricultural prices Income effect – depends on extent of previous subsidization and access to subsidy. Access – budget balance will improve due to the elimination of subsidization; could be used to improve access to public services. Employment effect – for employees of the boards Income effect -- more equitable land distribution will lead to higher wages, production, and growth. Employment effect – will be positive as opportunity cost of time for the former landless increases. Assets – the formerly landless will own a major asset post-land reform. Access – secure title to land provides collateral for credit.

Social impact assessment; SOCAT; supply analysis; household models; multi-market models; reduced form models; CGE; organizational mapping; social risk assessment As above

Income effect via impact on interest rates; also, growth can increase due to improved efficiency in financial system. Access – expansion to poor may not occur in the short run.

Social impact assessment; reduced form

56

As above, except for SIA.

As above

Social impact assessment; organizational mapping; social risk assessment.

Reform Lowering barriers to entry

Regulatory reform Privatization/closure of state banks

7

Promotion of financial institutions serving poor clients Labor Market Reform Minimum wage legislation

Job security regulation Active labor market programs

Selected Possible Transmissions Channels Access – overall access to credit will increase; including to financial intermediaries. Income effect via increased growth – due to increased efficiency in the system. Income effect from increased growth – due to improved efficiency in financial system as well as less vulnerability. Employment effect – lay-offs of bank workers. Access – rural branches may have represented welfare subsidy; may not be replaced if closed. Privatization may also not have a positive impact on access in the short run. Assets – transferred to private hands. Access – expansion for the poor; Price effect – lower interest rates are typical.

Most Useful Tools for PSIA Social impact assessment; SOCAT; social risk assessment

Employment effect – direction hotly debated Income effect – will affect incidence of low pay and earnings dispersion Employment effect – regulations have negative impact on levels against more stability for those employed

Reduced form model; CGE; social risk assessment.

Employment effect – evidence of positive impact is weak

Social impact assessment; SOCAT; organizational mapping.

Social impact assessment; social risk assessment Social impact assessment; reduced form; organizational mapping; social risk assessment; scenario analysis As above; minus reduced form.

As above; also institutional assessment tool.

Note: experimental techniques have been used to study the impact of ALMPs ex-post. 8

Utility Reform Restructuring state-owned utilities

Increased private participation in state-owned utility Full divestiture of utility 9

Privatization Lease of assets, management contracts

Full divestiture 10

Employment effect – short -term lay offs Price effect – tariff changes Impact on access – not easily predicted, e.g., dependent on changes in subsidies/tariffs and investment in network expansion. Price effect – higher tariffs, and connection fees Impact on access – dependent on regulatory framework Employment effect – short -term layoffs As above, also: Assets – transferred into private hands

SOCAT; benefit incidence analysis; demand/supply analysis (incl. willingness to pay); CGE; organizational mapping.

Price effect – generally higher Impact on access – dependent on regulatory framework Employment effect – short -term lay offs

Social impact assessment; demand/supply analysis; reduced form models; organizational mapping; social risk assessment; scenario analysis. As above.

As above, also: Assets – transferred into private hands

Civil Service Retrenchment Lay-offs

Employment effect – loss of jobs for affected workers

Reductions in wage bill

Income effect – loss of income for affected workers

11

Decentralization of public services

Change in access to public services

12

Social Safety Nets Targeted cash/in-kind transfers

13

51

As for restructuring state-owned utilities.

Social impact assessment; reduced form models; social risk assessment. Social impact assessment; reduced form models Social impact assessment; SOCAT; benefit incidence analysis; reduced form models; organizational mapping; social risk assessment.

Categorical benefits (e.g., to AIDS orphans, the disabled, etc). Contribution-based social insurance benefits (e.g., disability, health) Pensions Scaling back public pension schemes (higher contribution rates, lower benefits, increased retirement age)

Transfers to beneficiary group

PPA; benefit incidence analysis; application of Pensions Reform Options Simulation Toolkit (under development)51 ; organizational mapping; social risk assessment As above

Income effect on workers

As above

Income effect on workers and pensioners (current and future)

SOCAT; reduced form models (Pension Reform Options Simulation Toolkit); social risk assessment.

Increasing private provision

Income effect on current savers, future pensioners; Employment effect (via greater portability) Transfers to current pensioners

As above, minus SOCAT

Introduction of social pensions (cash assistance for vulnerable pensioners)

Transfers to beneficiary group; Employment effect (negative if transfers are not tied to employment)

As above; plus reduced form models and minus SOCAT.

World Bank, 2000b.

57

As above, minus SOCAT

References Agénor, P, 2002. "Macroeconomic Adjustment and the Poor: Analytical Issues and Cross-Country Evidence." Washington, DC: World Bank Staff Working Paper No. 2788. Agénor, P., A.Izquierdo, and H. Fofack. 2001. “IMMPA: A Quantitative Macroeconomic Framework for the Analysis of Poverty Reduction Strategies.” World Bank manuscript. Agénor, P. and J. Aizenman. 1999. “Macroeconomic Adjustment with Segmented Labor Markets.” Journal of Development Economics, 58:2:277-296. Alwang, J., Siegel, P. and Jorgensen, S. 1996. "Seeking Guidelines for Poverty Reduction in Rural Zambia". World Development, Vol. 24, No. 11: pp. 1711-1723. Baker, J., 2000. Evaluating the Impact of Development Projects on Poverty: A Handbook for Practioners. Washington, DC: The World Bank. Barnum, H. and L.Squire, 1979. A Model of an Agricultural Household: Theory and Evidence. Baltimore: Published for the World Bank by Johns Hopkins University Press. Barro, R. 1997. Determinants of Economic Growth: A Cross-Country Empirical Study. Cambridge: The MIT Press. Becker, G., 1965. “A Theory of the Allocation of Time.” Economic Journal, 75:493-517. Becker, Henk. 1997. Social Impact Assessment. London: University College Press. Benjamin, D., 1992. “Household Composition, Labor Markets, and Labor Demand: Testing for Separation in Agricultural Household Models.” Econometrica, 60:287-322. Bianchi, R. and S. Kossoudji. 2001. “Interest Groups and Organizations as Stakeholders.” Social Development Papers No. 35. Washington, DC: The World Bank. Binswanger, H. and J. Quizon, 1984. “Distributional Consequences of Alternative Food Policies in India.” World Bank, Agriculture and Rural Development Department. Discussion Paper No. 20. Binswanger, H. and J. Quizon, 1986. “Modeling the Impact of Agricultural Growth and Government Policy on Income Distribution in India.” World Bank Economic Review 1:103-48. Bolt R. and M. Fujimura, 2002. "Policy-based Lending and Poverty Reduction: An Overview of Processes, Assessment, and Options." Asian Development Bank Economics and Research Department Working Paper Series No. 2. 58

Carvalho, S. and H. White, 1997. “Combining the Quantitative and Qualitative Approaches to Poverty Measurement and Analysis: The Practice and the Potential.” Washington, DC: World Bank Technical Paper, No. 366. Cornia, A., R. Jolly, and F. Stewart, 1987. Adjustment with a Human Face. Oxford: Clarendon Press. Cox, D. and E. Jimenez, 1995. “Private Transfers and the Effectiveness of Public Income Redistribution in the Philippines,” in Public Spending and the Poor: Theory and Evidence, edited by Van de Walle, D. and K. Nead. Baltimore: Published for the World Bank by Johns Hopkins University Press. Christensen, L.R., D.W. Jorgensen, and L.J. Lau, 1975. “Transcendental Logarithmic Utility Functions.” American Economic Review 65:367-383. Davidson, R. and J. Duclos, 2000. “Statistical Inference for Stochastic Dominance and for the Measurement of Poverty and Inequality”. Econometrica, 68, No. 6:143564. Deaton, A. and J. Muellbauer, 1986. Economics and Consumer Behavior. New York: Cambridge University Press. de Janvry, A., M. Fafchamps, and E. Sadoulet, 1991. “Peasant Household Behaviour with Missing Markets: Some Paradoxes Explained.” Economic Journal, 101:14001417. de Maio, L., F. Stewart, and R. van der Hoeven, 1999. “Computable General Equilibrium Models, Adjustment and the Poor in Africa.” World Development, 27:453-75. Decaluwé, B., A. Patry, L. Savard, and E. Thorbecke, 1999. “Poverty Analysis Within a General Equilibrium Framework.” Montréal: Université Laval, Centre de Recherche en Économie et Finance Appliquées Working Paper No. 9909. Decaluwé, B., J.-C. Dumont,, and L. Savard, 1999. “Measuring Poverty and Inequality in a Computable General Equilibrium Model.” Montréal: Université Laval, Centre de Recherche en Économie et Finance Appliquées Working Paper No. 9920. Demery, L., 2000. “Benefit Incidence: A Practitioner’s Guide.” World Bank, Africa Region, Poverty and Social Development Group. Demery, L., M. Ferroni, and C. Grootaert, eds. 1993. Understanding the Social Effects of Policy Reform. Washington, DC: The World Bank. Dervis, K., J. de Melo, and S. Robinson, 1982. General Equilibrium Models for Development Policy. New York: Cambridge University Press.

59

Devarajan, S., and D. Go (with F. Charlier, A. Dabalen, W. Easterly, H. Fofack, A. Izquierdo, and L. Koryukin). 2001. “A Macroeconomic Framework for Poverty Reduction Strategy Papers, with an Application to Zambia. Washington, DC: The World Bank. Manuscript. Devarajan, S., W. Easterly, H. Fofack, D. Go, A. Izquierdo, C. Petersen, L. Pizzati, C. Scott, and L. Serven, 2000. “A Macroeconomic Framework for Poverty Reduction Strategies” World Bank manuscript. Devarajan, S., H. Ghanem, and K. Thierfelder, 1999. “Labor Market Regulations, Trade Liberalization, and the Distribution of Income in Bangladesh.” Journal of Policy Reform, 3:1:1-28. Devarajan, S. and S. Hossain, 1998. “The Combined Incidence of Taxes and Public Expenditures in the Philippines.” World Development, 26:963-77. Dollar, D. and A. Kraay, 2001. “Growth is Good for the Poor.” Washington, DC: World Bank Working Paper No. 2587. Dulamdary, E., M. Shah, and R. Mearns, with B. Enkhbat and L. Ganzaya. 2001. "Mongolia: Participatory Living Standards Assessment." National Statistics Office of Mongolia and The World Bank, Washington, D.C. Easterly, W. 1999. "The Ghost of Financing Gap: Testing the Growth Model Used in the International Financial Institutions." Journal of Development Economics, 60:423438. Egamberdi, N., P. Gordon, A. Ikhamov, D. Kandiyoti., and J. Shoerberlein- Engel, 2000. “Uzbekistan Agriculture Enterprise Restructuring and Development Program” in Social Assessment and Agricultural Reform in Central Asia and Turkey edited by A. Kudat, C. Keyder and S. Peabody. Finterbusch, K., J. Ingersoll; and L. Llewellyn, 1990. Methods for Social Analysis in Developing Countries. San Francisco: Westview Press. Foster, V. and C. Araujo, 2001. “Does Infrastructure Reform Work for the Poor? A Case Study from Guatemala.” The World Bank. [Unpublished] Gelbach, J. and L. Pritchett, 2000. “Indicator Targeting in a Political Economy: Leakier can be Better.” Journal of Policy Reform, 4:2:113-45. Gittinger, J. P., 1985. Economic Analysis of Agricultural Projects. Washington, DC: World Bank. Goldman, Laurence R. (Ed.). 2000. Social impact Analysis: An Applied Anthropology Manual. Oxford: Be rg Press.

60

Grootaert, C. and T. van Bastelaer, (eds). 2002. Understanding and Measuring Social Capital: A Multidisciplinary Tool for Practioners. Forthcoming. Washington, DC: The World Bank. Hammer, J., I. Nabi, and J. Cercone, 1995. “Distributional Effects of Social Sector Expenditures in Malaysia,” in Public Spending and the Poor: Theory and Evidence, edited by Van de Walle, D. and K. Nead. Baltimore: Published for the World Bank by Johns Hopkins University Press. Hammer, J. and A. Tan, 1989. “A Multimarket Model for Turkish Agriculture.” Washington, DC: World Bank, Agriculture and Rural Development Department WPS No. 285 Howes, S., 1993. “Mixed Dominance: A New Criterion for Poverty Analysis.” London School of Economics Suntory-Toyota International Centre for Economics and Related Disciplines Working Paper: DARP/2. Hunt, D. 1996. Process Mapping: How to Reengineer Your Business Processes, New York: John Wiley & Sons, Inc Ianchovichina, E., A. Nicita, and I. Soloaga, 2001. “Trade Reform and Household Welfare: The Case of Mexico”. Washington, DC: World Bank Working Paper 2667. Ironmonger, D. 1999. "An Overview of Time Use Surveys". Paper presented as part of the International Seminar on Time Use Studies. Ahmedabad, India, 7-10 December 1999. Jabara, C., M. Lundberg, and A. Sireh Jallow, 1992. “Social Accounting Matrix for The Gambia.” Ithaca: Cornell Food and Nutrition Policy Program Working Paper No. 20:1-45. Jalan, J. and M. Ravallion. 1997. “Spatial poverty traps?” World Bank Policy Research Working Paper 1862. Jalan, J. and M. Ravallion. 1999. “Are the Poor Less Well Insured? Evidence on Vulnerability to Income Risk in Rural China.” Journal of Development Economics, 58:1:61-81. Jensen, R., 1998. "Public Transfers, Private Transfers, and the Crowding-Out Hypothesis: Evidence from South Africa." Harvard University, Kennedy School of Government Research Working Paper, R98-08. Lampietti, J., A. Kolb, S. Gulyani, and V. Avenesyan, 2001. “Utility Pricing and the Poor: Lessons from Armenia.” Washington, DC: World Bank Technical Paper No. 497.

61

Lanjouw, P., M. Pradhan, F. Saadah, H. Sayed, and R. Sparrow, 2001. “Poverty, Education, and Health in Indonesia: Who Benefits from Public Spending?” Washington, DC: World Bank Policy Research Working Paper, No. 2739. Lanjouw, P. and M. Ravallion, 1995. “Poverty and Household Size.” Economic Journal, 105:1415-1434. Levinsohn, J., S. Berry, and J. Friedman, 1999. “Impacts of the Indonesian Economic Crisis: Price Changes and the Poor.” National Bureau of Economic Re search Working Paper No. 7194. Lockhart, C., 2001. Institutional Analysis: Russia Coal Case Study. Mimeo. López, R., J. Nash, and J. Stanton, 1995. “Adjustment and Poverty in Mexican Agriculture : How Farmers’ Wealth Affects Supply Response.” Washington, DC: World Bank Policy Research Working Paper No. 1494. López, R.1986. “Structural Models of the Farm Household That Allow for Interdependent Utility and Profit-Maximization Decisions.” in Agricultural Household Models: Extensions, Applications, and Policy by Singh, I; Squire, L; and Strauss, J. (eds). Baltimore: Johns Hopkins University Press for the World Bank . Lundberg, M. and L. Squire, 2002. “The Simultaneous Evolution of Growth and Inequality.” Forthcoming in Economic Journal. Lundberg, M., M. Over, and P. Mujinja, 2000. “Sources of Financial Assistance for Households Suffering an Adult Death in Kagera”. Washington, DC: World Bank Policy Research Working Paper No. 2508. Maack, J. 2001. “Scenario Analysis: A Tool for Task Managers” in Social Analysis: Selected Tools and Techniques, World Bank Social Development Papers, No. 36. Narayan, D. and P. Petesch, 2002. “An Empowering Approach to Poverty Reduction” in Voices of the Poor: From Many Lands edited by D. Narayan and P. Petesch. Oxford: Published for the World Bank by Oxford University Press. North, D. 1990. Institutions, Institutional Change and Economic Performance, New York:Cambridge University Press. Norton, A., B. Bird, K. Brock, M. Kakande, and C. Turk, 2001. A Rough Guide to PPAs: Participatory Poverty Assessment - An Introduction to Theory and Practice. London: ODI Publications. Orbeta, A. and M. Alba, 1998. “Simulating the Impact of Macroeconomic Policy Changes on the Nutritonal Status of Households.” Ottawa: Micro Impact of Macroeconomic and Adjustment Policies (MIMAP) Research Paper No. 21.

62

Pyatt, G. and J. Round (eds.), 1985. Social Accounting Matrices: A Basis for Planning. Washington, DC: The World Bank. Quah, D. and S. Durlauf. 1999. “The New Empirics of Economic Growth” in The Handbook of Macroeconomics edited by John Taylor and Michael Woodford. Radulescu, S. and M. Larionescu, 1999. “Social Assessment of Mining Restructuring in Romania”. World Bank Social Assessment. Mimeo. Rama, M. 2001. “The Gender Implications of Public Sector Downsizing: The Reform Program of Vietnam.” World Bank Policy Research Working Paper No. 2573. Forthcoming in the World Bank Research Observer. Ravallion, M. 1999. “Monitoring Targeting Performance When Decentralized Allocations to the Poor Are Unobserved.” World Bank Policy Research Working Paper 2080. Reinikka, R. and J. Svensson. 2002. “Explaining Leakage of Public Funds”. Centre for Economic Policy Research, Discussion Paper Series, No. 3227:1-45. Rickson, R., J. Western, and R. Burge, 1990. “Social Impact Assessment: Knowledge and Development”. Environmental Impact Assessment Review, 10:1-10. The World Bank. Robb, C. 2002. Can the Poor Influence Policy? Participatory Poverty Assessments in the Developing World. Washington, DC: The World Bank. Robillard, A.-S., F. Bourguignon, and S. Robinson, 2001. “Crisis and Income Distribution: A Micro-Macro Model for Indonesia.” Manuscript. Rutherford, M. 1994. Institutions in Economics: The Old and the New Institutionalism, New York: Cambridge University Press. Sahn, D. and H. Alderman, 1995. “Incentive Effects on Labor Supply of Sri Lanka’s Rice Subsidy,” in Public Spending and the Poor: Theory and Evidence, edited by Van de Walle, D. and K. Nead. Baltimore: Published for the World Bank by Johns Hopkins University Press. Shoven, J. and J. Whalley, 1992. Applying General Equilibrium. New York: Cambridge University Press. Singh, I., L. Squire and J. Strauss, 1986. Agricultural Household Models: Extensions, Applications, and Policy. Baltimore: Johns Hopkins University Press. Squire, L. and H. van der Tak, 1975. Economic Analysis of Projects. Baltimore: Published for the World Bank [by] Johns Hopkins University Press.

63

Stone, R., 1954. “Linear Expenditure Systems and Demand Analysis: An Application to the Pattern of British Demand.” Economic Journal 64:511-527. Strauss, J., 1984. “Joint Determination of Food Consumption and Production in Rural Sierra Leone: Estimates of a Household-Firm Model.” Journal of Development Economics, 14:77-103. Subramanian, S. and A. Deaton, 1996. “The Demand for Food and Calories.” Journal of Political Economy 104:133-62. Thorbecke, E. and H. Jung, 1996. “A Multiplier Decomposition Method to Analyze Poverty Alleviation.” Journal of Development Economics, 48:2:279-300. Timmer, C. P., W. Falcon and S. Pearson, 1983. Food Policy Analysis. Baltimore: Johns Hopkins University Press. Torero, M. amd A. Pascó-Font, 2001. “The Social Impact of Privatization and the Regulation of Utilities in Peru.” Helsinki: UNU-WIDER Discussion Paper 2001/17. Van de Walle, D., 2002. “Choosing Rural Road Investments to Help Reduce Poverty.” Forthcoming in World Development, 30(4). Van de Walle, D., 1998. “Assessing the Welfare Impacts of Public Spending.” World Development 26:3:365-379. Yitzhaki, S. and J. Slemrod, 1991. “Welfare Dominance: An Application to Commodity Taxation.” American Economic Review, 81:3:480-496. National Management Consultants. 2001. Second Annual Report of the Kecametan Development Project. Jakarta, Indonesia. Mimeo. World Bank, 2002a. “Monitoring and Evaluation: Some Tools, Methods, and Approaches.” Operations Evaluation Department. World Bank. 2002b. “Bosnia and Herzegovina Local Level Institutions and Social Capital Study: Findings and Recommendations.” ECSSD. World Bank. 2001. Malawi Public Expenditures: Issues and Options. Report No. 22440 MAI. Washington, DC. World Bank. 2000a. World Development Report 2000/2001: Attacking Poverty, New York: Oxford University Press. World Bank. 2000b. Modeling Pensions Reform: The World Bank’s Pension Reform Options Simulation Toolkit. HDNSP. Washington, DC.

64