Workshop Session Ii: Public Expenditure Financial Accountability (pefa) Assessment

Use of PEFA assessments

ICGFM Conference Miami – May 21, 2009 PEFA Secretariat

What can countries use the PEFA Framework for?  Harmonize information needs •

Common information pool

 Inform PFM reform formulation and priorities  Compare to and learn from peers  Monitor results of reform efforts 2

What can countries use the PEFA Framework for?  Harmonize information needs  Inform PFM reform formulation and priorities  Compare to and learn from peers  Monitor results of reform efforts 3

PEFA reports for reform formulation (1) 





4

PEFA report is one – out of several - inputs – Identification of main strengths and weaknesses – and potential impact on budgetary outcomes – Other factors: political economy, institutional, culture, constitution/legal, resources, capacity at entry Ownership means government decisions on priorities – Government to consider all factors in deciding priorities – Reform dialogue with donors to allow ample space Do not use Indicator scores simplistically – A low score is not sufficient justification for high priority reform – Other factors: relative importance of subject, complexity/timeframe for improvement, interdependence with other elements, weakest links

Coverage of PFM Performance Report in the PFM Reform Cycle Implement PFM reforms

High level performance overview

Formulate PFM reform program

PFM-PR Identify Identify main main PFM PFM weaknesses

Recommend PFM reform PFM reform measures measures Investigate underlying causes

5

PEFA reports for reform formulation (2)

6



Complementary analysis to PEFA required – Detailed analysis of underlying causes needed for formulation of detailed action plan – Limit such analysis to priority areas – Drill-down tools – some exist, others under development



PEFA is preparing guidance on use of PEFA reports in reform formulation – Stock-taking of conceptual approaches used – Country case studies illustrating good practice

Purposes of Standard Diagnostic Tools PER

Review of public expenditure policies & budgetary outcomes Review of political incentives High level overview of PFM performance Identification of PFM strengths/weaknesses In-depth analysis of capacity factors Recommendations for reform Assess risk to public funds Track progress over time

CFAA CPAR Fiscal TA ROSC report

FRA

PFMPR

X X

X X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X *

*

X

X

X *

X

* if performance indicators are incorporated

7

Country case - Norway  Findings of Norad-managed self-assessment presented to OECD-DAC in December 2007  The assessment showed low scores for seven areas of PFM system performance  Ministry of Finance reaction: – Weaknesses in procurement practices and follow-up to external audit findings need to be addressed. – Three areas of low scoring not considered priority at present (Multi-year program/sector budgeting, Limited extent of internal audit, no consolidated overview of risks from autonomous agencies and public corporations) – Two indicators scored low but are municipal responsibilities; central government will not get involved.

8

What can countries use the PEFA Framework for?  Harmonize information needs  Inform PFM reform formulation, priorities  Compare to and learn from peers  Monitor results of reform efforts 9

PEFA reports for country comparison (1) 

PEFA Framework developed for in-country use – –



Wide interest in country comparison using aggregation – – –



Researchers - learning on determining factors Donors - aid allocations Governments - peer learning

Aggregation requires three decisions – – –

10

‘Summary assessment’ provides nuanced overview of strengths and weaknesses as basis for reform prioritization No method given for arriving at one measure for ‘overall performance level’

Conversion from ordinal to numerical scale Weighting of indicators (generally and by country) Weighting of countries (for country cluster analysis)

PEFA reports for country comparison (2) 

No scientifically correct or superior basis for deciding conversion and weights exists – –



Preferred method of country comparison: – –



A nuanced comparison of two assessment reports Consider country context, ensure comparison of like with like

In case aggregation is desired: – – –

11

Each user takes those decisions on individual opinion PEFA program will not endorse any particular method

Be transparent on aggregation methods used Discuss reasons for choice Sensitivity analysis to illustrate impact on findings

What can countries use the PEFA Framework for?  Harmonize information needs  Inform PFM reform formulation and priorities  Compare to and learn from peers  Monitor results of reform efforts 12

Repeat Assessments  As

at March 2009, thirteen ‘real’ repeat assessment undertaken  Expected to become common from 2008/2009 onward i.e. three years after the first series of baseline assessments

What we want to determine?  Specific

changes in system performance – –

What has changed? How much?

Comparison of Indicator Scores  Indicator

scores will provide a crude overview of changes over time, but … – –

Dimensions may change differently Performance may not always change enough to change the score (use of arrow)

 So

more detailed explanation is required

Other Possible Reasons for Change in Scores  Changes

in definitions  Improved availability of or access to information  Different information sampling & aggregation  Scoring methodology mistakes in previous assessment

Reporting on Progress Made  Explain

all factors that impact a change in rating indicator-by-indicator  Identify the performance change  Ensure that any reader can track the change from the previous assessment – what performance change led to the change in a rating

Reporting on Progress Made Indicator Score Score Performance change 2005 2007

Other factors

PI-1

C

B

Performance appears improved based on 2005: last deviations 6%, 11%, 18% 2007: 5%, 11%, 6%

Not clear if all external project funds were excluded from data for 2005 assessment but probably insignificant issue.

PI-4 (i)

A

C

Performance change is uncertain, despite reported arrears increase from 1% in 2005 to 6% in 2007.

2005 assessment used data on pending payment orders only, not overdue invoices.

Organizational Requirements  Performance

tracking to be clearly reflected in the TOR  Use of same assessment team desirable but rarely possible  The lead agency of the previous assessment, if different, should assist with access to previous assessors’ notes  … and be part of the reference group for the repeat exercise

Country case - Mozambique  PEFA incorporated in PFM reform monitoring system – baseline 2005, repeat assessment 2007, planned follow-up 2010  Important performance improvements 2005-2007 in budget execution – revenue administration, cash management, internal controls  Improvements resulted from: – Reforms already well under way in 2005 (e.g. IFMIS/SISTAFE and revenue administration) – Small managerial/admin changes (including quick-wins identified on the basis of 2005 assessment) – New reform initiatives in pay-roll control (identified from the 2005 assessment as an important neglected area of reform)

20

Thank you for your attention