Thinken Thin

Thicken or Thin? The Consequences of Performance Accountability for Relationships between CSOs and Those They Serve Presentation to the Evaluation & Program Effectiveness Working Group (InterAction), Washington, DC 17 October 2008 [updated December 2008] Lehn M. Benjamin Assistant Professor Nonprofit Studies Department of Public and International Affairs

George Mason University [email protected]

Take Away • CSO long-term outcomes not only require technical competence but also requires relational ability. – Technical Competence: Delivering effective products and services – Relational ability: Building positive mutual relationships with those served

• Performance Accountability models do not recognize this relational work and consequently threaten to thin CSO relationships with beneficiaries: – Assumptions about Risk. Performance Measurement assumes transferring risk is the best way to guarantee results – Ambiguities about Role. Performance Measurement Training Material sends mixed messages about the role of those 2 served

My Research: Guiding Questions • 1) How do current efforts to ensure CSO accountability support or constrain their potential contribution to democracy? • 2) How can accountability systems better secure CSO potential contribution to democracy? • 3) Do CSOs merit distinct accountability systems, compared to public agencies or for-profit firms? 3

Research: Performance Accountability Accountability systems reward certain actions and discourage others Performance measurement promises to increase CSO effectiveness by… – By Ensuring Accountability. Put in maximum effort if held accountable for results. – By Encouraging Learning. Change routines in response to surprising data.

• Professional publications, training material consistently emphasize dual purpose: accountability and learning 4

Research: Key Ideas Civil Society Organizations are distinct from for profit firms and government agencies Civil Society Organizations can contribute to a democracy by playing roles and encouraging practices not well supported elsewhere but they can also undermine it Assumption: Their contribution lies with their relational work

5

Relational Work We don’t have a clear understanding of this work. Many Ideas help: • Social Capital (Putnam). • Free Spaces (Mansbridge, Skocpol, Evans and Boyte) • Home places. Development of Public Voice (Belenky, other womanist writers) • Listening as Everyday Social Policy (Forester, Barber, Arendt, Coles) • Relationships to accomplish tasks and tasks to build relationships (MacMurray) 6

Relational Work Relational work Establish a relationship so person comes back; so person is able to receive services/assistance Build a relationship so org can better respond to needs/experiences; so assistance is tailored/appropriate to individual Build a relationship so organization can better represent and advocate with and/or on behalf of those served Build a relationship to support a person in realizing their own leadership and capacity for change Build relationships to create strong community organizations that can take 7 action around problems over time

Thicken or Thin?

Thicken • Performance Accountability can Thicken relationships by including measures that capture relational work – obtain community input into work, – build capacity of local community groups to take collective action, – Identifies critical needs and advocates with/on behalf of community.

• If organizations are accountable for these results can motivate greater attention to these relationships. But measurement alone is not the answer… …need to look at how risk is perceived and managed in PM …need to look at how PM may change these relationships not just measure what they already do 9

Thin

RISK Performance Accountability assumes transferring risk is the best way to guarantee results

Risk Risk is the probability of gain or loss associated with a decision (Berstein 1996; March and Shapira 1987) Manage Risk by mitigating, avoiding, transferring or accepting

Performance Accountability • Involves using results based incentives (outputs or short-term outcomes) to transfer risk from one entity to another

11

Risk Principal-agent theory: • A relationship where one party, the principal, is dependent on another, the agent, for desired results Principal

Principal

+/-- Agent

Agent

+/--

Results

Results

12

Risk Principal-agent theory: • A relationship where one party, the principal, is dependent on another, the agent, for desired results

Funder

Funder

CSO

CSO

+/-Those Served

Results

Results

Those Served

+/-+/--

13

RISK Purpose of relationship s (McMurray 1961/Fieldin g 2001)

Funder’s Risk Management Strategy

Possible Implications for Nonprofit Practice

•Tasks drive relationships •Relationship s built to accomplish tasks

Funder transfers risk so the nonprofit bears at least some risk for results

1. Motivated nonprofits to achieve results 2. Lead nonprofits to take courses of action with more certain results

•Relationship s drive tasks •Tasks accomplishe d to build relationships

Funder willing to accept technical risks to support nonprofit relational work

1. Offer nonprofits the space to attend to relationships 2. Offer nonprofits the space to take courses of action with less certain results 14

RISK • Failure to achieve short-term results may not be about nonprofits learning the technical task, uncontrollable environmental circumstances or changing course but rather a strategic choice to accept greater risks for specific types of work. • Efforts to ensure effective nonprofits need to find a balance between accountability systems that focus on technical competency and relationship sensitivity. • The concept of risk can help us think more clearly about choices and trade-offs 15

Thin

ROLE Performance Measurement Training Material sends mixed messages about those served.

Why look at Resource Material to answer this question? • Analysis of commonly referred to and used resource materials on PM targeted to nonprofits • Documents as one carrier of normative messages and if these messages are coherent and forceful more likely to change org. practice ((Meyer and Rowan 1977; Powell and DiMaggio 1991; Scott 2002) • One phase of a larger study 17

Preliminary Findings

Caveats

Preliminary Findings performance measurement material encourages different degrees and types of beneficiary influence, ranging from deciding what nonprofit outcomes were most important, to improving transparency by reporting on results to beneficiaries, to completing customer satisfaction surveys. performance measurement material reinforces the ambiguous standing of CSO beneficiaries vis a vis the general public, donors and nonprofit staff/board and volunteers.

19

RELATIONAL Work

Dimension

Connection (attraction Establish a relationship so and retention) person comes back; so person receives services/assistance Responsive (satisfaction)

Build a relationship so org can better respond to needs/experiences; so assistance is tailored/appropriate to individual

Representation/Advoc Build a relationship so acy (policy) organization can better represent and advocate with and/or on behalf of those served Capacity Building – Individual Leadership (outcome 1)

Build a relationship to support a person in realizing their own leadership and capacity for change

Capacity Building – Strong Organizations (outcome 2)

Build relationships to create strong community organizations that can take action around 20 problems over time

Preliminary Findings

Overall:

• Resource material reframes the relationship between nonprofits and those they serve

Developing the System: • 1. Shadow. walk in their shoes • 2. Stakeholder. Input into OM system • 3. Constituent. OM as a way to enable community to ensure CSOs are accountable; to rebalance accountability relationships

Measuring the Work: • 4. Customer. Satisfaction and Attraction and Retention • 5. Target. Focusing on change in response to services • 6. Hero. Journey of the participant 21 and the role of the org in that journey.

Concluding Questions • What are the implications of reasserting the agency of those served in the image of customer? • We know from other studies that nonprofits negotiate stakeholder accountability expectations through their primary relationship with the communities they serve . Does the use of PM material support, constrain or simply inconsequential in this work? • What are the implications of measuring relational work as an instrumental outcome? • PM is intended to improve the technical work of nonprofits and to strengthen the relationship with its key stakeholders in the accountability environment, but does this 22 relational work needs more attention?

Take Away • CSO long-term outcomes not only require technical competence but also requires relational ability. – Technical Competence: Delivering effective products and services – Relational ability: Building positive mutual relationships with those served

• Performance Accountability models do not recognize this relational work and consequently threaten to thin CSO relationships with beneficiaries: – Assumptions about Risk. Performance Measurement assumes transferring risk is the best way to guarantee results – Ambiguities about Role. Performance Measurement Training Material sends mixed messages about the role of those 23 served.

Additional Resources • Benjamin, Lehn M. 2008 "Measurement Technician, Capacity Builder or Risk Manager: Evaluator's Role in Accountability Environments" Evaluation: International Journal of Theory, Research and Practice. 14(3): 323-343. • Benjamin, Lehn M. 2008. "Bearing More Risk for Results: The Implications of Performance Accountability for Nonprofit Relational Work." Administration & Society, 39(8): 959-983. • Campbell, David. 2002. “Outcomes Assessment and the Paradox of Nonprofit Accountability.” Nonprofit Management & Leadership 12(3): 243259. • Cutt, J., & Murray, V. (2000). Accountability and effectiveness evaluation in nonprofit organizations. London: Routledge. • Ebrahim, Alnoor. 2006. “Accountability Myopia: Losing Sight of Organizational Learning.” Nonprofit and Voluntary Sector Quarterly, Vol. 34, No. 1, 56-87 (2005) • Ebrahim, A. (2003b). NGOs and organizational change: Discourse, reporting and learning. Cambridge, UK: Cambridge University Press.

24

Additional Resources • Edwards, Michael and Gita Sen. 2000. “NGOs, Social Change and the Transformation of Human Relationships.” Third World Quarterly, 21(4): 605616. • Gao, Chao and Juliet Musso. 2007. “Representation in Nonprofit and Voluntary Organizations: A conceptual Framework.” Nonprofit and Voluntary Sector Quarterly, 36(20): 308-326 • Kilby, Patrick. 2006. “Accountability for Empowerment: Dilemmas Facing Nongovernmental Organizations.” World Development, Vol 34 (6): 951-963. • Lewis, David. 2007. “NGOS and the management of relationships.” Chapter 6 in Management of Non-Governmental Development Organizations. 2nd Edition. London: Routledge. • Ospina, S. W. Diaz and J. F. O'Sullivan. (2002). “Negotiating Accountability: Managerial Lessons from Identity-based Nonprofit Organizations.” Nonprofit and Voluntary Sector Quarterly, 31(1), 5-31 • Speckbacher, G. (2003). “The economics of performance management in nonprofit organizations.” Nonprofit Management & Leadership, 13(3), 267-279. 25