Learning through evaluation

(1)

Master of Science in Social Work 91-120 hp Social Work Programme

LEARNING THROUGH

EVALUATION

A CASE STUDY ON ORGANIZATIONAL

LEARNING WITHIN THE

NON-GOVERNMENTAL ORGANIZATION UYDEL

(UGANDA YOUTH DEVELOPMENT LINK) IN

KAMPALA, UGANDA

ELLEN FÜRST

ANNA MEITON

(2)

ABSTRACT

A CASE STUDY ON ORGANIZATIONAL

LEARNING WITHIN THE

NON-GOVERNMENTAL ORGANIZATION UYDEL

(UGANDA YOUTH DEVELOPMENT LINK) IN

KAMPALA, UGANDA

ELLEN FÜRST

ANNA MEITON

Fürst, E & Meiton,A. Learning through evaluation. A case study on organizational learning within the non-governmental organization UYDEL (Uganda Youth Development Link) in Kampala, Uganda. Master of Science in Social Work 30

hp. Malmö University: Faculty of Health and Society, 2013.

In a globalized social welfare market, where international and national NGOs influence social service delivery in community’s worldwide, the demands on mutual exchange, shared responsibilities, and transnational collaborations has increased. However, there are some challenges that NGOs face in this work. Lack in coordination of donor funded projects have enabled overlapping

responsibility in service provision and a rivalry between NGOs in retrieving funds, witch in it turn has hindered the communication of knowledge between NGOs in the social work field. To create good practice, the basic assumption is that NGOs have to be able to learn from past experiences, a process known as

organizational learning. One way of realizing organizational learning, is through evaluation. The success of an evaluation is further closely linked with the

utilization of the knowledge gained. The purpose of this study is to discover how knowledge gained through an evaluation made by the Swedish organization IOGT-NTO, has been used within the non-governmental organization of UYDEL, and how this knowledge is communicated to the organizations stakeholders. To realize our purpose, we conducted a field study within UYDEL in Kampala, Uganda. The material, collected through interviews and observations, was later transcribed and analyzed, in relation to our theoretical framework. The theoretical framework includes basic theories on organizational learning, evaluation and communication. Our results indicate that issues that initially introduced

themselves in this evaluation, later was reintroduced in our material, and thus still remained a problem within UYDEL. What seemed to hinder organizational learning was the organizations over-dependency on its donors, which affected its sustainability, and could be seen as a consequence of the misplacement of

(3)

Keywords: Communication, Evaluation, International social work, NGO,

Organizational learning, Sustainability

ACKNOWLEDGEMENTS

In August 2012 we got the opportunity, thanks to SIDA1_{and Malmö University,}

to travel to Kampala, Uganda to conduct a minor field study within the organization of UYDEL2_{. Minor field studies (MFS) is a SIDA financed}

scholarship, which aim to contribute to Swedish student’s awareness about developing countries and development work. This experience added to our understanding of the complexity that is modern day social work, and the

similarities and differences in social work practices in Sweden and Uganda. We would like to thank SIDA for providing us with the means to travel and work in Uganda, and UYDEL staff, youth clients and others, for showing us great hospitality and support in our work. We especially would like to extend our thanks to our field supervisor Mr. Rogers Kasirye for allowing us to conduct our field study within UYDEL and helping us better understand the Uganda society. We also would like to extend our thanks to our Malmö University supervisor Jonas Christensen for offering great support and leading us in trying to understand the subject of social work, in an ever-changing environment.

Ellen Fürst & Anna Meiton, January 2013

1_{Swedish International Development cooperation Agency} 2_{Uganda Youth Development Link}

(4)

INTRODUCTION

With this study we hope to gain the curiosity of people that are interested in the development of the social work field, but also social work students that are interested in social work in an international context. Hopefully our work will also be of interest of Swedish aid organizations working internationally, and the organization of UYDEL, where we have conducted this field study.

We would like to highlight some of the problematic issues in international development work, and especially in relation to NGOs. We find that social work in a globalized world is an interesting topic, and we would like to understand what kind of implications the internationalization of social work and social service delivery has on the community. We want to contribute to a discussion about the future of social work in an international context, but also make a contribution to the organization of UYDEL, by sharing their experience of social work practice in a changing environment.

We think it is important, as social work students entering the professional field, to always persist in learning and reevaluating social work theory and practice. That is why we think it is vital to also keep questioning conventional truths about our profession. Reevaluating social work also means taking part of discussing the future of the social welfare, both in Sweden and internationally. We recognize that our society is changing, and we think it is important for all practitioners to

(5)

ABBREVIATIONS

The following passage provides an overview of the abbreviations that are used throughout this study, and that are further introduced within each chapter.

NGO: Non-Governmental Organization

SNGO: Southern Non-Governmental Organization NNGO: Northern Non-Governmental Organization HSO: Human Service Organization

UYDEL: Uganda Youth Development Link UN: United Nations

SIDA: Swedish International Development cooperation Agency UNICEF: United Nations Children’s Fund

WHO: World Health Organization ILO: International Labor Organization

IOGT–NTO: Independent Order of Good Templars- Sobriety Movement CAN: Swedish Council for Information on Alcohol and Other Drugs PM&E: Participatory Monitoring and Evaluation

(6)

1. BACKGROUND

Promoting social protection and justice for vulnerable individuals or groups in society can be seen as a vital task within the social work profession. According to author Lynne M. Healy (2008) a prerequisite for achieving change in society is to use knowledge gained within social work practice. Modern day social work professionals operate in a globalized social welfare market, where international organizations influence social work policy and practice. This means that

international organizations play an important role in shaping the future of social welfare globally. This is also why it is of great importance for social work

professionals to understand not only this development, but the consequences it has on social work practice on a national and international level (ibid).

1.1 Social work in an international context

Healy (2008) argues that in order to discuss social work in an international context it is essential to understand the basic scope of globalization and global interdependency. Globalization can be described as “A process of

global integration in which diverse peoples, economies, cultures and political processes are increasingly subjected to international influences”

(Healy, 2008 p.26). The basic assumption is that a transnational exchange of production, information, ideas, authority and people generates global interdependency. Globalization and an increasing interdependency has specifically affected and shaped social welfare in different societies nationwide, most importantly through a vast increase in migration. In the field of social work this has led to greater demands on mutual exchange, shared responsibilities, and transnational collaborations, both in relation to organizations working internationally, but also on a national level (ibid). The idea of a globalized social work field, where the interventions and social problems of one society affects other society’s internationally, contribute to the perception that there is a shared social responsibility that cuts across national borders, something that in short can be described as

international social work. There are several different explanations as to

what international social work is. It can be identified as professional exchange of knowledge and experience between social workers working within an international programme, or as development work on an

international level. One basic assumption is that international social work is carried out by intergovernmental agencies (such as the UN) and includes governmental agencies and private or non-governmental

organizations (NGOs) which are in some way involved in developing social welfare internationally through programmes and interventions (Healy, 2008).

1.2 What are NGOs?

David Lewis (2007) recognizes that the term non-governmental organization originates from the international citizen organizations that were funded by United Nations after the Second World War. According to the UN definition any private, non-profit organization that is not part of a government can be stated as a NGO. With this said, there is not only one definition of the term NGO, for example some organizations that are in fact income generating can be defined as NGOs, as well as organizations that are working closely with the government. NGO’s have

(9)

specifically been playing an important role in the growth process of developing countries, including Uganda (ibid).

1.2.1 Challenges in NGOs

Roger Riddell (2007) explains that the merge of NGOs as the main contributors of social service has been subdued to extensive criticism, particularly in relation to international donors supporting community based NGOs in service delivery in development countries. Lack in coordination of donor funded projects have enabled overlapping responsibility in service provision and a rivalry between NGOs in retrieving funds, witch in it turn has hindered the spread of good

practices between NGOs in the social work field. The fragmentation of the social welfare due to the non-cooperation between donors and stakeholders can be seen one of the major problems that is evident within development work. This has also in many ways affected the quality of service that the beneficiaries in these

organizations receive, for example a shifting quality in service on a community level, due to lack of standardization of social work practice within the sector (ibid).

NGOs are often faced with specific challenges. One of them is that almost all of these organizations are donor dependent; this means that the space that enables an NGO to act is closely linked with the donors perception of what is needed to create a successful project. The assumption is that the NGOs have a close link to the local community, which makes them better suited for delivering quality service, based on the actual needs in a community (Riddell, 2007). Anthony J. Bebbington, Samuel Hickey and Diana C. Mittlin (2007) argue that since most NGOs rely on donors for funding it is crucial that they maintain good

relationships with these partners and at the same time also keep the commitment of delivering good quality of service to their clients. This may be problematic because NGOs often have a tendency to focus mainly on maintaining strong relationships with the donor community, which could divert the attention away from the needs of their target groups (ibid).

In discussing issues on NGOs, it is central to also discuss the concept of

accountability in relation to NGOs commitment to its stakeholders, according to

author David Fishel (2008). The basic notion of accountability in relation to the NGO context is that an organization should be held accountable for its actions towards its stakeholders, including staff members, donors, clients and others that are affected by the organizations interventions. To achieve accountability, the organization has to make sure that their mission and goals are realized. The process of creating accountability in an organization can be realized through assessing an organizations intervention and spreading information about the organizations past, present and current actions to its stakeholders. NGOs have assumed an important role in the delivery of social service and policy making; this also means that the demands on accountability in these organizations have

increased. Evaluation of projects and programmes within an organization can be seen as an important step in creating accountability. With this said, many NGOs often lack in an efficient evaluation and monitoring of programmes and projects, which can become and obstacle in achieving accountability (ibid).

1.3 Evaluation: definition and use

James Cutt and Vic Murray (2003) define an evaluation as a method of assessing the outcome of interventions executed in an organization, which should serve as a base for planning and decision-making regarding future interventions. They

(10)

further argue that we do not always know what consequences a project will carry in the long run, and the ideal evaluation would be able to show, without doubt, that the projects goal has been attained. However, what has become evident is that the impact of NGO projects and programmes are difficult to measure. The goals are often too broad, diffuse and long-term, which make it difficult to transform knowledge into practice (ibid).

Bruce Britton (2005) points out another problem that affects NGOs, which is that evaluations seldom serve as a base for decision-making, implementation and planning within these organizations. This seems to be a reoccurring problem even though many NGOs put great resources into evaluating their activities, which ultimately stands in the way of transforming information gained through evaluation, into practice (ibid).

When evaluating, there are many different kinds of technical methods to use to gain information, but not all of them are equally valid or reliable. In the case of the NGO sector this could be a problem, regarding the fact that most NGO does not have the time, money or expertise to develop and interpret these methods of evaluation. Another problem in the NGO sector is that lack of standards makes it difficult for the organizations to interpret data. This means that it would be difficult for an NGO to understand whether an evaluation indicates success or failure. In addition to this, there are almost no general standards or norms within this sector, which makes it difficult to compare evaluation results with other non-governmental projects and programmes. An evaluation is pointless unless it cannot serve as a base in an organizations decision making. To make a decision, the organizations have to be able to understand the results of their interventions and the impact they have on their beneficiaries. This is why it is important that every evaluation made include explicit models, which declare why the evaluation is being made, who are involved, what is being measured, how the data will be

interpreted and how the knowledge gained can be used within the organization

(Cutt & Murray, 2003).

1.4 Introducing the Ugandan society

Located in East Africa, neighboring Tanzania, Congo, Rwanda, Kenya and South Sudan is the country of Uganda3_{, a former British colony which has been tainted}

by political turmoil and un-equality. Years of dictatorship and civil war have affected the Ugandan society and have had a great impact on the social welfare of its people. The future of Uganda, one of Africa’s fastest growing economies, comes with a promise of democracy, freedom for individuals and increasing civil participation in the development of the country. In spite of the countries recent success, there are still some challenges that the country faces on its road to development. Ugandan society suffers from lack of employment opportunities, social inequalities, corruption and stagnation in health related indicators and weakness in the social sector management and administration. A weak social welfare system have had many implications on the Ugandan society, one

consequence of this is the merge of the third sector and NGOs in the delivery of social service to the Ugandan people (African Economic Outlook, Uganda, 2011). According to United Nations Population Fund´s report State of Uganda

Population Report (2011), almost half of the Ugandan population, seventeen

(11)

million people, is under the age of eighteen years. The increase in population growth affects the country’s ability to secure the social welfare and protection for young people (ibid). Sarah Okwaare and Jennifer Chapman (2006) recognizes, in the book NGO accountability - Politics, Principles & Innovations, that as a response to this population growth the numbers of NGOs working within the social work field have gone from 160 in the mid 1980's to 5,200 in 2004. Today the NGO sector in Uganda occupies more than 230,000 people (ibid).

The increase of NGOs can also, according to Richard Nuwamanya Butanmanya (2012), be seen as a consequence the efforts that were put in by international agencies during the beginning of 1980s, to provide social service in developing countries. This can be explained by a paradigm shift in international aid

development efforts towards investing and building capacity in NGOs working on a local level, which was something that was not offered by government during this period of time. The idea was that organizations working within the communities had a better perception of the needs of the beneficiaries, and thus could satisfy these needs in a more sufficient way (ibid). According to Riddell (2007) some of the important international donors and stakeholders that support NGOs in

developing countries include SIDA, UNICEF and WHO. These stakeholders also embody a substantial contribution to the social welfare in the Ugandan society (ibid).

1.4.1 Vulnerable children and youth in Uganda

Author and executive manager of UYDEL (Uganda Youth Development Link) Rogers Kasirye (2012) explains that the Ugandan government, in an effort to promote and protect the rights of vulnerable children and youth, has created an enabling environment for NGOs to deliver social service within this field.

Vulnerability can be described as a situation where "lack of security, susceptibility to risk and/or exploitation" (Kasirye, 2012 p.1) manifests itself in the life of an

individual. In Uganda, children and youth that are involved in child labor, trafficking, without education or living in poverty can be defined as vulnerable. With a growing population of children and youth there is a need to strengthen social protection within these groups, a challenge that have been proven difficult for various reasons, one being a lack in resources allocated towards these issues. Also, lack in coordination and collaboration between social protection providers have hindered this progress. Insufficient documentation in organizations working within this field has made it difficult to grasp their capacity and impact on society. This has also been standing in the way of collaboration, and thus the development of social protection among vulnerable children and youth in Uganda (ibid).

1.4.2 Uganda Youth Development Link

With the purpose of empowering vulnerable children and youth, and raising awareness on issues that affect young people in Ugandan society, the non-governmental organization of UYDEL was founded. The organization is

managing four main projects; child rights protection, HIV prevention, alcohol and substance abuse and adolescent sexual and reproductive health. The projects are implemented through five drop-in centers and outreach posts located in Kampala. The programmes target young people between ages 10-24 years as primary beneficiaries as well as parents, local leaders, teachers and other community members as key stakeholders in these different projects. UYDEL is a part of a vast network of national and international social work organizations and has

implemented a number of projects financed by different international as well as national organizations including: IOGT–NTO, UNICEF (The United Nations

(12)

Children's Fund), ILO (International Labor Organization), WHO (World health organization) among others (UYDEL Annual Report, 2011).

UYDEL is a part of a bigger context, a vast international social work field, where they can contribute to other organizations through communicating their

experience but also receiving knowledge from other actors within the field, and especially those who serves as important contributors to the organizations future and development. This was the case for the Swedish sobriety organization IOGT-NTO, one of UYDELs former donors, contributing not only with funds, but also with knowledge in the shape of evaluation and development workshops.

1.5 Knowledge gained

In October-December 2009 an evaluation4_{on the UYDEL Street Children Project}

was carried out by IOGT-NTO representatives Sven Persson and Mathias Kimiro (one of UYDELs main stakeholders). To further understand the concept of evaluation and monitoring, we have contacted the evaluators from IOGT-NTO, read and analyzed the material and identified some important issues that could be seen as relevant in this matter. The purpose of the evaluation was to collect information about the UYDEL Street Children Project results and impact on other prevention levels and help to form the future development of the project. The suggestions made in the evaluation include developing a dissemination strategy to spread the programmes to partners and other NGOs. To achieve this, they propose that UYDEL form a functional monitoring and evaluation plan. The monitoring

and evaluation process is aimed at helping the organization in setting priorities, providing baseline information, identifying problems when they appear,

eliminating unnecessary activities and redirecting resources. The IOGT-NTO representatives recognized that this would allow UYDEL to replicate successful projects and avoid making the same mistakes, to increase the accountability in their projects (ibid).

In the mentioned evaluation (2009) the IOGT-NTO representatives also identify that a sustainable organization plan and a review of the organization, including management and board, could create a more transparent management system. This means that evaluation and monitoring is important tool that should be used

throughout the whole of the organization. IOGT-NTO recognizes that UYDEL has good credibility as an NGO, and that many people and other organizations come to learn from their experiences. This is why UYDEL's activities could be extended locally, nationwide and internationally to build strong networks and partnerships with both national and international partners. To enable this, the organization has to be able to understand and interpret the results of the evaluation. In order to use the knowledge gained, the organization has to develop a functioning

communication strategy. This could also serve as an important base for exchange of knowledge and experience between local and international actors (ibid). In 2012 the IOGT-NTO evaluation was followed up by former IOGT-NTO representative Sven Persson, in collaboration with Swedish Council for

Information on Alcohol and Other Drugs (CAN). A development work shop was initiated by UYDEL in October 2012, with the purpose of assessing the

organizations capacity and progress.

4_{Street Children Project Evaluation Report. IOGT_NTO Movement International Institute (III) Regional}

(13)

2. PURPOSE AND RESEARCH QUESTIONS

The purpose with this study is to discover how the knowledge gained through the IOGT-NTO evaluation has been used within the organization of UYDEL, in relation to its staff members, beneficiaries and other external actors. Our aim is to contribute to an understanding about how human service organizations

communicate knowledge and experience to their stakeholders, working within the social work field. We want to find out how general recommendations from an evaluation made, can be adapted and translated into practice within UYDEL. This study is based on two research questions that we aim to answer, as seen below;

- How does the non-governmental organization UYDEL use the knowledge gained through the IOGT-NTO evaluation?

- How does the organization communicate and exchange knowledge and experiences with other organizations and stakeholders working within the field of social work?

3. PREVIOUS RESEARCH

In the publication Non-governmental Organization and Development, David Lewis and Nazneen Kanji (2009) explains how the NGO sector gradually became subject for research in the late 1980's, as the NGOs role in development work became more central. It is an interdisciplinary field of scholarship, where especially economist, sociologists, political scientist and anthropologists have been studying the whereabouts of the NGO world. Even though the NGO sector have been discussed and researched on from so many different angels, there is still a challenge to fully comprehend the phenomena that is non-governmental

organizations. As mentioned in the background, the diversity of the NGO landscape makes it hard to generalize on this subject (ibid).

Keeping this in mind, the following passage aims to provide an overview of previous research in the field of social work, with specific focus on the

development of NGOs in an international context. In presenting this material we have chosen to present publications that can be directly related to the study purpose and research questions, which include discussions on the use of evaluation within the NGO sector and the spreading of knowledge across organizational boundaries.

3.1 NGOs and international social work

As we have discussed NGOs, both national and international are playing an important role in delivery of social welfare in Uganda. Even though these organizations are providing vital services within the community, Issa G. Shivji (2007) argues in his book, Silence in NGO discourse, that there has been an unwillingness to criticize the impact that NGOs impose on society. He further identifies some of disadvantages that follow the merge of these organizations; firstly he establishes that many NGOs are donor funded, which means that their independence is restrained by the donor and their scope for action is restricted. This also contributes to the NGO responding to the donor’s opinion of what is

(14)

needed, rather than advocating for the needs of the beneficiaries. This adds to a top-down perspective on social welfare, that demotivates the participation of beneficiaries in the development of social policy and practice. This argument can be put in contrast to the supposed advantages that support NGOs, which is the ability to reach the most needy and vulnerable in society (ibid).

In the publication Nongovernmental organizations quest for development Richard Nuwamanya Butamanya (2012) distinguishes some of the challenges that NGOs often face in their work. Firstly, he points out a tendency in NGOs to be Urban centric, which mean that they often based in urban areas, rather than marginalized areas where service is often lacking. Since NGOs also often are restricted to a specific area, they are unable to make a wider effect on the social welfare as a whole. Other challenges in NGOs include inadequate technical capacity and database, which affect the spread of information internally and externally

throughout the organization, and thus limit the spread of knowledge gained in the NGOs different programmes and project (ibid).

3.1.1 Spreading knowledge

Since this study concerns itself with understanding how organizations

communicate and exchange knowledge and experiences with its surroundings we want to highlight some previous research on this subject. When studying

interaction and collaborations in the NGO sector, the literature mainly focus on partnership between northern NGOs (NNGO) and southern NGOs (SNGO), and the relationship/partnership between NGOs and the donors. In the article Power in

partnership? An analysis of an NGOs relationships with its partners, Sarah Lister

(1999) discuss the relationship between an southern NGO and one of its donors from a western country. Lister identifies some components that are important – not only in relation to the donors - for a successful collaboration between an NGO and its stakeholders. These include mutual trust and accountability, additive strength and clear, joint goals followed by mutual decision making, as well as both partners communicating and exchanging information. Furthermore, there must be functional tools to evaluate and monitor the work conducted, as well as transparency in relation to finances and reciprocal commitment to working together long-term (ibid).

Lister (1999) further state that many NGOs face difficulties in partnering with others. The most significant reason for this is the fact that donors have control over funding. The basic notion is that an NGO cannot demand of the donors what the donors can demand of the NGO. This asymmetrical relationship makes it difficult to create and obtain functional collaborations and partnership between these two. Lister also recognizes the importance of personal relationships in NGOs collaboration with its stakeholders. These relationships are mainly maintained by management. However, the importance of personal relationships makes NGOs more defenseless to alteration and challenges in leadership and management. The success of collaboration is closely connected to how strong the organizations relationships are. This in turn means that many NGO partnerships and collaborations are not formalized or institutionalized; instead they are built upon personal friendships, which mean that the NGO is dependent on those within the organization who maintain these relationships. Lister argues that these

personal relationships, as a base for interaction, are crucial when trying to understand how NGOs interact and collaborate within the field (ibid).

(15)

With this said, the discussion on NGOs is wide in range and there are many advantages as well as disadvantages in these types of organizations. As authors David Lewis and Paul Opoku Mensah (2006) discuss in their publication Moving

Forward Research Agendas on International NGOs: Theory, Agency and Context

the subject of NGOs has become complex and controversial, which can be seen as a result of lack in research within the NGO field. They argue that previous

research has been focusing on casework within organizations, with little or no emphasis on the organizations context (ibid).

In the book NGOs and Civil Society Anne C Hudock (1999) provides a conceptual framework for understanding NGOs and their role in the field of international development. Hudock recognizes that the relationships most NGOs have with their financial donors determine their capacity to a large extent, and minimize the control that NGOs have over their own space of action. This relationship is defined as an interdependent relationship. She further argues that most literature on NGOs focus on these organizations internal aspects, rather than acknowledging this problematic relationship. This ultimately leads away from the main question about how the organizational context controls NGO capacity. The author further argues that to realize the NGOs sector full capacity in delivering social welfare internationally, there is a need to understand and question the relationship NGOs have with external partners such as donors and other stakeholders, both

international and on a local level (ibid).

3.2 The use of Evaluation in NGOs

Evaluation is an essential concept in this study, and specifically the utilization of evaluation within NGOs. This is why we want to further pin point some of the challenges, advantages and issues acknowledged in previous research, concerning this subject.

In the article Who Counts Reality?, Participatory Monitoring and Evaluation: A

literature review, Marisol Estrella and John Gaventa (1998) discuss how the

growth of the NGO sector during the last two decades have raised a discussion about the need to identify the efforts made within the sector, through evaluation and monitoring. As mentioned in the background, the NGO sector faces different difficulties in relation to evaluation and monitoring. Even though the interest in evaluation and monitoring NGOs has increased, there is still much to be desired for in reliable evidence on the impact that NGO have on development work (ibid). Bruce Britton (2005) identifies three main reasons for evaluating and monitoring the activities of NGOs. Firstly, the purpose of evaluation and monitoring is to improve the work of the organization, by showing how resources are being used and acknowledge difficulties, improve management and ensure that the projects goals and objectives have been achieved. Secondly, Britton identify

communication, between the NGO and its stakeholders, as a reason for evaluating and monitoring. According to Britton communication is the key to overcome the difficulties NGOs often face in translating information gained through evaluation into practice. Thirdly, Britton argues that evaluation and monitoring should serve as a base for learning, by spreading the knowledge gained through evaluation to other organizations operating in the same field. This could help other NGOs repeat the successful interventions, and avoid failures (ibid).

In contradiction to this, Roger C. Riddell (2007) argues that NGOs, as a result of the liberalization of the NGO market, are forced to compete for donor funding to

(16)

survive, and therefore tend to be more unwilling to share gained information between each other. Evaluation and monitoring is crucial in creating

accountability in relation to donors and clients. This is also noted by Estrella and Gaventa (1998), who focus on the participatory monitoring and evaluation (PM&E), which is based on the idea that the NGO sector is a central arena for development work and therefore should involve all central actors in evaluation. PM&E acknowledge the fact that an NGO have different stakeholders who directly and in-directly are affected by the intervention implemented by the organization, and because of this they should be able to express their needs, interest and expectations and how these can be realized in a specific project. The stakeholders include for example donors, staff members, government agencies and beneficiaries (ibid).

In the article Accountability in Practice: Mechanisms for NGOs, Alnoor Ebrahim (2003) distinguish between external and internal evaluation in relation to

monitoring and evaluation in NGOs. At the end of an intervention, project or grant, it is common for the donor to do an external evaluation of the

achievements, often in relation to the programme objectives and goals. This kind of evaluation often serves as a base for the donor in the decision whether to sustain the programme with funds. Internal evaluation is also common within NGOs, where staff members evaluate their own achievements in relation to the goals or objectives of a certain project. Furthermore, how to use evaluation and for what reasons, can sometimes differ between donors and NGOs. When an NGO wants to evaluate processes of participation, impact or empowerment, the donors is often more interested in evident facts and quantifiable results, for example how many children have been reached or how many schools have been built (ibid). Ebrahim (2003) also acknowledge the fact that many NGOs are skeptical to evaluating and monitoring their activities. The idea is that evaluation and

monitoring is something that is being used, for example by donors, to control the activities of an organization, and not something that could be a used as a

possibility to improve the work within the organization. There is also a tendency within the NGO sector to act instead of analyzing. A common notion is that the NGO staff see them self as “doers”, and they get their legitimacy, not by engaging in time and money-consuming evaluations, but in being in the field and helping the people they are set out to help (ibid).

4. THEORY

In the previous chapters we have highlighted some of the central issues that many NGO face in their work. What we have seen is that these issues cannot solely be attributed to organizations internal affair, but also to circumstances outside the organization. With this said, the purpose and study questions are based on two different levels; on one hand we are concerned about the use of evaluation within the organization, and on the other hand we want to discover how the organization communicates with its surroundings. These different levels have furthermore played an important role in our choice, and demarcation, of theories.

Keeping our research questions and purpose in mind, we will start out in a system theory approach to shed light on some basic ideas on how organizations interact

(17)

and exchange information. This will also enable an understanding, through network theory, of how the surrounding context influence one organization, and vice versa. Furthermore, we will highlight some of the basic definitions of a Human Service organization. To create an understanding about how an organization can use the knowledge gained from evaluation, we will study theories concerning organizational learning and communication. Since we are interested in organizational learning, we feel the need to recognize certain theories that could help us explain how knowledge can be exchange across organizational boundaries, through internal and external communication. We will conclude this chapter with presenting theories on evaluation, and more specifically the use of evaluation in Human Service Organization.

4.1. Understanding organization through System Theory

We recognize that in order to discover how organizations interact we need to adapt theories that can explain this relationship. Kathrine Miller (2006) establishes in the book Organizational Communication – Approaches and

Processes that the system theory derives from the field of biology and

engineering. Applied to organizational theory, the idea is that organizations resemble a living organism, a living system. The basic concept is that

organizations – in similarity to organisms – are build up by components. In a biological system, these components consist of cells and organs. In an

organizational system, the components are the people and departments that form the organization. The organization, in turn, operates in a larger system; society. The basic notion is that the organization affects its surroundings and vice versa. Therefore every organization has to find a way of interacting, and communicating, with its surrounding in order to survive (ibid).

According to the system theory, organizations are complex social system and therefore we have to study them as a whole - separating one part from the system will only reduce the effectiveness of the whole system. Paul Flaa, Dag Hofoss, Finn Holver-Hoven, Torstein Medhus and Rolf Ronning (1998) argues in the book

Introduktion till organisationsteori (Introduction to Organizational Theory) that it

is important, in relation to organizations, to draw the attention to the exchange of ideas, funding/money, services, information and human resources between an organization and its surroundings. This is necessary because no organization is self-sufficient in relation to information and resources. All organization is, to some extent, dependent on others to attain their goals and secure their survival. However, this could signify the development of dependent relationships and imbalance between organizations, and give external actors the opportunity to control and influence the organizations actions (ibid).

Lars Skyttner (2006) discuss in his book General System Theory – Problems,

Perspectives, Practice, that an organization is a product of the needs of the

community that they operate in. To survive and succeed, an organization must know and keep on learning about the society that they work within, and at the same time remain true to its identity. The organization has to be able to control its surroundings in order to endure, due to the great influence the environment has on the organization (ibid). Miller (2006) uses the conception of interdependence. The idea is that one component of a system rely on other components in the system in order to function. This applies both within an organization, and for relationships between the organization and its surroundings. In a globalized world, the concept of interdependence is apparent to describe the complex relationships between organizations, and their environment (ibid).

(18)

Skyttner (2006) further discuss that all system have boundaries, separating them from their surroundings. There are open systems where the boundaries are

difficult to define, and closed systems with tangible boundaries. Organizations are mostly categorized as open systems, and the boundaries coincide with the

different events that cause a chain of reactions within the organization. Skyttner points out that the important thing is not the actions of one individual within the organization, but the arrangement of these actions. In other words, what we need to study is how people integrate and work together (ibid). Miller (2006) uses the term permeability boundaries to describe how organizations let information flow in and out. How permeable the boundaries are depend on how closed, or open, the system is – but in order to survive, all social system need to be somewhat permeable to incoming information both within the system and in relation to its surroundings (ibid).

4.2 Understanding the Human service organization

Since this study is conducted within an NGO, working with vulnerable children and youth, we appreciate that we have to recognize the basic scopes of this type of organization, which can further be defined as a Human Service Organization. Ingela Thylefors (2007) recognizes in her book Ledarskap i human service

organisationer (Leadership in Human Service Organizations) that human service

organizations (HSOs) can be understood through several different perspectives. HSOs can be defined as organization that contributes to the social welfare in society, which main purpose is to influence and transform people into independent well-functioning citizens. These organizations can be both under the control of government, or non-governmental. The basic thing that these organizations have in common is that they are dealing with people in one way or the other. One perspective within organizational theory is the theory of network based

organizations. This theory focuses on the importance of what happens between

organizations, rather than concentrating mainly on actions taking place within the organization. The collaboration between different professionals within different organizations is crucial in the development of the network based organization. She argues that the gap between needs and resources within these organizations, has contributed to an increasing need to collaborate with other organizations working within the same field (ibid). The question then remains, how we can determine whether a HSO is equipped to collaborate or communicate with other

organizations?

4.3 Network Theory

To shed light on how communication can be used as a tool for interaction with other actors within the field, and how these affect the work conducted within the organization we will apply the network theory. According to George Ritzer (2009), the network theory aims to explain how the behavior of individuals and organizations can be derived to the structure of the social relationships they are part of. The basic notion is that these structures have a greater impact on behavior than norms and values. When organizations interact and network with other organizations, they tie a connection with each other. This connection in turn motivate organizations to develop and work together, hence they will reach consensus (ibid).

Ritzer (2009) further argues that these connections are critical for constructing communication channels between organizations. These connections can be strong or weak, nevertheless they fulfill the same function; to communicate and

(19)

organization become weak and isolated due to the fact that they will not receive enough information from its surrounding, and eventually the organization will cease to exist. In order to maintain these connections all involved partners

contribute with information, as well as receive information from the others. When this happen, there will be collaboration between organizations that is characterized by the willingness of working towards the same goals, regardless of subjective ideals (ibid).

To further understand collaboration between organizations and its context we need appreciate how an organization can receive knowledge. With this said, we would also like to recognize that the collaboration and exchange of knowledge must be seen from an internal and external perspective, where the collective as well as individual learning process holds great importance in the development of an organization.

4.4 Communication

As mentioned in the background, NGOs faces some challenges that can be attributed to their ability to communicate knowledge and experience, gained through evaluation. Since we are interested in understanding how an organization like UYDEL communicates and share knowledge, we will apply theories on communication and organizational interaction.

Cindy Gallois, John Gardner, Elisabeth Jones and Bernadette Watson (2004) declare in the article Organizational Communication: Challenges for the New

Century that there are as much theories about communication and organizations,

as there are theories about communication in general. Communication is a central conception when analyzing organizations. They further argue that organizations today are under constant pressure to adapt to economic and environmental changes. To do this, modern organizations must be able to transform their communication channels in order to make, and reflect on, new structures, processes and relationships. As mentioned earlier, boundaries between

organizations and their surrounding have become more and more vague, hence an increasing need of communication channels overlapping the organizational boundaries (ibid).

André Jansson (2009) recognize in his book Kommunikation (Communication) that communication have three essential components. Communication is a process, not a state of things. Communication happens between individuals, linking them together. Last, all communication have an object, it contains of something. Jansson further acknowledge three levels, or structures, for

communication. First, he declares that we need some sort of “code” to be able to understand each other. Language is the most important tool – or code - for communication. The second levels consist of infrastructure, different technics we use to exchange information. In a modern society this includes for example computers, internet, telephones, radio and TV. The third structure is power. Jansson claims that all communication takes place between individuals and often is a reflection on our position in society. What one individual asset material- and culture wise, determines to what extent that person is able to control their own communication. At the same time, communication itself helps maintain and re-produce these structures. However when communicating, it is not always clear where the power lies (ibid).

Flaa, Hofoss, Holver-Hoven, Medhus and Ronning (1998) claim that

(20)

organizations. However, it is not just a matter of communicating. To succeed it is also important to create a functioning structure for communication. In order to make a good decision, there is a need to attain adequate information. To avoid miscommunication and misunderstandings Flaa, Hofoss, Holver-Hoven, Medhus and Ronning recognizes some possible moves an organization can make in order to reduce the risk of dysfunctional communication. For example, it is important for an organization to carefully plan the events of the organization in relation to its goals, distribution of work and assignments. This will contribute to a better

understanding of where information lack and where there is a need of increased or decreased communication within the organization, as well as between the

organization and the environment it operates in (ibid).

Authors Jesper Falkheimer and Mats Heide (2007) present another theory on organizational communication, where they identify three basic types of

communication within an organization. Communication can be hieratical in the sense that communication is centralized. This means that all decisions are made within management and eventually spread throughout the organization. This means that its management who has to communicate information and help the organizations members to interpret the information gained. Media communication is defined as internal communication enabled through meetings, newsletters and other forums for exchanging knowledge. The purpose of this type of

communication is to make information easy to access. Another type of communication is the one that consist of informal meetings within the organization and is therefore defined as informal communication. Informal communication is important in spreading information throughout an organization and enables the organizations members to create a better understanding about the organizations goals and interventions. With this said, they also point out that building structures for communication within the organization is often a low priority. Organizations tend to access statistical results and the effects that communication have might be hard to evaluate within an organization, this ultimately means that creating structured communication within the organization is often overlooked (ibid).

4.5 Knowledge and learning within the organization

Author Nick Gould (2004) argues in his book Social Work, Critical Reflection

and the Learning Organization that individual learning is important, but not

imperative in organizational learning, it is a collective process that comes about within the organization. His interpretation of the organizational learning

experience is closely interlinked with a three stage process of learning which include; elaboration, expansion, externalizing and a constant reevaluation of the organizations standpoints. The Elaboration face is defined as organizational learning through practice; The Expansion takes place when an organization transforms existing knowledge into deeper understanding through research and

Externalizing is the face where gained knowledge and experience lead to a

development in the organization. He further argues that this theory combines individual learning and collective learning, through focusing on the actual faces in which members of the organization collectively move toward a mutual

understanding of the information that has presented itself. In short this means; when groups come together and share their ideas and experience there can be collective learning, which can contribute to the externalization of learning between different organizations. Thus, collective reflection can be seen as a prerequisite for organizational learning. The basic notion of collective learning is

(21)

that the gained knowledge will be translated into action, and that this transformation will change the organization for the better (ibid).

Yeheskel Hasenfeld (2010) defines organizational change as “a process that that

occurs in an HSO as a result of external constrains imposed on it or as a result of internal organizational pressures that cause alterations and modifications in the organization’s core activity, goals, strategies, structure, and service

programmes” (Hasenfeld, 2010 p.456). The basic notion is that change is brought

about by internal and external factors. The individual influence can be defined as the meaning that individual members in the organization attribute to the suggested change or basically the attitudes towards change. The basis for creating attitudes among the organizations members is access to information and knowledge. The information and knowledge create a necessary base for interpretation of the suggested change, without information the individual is prone to discard the suggested change due to lack of understanding (ibid).

Hasenfeld (2010) introduces a perspective that put emphasis on the more

objective or external oriented perception of bringing about organizational change, which focuses on measurable modifications. The basic notion is that the

organization can bring about change through changing structures, goals and programmes within the organization. He further presents different theories that might be useful in explaining organizational change through knowledge and information. One approach that might be interesting in particular in this study is the Stakeholders Approach. This approach recognizes that the HSO is dependent on its context, which contain of donors, local authorities, governmental agencies and so on. These stakeholders all have their set of demands and expectations on the organization. Hasenfeld notes that the more these actors have to gain from the organization, the more legitimation and recognition the organization receive. This means that the organization must chart the interests of these actors and respond to these interests. This can be problematic since stakeholders often have conflicting interests in relation to the organization, which ultimately makes the process of organizational change relatively complex since the organization has to satisfy the needs of many different actors (ibid).

4.6 A building block for organizational change

Thylefors (2007) recognizes the use and spread of knowledge and information as a key factor in developing an organization. She also establishes that there is a need for structure in face of change in an organization, especially in relation to its staff, which can be realized through offering set routines, instructions and the access to information necessary to create adequate practice. Thylefors further introduces a model for organizational change, as shown on the next page (ibid). This model, or building block, can be useful in trying to understand the basics of what motivates organizational change. With this said, organizational change in this theory is seen as something positive and rewarding in the development in an organization, even though Thylefors (2007) notes that change can often pass through different faces of resistance, trial and error, unfreezing, actual change and stabilization. Decisions that lead to change might have negative impact which, and with this said, it is important that the organization try new solutions but also reject ideas that does not work. Resistance, as we have discovered, can be a consequence of lacking spread of information but also fear of trying new solutions within the organization. The unfreezing-process can be defined as motivation

(22)

among members of the organization to provoke change, for an example higher level of efficiency or bettering the organizations reputation. The actual change can be brought about when organizational structures are changed and reevaluated in a process of trial and error. Stabilization of change in return can be seen as the point in time when the organization has adapted and accepted the changes made (ibid). Thylefors (2007) presents five basic building blocks that are important in

changing an organization. Firstly, there is the need for a clear purpose or vision of the intended change. The vision is the idea of the organization that can only be realized through competence. Competence is important because it is the key factor in translating an idea into practice, and to be able to realize this, an organization needs to attract a fair amount of competence. This brings us to the next building block which explains the need for motivation in attracting competence, or human resource. Motivation can be realized through certain rewards, such as education or wages that are meant to motivate members of the organization towards realizing change. Another important building block is the organizations resources. Attaining

Resources is crucial in order for the organization to maintain projects running.

When resources are limited or phases out in short term projects it can be difficult to bring about lasting change in the organization. Resources that are long term are more lightly to bring about lasting change. Furthermore, a systematic plan of action is needed to bring about change, which can be realized through identifying

what needs to be change. The lack of these basic building blocks can, as shown

above, lead to confusion among members of the organization, anxiety and insecurity about the future, resistance, frustration and the feeling of not getting anywhere or being stuck in the “treadmill” (ibid).

To create organizational change can be complex, as we have learnt, but the basic realization of bringing about change is the notion of what actually needs to change. To be able to recognize what needs to be changed within an organization we are going introduce some theories about the basic concept of evaluation in HSOs.

4.7 A definition of evaluation

As we have established previously, evaluation can be defined as a method of assessing the outcome of interventions executed in an organization, and which should serve as a base for planning and decision-making regarding future

interventions. Björn Blom, Stefan Morén and Lennart Nygren (2011) argues that most social work practice concerns itself in realizing changes in the society, and

Vision Competence Reward Resources Plan of action Change

Competence Reward Resources Plan of action Confusion

Reward Resources Plan of action Anxiety

Resources Plan of action Resistance

Plan of action Frustration

Vision

Vision Competence

Vision Competence Reward

(23)

the underlying logic is that these changes are supposed to be positive. To

understand whether these changes are positive or not, is on the other hand almost impossible to determine, because changes can often be attributed to its context. The basis for this discussion can be understood through non-realistic theory, which sees reality as socially constructed, and therefore impossible to interpret without placing one’s own perspective as a mold for understanding. The basic assumption is that in evaluation, it is critical that the evaluator realizes that the interpretation of what is happening in an organization is not always the “correct” interpretation. This means that looking at the organizational context is important in evaluation of an organization. By understanding the context, one can give a truer picture of reality through evaluation, which eventually means that this knowledge can be translated into practice (ibid).

The use of knowledge and information is, as we have discovered, a complex question. Organizational change is dependent on internal and external factors, and is bound to its context in many ways. In order to reach a deeper understanding about what it takes to go from gained knowledge to practice we are going to study some basic theories about the use of evaluation.

4.7.1 The utilization of evaluation

According to Evert Vedung (2009), the utilization of evaluation can be described as a vital part of any HSO. The basic notion is that evaluations can be used as a tool for systematically examine an organization, with the purpose of discovering its challenges and opportunities, and ultimately serve as a base for planning and decision making. Vedung further argues that the underlying purpose of an

evaluation can affect the utilization and how it is received within the organization. An evaluation can be used to develop the organization and also contribute to positive change and organizational learning, but at the same time it can be used with the purpose of controlling whether the organization is reaching its set goals and standards (ibid).

Vedung (2009) argues that the concept of accountability is central in the

utilization of evaluation. Evaluations that have the primary purpose of controlling an organization or create accountability, are basically performed because a

specific actor wants to be able to assess the success of an organization and being able to control the outcome. The basic purpose of this type of evaluation is creating accountability upwards and placing supervision downwards. Regardless what the underlying purpose is, an evaluations success is closely linked with the utilization of the knowledge gained and how the suggested change can be implemented within the organization (ibid).

Vedung (2009) further identifies some possible ways in which an organization can utilize evaluations. Firstly, evaluations can be used instrumentally, meaning that the recommendations that emerge from the evaluation are implemented and used as a basis for decision making within the organization. The basis for instrumental

use of evaluation is that all recommendations and results have been researched in

a scientific manner, often in relation to a so called engineer model. An engineer model can be understood as a model or tool for interpreting the relationship between research, decision making and implementation processes, in a general sense. The basis notion for this model is that organizations are rational and that it is possible to generalize processes that take place within organizations through scientific research, which ultimately mean that organizations would be able to adapt all knowledge gained. Vedung recognizes that the use of an engineer model

(24)

should be purely used as an ideal for understanding the utilization of evaluations, because it goes against the basic idea that organizations are active participants in organizational learning. Secondly, he identifies the conceptual use of evaluation, which means that evaluations serve as a base for creating discussion about interventions and methods, creating new terminology and encourage critical thinking within the organization. In this sense the evaluation can be used as a tool for organizational learning, but may not be used as a base for decision-making or acting on the basis of knowledge gained (ibid).

Vedung (2009) further explains that evaluation can also be used for the purpose of

legitimizing an organization; in this case the evaluation plays an important role in

motivating specific interventions or methods used within the organization. This means that the evaluation is used to promote existing interventions and the organizations use the knowledge gained selectively in order to legitimize its actions. A tactical use of evaluation means that the evaluation process is used as legitimizing or with the purpose of delaying decision-making within an

organization. The organization, in this case, uses evaluations to display that changes and development will materialize within the organization, even though this might not be the case. Some evaluations can be recognized as nothing more than a routine activity, which has little or no impact on the organization; this can be defined as discursive use of evaluation. The basic notion is that evaluation is something that an organization should do, and can correlate with the legitimizing aspect of utilization of evaluation. With this said, Vedung points out that the same evaluation may be utilized in different ways over time, which means that an evaluation that is used to legitimize an organization can eventually also lead to conceptual use and organizational learning (ibid).

Vedung (2009) acknowledges some of the difficulties that might hinder the utilization of evaluation within an organization. Evaluations are largely based on

subjective assessments; this means that the actor conducting an evaluation sets the purpose or criteria in the evaluation. The pursuit of general evaluation criteria or models that can be transferred into operational guidelines can thus be questioned. The use of assessments can also be affected by the lack of resources within the organization or misused in political power struggles, which makes it even more necessary to distinguish the utilization of evaluation. Organizational development is based on active participation and dialogue between actors such as evaluators, the organizations members and stakeholders. This means that the evaluation could play an important role in learning and the spread of knowledge within an

organization (ibid).

5. METHOD

This study concerns itself with understanding of the structural aspects of the organization of UYDEL, its work and surrounding context. To realize this we traveled to Kampala, Uganda and conducted a field study within the organization. In the following text we intend to describe the process, from planning to

execution, in which we collected the material. We would like to acknowledge that the process of writing this study is based on a shared responsibility between the two of us, from gathering the material and analyzing, to the finished product. This

(25)

means that all the text and different passages are jointly put together and written by us, in collaboration.

5.1 Field Study, preparation and process

In the end of August 2012 we traveled to Uganda to collect material for our field study, but the process of planning and preparing the study has been an ongoing process tracing back to approximately seven months previous to our journey. According to Ulf Jakobsson (2011) the concept of a field study derives from the field of anthropology, and it is a method where the researchers are collecting the material in a natural setting - in real life. What the field consist of is decided by the research subject. Jakobsson defines a case study as method where the

researchers study a phenomenon (for example a process, a state of condition or an event) in its natural context, during a limited period of time. However, it is not only the phenomena itself that is being studied, but the relations and connection between different variables. The collection of material can be done by using different methods, such as interviews, observations and through journals (ibid). We would like to establish that this study can be defined as a field study in the sense that we are operating within the field, in the reality of the study object. We have limited our study to a specific case or example which means that our study is also likely to be labeled as a case study.

Regarding the process of our study, we initially wanted to find information on the country of Uganda, its history and culture, in order to increase our understanding about the context in which we would conduct our study within. Previous to our field study we also did a literature review on the annual reports, previous evaluations and steering documents of UYDEL, found on the organizations

website. In May (2012) we met with UYDEL’s executive managers in Stockholm, Sweden, this gave us the opportunity to present our study and make arrangements for our stay.

Previous to this field study the goal was primarily to identify reoccurring topics and results that had presented themselves both in evaluations (such as the IOGT-NTO evaluation) and annual reports. We wanted to create an understanding about common dilemmas in the use of evaluation within UYDEL. After reading and studying this material we started to put into words the basic purpose of our study. After gaining more knowledge about the organization we realized that our purpose might not be relevant in relation to the present and ongoing work within the organization. This led to a reformulation of our purpose, including a broader perspective on the primary problem. With this we started formulating question guides and trying to compile as much material as possible previous to our travel to Uganda. Arriving in Uganda we started collecting material through interviews and observation, which later was transcribed and analyzed.

5.2 A Qualitative methodology and Hermeneutic approach

This study will take on a qualitative methodology with a hermeneutic approach. Steinar Kvale and Svend Brinkman (2009) explain that qualitative method can be used for creating a basic understanding about human nature, through looking at different characteristic of a specific phenomenon. Understanding these

characteristics can be realized through analyzing material that can be retrieved in text, interviews and observation. This can be put in contrast to a quantitative method that involves itself in studying numeric data and identifying variances and correlations between characteristics or variables, through statistic analyze

Learning through evaluation - a case study on organizational learning within the non-governmental organization UYDEL (Uganda Youth Development Link) in Kampala, Uganda