Indigenous approaches to evaluation and research

Western approaches to evaluation tend to focus on objectively assessing the relevance and performance of programs through frameworks that seek systematic collection and analysis of evidence on the outcomes identified during the design of the program. They focus on the outcomes and metrics of programs and on topics of study, respectively, without an in-depth understanding of a communities’ perception of these issues. Western-based evaluations tend to assess programs from the perspective of the funding agency, which is often given more value than the community perspectives. Indigenous communities, organizations, and researchers have been vocal in the misalignment of these western methods in terms of their ability to identify meaningful outcomes that contribute to wellness and a holistic understanding of results.

The four expert papers highlight how Indigenous evaluation approaches enable communities to identify relevant program and community outcomes that respond to the needs of their community rather than the needs of the funding agency. They also provide insight into mechanisms and processes that are meaningful for individual and collective healing, health, and wellness, and that make space for stories that share the complexities of transformation and innovation – rather than a linear reporting of outcomes.

The expert papers highlight how Indigenous approaches to evaluation are about uncovering the truth about how an issue, program, or system functions from different perspectives. Specifically, an Indigenous approach to evaluation:

is a process of deep reflection and contemplation, a process of looking back and seeing what worked, what didn’t and then determining the path ahead… [it] does not employ an external set of indicators upon these questions of where you ‘should’ be. Rather this creates space for people to learn from their experiences, reflect on what has worked for them, celebrate the journey, and take that learning into their future (Rowe, 2019, p.10).

More broadly, Indigenous approaches to both evaluation and research are about:

determining who will set a knowledge seeking agenda, whose voice will lead the process, whose knowledge will be sought and valued, what methods will be used to gather the knowledge, and the ultimate use and distribution of the results of the knowledge gathering are all important elements (Rowe, 2019, p. 3).

In this way of looking at evaluation, Indigenous people are actively involved in conducting the evaluation or research rather than subjects of the work. The evaluation becomes a process of self-determination and self-governance, which becomes a decolonizing approach.

Indigenous peoples of Canada have many different languages and cultures. While there are many common elements to Indigenous worldviews such as spirituality and relationality, there is no one approach. Indigenous worldviews and ways of knowing are rooted in their individual contexts, histories, locations and experiences. Indigenous approaches to evaluation and research are grounded in, and inseparable from, Indigenous ways of knowing, worldviews and perspectives.

This highlights the importance of considering a community’s unique context at the outset of a project and that a one-size-fits-all approach to evaluation and research will not work when designing, implementing, or sharing the findings of a study. Context includes the unique economic, environmental, institutional, social, spiritual, and political realities of each community. Evaluators and researchers need to take care to appreciate why and how generalizations can reduce the truth and value of a given community’s positive contributions and dynamics. Individually tailored evaluations allow for the unique aspects of the community and program to determine the approach, methods and questions. It is essential that time and resources are set aside to understand and build competency about the local context in advance of determining the approach.

The following sections provide a brief summary of some of the key principles in Indigenous approaches to evaluation and research from the four expert papers.

Guiding principles

While recognizing that each Indigenous community possesses unique views, involving unique cultural, political and historical contexts, the four subject matter experts highlighted the following broad guiding principles as especially important to Indigenous approaches to evaluation and research. This list is not exhaustive, and additional detail with respect to the guiding principles presented here are included in the four papers appended to this report.

Relationships are central

Relationships are central to Indigenous approaches to evaluation and research practices. This includes relationships with the land, culture, community, people, ancestors, and spirituality.

Respect, trust, and responsibility are key factors when it comes to building relationships. Evaluators and researchers have responsibilities in the relationships they create in the community that includes how they remain accountable to those who have shared their knowledge. Developing trusting and respectful relationships with community members, program staff, and program participants can be achieved by participating in face-to-face meetings, listening and speaking, sharing and hosting, being generous and respectful in sharing one’s own knowledge. Evaluators and researchers can also demonstrate accountability by being actively present during the sharing of information, allowing time for reflection after information is shared, and ensuring that the information will serve the community and organization in the end.

Reciprocity is also a key part of being respectful and accountable to the relationships developed through the evaluation or research process. Evaluators and researchers have the opportunity to build capacity in a community, and it is important that they determine what will be left behind to ensure a deeper understanding of the role of evaluation or research and the skills necessary to do this work. This is also important when it comes to sharing and disseminating the findings; it is important to ensure that the results are disseminated in a way that is meaningful to the community. This can be done through community events or gatherings, videos or user-friendly materials.

Culturally responsive and community driven

Culturally responsive means that programs and services are respectful and relevant to the beliefs, practices, culture, and linguistic needs of diverse client populations and communities. It also requires the knowledge and capacity to respond to these needs. A culturally responsive evaluation or research study recognizes the need to bring to the forefront the land, language and cultural practices that are specific to those involved.

Recognizing the importance of community and the context of a program, community leaders as well as program staff and/or participants need to be engaged at the outset of a project. This means that members of the community would be the ones to determine the purpose of the evaluation or research study as well as the methods that will best capture information that would be most useful. This would ensure that the study meets the needs of the program and community. They would then be involved in all aspects of the research or evaluation from planning the scope and methodology to sharing the findings.

A key aspect of community-driven and culturally responsive approaches to evaluation and research is engaging an advisory group throughout the process. This approach recognizes that although the evaluation or research team may bring technical expertise to the project, it is the community and program that provide direction throughout the process. The advisory group can include program staff, community leaders, Elders, knowledge keepers, and other partners directly involved in the program. The role of the advisory group can include determining and validating the approach, the questions, and the evaluation/research results. They should also determine the best way to share the information among members of their community. This ensures that a community-driven approach is ultimately used.

Elders, knowledge keepers, and healers

Developing partnerships with Elders, Indigenous knowledge keepers, and healers is essential because it ensures that their insights guide the work of the evaluation or research. It also allows for the inclusion of ceremonies and the sharing of medicines or sacred objects that are appropriate given the community context (Johnston, 2019; Rowe, 2019), and ensures that knowledge and sacred stories shared are not given away without permission.

Respecting community protocols is also important when in communities. This may include the giving of tobacco or wild rice to show respect and to ask for guidance during the study or the use of a talking piece/stick in talking circles. Elders, knowledge keepers, and healers can provide guidance with these protocols as it is also important that the person who is giving these medicines/gifts understands the teachings related to the offering.

Sharing personal experiences

Holding and honouring the stories of participants, communities and organizations is sacred and not to be taken lightly in Indigenous methods. How a community’s stories are used and presented is crucial to being accountable to the relationships established during the process. Evaluators and researchers have a responsibility to ensure that personal experiences and stories are represented accurately. This can be done by sharing preliminary findings with participants and asking that they review the draft report to validate the findings.

Strengths-based perspectives

Indigenous approaches to evaluation and research use strengths-based perspectives rather than deficit-based ones. A strengths-based perspective focuses on identifying the resources available to address problems in a positive way. Whereas, deficit-based perspectives focus on problems with outcomes, without taking into account the social or structural issues underpinning the conditions for Indigenous peoples. Evaluation must focus on strengths, recognize challenges, but also consider individual and community resilience. Strengths-based perspectives look for opportunities for growth, emphasize a community’s assets, and identify solutions to issues.

Decolonized approach

Indigenous approaches and methodologies to evaluation and research must take a decolonized approach that recognizes the intergenerational impacts of colonization on Indigenous peoples, their families and their communities. These approaches must consider the historical trauma and cultural repression experienced by Indigenous people.

Trauma-informed approach

Evaluators and researchers need to be aware of and understand a community’s history and understand the intergenerational impact of colonization and its associated negative impacts on the lives of Indigenous people. This will help to ensure that evaluation and research approaches or processes that have alienated Indigenous peoples in the past can be avoided. When a trauma-informed approach is used, the process can contribute to the well-being of the community, decolonization, and reconciliation.

Ensuring appropriate timelines and resources

Indigenous approaches to evaluation and research cannot be rushed. It takes time to understand the context of a community or program; build meaningful, respectful and trusting relationships and approaches; allow for community engagement, hosting and attending ceremony; undertake meaningful data collection and analysis of results; and fulfil the need for reciprocity not only of the results but also for capacity-building. There also needs to be a sufficient budget to allow for relationship building, food, cultural protocol items, knowledge keeper and Elder gifts as well as travel.

Ways of knowing: Evaluation and Research Methods

Community context should inform the methods used for evaluation and research. Given the diversity of Indigenous communities in Canada, it is critical for evaluators and researchers to become familiar with the community’s past and current context, and understand community protocols, social, cultural and spiritual values before beginning a project. This will help to inform the process and more specifically, what methods or approaches should be used.

The following are a few examples of methods that the subject matter experts highlighted in their individual papers.

Case studies

A case study approach allows for an in-depth understanding of a program or community and how it operates. This method recognizes the uniqueness of programs and communities, and allows for an exploration of the ways that individuals experience a program.

Participant observation

Participant observation uses techniques to gather information without influencing the environment studied. In an Indigenous context, participant observation requires the building of reciprocal and respectful relationships through face-to-face interactions and the sharing of daily-lived experiences.

A form of participant observation can also be used as a conversational tool in discussions with program staff, to cross check lists completed by a researcher or evaluator through observing a program or process. This ensures validity of the check list and supports the building of a healthy line of communication between the researcher and participants. Doing this exercise early-on allows the process to act as a tool that contributes to gaining buy-in into the evaluation process. This also serves as a means for continued healthy communications through the longer evaluation process.

Dialogue and conversational methods of knowledge gathering

Conversational methods include such tools as facilitated self-reflection, storytelling, land-based activities, participation in ceremony such as the use of sacred fire, drumming, singing and the use of traditional medicines. These methods contribute to building relationships and are in line with the oral nature of Indigenous ways of knowing. These methods require a commitment from all participants to learn and share within a collective tradition.

Storytelling as a method is a form of decolonizing research because it supports Indigenous ways of knowing. By asking others to tell their stories, the evaluator or researcher must also share their own which demonstrates respect, reciprocity, and relationship building. This can be done by developing evaluation tools that allow evaluators to share their personal story at the same time as sharing teachings on conducting evaluation, such as self-assessment or self-evaluation. This can support and enhance the goals of the program, since this process contributes to capacity building to share data in an organized fashion, such as using a medicine wheel, basket teaching, blanket teaching, or tree of life teaching.

Talking circles, which involve individuals sitting in a circle to discuss a topic, is a newly accepted research technique. Unlike focus groups, talking circles allow each participant to have an opportunity to take an uninterrupted turn to discuss a topic. Another method involves walking around communities and talking to individuals, which provides an opportunity for listening and learning.

There are also visual or arts-based methods that use photos or drawings to facilitate a group dialogue about a topic as well as facilitated self-reflection.

Importance of ethics

Subject matter experts emphasized the importance of ethics when undertaking evaluation or research. There is an inherent duty to ensure that no harm is caused, that respect guides all the work, that knowledge is protected, and that those involved in the process benefit from their participation.

The Report of the Royal Commission on Aboriginal Peoples (1996) includesFootnote 3 guidelines for research sponsored by the Commission to ensure that the appropriate respect is given to cultures, languages, knowledge, and values of Indigenous peoples to legitimate knowledge.

In 1998, the First Nations Information Governance Centre established the Ownership, Control, Access and Possession (OCAP®) principles,Footnote 4 as a standard on how research should be conducted with First Nations and how data should be collected, protected, used, or shared. The principles influence how research ethics boards conduct ethical reviews of Indigenous related research, how community-based research information is accessed, and how research is conducted.

In 2010, the Tri-CouncilFootnote 5 Policy Statement on the Ethical Conduct of Research Involving Humans published a specific chapter outlining core principles when engaging research with Indigenous participants, which includes reference to OCAP® principles. The purpose of the guidelines is to ensure that research involving Indigenous people is undertaken through respectful relationships and encourages collaboration between researchers and participants.

Questions to consider when planning an evaluation or research study

The authors identified questions for consideration to help guide the planning and implementation of an evaluation or research study. While not an exhaustive list, these are some key questions to be asked throughout the process to ensure that the approach is community driven and prioritizes relationship building. These questions provide another way to emphasize the key principles that the authors identified.

  1. How will the community and program staff be engaged at the outset and throughout the study to ensure that they are contributing significantly to the evaluation or research? How will these relationships be developed?
  2. What are the reasons for undertaking the evaluation or research study? How does it give back to the community or address the needs of the community or program?
  3. Who are the intended audiences?
  4. What is your role as evaluator or researcher? How will you fulfil this role and obligations to those involved? What are you contributing or giving back?
  5. What is the role of program staff, participants and community in the design and implementation of the study as well as dissemination of the results? How will they be engaged to ensure a partnership is fostered?
  6. Who will be part of an advisory group or actively involved in the study? Are there leadership, community members, Elders, knowledge keepers or healers, other professionals or organizations that should be involved? How will these relationships be fostered and nurtured to ensure that the needs of the community are at the core of the work being completed?
  7. How will you ensure that you have a deep understanding of the program, community, their current and historical contexts prior to designing the evaluation or research study?
  8. What appropriate methods or tools can be used to document the story of the program?
  9. Do you understand the meaning of protocols and how values and principles are placed into action within local cultures? How will you incorporate protocols, ceremony, and spirit into the evaluation or research study?
  10. How will ethics and respect guide the study?
  11. How will the principle of reciprocity be implemented? How will you give back to those involved in the process (e.g., gift giving, capacity-building through mentorship, hiring or training of those within the community)?
  12. How will the holistic experiences of the program be shared? What type of follow-up will be used with the program, participants and community (e.g., written report, video, presentation, meeting, town hall)?
  13. Do you have the necessary budget and time to conduct the evaluation or research study in a good way?