Restoration Magazine

Considerations for Evaluating Tribal or Native-Focused Victim Service Programs

Introduction

Victim service programs operating on or off American Indian and Alaska Native (AI/AN) communities are including evaluations in their programs to establish best practices, which are called practice-based evidence germane to AI/AN Tribes. Evaluation helps programs turn PBEs into EBPs rooted in longstanding cultural values, knowledge, methods, approaches, and worldviews. Tribal governments and grassroots data warriors have developed guiding principles for conducting culturally respectful Indigenous evaluation and research.¹ Victimization in AI/AN communities against or by Native people requires understanding from historical, political, economic, environmental, and social perspectives as it impacts individuals, families, and communities. Also, historical trauma and cultural oppression have contributed significantly to social problems in Native communities and among Native people wherever they live.

What is Evaluation?

Program evaluation employs various methods to assess program effectiveness in meeting intended objectives. Comprehensive evaluations encompass all program aspects of design, administration, implementation, planning, and impact. Evaluating programs involves numerous methodologies, such as experimentation, document reviews, questionnaires, observations, etc. Often selected methods present unique advantages and challenges for Native-specific evaluations. Therefore, culturally informed victim-related evaluation designs created by evaluators with extensive knowledge of AI/AN governments, communities, and experience evaluating Native programs are essential. Program evaluation helps programs document how they integrate Native-defined PBEs with mainstream EBPs or new practices into programs. Many programs use Tribal- or culture-based PBEs to apply cultural knowledge, assets, beliefs, traditions, and practices and identify appropriate usage strategies. Identifying PBEs or EBPs for Native programs requires understanding the culturally complex Tribal context and the state and national-level mechanisms that may help or hinder service delivery. Indigenous program evaluations must be participatory, integrate culturally appropriate methods into study designs, and promote self determination in the evaluation process.

What Can Programs Learn From Evaluation?

Evaluation findings and results provide information to draw solid conclusions, make informed decisions, and help staff understand their programs’ effectiveness.

Evaluations document program performance and identify what’s working well, what needs adjustment or more attention, and what to put on hold or move.

Evaluations help a program better understand what it can or cannot provide, given its current capacities, and consider referring victims to other resources and services.

Evaluation helps programs justify the support needed from partners to deliver comprehensive victim services, assess connections with service providers, and provide suggestions to strengthen relationships.

Evaluation validates cultural measures and strengthens culturally responsive programming by creating and improving data collection tools to capture relevant measures.

Funders include evaluation in AI/AN solicitations to better support programs with training and implementation. These efforts increase the knowledge base of effective Native programs through culturally responsive evaluation methods.

Evaluation Types

Some agencies fund single-program evaluations, while others sponsor multisite or comparative cross-site evaluations. Four common types include:

• Evaluability assessments are preparatory steps for new or more mature programs to showcase their value and feasibility before an evaluation. These assessments identify program goals, resources, activities, and areas for improvement. They assess program readiness by ensuring the program structure, data, staffing, etc., are available for a process or outcome evaluation.
• Formative evaluations are tailored for early-stage projects and focus on design, implementation, and influencing factors. They prioritize learning and reflection to identify project strengths and challenges and make necessary adjustments.
• Process evaluations assess a program’s progress and implementation. They measure significant milestones, objectives achieved, procedures used, services offered, reaching the intended recipients, and getting feedback on service satisfaction.
• Outcome evaluations measure program success and service impact by tracking the achievement of core goals and measuring the changes in the intended population affected by the interventions or program.

What are Some Tribal or Native-Focused Evaluation Challenges?

Challenges center on applying culturally informed and respectful Indigenous-based evaluation principles, approaches, and methods. Following Indigenous approaches enables evaluators to intentionally include cultural and Tribal-based ideals, PBEs, and strategies for protecting cultural information.

Evaluation sometimes describes an issue like human trafficking as recent when Native people know it existed long before the new term was applied. Similarly, traditional cultural methods and approaches are not new. For example, Indigenous peacemaking has victim-centered processes that are still in practice. Identifying cultural indicators for longstanding PBEs presents challenges since they are only evident in oral traditions and practices, requiring intentional documentation. Also, some cultural practices contain “sacred knowledge” that cannot be shared and needs protection.

Costs, time, local expertise, and staff participation are competing factors, especially when funding only covers personnel’s service duties. Many grants don’t allocate resources for evaluation participation.

Funders could clearly outline the evaluation process and the evaluation type anticipated and require that evaluators with extensive Native evaluation experience be selected. Federal agencies often combine evaluation opportunities given to corporate evaluation firms, limiting the selection of Native evaluators. This practice hinders Native evaluators from fairly competing for evaluation opportunities to support programs and communities requiring their expertise.

How Can Programs Incorporate Evaluation?

It’s essential to plan for different evaluations. Self-evaluations start at program onset, while nationally sponsored evaluations involve external evaluators and begin later. Evaluators use logic models to describe a program’s structure and its intended activities for change. Indigenous logic models integrate culturally relevant elements that are aligned with Native ways of knowing and understanding.

National funders include evaluations in the Native-focused victim service programs they fund. Programs are asked to describe how they plan to evaluate their programs, the data sources, tools, and processes, and who is responsible for collecting and reporting data. They must explain how they expect to succeed and specify the anticipated outcomes. By incorporating this step into every grant application, programs plan for meaningful evaluation.

Even when federal funders expect Tribal programs to participate in a national evaluation, this does not mean the Tribe has approved participation. Applicants must outline their strategy to obtain Tribal approval in their applications.

Before an evaluation, programs should identify internal and external data sources, data-sharing agreements, tools, and collection procedures and describe data protection and privacy procedures.

Programs can identify participants like service recipients, staff, partners, leadership, and the community to gain their support and perspective on program operations, services, and evaluation design.

Culturally-informed evaluators should evaluate Tribal and Native programs. Programs must assert this standard of practice to their Tribal and organizational leadership and funding agencies.

Using Evaluation Results

When programs start with an evaluability or formative evaluation, they lay the groundwork for the next steps in evaluation, such as a process or outcome evaluation. Programs can use the findings to make necessary adjustments and ensure the effective dissemination of results. Program evaluators must give back data or information to the community in creative and culturally-appropriate ways.

Tribal governments intentionally setting aside time and resources to evaluate their responses to victims of crimes, including domestic violence, sexual assault, stalking, sex trafficking, and homicide, is reflective of and supports the increasing capacity to take better care of our citizens and community members.

1. See bit.ly/4eA4WVS