National Anti-Drug Strategy Implementation Evaluation, Final Report

EXECUTIVE SUMMARY

1. Introduction

The National Anti-Drug Strategy (the "Strategy") is a horizontal initiative of 12 federal departments and agencies, led by the Department of Justice Canada (DOJ). As described in the Speech from the Throne in October 2007, the Strategy is a focused initiative whose goal is to "contribute to safer and healthier communities through coordinated efforts to prevent use, treat dependency and reduce production and distribution of illicit drugs."[1] The Strategy seeks to achieve this goal through three action plans devoted to prevention, treatment, and enforcement.

The Implementation Evaluation of the Strategy was conducted between December 2008 and September 2009 in accordance with Treasury Board requirements. The primary purpose of the evaluation was to assess whether the Strategy has been implemented as planned. This report summarizes the evaluation findings, draws conclusions, and provides recommendations.

2. Methodology

The evaluation was guided by the evaluation framework for the Strategy contained in the Results-Based Management and Accountability Framework (RMAF). The evaluation methodology consisted of document and file review, interviews (n=85), and three focus groups with departmental personnel (n=19). Triangulation was used to verify and validate the findings obtained through these methods and to arrive at the overall evaluation findings.

3. Findings and Conclusions

This section of the report summarizes the evaluation findings and draws conclusions.

3.1. Relevance

The evaluation confirmed a legitimate and necessary role for the federal government in prevention, treatment, and enforcement of illicit drug issues. The federal government's role in enforcement is defined by the Constitution, domestic legislation such as the Criminal Code and the Controlled Drugs and Substances Act, and by various international conventions Canada has ratified. While prevention and treatment are areas of primarily provincial jurisdiction, there is general agreement among those interviewed that the federal government can and should be active in these areas, since the sheer magnitude of the illicit drug problem demands sizeable resources and because the problem is not contained within local, provincial/territorial, or regional boundaries.

At the federal level, continued federal commitment to the Strategy has been demonstrated by recent public statements by the Ministers involved in the Strategy and by the reintroduction of the Mandatory Minimum Penalties (MMP) legislation in February 2009. More broadly, the Strategy links to other federal government initiatives, including its Tackling Crime and Safe and Healthy Communities agendas. By comparison, the provinces and territories focus on substance abuse in general rather than abuse of illicit drugs, support harm reduction, and take a more holistic approach to substance use issues.

The evaluation also confirmed an ongoing need for a coordinated response to illicit drug issues, such as the National Anti-Drug Strategy, with elements of prevention, treatment, and enforcement. With respect to enforcement, illicit drug trafficking continues to be a major concern both nationally and internationally, and recent reports indicate that Canada has become a source country for illicit drugs such as methamphetamine, ecstasy, and marijuana. Similarly, the most recent Canadian statistics on rates of illicit drug use, particularly among youth, indicate an ongoing need for action in the areas of prevention and treatment. That being said, many external stakeholders believe that the prevention and treatment components of the Strategy would better serve the public interest if they covered substance use in general, rather than illicit drug use specifically.

Finally, the evaluation found the Strategy to be relevant to international objectives and priorities. The Strategy has been well-received as a model by the international community, and at least one country (Chile) has developed a national strategy very similar to Canada's. Canada actively advances Strategy principles at regional and international drug negotiations and has found receptive audiences. Capacity-building, including the training provided by the RCMP, is in increasing demand internationally. However, the international community is divided on the most appropriate direction for illicit drug policy, and in particular is divided on questions such as harm reduction and the appropriate balance between supply reduction (enforcement) and demand reduction (prevention and treatment). In addition, many countries have broader drug strategies in place that address substance use in general, rather than focusing on illicit drugs specifically.

3.2. Design

The Strategy is intended to cover illicit drugs, defined as substances controlled by the Controlled Drugs and Substances Act (CDSA). This includes street drugs such as cocaine, heroin, marijuana, and ecstasy, as well as pharmaceuticals. The evaluation found that some stakeholders, both within and outside the federal government, are operating on the assumption that the Strategy covers only street drugs and not controlled pharmaceuticals. There is therefore a need to clearly communicate to stakeholders that the Strategy covers all controlled substances, including pharmaceuticals.

The evaluation also found that about half of the projects funded under the Strategy address substance use in general, rather than illicit drugs specifically. Recognizing that the objective of the Treatment Action Plan is to "support effective treatment and rehabilitation systems" and that as such, its focus is legitimately broader than illicit drugs, this finding nevertheless suggests that more attention should be paid to the way Strategy priorities are communicated to stakeholders.

3.3. Governance and Horizontality

Among departmental interviewees, there is general agreement that the roles and responsibilities of the Strategy partners are appropriate and clearly defined, and there is a common understanding of these roles and responsibilities among the partners. Initial challenges related to the transfer of the lead role from Health Canada to the DOJ have been resolved.

The Strategy's formal governance structure, consisting of an ADM Committee and four Director General working groups, may not be functioning as effectively as intended. The working groups are not all meeting regularly or as often as mandated, and their focus tends to be information-sharing and reporting rather than operational collaboration and coordination. It has not been uncommon for lower ranking personnel to attend working group meetings in place of the intended more senior personnel.

That being said, the evaluation did find some examples of operational collaboration among Strategy partners, including the Aboriginal Issues Committee developed in the context of the Treatment and Prevention Working Group, the Synthetic Drug Initiative spearheaded by the RCMP, and the international work. While other issues might benefit from increased coordination and collaboration among Strategy partners, not all circumstances warrant strong coordination. When contemplating mechanisms for improving horizontality, the driving consideration should be whether increased collaboration and coordination are likely to produce greater success in meeting the objectives of each action plan.

Possibilities for improving horizontal functioning include establishing, in addition to the Director General-level working groups, working groups consisting of program-level representatives to foster collaboration and coordination at the operational level; to share information among all, or a relevant subset of, Strategy partners on what organizations have applied for or received funding through the Strategy funding programs; and to develop a Strategy extranet or intranet site for information exchange among the partners. This will enable Strategy partners to work with funding recipients on their initiatives and thus maximize the impact of Strategy resources.

3.4. Implementation

The evaluation found that the Strategy has been implemented largely as intended. All components of the Strategy have been implemented, with the exception of elements contingent on passage into law of the government's proposed Mandatory Minimum Penalty (MMP) legislation. However, many components, particularly those that are part of the Prevention and Treatment Action Plans, have experienced delays and other challenges, and are behind schedule in implementation. The main challenges to implementation have been: circumstances beyond the control of the component programs; human resource issues; and the need to reorient existing programs for the Strategy.

As a result of these challenges, approximately one-third of the funds allocated to the Strategy were not spent in the first two fiscal years. There were significant differences in the capacity of the three action plans to expend funds. Implementation was least problematic for the Enforcement Action Plan because it consists primarily of capacity enhancements that did not require any program development or reorientation. Conversely, implementation was most problematic for the Treatment Action Plan, which consists of several new and pre-existing programs, some of which have spent less than half of their planned spending. Across the Strategy as a whole, some $55 million were either re-profiled into future fiscal years or lapsed, which may have implications for the rate of progress toward expected outcomes.

3.5. Resources

Although some components of the Strategy have lapsed funds, there are some areas in which resources may be insufficient. One area is the international component, which did not receive any new funding under the Strategy. This area is experiencing growing resource challenges as Canada comes under increasing international pressure to deal with a range of international drug issues, including emerging issues such as synthetic drugs and diversion of precursor chemicals. There are also resources expended in ensuring that Canada meets its international obligations and engages in advocacy efforts internationally.

The evaluation also found that resources for performance measurement, reporting, and evaluation may be insufficient, particularly for Strategy-level reporting. Most Strategy partners did not receive resources for performance reporting and as a result, most components of the Strategy will not undergo separate evaluations. Given that the DOJ's role in Strategy-level evaluation is likely to be more substantial than "rolling up", or integrating, a series of evaluations completed by the partners, the level of resources it received for this purpose ($100,000 per year plus one FTE) seems inadequate in relation to the task.

Finally, based on key informant opinion, the DOJ may be under-resourced to carry out its role as the lead department for the Strategy, with responsibilities including overall lead, policy development, coordination among partners, communication, and reporting and evaluation. Through the Strategy, the DOJ received four FTEs (including the one FTE mentioned above) to support its lead role, as well as some operations and maintenance (O & M) funding. The Department has had to borrow from other functional areas to fulfill its responsibilities as the Strategy lead.

3.6. Performance measurement and reporting

Although it is too early to report on outcomes given the delays in implementation, substantial work has been done to support performance measurement, reporting, and evaluation of the Strategy. The DOJ, as Chair of the Subcommittee on Evaluation and Reporting, has provided considerable assistance to the partners by developing reporting templates and tools and by coordinating and leading SER meetings. Many Strategy partners have put systems in place to collect and report on performance information, and all contributed to the DOJ's Departmental Performance Reports (Horizontal Initiatives) for 2007-2009.

Despite the quite sophisticated structures and mechanisms that have been put in place to facilitate performance reporting and evaluation, components of all three Action Plans will have challenges in reporting results for the impact evaluation scheduled for 2011. Challenges include a lack of baseline information, difficulties in isolating outputs and outcomes of Strategy funding, difficulties in establishing a causal link between a department's activities and outputs and any outcomes that may be observed (particularly for longer-term outcomes), reliance on funded projects to provide performance information, and the relatively short time frame of the Strategy, particularly since behaviour change occurs over an extended period of time.

These challenges will be exacerbated by the delays in implementation as many programs will likely not be in a position to report on outcomes by 2011. Therefore, alternative approaches to the impact evaluation should be considered. Some possibilities include greater than anticipated reliance on qualitative approaches and/or reporting on selected components or outcomes as opposed to all components or outcomes.

[1] DOJ. (2008, May). Results-based Management and Accountability Framework: National Anti-Drug Strategy: Final Report.