Top of Page
In order to meet the information requirements of senior management at Environment Canada and to fulfill a commitment to Treasury Board, an evaluation of the Enforcement Program was conducted in 2008–2009. This evaluation was part of Environment Canada’s Audit and Evaluation Plan for 2007–2008 to 2009–2010 as approved by the Departmental Audit and Evaluation Committee on April 18, 2007. The evaluation addressed four issues1:
The evaluation covered the four-year time frame from 2004–2005 to 2007–2008, focusing primarily on the period following the creation of the Enforcement Branch in June 2005 and also addressing significant developments to the Enforcement Program that occurred in 2008–2009. In addition, the evaluation of the Environmental Enforcement Intelligence Program, previously planned for 2007–2008, was integrated into this broader evaluation of the Enforcement Program2.
Multiple lines of evidence were used to conduct this evaluation, including key informant interviews, group discussions, case studies, document reviews and analysis of program performance measurement information.
The key findings and conclusions of the evaluation are presented by evaluation issue.
The evaluation evidence shows that the Enforcement Program is relevant to federal government priorities and addresses a valid need to manage risks to the environment, wildlife and human health through the enforcement of federal legislation and regulations. There is a clear, legislated mandate for the federal government’s involvement in the enforcement of environmental laws and the enforcement of these laws is necessary for the achievement of the government’s environmental objectives. Furthermore, Canadians support an active role for government in this area.
Evidence collected for the evaluation was largely qualitative, based heavily on the views of interviewees supplemented by some documentation and data. Conclusions regarding the achievement of intended Enforcement Program outcomes were based on the best available evidence. However, a lack of documentation and performance information on several outcomes limited the degree to which interviewees’ observations on outcomes could be corroborated. Accurate and timely performance data on key measures such as rates of compliance with regulations were not available from the Enforcement Program. This lack of meaningful performance measurement is a serious shortcoming, as it inhibits the Enforcement Program’s ability to determine the degree to which it is meeting its objectives, which in turn creates a gap in the information available for decision making.
Based on available information, the Enforcement Program appears to have made considerable progress on most of its outcomes. There is also evidence that, beyond the time frame of the evaluation (2004–2005 to 2007–2008), further efforts to address shortcomings are being undertaken.
The evaluation findings indicate that the Enforcement Program has been most successful in contributing to the achievement of the following outcomes:
In addition, progress has been made toward the other immediate and intermediate outcomes. A number of memoranda of understanding and agreements with external partners have contributed to improved knowledge of, and engagement in, enforcement issues and strategies among various jurisdictions. These memoranda and agreements have also led to better integrated strategies with other government departments (OGD), partners and stakeholders. Collaborative working relationships have contributed to progress toward better integrated enforcement strategies with these external partners and a more efficient use of resources. There are, however, some regions within the Wildlife Enforcement Directorate where certain provincial and federal partner relationships have weakened and there is an ongoing need to nurture those relationships that are currently strong and work to improve those that are not optimal.
Efforts are being made by the Enforcement Program to ensure that its views are included in regulatory reviews in order to improve the enforceability of regulatory instruments, although this is not always occurring. Evidence indicates that progress has been made toward more strategically targeted enforcement activities, through the use of a consultative annual planning process to identify priorities for enforcement and the use of intelligence to focus efforts on the detection of non-compliance. However, the success of these planning efforts has been limited by perceived weaknesses in the Enforcement Program’s coordination with its key internal partners within Environment Canada—the Compliance Promotion and Analysis Division (CPAD), also referred to as Compliance Promotion, environmental protection programs, and the Canadian Wildlife Service of the Environmental Stewardship Branch. This is discussed further below in the Design and Delivery section.
Evaluation findings indicate that, although some progress has been made toward improved awareness of the responsibilities of the Enforcement Program and the benefits of enforcement, there continue to be gaps in understanding of the Enforcement Program within and outside the Department. There is also evidence that progress has been made toward the development of staff knowledge and skills, though there are some gaps, including the lack of a formalized basic training course for Wildlife Enforcement Directorate (WED) officers3 and an overall need for more regulation-specific training and specialized knowledge and skills. As well, through the use of intelligence4, progress has been made toward increasing the knowledge available to inform Enforcement Program planning and decision making, but the capacity of the intelligence function varies by region and there is a need to develop a more strategic approach to this function in the Wildlife Enforcement Directorate.
In the view of a majority of Environmental Enforcement interviewees, the Enforcement Program has resulted in increased regulatee compliance with laws and regulations. However, as noted, reliable data on compliance rates were unavailable and Wildlife Enforcement interviewees indicated that it is difficult to determine the level of compliance with wildlife regulations.
The available evidence suggests that the Enforcement Program is delivered in a generally cost-effective and efficient manner. The Enforcement Program operated under resource constraints for most of the evaluation time frame and strategically targeted its activities to focus efforts on the highest-priority areas.
In order to reduce duplication and support cost-effective delivery, the Enforcement Program collaborates with other federal departments as well as agencies and departments in other jurisdictions that have similar objectives and activities. Although a detailed study of the cost-effectiveness of alternative approaches was beyond the scope of this evaluation, no evidence of more cost-effective approaches was provided by representatives of the Enforcement Program, internal partners or external partners interviewed.
Evidence suggests the Enforcement Program could benefit from some efficiency improvements. In particular, continued improvements to communications and coordination with internal and external partners would contribute to greater efficiency.
For the most part, the Enforcement Program is being delivered adequately and as intended. As discussed below, however, the Enforcement Program has experienced some challenges with communications, information exchange and coordination with its key internal partners that influence delivery.
Ensuring a clear understanding of the respective roles and responsibilities of the Enforcement Program and its internal partners is critical for regulatory development and review, the establishment of priorities for enforcement, and supporting the achievement of regulatory compliance. Internal partners, however, do not have a complete understanding of precisely where the responsibilities of Compliance Promotion, the environmental protection programs, the Canadian Wildlife Service and the Enforcement Program begin and end in the continuum of activities required to support regulatory compliance. This lack of understanding results in gaps where compliance promotion is not occurring and instances where the Enforcement Branch’s viewpoint is not included in the development or amendment of regulations. Findings also point to a lack of understanding within the Enforcement Program regarding the division of responsibilities between National Headquarters and the regions for the major functional areas—inspections, investigations and, in particular, intelligence. A lack of finalized policy documents clarifying the operations of the Enforcement Branch is a contributing factor to this lack of understanding.
Related to the need to ensure a clear understanding of roles and responsibilities, communications and coordination with internal partners are another area for improvement for the Enforcement Program. As noted, the Enforcement Program’s strategic targeting of enforcement activities relies in part on an annual consultative planning process to identify priorities for enforcement. The concept of this approach of consulting with key stakeholders to determine priority areas for enforcement is widely supported. However, perceived weaknesses in the Enforcement Program’s coordination with key internal partners involved in the planning exercise, coupled with insufficient formal processes for communications and information exchange, have resulted in shortcomings in the implementation of this targeting strategy. Internal partners do not fully understand or feel fully engaged in the process, thereby limiting their level of commitment to the plan and the degree of alignment between their activities and the Enforcement Program’s priorities.
With respect to the organization and governance of Enforcement, the creation of a separate Enforcement Branch, with clear lines of authority under the direction of the Chief Enforcement Officer and a centralized reporting structure, is an effective and efficient structure for the Enforcement Program, as is the placement of Wildlife Enforcement and Environmental Enforcement within the same organization. However, the wildlife and environmental enforcement functions continue to operate independently of each other in many respects, despite being placed organizationally within the same Branch. Although this was not specifically a focus of the evaluation, it became clear through interviews and the document review that there are a number of inconsistencies between these two directorates. Consequently, there may be opportunities to standardize some aspects of operations and reporting and to leverage efficiencies, while still recognizing their uniqueness.
Finally, in interviews for the evaluation, resource shortages were cited by representatives of the Enforcement Branch and internal partners as a factor exerting a negative influence on Enforcement Program delivery and success. This includes resource shortages on the part of both the Enforcement Program and its internal partners, thereby limiting their ability to respond to Enforcement’s needs. Additional resources approved in 2007 and 2008, to be allocated over a five-year period, have been provided to the Enforcement Program and are currently being used to address some of these deficiencies.
The following recommendations were developed for action by the Ecosystem Sustainability Board (ES Board) and Environmental Protection Board (EP Board) based on evaluation findings and conclusions.
Recommendation 1: It is recommended that a useful performance measurement strategy for the Enforcement Program be developed and implemented. While it is acknowledged that the Enforcement Program has already initiated the development of an improved approach for measuring its results, more work is required to provide meaningful, accurate and accessible data on the delivery of the Enforcement Program’s outputs and degree of achievement of its intended outcomes. Although some performance data were available in the Enforcement Program’s database, called National Enforcement Management Information System and Intelligence System (NEMISIS), there was insufficient evidence to demonstrate the Enforcement Program’s progress towards its intended outcomes. A standardized process for collecting data and reporting on specific, measurable indicators that are linked to outputs and outcomes in the Enforcement Program logic model would be useful to senior management for decision making and would enable the Enforcement Program to tell its performance story.
Agree. The Strategic Enforcement Framework (and work now underway to implement the vision it defines), developed by the Enforcement Branch (EB) and approved by the Executive Management Committee (EMC), is designed explicitly to respond to these gaps. The Branch has recognized that clear performance goals and the data and requisite data collection/analysis processes to measure progress against those goals are imperative for enabling strong management of the Enforcement Program, and to more clearly portray to Canadians, senior management, and parliamentarians the results realized by the Enforcement Program. Work is now underway to
Performance indicators and data collection/management will be in place for all immediate outcomes for the beginning of 2010–2011, with reporting featuring new performance indicators delivered by 2011–2012 (reporting on the 2010–2011 year).
Recommendation 2: It is recommended that roles and responsibilities be clarified and articulated, particularly the respective roles and responsibilities of the Enforcement Program and its key internal partners for the achievement of regulatory compliance and the division of the Enforcement Program’s responsibilities between National Headquarters and the regions. The Enforcement Program requires a clear and coordinated approach when carrying out the enforcement function due to the breadth of its mandate and responsibilities. There is ongoing confusion, however, both within the Enforcement Branch and among its internal partners, on the division of responsibilities for various roles in the achievement of regulatory compliance, including regulatory development, compliance promotion and the establishment of enforcement priorities. There are gaps where compliance promotion is not occurring and a clear compliance plan and enforcement plan do not exist for each regulation. Although compliance and enforcement policies are intended to outline what is expected of those who share a responsibility for protection of the environment, the policies for the habitat protection and pollution prevention provisions of the Fisheries Act and for wildlife legislation are very general in terms of who conducts compliance promotion. The policy for the Canadian Environmental Protection Act, 1999 does not delineate the respective responsibilities of Compliance Promotion and the programs. The evaluation also points to a need for a clearer articulation of the division of responsibilities between National Headquarters and the regions for the functional areas of inspections, investigations and, in particular, intelligence to ensure effective and efficient Enforcement Program delivery. A need was identified for finalized written policy documents to clarify the operations of the Branch. For instance, what are the exact responsibilities and relationships between national managers and regional managers of these three functional areas? Clarification of roles and responsibilities both within and outside the Enforcement Program would improve relationships and the Enforcement Program’s ability to carry out its enforcement activities.
With respect to the first component: Working with internal partners
Agree.
Where the Environmental Enforcement Directorate (EED) is concerned, the Compliance Promotion and Analysis Division (CPAD) and the Enforcement Branch are increasing the scope and depth of their collaboration. Particularly in planning for 2009–2010, CPAD and Enforcement have worked together to streamline the program priorities submission process. Improvements in this respect will continue as we plan for 2010–2011, when CPAD and the Enforcement Branch will develop a fully integrated proposal for compliance promotion and enforcement priorities, and a high-level integrated plan for delivering on those priorities.
In the 2009–2010 fiscal year, work is proceeding jointly on the development of, and plans for implementation of the Interim Compliance Strategy for the Wastewater Sector. The Compliance Analysis and Planning (CAP) program in CPAD is also actively involved in the implementation of the Enforcement Program performance measurement strategy and will be offering online performance measurement reports through its data warehouse technology to the Enforcement Branch.
Where the Wildlife Enforcement Directorate (WED) is concerned, the Directorate’s key internal partner is the Canadian Wildlife Service (CWS), which plays a leading role in regulatory development and compliance promotion. WED also plays a role in compliance promotion, and we acknowledge that these interdependencies and shared responsibilities necessitate a better coordinated approach. To that end, the Enforcement Branch and CWS will conclude a formal agreement that clearly articulates the expected roles and responsibilities of each organization with respect to regulatory development, compliance promotion and establishment of priorities for enforcement, including the roles of regional and National Headquarters units within both organizations. Annual processes between the two organizations will also be described. This will be completed by June 2010.
Additionally, CWS and WED will develop a plan that articulates short-term enforcement priorities and a three-year overall compliance promotion plan for wildlife legislation—the Migratory Birds Convention Act, 1994 (MBCA 1994), the Species at Risk Act (SARA), the Wildlife Animal and Plant Protection Regulation of Interprovincial and International Trade Act (WAPPRIITA), and the Canada Wildlife Act (CWA). This will be completed by June 2010. The subsequent delivery of such a plan will be contingent on effectively determining the respective roles and responsibilities of CWS and WED, as well as the human and financial resources available from both organizations.
With respect to the second component: Clarifying roles and responsibilities within the Enforcement Branch, with a focus on intelligence
Agree.
The Enforcement Branch is now finalizing the Enforcement Operations Manual, which will spell out roles and responsibilities for all staff, and which will be disseminated throughout the Branch. The regional restructuring undertaken in 2009–2010 is integrating the inspections and investigations functions in all regions, and the national directors are now formalizing within their respective organizations the processes for fully completing this change. Finally, with respect to intelligence, the Enforcement Branch has created the Intelligence Working Group co-chaired by directors from Environmental Enforcement Directorate (EED) and Wildlife Enforcement Directorate (WED) with a mandate to examine intelligence roles and responsibilities, and make recommendations for improving consistency and overall performance in the intelligence function.
Moreover, and particularly in light of the new regional structure and the Intelligence Working Group, the Enforcement Branch will produce a document describing the roles and responsibilities of all organizations within the Branch. This will be completed and communicated to staff by the end of 2009–2010.
Recommendation 3: It is recommended that mechanisms and processes for improving the Enforcement Program’s communications and information sharing with internal and external partners be examined and implemented at both the National Headquarters and regional levels. Because the Enforcement Program’s strategic targeting of enforcement activities relies on a consultative planning process and coordination with partners, effective communications with internal partners are essential for the successful implementation of this targeting approach. Similarly, good communications with external partners are required for the Enforcement Program’s coordination of enforcement operations and strategies with those in other jurisdictions. The development of an overall communications strategy could help to improve information exchange and coordination as well as strengthen relationships with partners. The development and consistent implementation of more formal communications and coordination mechanisms could be focused on meeting the information needs of all parties and improving communication flow.
Agree.
In partnership with the Communications Branch, the Enforcement Branch is now finalizing a long-term communications strategy, consistent with, and to deliver on, the new Strategic Enforcement Framework. This strategy will focus particularly on communication/engagement with external partners, with the goals of building capacity through partnerships and raising public awareness of the Enforcement Program. The strategy will be approved and implementation begun in fall 2009. Communications will adopt an ecosystemic approach in portraying the impact of enforcement work on species and habitat conservation.
Moreover, and as noted in the response to recommendation 2 above, the joint EED –CPAD annual priority-setting and planning process planned for 2010–2011, and consequent regular communication/coordination with risk managers within the Department, is a significant vehicle for internal communication and information sharing.
Concerning WED and CWS, as noted in the response to recommendation 2, the two organizations will collaborate to develop both formalized roles and responsibilities and annual planning/communication processes. These annual processes will include national and regional coordination of strategic and operational planning, exchanges of letters regarding priorities and results, and communications commitments. This agreement will be concluded by June 2010.
In addition, the Chief Enforcement Officer’s membership on the ES Board and EP Board provides for important information exchange and decision-making coordination at a strategic level.
Recommendation 4: It is recommended that opportunities to reduce imbalances and increase consistency and standardization be explored between the environmental enforcement and wildlife enforcement components of the Enforcement Program, and implemented where appropriate. Although Environmental Enforcement and Wildlife Enforcement have been combined into one Branch, some imbalances and inconsistencies between the groups remain in areas such as reporting (e.g., different formats of planning documents), training (e.g., provision of the standard Basic Enforcement Training (BET) program for new officers for the Environmental Enforcement Directorate but not the Wildlife Enforcement Directorate), human resources (e.g., classification of enforcement officers) and intelligence (e.g., differences in the extent of development of this function). While it is acknowledged that the focus and requirements of these two directorates within the Enforcement Branch are different and may vary by region in some cases, there is the potential for further benefits to be achieved through greater standardization. This could improve the Enforcement Branch’s efficiency, effectiveness and ability to report on results for the overall Enforcement Program.
Agree.
With respect to human resources, the Enforcement Branch moved in 2009–2010 to complete the integration of the Branch by means of a nation-wide organizational restructuring initiative designed to harmonize organizational structures across regions and the two operational directorates. Classification and organizational structures are now consistent across the Branch. The Branch is also developing a corporate human resources framework that will define a single human resources management vision across both operational directorates. The framework will be in place by the end of 2009–2010.
With respect to planning and reporting, the performance indicator/reporting project now underway is building performance indicators for both wildlife and environmental enforcement that measure the environmental benefit of enforcement actions in each directorate. As well, the results of the framework are integrated, down to the intermediate outcome level. This will enable a consistent reporting and performance management approach for the entire Enforcement Branch. In addition, priority setting will be integrated between the two directorates for 2010–2011. Indicators will be fully developed by the end of 2009–2010, with data collection beginning the following year to enable reporting against new indicators in 2010–2011.
With respect to training, the Branch acknowledges that strengthening training for WED is a priority. In 2009–2010, the Branch is developing corporate training priorities that reflect the needs of both directorates, and will seek to integrate the provision of Basic Enforcement Training to the greatest extent possible, including the development of a common curriculum, where appropriate. In addition, and in order to build greater consistency across the Branch, an enforcement operations manual is currently being developed and will be implemented as a pilot project by the end of 2009–2010. All enforcement officers will be trained in how to use the manual, resulting in the implementation of common practices not only between WED and EED, but also for all regions. The manual will also help protect enforcement officers through common, sound occupational safety and health practices.
With respect to intelligence, as noted above, the Enforcement Branch has created the Intelligence Working Group, co-chaired by directors from EED and WED, with a mandate to examine intelligence roles and responsibilities, and make recommendations for improving consistency and overall performance in the intelligence function. The Working Group will produce a report by October 2009, with implementation of accepted recommendations by the end of 2009–2010.
1 The planning and data collection for the Enforcement Program evaluation was carried out in the 2008–2009 fiscal year, prior to implementation of Treasury Board’s new Policy on Evaluation in April 2009. The current evaluation reflects the issues outlined in Treasury Board’s 2001 evaluation policy, which was in effect at the time this evaluation was conducted.
2 The mandate of the Environmental Enforcement Intelligence Program had been previously expanded to include the Public Safety and Anti-Terrorism (PSAT) agenda, and funding was provided to enhance and improve this program. A decision was taken to evaluate the entire Intelligence Program—rather than just its PSAT component—as part of the evaluation of the Enforcement Program.
3 While Wildlife Enforcement Directorate (WED) officers did not receive a standardized basic training course during the time frame of this evaluation, WED recently created and launched Standardized Wildlife Enforcement Officer Training (SWEOT) as a prototype in January 2009.
4 Intelligence involves ongoing information collection and analysis of emerging non-compliance issues within regulated sectors to support inspections and investigations.