Evaluation of the Environmental Damages Fund

July 2009

Report Clearance Steps

Planning phase completed:

November 2008

Report sent for management response:

June 2009

Management response received:

July 2009

Report completed:

July 2009

Report approved by Departmental Evaluation Committee (DEC):

July 2009

Acronyms used in the report

AEB
Audit and Evaluation Branch
ADM
Assistant Deputy Minister
CATI
Computer-assisted Telephone Interviewing
CA
Contribution Agreement
CAPE
Community Action Programs for the Environment
CEPA 1999
Canadian Environmental Protection Act, 1999
DEC
Departmental Evaluation Committee
EC
Environment Canada
ECCAP
Environment Canada Courts Awards Project
EDF
Environmental Damages Fund
EPAM
Environmental Protection Alternative Measures
FTE
Full-time Equivalents
G&C
Grant and Contribution
MBCA 1994
Migratory Birds Convention Act, 1994
MIS
Management Information System
OPG
Outcome Project Group
OPP
Outcome Project Plan
OPSC
Outcome Project Subcomponent
RDG
Regional Director General
RMAF
Results-based Management and Accountability Framework
RMS
Results Management Structure

Acknowledgments

The Evaluation Project Team, including Robert Tkaczyk and Conrad Winn, the lead researcher from COMPAS Research Inc., led by William Blois under the direction of the Director, Shelley Borys, would like to thank those individuals who contributed to this project and particularly all interviewees who provided insights and comments crucial to this evaluation.

Prepared by COMPAS Research Inc. and the Evaluation Division, Audit and Evaluation Branch.

Table of Contents


Top of Page

Executive Summary

Environment Canada’s (EC) Evaluation Division, Audit and Evaluation Branch, contracted COMPAS Research Inc. to conduct an evaluation of Environment Canada’s Environmental Damages Fund in 2008–2009. This project was selected for inclusion in Environment Canada’s Risk-based Audit and Evaluation Plan 2008–2011, approved by the Departmental Evaluation Committee in May 2008, to fulfill a requirement for an evaluation prior to the renewal of the terms and conditions of the Environmental Damages Fund program.

The Environmental Damages Fund (EDF) was first established in 1995 to provide a general framework and guidelines for the management of monies stemming from the prosecution of environmental infractions under federal law in cases where the courts have stipulated that the awards are to be used for specific purposes. The purpose of any contribution to the EDF is to remediate pollution-damaged natural environments in a cost-effective way, in accordance with conditions specified by the courts or other compensation regimes. Funding to the EDF may come from court awards or out-of-court settlements, voluntary contributions, and contributions from national and international liability funds.

The objectives of the EDF are achieved through financial support to appropriate restoration projects or alternative activities that are proposed by eligible organizations. Program delivery flows from four streams of activity and outputs related to: communications and promotion; partnerships; proposal planning and delivery; and project delivery. The three broad outcomes of the program are:

Evaluation Issues

The Evaluation of Environment Canada’s Environmental Damages Fund assessed the relevance, success, cost-effectiveness, and design and delivery of the program. The evaluation was designed to determine whether the program is:

Methodology

Data were collected for the evaluation using multiple lines of evidence. These included a document/file review, a review of project files (n = 35), and a total of 116 key informant interviews conducted with Environment Canada personnel (n = 46), funding recipients (n = 40), judicial respondents (prosecutors, judge and federal counsel) (n = 6), environmental academics (n = 9) and potential funding recipients (n = 15). Key challenges to conducting this evaluation included the absence of a centralized information management system, limited information with which to assess cost-effectiveness, and the absence of a comparison group for funded projects.

Evaluation Findings

Overall, evaluation findings indicate that the EDF is highly relevant to the Government’s mandate and departmental priorities. The program, however, is not adequately resourced to ensure its consistent regional implementation and in particular, its promotion among the judiciary and potential project proponents. The program nonetheless demonstrates progress toward the achievement of outcomes in a cost-effective manner. Program improvements may be realized through the dedication of adequate resources for program delivery in all regions, refinement or development of existing tools and processes (e.g., application forms, training), the initiation of a communications strategy targeting key stakeholder groups, and stronger national oversight to ensure consistency of program delivery.

Evaluation findings are summarized in the following sections by evaluation issue.

a) Relevance

The EDF is highly relevant in terms of supporting federal priorities related to environmental enforcement and departmental priorities related to natural resource rehabilitation, environmental quality improvement and conservation, research and development, and education and awareness. The vast majority of key informants feel the EDF serves an important role and does not duplicate other existing programming. As well, the evidence suggests the program is the only federal mechanism of its kind for managing court awards and financial penalties arising from federal environmental enforcement activities, is expected to play an expanded role in the management of court awards prosecuted under different federal statutes, and supports the court’s use of creative sentencing in environmental damage cases.

b) Design and Delivery

The EDF is generally perceived to be delivered as designed, although low familiarity with the program and a lack of supporting documentation for its use in the courts may reduce its application when prosecuting environmental infractions.

With the exception of Atlantic Canada, the absence of dedicated resources has resulted in less than full implementation of the EDF program in the regions, such that the pace at which funds are being diverted to the program through the courts outstrips the program’s ability to allocate funds to EDF projects. The solution to uneven regional implementation is generally thought to involve an increase in human resources dedicated to the program, which should improve program promotion in the courts and among potential applicants, accessibility of staff to support funding recipients and the ability of program staff to respond to court-imposed funding conditions.

Awareness of the EDF is low among both the courts and potential applicants to the program, and this appears to be largely the result of a lack of dedicated resources. Low program awareness is thought to limit the number of court awards being diverted to the program, lead to a higher rate of court-imposed conditions, and result in fewer applications for EDF funding. Possible strategies for raising awareness among the courts include training for prosecutors, communications activities targeting judges, and engaging enforcement officers in efforts to educate the judiciary.

The evaluation also found the following with respect to various specific aspects of the EDF’s delivery:

c) Success

The EDF is widely perceived by recipients and Environment Canada personnelto be successful in the achievement of program outcomes—a finding confirmed by administrative files demonstrating clear achievement of project deliverables and outcomes in a majority of cases. Evidence from key informants and project files suggests that the program is most successful in contributing to restoration objectives, and is somewhat successful in contributing to education and awareness and research and development outcomes as well. The primary factors associated with the achievement of different outcomes concern the resources, number of projects and number of applications targeting the priority area.

The existence of clear documentation pertaining to the role recipients are required to play in the financial oversight of funded projects, as well as the high rates of projects conforming to audit and payment authority specifications, suggests the program is being administered with good financial oversight and accountability. While generally clear and well understood, the transparency of the funding process could be enhanced through more consistent staff support and better communications about court awards and funding decisions between Enforcement and EDF personnel.

Resources to support the administration of the program overall are not tracked in any systematic fashion, although regions are tracking the number and value of court awards and contribution agreements. Past research suggests that overall financial accountability and transparency would be enhanced through the development of an information management system, which is already under development.

There has been no apparent progress in increasing awareness of the program among the legal community and potential applicants or in increasing the court’s use of the EDF as a sentencing tool. Although there has been some effort to make information related to EDF projects in the Atlantic region public, this information was incomplete and out-of-date and no similar information was found for other regions.

Efforts have been made to expand the use of EDF in the context of federal legislation controlled by other departments. Although implicated departments (i.e., DFO and Transport Canada) are involved in the technical review of project applications, information transfer between federal partners could be improved by sharing more information on final funding decisions.

The EDF program employs a number of controls and processes to ensure the quality and feasibility of project proposals. Evidence suggests also that guidance documentation and program staff support are contributing to the acquisition of proposal and planning skills by community groups, as evidenced by the high rates with which projects achieve intended deliverables, the consistency of proposals with EDF objectives, the use of partnerships, and the ability of projects to leverage partner resources.

d) Cost-effectiveness

Overall, the evidence suggests that the EDF is cost-effective. Projects have been successful at leveraging funds from other sources and some limited file review evidence suggests that individual projects are more likely than not to be evaluated as cost-effective. At the program level, no other federal programs exist to manage specified-purpose funds to carry out environmental restoration activities and overall administrative spending is low, particularly in light of the national scope and specificity of court conditions related to funded projects.

Considering the low average dollar value of many EDF contribution agreements, the evaluators also explored the merits of introducing a minimum threshold value for project funding in order to increase overall program efficiency. The evaluation evidence is inconclusive concerning the advantages of such a minimum value for EDF contribution agreements. While cost-efficiencies might be realized by imposing a minimum value for EDF-funded projects, doing so could impair small projects whose requirements for restoration or other environmental contributions could be lower than a potential award threshold.

Recommendations

Six recommendations were developed for the ES Board based on the evaluation findings and conclusions.

  1. An allocation of dedicated staff and budget to support the ongoing management and administration of the EDF program should be considered. Uneven regional implementation of the EDF program can be traced, at least in part, to the absence of dedicated resources to support its implementation, and this in turn may affect the consistency of support to recipients. Further, resource constraints are felt to represent a key barrier to the proper promotion of the program among potential applicants, thus limiting the pool of potential funding recipients able to carry out court requests through the program.
  2. Standardized tools and program documentation for managers and funding recipients should be updated to ensure consistent program implementation. In updating these tools, consideration should be given to:
  3. Means of streamlining the program application process and forms should be explored to reduce the burden on recipients and enhance consistency with other community partnership programs. To this end, the EDF could consider research (surveys, focus groups, etc.) with recipients to explore ways in which the application form could be improved. Alternatively, a comparison with the application forms for similar funding programs (e.g., EcoAction) may yield information to guide such changes (e.g., word limits and question simplification).
  4. A communications plan should be developed and implemented to promote the EDF among potential recipients, the judiciary and enforcement officers. The communications plan should consider the development of standardized communications materials that can be modified, as appropriate, to address regional circumstances, as well as the development of national communication activities to enhance efficiency and consistency of messaging nationally. Efforts to promote the program among potential recipients would likely yield higher quality and a greater variety of project proposals. As well, communications efforts targeted toward judges and, to a lesser extent, enforcement personnel may increase the number of awards and other contributions to the EDF and help to reduce the number of court-prescribed funding restrictions.
  5. A centralized national information management system should be developed to track financial and performance information for EDF contribution agreements across regions. The development and implementation of such a system is essential for proper measurement, evaluation and reporting of program performance.
  6. A formal process for annual EDF performance measurement and reporting should be implemented. No consistent national process exists for reporting on the program’s impacts and effectiveness. In particular, such a reporting process is necessary to monitor progress toward full national implementation and the timely disbursement of funding in proportion to program awards. As a first step in developing this process, the program logic model should be revised to better reflect intended outcomes and more clearly distinguish outcomes from activities and standard corporate functions (e.g., financial oversight).

Management Response

The ES Board accepts the evaluation and its recommendations, and the EDF program has provided a plan to implement the following management actions in response to the evaluation recommendations within the context of EDF program renewal.

Note to Reader

Bill C-16, the Environmental Enforcement Act, received Royal Assent on June 18th, 2009. Among other things, this Bill amends six EC statutes and three Parks Canada statutes to direct all fines to the EDF. Once these amendments come into force new minimum fines will start at $5,000 for individuals and $25,000 for corporations and maximum fines will increase to $1 million for individuals and $6 million for corporations. The amendments also allow for a portion of the fines to be used for administering the EDF Program.

It is anticipated that these amendments could significantly increase the monetary value and number of awards directed to the EDF and these changes will be factored into the renewal of the program.

  1. An allocation of dedicated staff and budget to support the ongoing management and administration of the EDF program should be considered. Uneven regional implementation of the EDF program can be traced, at least in part, to the absence of dedicated resources to support its implementation, and this in turn may affect the consistency of support to recipients. Further, resource constraints are felt to represent a key barrier to the proper promotion of the program among potential applicants, thus limiting the pool of potential funding recipients able to carry out court requests through the program.

The ES Board agrees with this recommendation.

Under the Environmental Enforcement Act (Bill C-16), which is expected to come into force in 2011–2012, the program will have the option to use funds directed to the EDF for administrative purposes.

The business case to support program renewal will be requesting administrative resources, including dedicated staff and operational and management costs. The request for program renewal will be presented in September 2009.

Securing resources for the EDF will ensure that each region has dedicated staff to deliver the program. This in turn will help to bring national consistency in the delivery of the program and ensure that proper support is available to clients. These resources will enable staff to effectively promote the EDF program to targeted audiences and diversify the pool of potential funding recipients.

Timeline

Deliverable

Responsible Party

September 2009

Present the EDF business case and documentation related to program renewal.

Director, Outreach

2. Standardized tools and program documentation for managers and funding recipients should be updated to ensure consistent program implementation. In updating these tools, consideration should be given to:

The ES Board agrees with this recommendation.

As part of the program renewal package, the program is recommending the approval of the EDF Management Framework and the Funding Agreement Terms and Conditions for the Environmental Damages Fund Program. The management framework provides an overview of how the program works, the program logic model, performance measurement, and reporting and risk management strategies. The funding agreement terms and conditions set forth the terms and conditions under which the EDF program will provide funding for projects.

The program renewal package outlines a number of commitments to streamline administrative processes to ensure consistency across all regions:

  1. Development of operational guidelines to outline program governance, roles and responsibilities of program staff and clarify partnership requirements for the EDF program, as partnerships are considered an advantage but not essential.
  2. Development of regional management plans that define top priorities for potential EDF projects and enable the program to direct funding to local environmental priorities that represent the best use of funds in response to environmental damage or damage-to-wildlife incidents. Regional management plans will be prepared in consultation with EC regional management teams and include other government departments as required. The plans will be approved by the RDG.
  3. Development of standard project indicators to guide managers and project proponents in measuring project success. These indicators will contribute to the evaluation of program results.

Timeline

Deliverable

Responsible Party

September 2009

Present the EDF business case and documentation related to program renewal

Director, Outreach

November 2009

Develop program operational guidelines

Director, Outreach

Fall 2009

Develop regional management plans

Regional Directors General

Fall 2009

Conduct project indicators workshop to develop standard project indicators

Director, Outreach

3. Means of streamlining the program application process and forms should be explored to reduce the burden on recipients and enhance consistency with other community partnership programs. To this end, the EDF could consider research (surveys, focus groups, etc.) with recipients to explore ways in which the application form could be improved. Alternatively, a comparison with the application forms for similar funding programs (e.g., EcoAction) may yield information to guide such changes (e.g., word limits and question simplification).

The ES Board agrees with this recommendation.

The EDF program will take a more directed approach to soliciting applications for projects. As outlined in the response to the second recommendation, the program will develop regional management plans that define top priorities for potential EDF projects and enable the program to direct funding to local environmental priorities that represent the best use of funds in response to environmental damage or damage to wildlife. Project proposals will be solicited on an on-going basis as EDF awards are received.

As a part of the EDF program renewal, the program has committed to streamlining the EDF administrative processes, including reducing the burden of application and reporting processes on funding applicants. The EDF program will participate in the Departmental Action Plan to Reform the Administration of Grants and Contributions, which has the goal of simplifying and streamlining application processes and improving forms for grants and contributions (G&Cs), while strengthening accountability. It is important to note that the scope and timelines of the commitments in relation to the Departmental Action Plan to Reform the Administration of Grants and Contributions are beyond the program’s control.

Timeline

Deliverable

Responsible Party

Fall 2009

Develop Regional Management Plans

Regional Directors General

Fall/winter 2009–2010

Review application and reporting processes

Director, Outreach

2011–2012

Adopt applications and reporting processes resulting from G&C reform

Director, Outreach

4. A communications plan should be developed and implemented to promote the EDF among potential recipients, the judiciary and enforcement officers. The communications plan should consider the development of standardized communications materials that can be modified, as appropriate, to address regional circumstances, as well as the development of national communications activities to enhance efficiency and consistency of messaging nationally. Efforts to promote the program among potential recipients would likely yield higher quality and a greater variety of project proposals. As well, communications efforts targeted toward judges and, to a lesser extent, enforcement personnel may increase the number of awards and other contributions to the EDF and help to reduce the number of court-prescribed funding restrictions.

The ES Board agrees with this recommendation.

As part of the program renewal strategy, the program has committed to developing a promotional strategy that targets judges and prosecutors, enforcement officials from other federal government departments and potential funding recipients. In addition, a national communications plan will be created to ensure national consistency in efforts to promote the EDF program.

Overview of the EDF Promotional Strategy

Judges and Prosecutors

Enforcement

Potential Recipients

Timeline

Deliverable

Responsible Party

Fall 2009

Implement national EDF communications plan

Director, Outreach

Summer 2009

Develop EDF promotional strategy

Director, Outreach

July 2009

Participate in the CAPE Web portal

Director, Outreach

August 2009

Develop and distribute EDF fact sheet

Director, Outreach

Fall 2009

Disseminate EDF letter to prosecutors by the Department of Justice National Environmental Prosecutions Coordinator

Director, Outreach

Fall 2009

Complete Web site improvements

Director, Outreach

Winter/spring 2010

Develop enforcement training materials and tools

Director, Outreach

Spring/summer 2010

Develop information in conjunction with the Judicial Outreach Program for the National Judicial Institute

Director, Outreach

Fall/winter 2009–2010

Develop EDF brochure

Director, Outreach

5. A centralized national information management system should be developed to track financial and performance information on EDF contribution agreements across the regions. The development and implementation of such a system is essential for proper measurement, evaluation and reporting of program performance.

The ES Board agrees with this recommendation.

As a part of the Department’s G&C reform initiative, an online application and information management system is being developed for its G&C programs. This system will improve program efficiency, enhance alignment with departmental priorities and improve the ability to report collectively on the results of departmental funding programs. This system is scheduled to be in operation by 2011–2012, after which the EDF program will adopt it to collect program data.

In the meantime, the program acknowledges that a consistent and effective mechanism for collecting and tracking program information and results is required. In April 2009, the program implemented a Management Information System (MIS), cloned from the existing EcoAction Community Funding Program MIS. The MIS allows program staff to track project information including recipient information, funding agreements, project funding, and project results. Planned modifications and improvements will allow the program to track the source of monetary contributions, the legislation used and whether EC enforcement officials were involved. The system will facilitate reporting and allow project managers to track communications with funding recipients. The use of this tool will ensure consistency of data collection across the regions and significantly improve the program’s ability to track and report results. It has been launched in all EC regions and is currently being populated with historical as well as current project information. This system will be used until the new G&C online system is up and running.

Timeline

Deliverable

Responsible Party

Fall 2009

Complete implementation of MIS

Director, Outreach

2011–2012

Adopt the departmental G&C information management system

Director, Outreach

6. A formal process for annual EDF performance measurement and reporting should be implemented. No consistent national process exists for reporting on the program’s impacts and effectiveness. In particular, such a reporting process is necessary to monitor program progress toward full national implementation and the timely disbursement of funding in proportion to program awards. As a first step in developing this process, the program logic model should be revised to better reflect intended outcomes and more clearly distinguish outcomes from activities and standard corporate functions (e.g., financial oversight).

The ES Board agrees with this recommendation.

The program recognizes that a formal reporting process is required to reflect the success and progress of the EDF program. In response to this need, a revised logic model, performance measurement framework and reporting strategy have been developed and included in the Management Framework as part of the program renewal. The program has also committed to developing project-specific indicators to assist in the effective measurement of project results.

Program results will be collected by NCR program staff on an annual basis and presented to the ES Board. The information will capture the number of projects that were funded and the results achieved, including environmental impacts. The program will also demonstrate its administrative effectiveness with respect to national implementation as well as the timely allocation of funding.

A departmental performance measurement framework has also been developed and the EDF has identified three expected results and three corresponding indicators against which it will report on an annual basis in the Departmental Performance Report.

The expected results are:

The indictors that will be used to measure these results are:

As previously mentioned the program has developed an electronic management information system that will be used to track program results and help facilitate reporting and ensure consistency in reporting practices across all regions. The system will be used to roll up annual results and for program evaluation purposes.

Timeline

Deliverable

Responsible Party

September 2009

Present the EDF business case and documentation related to program renewal

Director, Outreach

Fall 2009

Conduct project indicators workshop to develop standard project indicators

Director, Outreach

Fall 2009

Complete implementation of MIS

Director, Outreach


Top of Page

 

1.0 Introduction

The Evaluation Division of the Audit and Evaluation Branch of Environment Canada commissioned COMPAS Inc. to conduct an evaluation of Environment Canada’s Environmental Damages Fund (EDF) program. This project was selected for inclusion in Environment Canada’s Risk-Based Audit and Evaluation Plan 2008–2011, approved by the Departmental Evaluation Committee in May 2008, to fulfill a requirement for an evaluation prior to the renewal of the terms and conditions of the EDF program. The previous evaluation was completed in 2002.

This document presents the findings and recommendations of the evaluation 1. It is organized as follows: background, including the origins and administrative setting of the Environmental Damages Fund in Section 2; evaluation design, including issues and sources of evidence in Section 3; findings, including sub-sections devoted to relevance, design and delivery, terms and conditions for EDF contribution agreements, success, cost-effectiveness and Environmental Protection Alternative Measures, the negotiated settlements known as EPAMs, in Section 4. Sections 5 and 6 respectively present the conclusions and recommendations stemming from the evaluation.


Top of Page

2.0 Background

The overarching goal of the Environmental Damages Fund (EDF) is to improve the quality of Canada’s environment. This is accomplished through the financial support of projects that restore environments damaged by spills of hazardous substances, discharges of industrial and municipal wastes, and the disruption of fragile habitats by human activities, as well as studies and conservation actions with the special purpose of protecting and conserving migratory birds and species at risk.

COMPAS Research Inc. was contracted to undertake an evaluation in concert with the Evaluation Division of the Audit and Evaluation Branch of Environment Canada. COMPAS performed the evaluation with oversight from an evaluation committee that consisted of representatives of the Evaluation Division of the Audit and Evaluation Branch and the EDF program.


Top of Page

2.1 Environmental Damages Fund (EDF): Origins, Functions and Evaluation

2.1.1 Origin and Legal Setting

The accidental release or deliberate discharge of environmental contaminants such as oil, chemical products, and industrial wastes can have a variety of adverse effects on the biophysical components of the environment. When pollution incidents occur resulting in environmental damage, courts may use a number of pieces of federal environmental legislation to obtain compensation from the responsible party for damages incurred. The compensatory awards received as a result of prosecutions may be directed to the Environmental Damages Fund for the purpose of rehabilitating or restoring damaged environments.

In 1993, Environment Canada was requested to work with the Treasury Board of Canada Secretariat to develop guidelines for the management of court orders, awards or other financial compensation for damage to the environment. In response to this request, the Environmental Damages Fund was first established in 1995 to provide a general framework and guidelines for the management of monies stemming from the prosecution of environmental infractions under federal law in cases where the courts have stipulated that the awards are to be used for specific purposes 2 (see Annex 2 for a summary of EDF guidelines for contributions). Through the EDF, sub-accounts are established for individual awards, with the funds being dispersed as appropriate to support the specified purpose for which the award was originally made. The funds can be dispersed either through contributions or contractual payments that are consistent with the purpose to which the funds are required to be applied 3.

In addition to court awards, funds can be diverted to the EDF through the use of Environmental Protection Alternative Measures (EPAMs). A 2007 process analysis of the EDF program describes EPAMs as:

“an alternative to court prosecution for a violation of the Canadian Environmental Protection Act, 1999. EPAMs divert the accused away from the court process after charges have been made by allowing for the negotiated settlement of certain offences between an accused and the Crown after charges are laid but before a prosecution is conducted. The accused must consent to negotiate and must accept responsibility for the offences. Under the EPAM process an agreement is filed with the court that outlines the agreement penalties. Although EPAMs have resulted in funds to the Environmental Damages Funds, the majority of the funds continue to result from court-directed contributions.” 4

The purpose of any contribution to the EDF is to achieve remediation of pollution-related damages to the natural environment in a cost-effective way, in accordance with conditions specified by the courts or other compensation regimes. Funding to the EDF may come as a result of court awards or out-of-court settlements, voluntary contributions, and contributions from national and international liability funds, such as the Ship-source Oil Pollution Fund 5 and the International Oil Pollution Fund 6. Court awards are directed to the EDF through various legislative acts, including the Fisheries Act; the Canada Shipping Act, 2001; the Migratory Birds Convention Act, 1994; and the Canadian Environmental Protection Act, 1999 7.


Top of Page

2.1.2 Program Delivery

The goals and objectives of the EDF are achieved through financial support to appropriate restoration projects or alternative activities that are proposed by eligible organizations. To receive funding, project proponents must demonstrate that the project is “technically sound, feasible, and cost-effective” and that the proponents themselves possess “the organizational capacity, skills and knowledge necessary to successfully implement and carry out the planned activities.” 8

Activities and outputs related to the delivery of the program fall within one of four categories:

EDF activities and outputs contribute to the successive attainment of short and medium-term outcomes, which in turn are logically associated with the achievement of three long-term outcomes:

A detailed logic model of the EDF program, which provides a visual representation of the linkages between the program’s activities and outputs and the achievement of its outcomes, is presented in Annex 1 9.


Top of Page

2.1.3 Eligible Organizations and Priority Projects

Organizations that are eligible to receive EDF funding include non-governmental organizations (e.g., community-based environmental groups), Aboriginal groups, universities, and provincial, territorial and municipal governments 10. EDF funding is used to support projects and initiatives that fall within one of the four following priority areas: 11


Top of Page

2.1.4 National Implementation of the EDF

At the time of a 2002 evaluation of the EDF, Atlantic Canada was the only region where the program was fully operational. As such, the 2002 evaluation focused only on this region. The four principal recommendations stemming from the 2002 evaluation aimed at encouraging broader and more consistent implementation of the EDF were:

In the years following the 2002 evaluation, results of follow-up exercises to ascertain progress toward achieving these management actions revealed that management actions related to the first two recommendations had been implemented. Management actions related to the latter two recommendations, however, are not yet fully implemented and so are considered in the context of the current evaluation. In particular, while the EDF has clarified the statutes under which prosecution may make use of the provisions of the Environmental Damages Fund, it is unclear how or even if this information is being communicated to the Department of Justice’s Regional Office Group Heads for Prosecution for dissemination to all prosecutors within their areas of responsibility. As well, the management response to the fourth recommendation also noted the need to further investigate and clarify applications to Environmental Protection Alternative Measures (EPAMs), but it remains unclear whether or not and to what extent this action may have been taken 12.

There exist no national or regional resources dedicated to the delivery of the EDF in the Department’s regions. The EDF is delivered by the Community Funding program, which has responsibility for the delivery of several other funding programs including EcoAction, Ecosystems Initiatives and various other education and engagement activities 13. Resources to manage and deliver the EDF are drawn from A-base funding allocated for the delivery of these other community-based funding programs. Consequently, the EDF has been rolled out to differing extents in the other Environment Canada regions (Pacific and Yukon, Prairie and Northern, Ontario, and Quebec) as not all regions have had the same capacity and ability to support the program’s administration. The absence of dedicated resources for program administration has also meant that administrative costs for program delivery have not been tracked in a consistent fashion.

Furthermore, implementation of the EDF may have been delayed somewhat in Ontario Region as a result of the existence and use of a competing program. Until 2007, when imposing fines for transgression of the Canadian Environmental Protection Act, 1999 (CEPA 1999) in Ontario, the courts could use the Environment Canada Courts Awards Project (ECCAP), a separate mechanism for making awards that was available only in Ontario Region. Under ECCAP, EC enforcement officers in Ontario would recommend to the court an appropriate beneficiary of an award. The award went directly to the recipient organization 14, which was not subject to the oversight possible under awards transmitted through the EDF.


Top of Page

2.2 EDF Administration

EDF activities are undertaken under the direction of the Ecosystem Sustainability Board. Within the departmental Program Activity Architecture, EDF is under the sub-activity Outreach Program and sub-sub-activity Community-based Funding. Funding agreements are established under the terms and conditions of a class grants and contributions authority Contributions to support Environment and Sustainable Development Initiatives. It is designed to be a national program, delivered regionally across the Department, with representatives in all five regions of the country.


Top of Page

2.2.1 Roles and Responsibilities

Applications for EDF funding are reviewed for their scientific and technical merit by experts from Environment Canada, and in some cases, experts from other government departments. Funding recommendations are made by the EDF regional program managers to the regional directors general of Environment Canada.

Over the time period examined for the current evaluation, departmental activities were organized using a results management structure (RMS) divided into: Outcome Project Subcomponents (OPSCs), the smallest element of the results structure where short-term results are articulated; Outcome Project Plans (OPPs), which embody medium-term results statements to which OPSCs were attached; and higher order outcomes represented by Outcome Project Groups (OPGs), comprised of groupings of OPPs. All levels of the RMS were aligned with the Department’s strategic outcomes.

Formal responsibility for program administration and accountability rests with the Minister of the Environment, as detailed in Table 2.1. The OPG lead (i.e., director general) and regional directors general bear responsibility for broad program oversight of such things as project solicitation, funding approval and allocation, and overall program performance. Mid-level accountabilities for national implementation, performance reporting, oversight of audits and evaluations, and guidance for program management, delivery and promotion rests with the relevant OPP and OPSC leads (i.e., director and manager, respectively). The day-to-day operations of the fund are managed by regional program managers. Regional program managers are also responsible for coordinating regional technical review working groups, evaluating proposals, tracking awards and disbursements, promoting regional awareness, and other operational tasks.

Table 2.1 Program Administration: Roles and Responsibilities

Program authority

Responsibilities

Minister of the Environment

OPG lead and regional directors general

OPP lead 1C3D and OPSC lead L266

Regional program managers

Source: Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework, Table 2. Ottawa.


Top of Page

2.2.2 Monetary Contributions to Date and Prospective Financial Resources

Table 2.2 displays financial data (court-funded awards and disseminated contributions) nationally and by region for each of the nine years of the program’s operation. This distribution demonstrates an uneven regional pattern of both awards directed to the EDF and contribution agreements disbursed through the program. As discussed in Section 4.2, both the number of awards (i.e., as a result of program promotion in the courts) and contribution agreements are contingent to some extent on the resources available for regional administration of the program.

The other especially noteworthy pattern in Table 2.2 is the evidence of greater funds assigned to EDF than contributions made by the EDF. The total value of court awards and other funds directed to the EDF to date amounts to approximately $4.1 million, compared to approximately $3.6 million in contributions disbursed by the EDF.

Future monetary contributions from court awards and other sources cannot be predicted with certainty. Awards may nonetheless grow to the extent that the EDF becomes better promoted to the legal community and more fully utilized by the courts. As well, recent amendments (Bill-C15) to the Migratory Birds Convention Act, 1994 (MBCA 1994) call for fines and penalties resulting from prosecutions of offences under the Act to be credited to the EDF. In addition, Section 5.1 of the Act with respect to the pollution of waters or areas frequented by migratory birds has been strengthened. As such, fines received under this section, although less common, have the potential to be of a sizeable amount, up to a maximum of $1 million. In the ten years before the amendments came into effect, only one award, in the amount of $5,000 was made. Since the amendments came into effect four years ago, 11 awards totalling $90,143 have been directed to the EDF, resulting in nine projects across Canada.

The Policy on Specified Purpose Accounts stipulates that “Administrative charges may be imposed on administered funds (for example, trust funds, endowments, joint project funds) when the legislation or other enabling authority allows it.” However, the legislation under which court awards and other funds can be directed to the EDF do not specifically authorize the cost of administration to be charged to the account, and so the costs for the administration of the EDF are “charged to a departmental appropriation.” 15

Table 2.2: Number and Value of EDF Awards and Contributions by Region


Top of Page

3.0 Evaluation Design

3.1 General Purpose

The evaluation assessed progress in achieving the Department's expected environmental outcomes for the EDF, with specific emphasis on developments since the last evaluation. The timeframe for the evaluation focuses on the period between August 2002, when the last evaluation took place, and March 2008.

The study explored the relevance, design and delivery, success and cost-effectiveness of the EDF program. Specifically, the evaluation sought to determine whether the program is:

Furthermore, the evaluation explored the extent to which 13 terms and conditions for the EDF contribution agreements (see Annex 2), established from the outset of the EDF program, had been satisfied 17.

To address these questions, the evaluation involved a variety of data-collection approaches, including a document review, a review of contribution agreement files and key informant interviews. The review of EDF project files is the sole source of data for determining if most of the EDF terms and conditions for contribution agreements had been met. As well, the document review was the primary source of evidence used to address issues related to the relevance of the program, as this issue was fully explored in the 2002 evaluation and the 2007 process analysis 18. The relevance-related conclusions stemming from these two studies remain pertinent because there has been no change in the Government of Canada’s constitutional role on matters of the environment or in EDF’s role relative to the courts. Nonetheless, some qualitative evidence from key informant interviews was gathered to further corroborate these conclusions.

Annex 3 presents the full series of evaluation questions, enumerates sources of evidence, and identifies relevant challenges considered by the evaluation team in investigating each question.


Top of Page

3.2 Evaluation Approach and Methodology

Three sources of evidence were necessary for the evaluation: background documents, administrative and financial files for projects, and key informant interviews. Each of these data-collection approaches is described in turn below.


Top of Page

3.2.1 Document Review

A primary source of information for the evaluation involved a review of existing documentation. The review included:

The document review responded to a number of evaluation issues primarily related to the EDF’s ongoing relevance and design and delivery. A bibliography of documents included in the review appears in Annex 4.


Top of Page

3.2.2 Review of Administrative and Financial Files

Administrative and financial files from EDF-funded projects were reviewed to explore issues related to the success and design and delivery (i.e., performance measurement, partnerships and accountability) of the EDF, as well as to assess the degree to which EDF agreements satisfy EDF terms and conditions for contribution agreements.

The files from a sample of 35 EDF-funded projects were analyzed using a customized file review guide to ensure the accuracy and consistency of the review (see Annex 5). The sample of 35 projects was selected to reflect: (a) projects from all regions, with oversampling for regions with fewer projects 19, (b) the diversity of purposes and subjects of funded projects, and (c) the diversity of recipients that received EDF funding.

Overall, a large majority of project files were found to contain information to address evaluation questions related to nearly all evaluation questions addressed through this methodology. Only one in four (23%) project files contained any information on unintended outcomes, although this is not surprising considering that this information is not required nor is it necessarily relevant (i.e., when no such outcomes exist) for all projects.

With regard to information demonstrating compliance with the terms and conditions of the EDF contribution agreements, large majorities of project files provided information on recipient eligibility/qualifications, project eligibility (i.e., maximum value, allowable expenditures), use of partnerships, and proposal review (see Table 3.1). Less evidence was available for those terms and conditions that may not be applicable to all projects, such as consistency with court conditions and ownership of equipment purchases. As well, little evidence was found concerning project cost-effectiveness or payment invoices.

Table 3.1: Existence of Information from Project Records
by Evaluation Question and EDF Contribution Guidelines
Evaluation Question % of
projects
*This criterion appears as a single item in the EDF terms and conditions, but was
assessed separately for the evaluation.
**The prohibited basic costs are those costs that are necessary for maintaining the operation of the recipient organization.
1. Does the project identify clear deliverables and expected results?
100
2. Is performance data collected against program activities?
94
3. Is performance data collected against program outcomes?
83
4. Program outcome targeted by the EDF-funded project
100
5. Evidence of intended outcome achievement
83
6. Is there appropriate accountability framework (e.g., for multi-stakeholder agreements)?
100
7. Is the project delivered in partnership with any other individuals, groups or organizations?
74
8. Have there been any unintended (positive or negative) outcomes?
23
Terms and Conditions of Contribution Agreements
1. Recipients must be eligible to receive the monies
97
2. The contribution is consistent with application and decision
6
3. The project meets court conditions
6
4. The project is cost-effective
29
5. The recipient possesses the skills necessary to deliver the project
100
6. Projects undertaken build on partnerships
74
7. Proposals are evaluated by program managers / are signed off by RDG*
91/83
8. The maximum amount payable is under $5 million
100
9. Awards cover allowable expenditures (not basic costs)**
89
10. Payment methods are on invoice and with statement of expenditures
29
11. Equipment purchased belongs to the project
51
12. There exists a right to audit each project
91
13. There exists a payment authority
94

Top of Page

 

3.2.3 Key Informant Interviews

Key informant interviews were used to gather in-depth information on all the evaluation questions and to supplement information collected through the document and file review. A total of 116 key informant interviews were conducted with representatives from the following groups:

Customized key informant interview guides were developed in consultation with the project authority for each respondent group to ensure the guides reflected their level of familiarity with the program (e.g., past vs. potential applicants) and occupational perspectives (e.g., EC personnel vs. university-based experts) (see Annex 6). For example, detailed design and delivery-related questions were asked of EC personnel but not of potential funding recipients or key informants in the court and justice segment. Each interview guide was programmed and delivered using a Computer-assisted Telephone Interviewing (CATI) system that allowed for real-time data collection. Interviews lasted approximately 45 minutes each.

The sample frames for EC personnel, funding recipients and justice system respondents were supplied by the EDF program. The sample frame for potential applicants 21 and academic experts was developed by COMPAS. Potential applicants included community-based environmental groups, Aboriginal groups, universities, and municipal governments.

Given the busy schedules of individuals in professional or executive occupations, potential respondents from these cohorts were often unavailable to take part in interviews during the timeframe for evaluation fieldwork. For this reason, the lists of potential key informants available from administrative files were not large enough to complete an adequate number of interviews in the court and justice segment. To enhance the size of the justice-segment sample frame, the researchers also employed a referral (or “snowball”) method. Using this approach, respondents from this group were asked for the names and contact information of other potential key informants involved in the justice system and familiar with the EDF program.


Top of Page

3.2.4 Challenges and Limitations

The evaluation project involved a large number of key informants (116) and a large number of administrative files related to each of 35 EDF-funded projects. Any study of this magnitude will necessarily encounter some challenges that may or may not be possible to remedy. The following is a description of challenges and limitations of the current research, as well as steps taken to address them.

1) The identification of external experts.

A potential challenge for any evaluation arises from the tendency for experts and expertise to be concentrated within the program being evaluated. In order to satisfy a need for independent experts, COMPAS developed a list of professors whose specialization in environmental matters would provide them with some degree of expertise in the program or its subject matter. Key informants at universities who fulfilled the need for independent expertise were drawn from teaching and research faculties. Key informants at universities who fulfilled the requirement to consult potential applicants were drawn from senior university administrators.

2) The absence of a centralized information management system.

The major challenge associated with project records is a sometimes high rate of incomplete records, as shown in Table 3.1. All project records were lodged in regional offices without duplicates assembled at a central location. This geographic dispersal of record-keeping made it impossible to verify if more complete project records were available than those transmitted to the researchers under short timeframes. Given the incompleteness of the records provided, there remains some question about whether or not the findings may be underestimating compliance with EDF terms and conditions (e.g., compliance with court conditions) as well as EDF’s success in achieving various evaluation goals (e.g., related to deliverables and impacts).

3) Limited information to assess cost-effectiveness.

Limited information related to financial and human resource expenditures, possibly the result of a lack of dedicated resources for the administration of the EDF, meant a thorough assessment of the cost-effectiveness of the program was not possible. Although key informants were able to provide qualitative feedback on the cost-effectiveness of the program, very little administrative data exist to support these views.

4) The absence of a comparison group for funded projects.

Court restrictions and limited program promotion among potential applicants meant that an adequate pool of unsuccessful applicants was not available to develop a comparison group against which to assess the success of EDF-funded projects. In this manner then, it is not possible to determine conclusively whether progress toward the realization of program objectives is uniquely attributable to the EDF.


Top of Page

4.0 Findings

Findings for the following four main evaluation issues are presented serially: relevance, design and delivery, success, and cost-effectiveness. A discussion of project compliance with EDF terms and conditions for contribution agreements follows.

A rating is also provided for each evaluation question. The ratings are based on a judgment of whether or not the findings indicate that:

In addition, a tilde (~) is used to denote instances where outcome achievement ratings are based solely on subjective evidence. A summary of ratings for the evaluation issues and questions is presented in Annex 7.


Top of Page

4.1 Relevance

Overall Findings:

The EDF is highly relevant in terms of supporting federal priorities related to environmental enforcement and departmental priorities related to natural resource rehabilitation, environmental quality improvement and conservation, research and development, and education and awareness. The vast majority of key informants feel the EDF serves an important role and does not duplicate other existing programming. As well, the evidence suggests the program is the only federal mechanism of its kind for managing court awards and financial penalties arising from federal environmental enforcement activities, is expected to play an expanded role in the management of court awards prosecuted under different federal statutes, and supports the court’s use of creative sentencing in environmental damage cases.

Evaluation Issue: 1. Is there a legitimate and necessary role for government in this program area or activity?

Indicator(s)

Methods

Rating

Recent federal budgets have invested considerably in environmental stewardship and enforcement of environmental laws. In the 2007 budget, the Government invested $110 million for more effective implementation of the Species at Risk Act and $22 million to improve the enforcement of environmental protection laws 22. Similarly, the 2008 federal budget committed $21 million to enhance the effectiveness of environmental law enforcement; $12 million for law enforcement within national parks; $62 million for the health of the oceans, water pollution prevention, and improved surveillance and enforcement along Canada’s coasts; and $15 million for protected areas in the Northwest Territories 23.

The EDF is an important mechanism for the management of court awards and financial penalties arising from federal enforcement activities. The EDF is specifically identified in amendments to the Migratory Birds Convention Act, 1994 (MBCA 1994) as the mechanism to be used in managing fines and penalties resulting from prosecutions of offences under the Act. While not identified by name, the EDF can also be used for the management of court awards and penalties meted out through a variety of other legislation, including the Fisheries Act, the Canada Shipping Act, 2001, the Species at Risk Act, and the Canadian Environmental Protection Act, 1999. Furthermore, the Government has signalled its intent to expand the use of the EDF by considering amendments to six Environment Canada statutes 24 and amending three Parks Canada statutes 25 to enable awards and fines to be directed to the EDF.

The Environmental Damages Fund is the only federal mechanism of its kind for the management of court awards and financial penalties resulting from environmental damage events that have been prosecuted under these and other legislative authorities 26. What distinguishes the Environmental Damages Fund from other programs that support third parties in implementing environmental projects, such as grants and contributions programs, is its role as a tool to support the use of creative sentencing 27 by the courts in environmental damage cases. Creative sentencing allows the courts to specify how funds derived from fines and penalties for environmental infractions are to be used for beneficial environmental purposes 28. Although the creative sentencing provisions of some federal legislation allow the courts to order monies to be paid directly to local community groups to help reverse the effects of environmental damage, these funds are not managed with either the oversight or the stringent conditions on their use imposed by the EDF 29. In particular, the EDF ensures that:

The creation of the EDF offers the courts the option of using this national program as an intermediary capable of overseeing and auditing the work of organizations engaged in using EDF funds for their specified purposes, as well as finding geographically niche organizations suited to carrying out a court’s geographically specific requirements 31.

Almost all key informants consulted believe that the EDF serves a valuable societal need. Most feel the EDF does not duplicate the functions of other organizations and is uniquely placed to carry out its functions 32. Furthermore, the EDF priorities of natural resource rehabilitation; environmental quality improvement and conservation; research and development; and education and awareness are well aligned with the Department’s strategic outcome for the restoration and conservation of Canada’s natural environment.

Only two key informant respondents noted vaguely similar programs which operate at the provincial level. One judicial key informant suggested that the EDF is “roughly analogous” to the British Columbia Habitat Conservation Trust Fund, although this fund operates exclusively within the province and is funded and supported by outdoors sportspeople (e.g., hunters and anglers) rather than the courts. Similarly, one program respondent mentioned Ontario’s Community Environment Fund, which was launched in 2007 to support restoration projects using penalties exacted by the Ontario Ministry of the Environment (not the courts) for industrial wastewater spills, but not for other types of environmental damage incidents. Environment Canada’s Ontario Region also managed the Environment Canada Courts Awards (Pilot) Project until 2007. This program, however, was limited to offences prosecuted under CEPA 1999, with penalties disbursed directly to the award beneficiary 33.


Top of Page

4.2 Design and Delivery

The following section of the report presents evaluation evidence related to various aspects of the EDF program’s design and delivery. The section begins with an overview of evidence related to EDF’s general delivery, including administrative resourcing and program awareness among key stakeholder groups.

Overall Findings:

The EDF is generally perceived to be delivered as designed, although low familiarity with the program and a lack of supporting documentation for its use in the courts may reduce its application when prosecuting environmental infractions.

The absence of dedicated resources for the EDF program has resulted in less than full implementation of the program in most Environment Canada regions, with the exception of Atlantic Canada. The solution to uneven regional implementation is generally felt to involve an increase in human resources devoted to the program, which should improve promotion in the courts and among potential applicants, accessibility of staff to support funding recipients, and the ability of program staff to respond to court-imposed funding conditions.

Awareness of the EDF is low among both the courts and potential applicants to the program, and this appears to be largely the result of a lack of dedicated resources for program delivery. Low program awareness is thought to limit the number of court awards being diverted to the program, lead to a higher rate of court-imposed conditions, and result in fewer applications for EDF funding. Possible strategies for raising awareness among the courts include training for prosecutors, communications activities targeting judges, and engaging enforcement officers in efforts to educate the judiciary.

The evaluation also found the following with respect to various specific aspects of the EDF’s delivery:


Top of Page

4.2.1 Program Delivery

Evaluation Issue: Is the program delivered as designed?

Indicator(s)

Methods

Rating

Key informants from Environment Canada and the justice and courts segment were asked to rate on a 7 point scale the extent to which they perceive the EDF program is being delivered as designed. As shown in Figure 4.1, four of six key informants from the justice and court segment were strongly of the view (i.e., responding with a 6 or 7 on a seven-point scale) that the EDF is being delivered as designed, compared to only one who felt the program is delivered as designed to only a moderate extent (i.e., responding with a 3, 4 or 5). In contrast, Environment Canada staff were divided as to whether the program is being delivered as designed to a large (43% responding with a 6 or 7) or moderate extent (40% responding with a 3, 4 or 5). Almost no key informants in either group felt the EDF is not functioning as it was intended (i.e., responding with a 1 or 2 on a seven-point scale).

Figure 4.1 Perceived Consistency of Program Delivery with Design

The six key informants in the justice and court segment were also asked to describe how the EDF’s use by the courts has been consistent or inconsistent with the program’s original design. The majority of these respondents expressed some uncertainty of the program’s original design and how it was to be used, and so had difficulty in assessing whether it was being delivered accordingly. Based on what these respondents do understand of the program, most interviewees felt that the courts used the program as designed, particularly in terms of specifying conditions for how penalties are to be used to address the environmental damage. Some respondents suggested, however, that the absence of specific examples of how the funds can be and have been used means the EDF is less likely to be well received by the courts as a sentencing option and that the courts are left to their own devices to invent ways of applying the program.

Many Environment Canada key informants volunteered the view that insufficient staff support and stringent court conditions, especially geographically tight restrictions, are the biggest challenges to delivering the program as designed. In some regions, insufficient staffing is seen by many of these key informants as impeding promotion of the program among potential fund applicants, with the result that the assignment of contributions does not keep pace with the receipt of funds from the courts.

Summary: The EDF is generally perceived to be designed as delivered, although low familiarity with the program and a lack of supporting documentation for its use in the courts may reduce its application when prosecuting environmental infractions. The primary barrier to delivering the EDF was perceived by Environment Canada staff to be insufficient staff support, which has also resulted in poor program promotion among potential recipients.

4.2.1.1 Administrative Resources and EDF Development across Canada

There exist no national or regional resources dedicated to the delivery of the EDF. The EDF is one of several programs (including EcoAction and other education and engagement activities) delivered by the Community Partnerships program at Environment Canada. In the absence of dedicated resources, the program’s management is a shared responsibility among all 31 headquarter and regional Community Partnerships staff. According to administrative information, it is estimated that collectively, the Community Partnerships personnel devote the equivalent of four full-time staff to administering the program. This amounts to an average assignment of one-eighth of each Community Partnership staff person.

EC employees generally feel that the absence of dedicated resources has tended to result in less than full program implementation and uneven national delivery. In particular, while approximately $4.1 million in court awards and penalties have been diverted to the EDF over the nine years of the program’s operation, only $3.6 million has been allocated to support EDF projects over the same period (see Table 2.2). The one exception has been the virtually full delivery of the program in Atlantic Region, even though in 2008–2009, 54% of recorded pollution incidences (1,611 of 2,958 events) in Environment Canada’s NEMISIS data base on environmental emergencies occurred in the Atlantic region 34. Although not addressed directly through interviews, EC respondents commented variously that the Atlantic region’s success in implementing the EDF is attributable to its resources (i.e., experienced staff, Web site and toll-free phone number) and established relationships with partners and organizations external to the program (e.g.,community groups, EC enforcement officers and other departments). In particular, these relationships facilitate the identification of eligible and qualified project proponents, and enhance the likelihood that an informed judiciary will direct court awards and penalties to the EDF.

The predominant recommendation among EC respondents to address the issue of uneven regional implementation was an increase in human resources assigned to the program, believing that insufficient personnel in key regions was the root cause of the program’s underdevelopment in these areas. Insufficient personnel in key regions hampered the program’s ability to promote the EDF in the courts and hence increase the number of awards to the program, in their view. Insufficient personnel also hampered EDF’s ability, in the view of some key informants, to promote itself among the population of potential applicants and hence expand the pool of potential beneficiaries able to carry out court requests.

EC personnel and past EDF beneficiaries alike also reported problems of accessibility to EDF personnel for both applicants and approved beneficiaries. Both groups of key informants reported navigational problems with the Web site and delays in processing applications and paperwork. It is also possible that gaps in the records for some EDF project files, as outlined in Section 4.2.2.3 on compliance with program terms and conditions, are due in part to the absence of dedicated resources.

Another characteristic of the program which may place further strains on these limited resources involves the imposition of stringent court conditions. Court-imposed conditions on the types of projects that can be supported through court awards and penalties diverted to the EDF can require significant effort on the part of program staff to find potential recipients who can satisfy the court’s requirements and who also have the organizational and technical capabilities necessary for the tasks. To meet court-determined requirements, EDF personnel are sometimes required to search for potential funding recipients. As such, the EDF may have a greater need for administrative capacity than a program of its size would normally require.

Summary: With the exception of Atlantic Canada, the absence of dedicated resources for the EDF program has resulted in less than full implementation of the program in most regions, such that the pace at which funds are being diverted to the program through the courts outstrips the program’s ability to allocate funds to EDF projects. More resources and established relationships with stakeholders are thought to underlie the Atlantic region’s successful implementation of the program. The solution to uneven regional implementation is generally thought to involve an increase in human resources devoted to the program, which should improve promotion in the courts and among potential applicants, accessibility of staff to support funding recipients and the ability of program staff to respond to court-imposed funding conditions.

4.2.1.2 External Awareness

Evidence from key informants reveals that awareness of the EDF is low among potential EDF applicants and the courts (see Section 4.3.1 for a detailed discussion of the achievement of communications and program promotion outcomes). Previous research 35 suggests that low awareness of the program may have several negative consequences for its effectiveness. The most obvious drawback is fewer court awards being diverted to the EDF, such that the application of these funds to address environmental damage is not overseen in an accountable and neutral fashion. As well, some key informants among funding recipients, judicial respondents and EC personnel suggested that excessive court-imposed restrictions on awards and penalties diverted to the EDF may be attributable to low awareness of the program. In particular, some individuals in the small judicial segment of key informants noted that all sides in the courtroom appear to share a common mistrust of government and hence an inclination to impose restrictions. Respondents among these various groups note that if communication to the judiciary were framed in such a way that crown attorneys and judges alike understood the impacts of stringent court conditions, a supplementary benefit could be a reduction in court-mandated restrictions that make EDF projects difficult to implement. Some EC key informants further note that this could in turn lower administrative costs and reduce delays, as the absence of stringent conditions would mean projects could be initiated without searching out or even assembling recipient organizations to carry out the court-mandated task.

Low awareness among potential beneficiaries may also explain the low rate of proposals and consequent high rate (more than 90%) of acceptance of proposals. Increased awareness of EDF among potential funding recipients could lead to an enlarged pool of submissions and hence the possibility of support for more projects. This would lead to a greater balance between EDF funding from awards, approximately $4.1 million in total across the nine years of the program, and EDF spending on projects, approximately $3.6 million over the same period (see Table 2.2). It remains unclear, however, if heightened program awareness among potential recipients would result in a greater number and quality of proposals to the program, as the conditions imposed on many court awards necessarily limits the number of potential recipient organizations equipped to meet these conditions.

Key informants in the judiciary suggest several potential strategies by which EC could increase awareness of the EDF in the courts. One strategy involves working with prosecutors to develop and provide opportunities for training in environmental legislation. Other suggestions from several key informants in the judiciary include separate communications campaigns to crown attorneys and judges, as well as communicating to judges by means of a newly created workshop or at annual or semi-annual meetings and retreats attended by judges 36. One key informant spoke of a chain of education efforts beginning with enforcement officers: “We've talked to the enforcement people about the need to encourage prosecutors to recommend to judges that they not be too rigid in the conditions attached to the money.”

Summary: Awareness of the EDF is low among both the courts and potential applicants to the program, and this appears to be largely the result of a lack of dedicated resources for program delivery. Low program awareness is thought to limit the number of court awards being diverted to the program, lead to a higher rate of court-imposed conditions, and result in fewer applications for EDF funding. Possible strategies for improving awareness among the courts include training for prosecutors, communications activities targeting judges, and engaging enforcement officers in efforts to educate the judiciary.


Top of Page

4.2.2 Administration of EDF Projects

This section explores various aspects of the management and oversight of EDF-funded projects, including the application process, compliance with EDF terms and conditions for contribution agreements and the use of partnerships to support funded projects. The potential to apply a minimum project value standard for EDF-funded projects was also explored.

4.2.2.1 The Application Process

EC personnel and funding recipients tend to perceive the application process and the application form favourably, with many recipients noting that it is no more onerous than applications for other funding programs and fairly straightforward. A few funding recipients and EC personnel, however, consider the application tedious and repetitive, and disproportionate to the size of the EDF contributions: “A lot of administration for not a lot of money.”

A comparison of the EDF and EcoAction Community Funding program application forms suggests that the EDF application might be considered marginally more burdensome at most. For example, EDF applicants are asked in two separate places to describe how the environment would benefit 37. The EDF application also requires slightly more writing than the EcoAction process. The latter includes a checklist of potential evaluation indicators while the former requires applicants to describe their evaluation indicators in essay form.

EC personnel and funding recipients tended to characterize EC Community Partnerships employees who manage the EDF as being helpful and solicitous. EDF staff assistance is sometimes seen as essential. “If I hadn’t helped the applicants out,” volunteered one EC key informant, “they would still be trying to navigate the bureaucratic paperwork.” However, a few key informants note that the uneven accessibility and expertise of personnel assigned to the EDF are other factors which may contribute to the burdensomeness of the process. Some funding recipients emphasize their frustration trying to reach EC staff and elicit useful answers to their questions. As outlined in Section 4.2.1 above, it is likely that the variable quality of support to recipients is the result of a lack of dedicated resources for EDF program delivery and the consequent shared responsibility for program administration among Community Partnerships staff.

Summary: The application process is generally perceived to be appropriate, although some funding recipients and EC personnel feel it is burdensome. A comparison of the EDF application form with similar forms of other programs suggests there may be some limited room to streamline it. EDF program personnel are perceived to be quite helpful in navigating the application process, but their uneven accessibility and expertise, likely resulting from a lack of dedicated resources, may dampen the effectiveness of support to applicants.

4.2.2.2 Partnerships

Terms and conditions for EDF contribution agreements require that projects “build on partnerships,” but EDF program documentation is ambiguous on the issue. Neither the EDF Applicant’s Guide 38 nor any other literature treats partnerships as an explicit requirement. The evaluation point structure allots up to 14 percentage points for partnerships, thereby communicating by inference that partnerships are desirable but not mandatory.

Feedback from key informants is divided about the value of such partnerships. Funding recipients and EC personnel tend to be favourably disposed to the idea of partnerships while academics and potential applicants tend to view partnerships unfavourably. Key informant advocates of partnerships perceive the benefits of partnerships as contributing more resources, more technical know-how, and a greater chance of leaving behind capacity in the community after the project has been completed. The main disadvantages are thought to be the difficulties in finding suitable partners, a potential for conflict or poor performance among partners with fewer resources and less experience, and the artificiality of some apparent partnership relationships. As one academic expert pointed out, “A lot of grantsmanship [the art of successful applications] goes into giving the appearance of leveraging funds and in many cases that's not really happening.”

The review of administrative files found that just more than half (54%) of the sample of projects examined provided evidence of partnership support, either in terms of the identity of partner organizations or partners’ contributions to EDF-funded projects. Partners identified in project files tend to include a wide range of organizations, including environmental organizations, municipalities, businesses, and community organizations, and to a somewhat lesser extent post-secondary educational institutions (see Table 4.1) 39. Of these partners, more than half (58%) contribute funding to the project, while nearly half (47%) assist in service delivery and one-third (32%) contribute research. For one in five projects (22%) that identified partners, there was either no statement of the partners’ contribution (11%) or the partners’ contribution was unclear from the documentation supplied (11%).

Table 4.1: Profile of EDF Project Partnerships
Type of Partner % of sample (n=35)*
* Sum of partnership categories may exceed 100%, as multiple partners and types of partnership arrangements are possible.
Environmental organization
34
Municipality
23
Business
23
Community organization (excl. environmental organization)
23
Post-secondary educational institution
14
No partners
20
Insufficient information
26
Nature of Partnership % of partnered projects (n=19)
Funding
58
Assistance in service delivery
47
Research
32
Unclear description of partners’ obligations
11
No statement of their obligations
11

Summary: Program documentation is not clear about whether partnerships are a requirement for EDF-funded projects, as suggested by the terms and conditions for EDF contribution agreements. Key informant assessments of the value of partnerships are mixed. The key strengths of partnerships are perceived to be the leveraging of partner funding/expertise and project sustainability, while the weaknesses are felt to involve difficulties in finding suitable partners and the potential for conflict with or poor performance of partners. Nonetheless, a slim majority of EDF-funded projects were implemented in partnership, and the most frequently cited partner contributions included funding, service delivery or research.

4.2.2.3 Compliance with Terms and Conditions for EDF Contribution Agreements

The 2002 EDF evaluation found that for the Atlantic region, the program and EDF projects were being administered in compliance with all EDF terms and conditions for contribution agreements 40. Now that implementation of the program is further evolved in the other regions than was the case in 2002, the current evaluation sought to explore the extent to which funded projects across all regions are in compliance with these terms and conditions.

Table 4.2 presents data from the administrative files to assess project compliance with each of the 13 EDF terms and conditions. In several instances, the administrative files contained too little information to verify whether the conditions were satisfied; in none was there evidence of noncompliance.

Overall, information from the file review demonstrates compliance with EDF terms and conditions in the following areas, where a very high percentage of project files were found to be compliant:

For the remaining five terms and conditions (described in bulleted text below; also see Table 4.2), the high rates of incomplete information in the files, when considered in combination with evidence from other sources, do not allow researchers to reach a conclusion concerning compliance with the EDF guidelines. In particular, the absence of a centralized information management system (see Section 3.2.5 Challenges and Limitations) suggests that it is possible that the high rate of missing information reflects the poor accessibility of this information given current data management practices rather than noncompliance with these conditions. For these five guidelines, the file review found that:

Table 4.2: Project Compliance with Terms and Conditions for Contribution Agreements
Terms and Conditions Compliance Rating (%) Overall Rating
Full Part Non No Info. N/A
* In roughly 90% of cases for which ownership of equipment purchases was identified, the contribution agreement states that “the ownership of any equipment purchased by the Recipient with funds provided by the Minister under this agreement shall rest with the Recipient.” This was interpreted to mean that the equipment purchased belongs to the project.
1 Recipients must be eligible to receive the monies 97 0 0 3 0 Compliant
2 The contribution is consistent with application and decision 3 3 0 94 0 No conclusion
3 The project meets the court’s conditions 3 3 0 94 0 No conclusion
4 The project is cost-effective 20 6 3 71 0 No conclusion
5 The recipient possesses the skills necessary to deliver the project 74 26 0 0 0 Compliant
6 Projects undertaken build on partnerships 54 0 20 26 0 Compliant
7(a) Proposals are evaluated by program managers 83 9 0 9 0 Compliant
7(b) Proposals are signed off by RDG 83 0 0 17 0 Compliant
8 A ceiling of $5 million per award 100 0 0 0 0 Compliant
9 Awards cover allowable expenditures (not basic costs)* 89 0 3 9 0 Compliant
10 Payment methods are on invoice and with statement of expenditures 9 0 20 71 0 No conclusion
11 Equipment purchased belongs to the project 52* 0 0 49 0 No conclusion
12 There exists a right to audit each project 91 0 0 9 0 Compliant
13 There exists a payment authority 94 0 0 6 0 Compliant

Summary: The EDF is subject to 13 terms and conditions of operation 43. The review of project files confirms that the vast majority of projects are in compliance with most of the terms and conditions for EDF contributions. For five terms and conditions, the high rates of incomplete information, as well as evidence from other sources that suggest the conditions may have been met, do not allow the evaluators to reach a conclusion concerning compliance.


Top of Page

4.2.3 Environmental Protection Alternative Measures (EPAMs)

As previously discussed in Section 2 of the present report, Environmental Protection Alternative Measures (or EPAMs) allow for the negotiated settlement of certain offences between an accused and the Crown after charges are laid for a violation of the Canadian Environmental Protection Act, 1999 (CEPA 1999). 44

Key informants from the judicial segment and from EC 45 were asked if the use of EPAM-negotiated settlements was being encouraged and if their frequency of use would likely rise or fall in the future. EC respondents most frequently reported that they did not know and so could not comment on the promotion or expected use of EPAMs, either now or in the future. Some also indicated that they do not know what an EPAM is, which may not be surprising since they are rarely used and only then for offences prosecuted under CEPA 1999.

A lack of familiarity with EPAMs was also evident among key informants in the judicial segment, some of whom remarked that they had never been involved with an EPAM settlement or were unaware of Environment Canada’s connection to such activities. Even one judicial key informant with prior experience using EPAMs was uncertain about the current promotion of EPAMs or if prosecutors were encouraging their use.

Of the few key informants who were knowledgeable enough to comment on the use of EPAMs, opinion was divided about whether the frequency of their use would grow or diminish in the future. One EC respondent noted that with more enforcement of environmental laws, more charges will likely be laid and so their use would likely rise, while others felt that their use would diminish given that the process is perceived to be “large and unmanageable” or that pollution events are on the decline.

Most judicial respondents were equivocal about whether the use of EPAMs will increase. A couple of these respondents noted that any increase in their use will likely be limited by the possibility of EPAMs being perceived as a means for offenders to buy their way out of breaking environmental laws. Other factors that judicial respondents think may result in a decreased use of EPAMs include:

Alternatively, some judicial respondents suggested that EPAMs might be used more in light of increasing public pressure to deal with environmental offences and to demonstrate some beneficial outcome in the communities where the offense took place, as well as if heightened enforcement of environmental laws occurs for both serious and less serious environmental infractions where the use of EPAMs may be more appropriate.

Summary: EC personnel and, to a lesser extent, the judicial respondents report very low levels of awareness of the function and/or use of EPAMs. Of those respondents who are familiar with EPAMs, opinion was divided as to whether their use would grow or diminish in the future. While increased enforcement activity and pressures to deal with environmental offences suggest their use may rise, the cumbersomeness of the EPAM process and the possibility that the number of cases for which EPAMs can be appropriately applied does not increase are equally compelling reasons to predict that their use may diminish.


Top of Page

4.2.4 Accountability

Evaluation Issue: Who is accountable for the program?

Indicator(s)

Methods

Rating

A number of EDF program documents outline the roles and responsibilities of the various parties involved in the administration of the program and delivery of individual projects:

Most EDF managers and federal respondents report that the roles and responsibilities of various players involved in the EDF are clear, most often noting that there is good documentation to outline expectations and roles, which are also actively communicated to affected staff and partners. Nonetheless, roughly three in ten respondents from this group felt roles and responsibilities are not clear, or expressed some reservations in this regard. These respondents most often felt there is need for more consistency and oversight in the way the program is delivered across regions 51, noting also that efforts to provide national program oversight and coordination are relatively new. These respondents also commented on the need for better communication between enforcement officers and EDF staff to track what funds are flowing to the program and seek input on funding decisions, training for program officers, and more information about project selection, approvals and funding processes.

Similarly, while the majority of funding recipients felt their roles and responsibilities and those of the various partners involved in the delivery of the EDF program were clear, they were not always sure about the roles and responsibilities of personnel assigned to the EDF. Some recipients expressed doubt about the roles of staff because of the uneven accessibility of EDF-assigned personnel or the perception that EDF-assigned personnel were not equally familiar with the program: “Different people had different answers to some of my questions.” According to other recipients, EDF-assigned personnel seemed confused about whether their first obligation was to the courts or to the Department.

At the level of individual EDF projects, the review of project files demonstrates that the roles and responsibilities of proponents and partners are typically very well articulated. All files were found to include an accountability framework, the vast majority of which included a very clear (83%) or moderately clear (14%) description of the roles and responsibilities of project stakeholders.

Summary: A great deal of program documentation exists which outlines the roles and responsibilities of different parties involved in the delivery of the EDF program, and these are perceived to be clear for most EC personnel and funding recipients as well. Nonetheless, a number of respondents expressed some confusion in this area. In particular, respondents from both groups suggested a need for more consistency and oversight in the delivery of the program across regions (noted by EC personnel) and EDF program staff (noted by recipients).


Top of Page

4.2.5 Performance Measurement and Reporting

Evaluation Issue: Does the program identify clear deliverables and expected results?

Indicator(s)

Methods

Rating

Both administrative and key informant data suggest that the program’s intended deliverables and intended impacts are fairly clear. Key informants among EC personnel and funding recipients appear to have a generally positive assessment of the clarity of program deliverables and impacts, as two-thirds of recipients (66%) and nearly half (46%) of program personnel reported that these were very clear (i.e., responding with a 6 or 7 on a seven-point scale). There is room to improve the clarity of deliverables and results, however, as more than one-third of both recipients (35%) and EC personnel (35%) feel the program’s deliverables and expected results are only somewhat clear (i.e., responding with a 3, 4 or 5 on a seven-point scale) (see Figure 4.2).

Figure 4.2 Perceived Clarity of EDF Deliverables and Expected Results

At the project level, analysis of administrative data shows that projects identify clear deliverables and expected impacts in 97% of cases. Likewise, feedback from program personnel suggests that the large majority feel performance indicators are developed and clearly identified for EDF-funded projects. Several respondents, however, suggest that the process of identifying indicators for EDF projects may not be as evolved as for more established programs, such as EcoAction, where standard indicators have been developed. One respondent also suggested that the identification of performance indicators may present more of a challenge for projects involving education and awareness, as the results of these projects are “a little less tangible.”

In terms of the collection of performance data for EDF-funded projects, key informant and administrative data both provide evidence that this is being done to a large extent. The review of administrative files found evidence that performance data were collected for activities and outputs in 95% of project files reviewed and, to a lesser extent, for outcomes in 83% of cases. Similarly, more than four in five funding recipients (83%) reported that performance data was collected for their projects.

While the program is systematic in eliciting performance data on deliverables and outcomes, Environment Canada key informants suggest that both types of performance data are only transmitted irregularly or informally to senior management. In Atlantic Canada, where the upward transmission of performance data seems relatively formal, all the information is passed on in a report to the RDG of Atlantic Region. Elsewhere, transmission of information to management is typically informal and may not include information on small projects. In these regions, reporting of performance information may take the form of presentations or briefings that are sporadically made to the Minister or senior management. Some EC key informants expect the flow of performance data to senior management to become more formal and regular as a consequence of the installation of a dedicated information management system, which is currently under development.

Summary: Most EC personnel and recipients feel the intended deliverables and impacts of the EDF program are at least somewhat clear, while project deliverables and impacts are clearly identified in the vast majority of project files. Strong evidence emerges from both key informants and administrative files that performance data are collected for virtually all projects with respect to activities and outputs and are collected for most but not all projects with respect to outcomes. In most regions, no systematic process exists for communicating program results to senior managers and decision-makers.


Top of Page

4.2.6 Strengths and Challenges

4.2.6.1 Program Strengths

In the view of several Environment Canada personnel, the EDF’s focus and design is seen as its greatest strength, as the program allows fines to be applied directly toward addressing the environmental damage that was done. A second commonly cited strength among EC respondents is the perceived simplicity of the program, as it is felt to be straightforward and easier to manage given the consistency of the EDF’s design with another Community Partnerships program called EcoAction. This too may explain another key strength noted by many recipients, which is the simplicity of the application process and well-defined deliverables.

Another key strength of the program identified by some recipients concerns the establishment of partnerships with environmental, community and government organizations. Similarly, several EC personnel note also that one of the program’s greatest strengths as it is delivered in the Atlantic region is the partnerships that have been built within EC and with other government departments. As presented here in Section 4.2.2.2, however, academics and potential recipients are less favourably disposed to the idea of partnerships, believing partners are difficult to find, can create conflict or perform poorly.

Potential recipients, academics and members of the judiciary were also asked to identify strengths and best practices in the design and delivery of funding programs generally, based on their experiences with programs similar to the EDF. Not surprisingly, each group of respondents tended to identify somewhat different strengths and best practices, as described in turn below:

4.2.6.2 Key Challenges

In terms of program challenges, respondents from all groups identified many of the same issues, chiefly a lack of resources for the program, court restrictions, internal departmental processes, and engendering buy-in among key program partners. For managers, challenges relating to resources concerned a lack of funds and staff to support program delivery, while for recipients and potential applicants, mention was most often made of the low levels of EDF project funding available, the need to rely on other partners and the short timelines for funding. Both managers and recipients also noted that it was challenging to use and access EDF resources. This latter challenge likely stems from both the absence of resources for administration of the program, as well as from court conditions imposed on EDF funding. Challenges related to addressing court conditions were frequently mentioned by managers, recipients, and justice respondents, as they caused difficulties in matching project submissions to the narrow conditions and criteria for funding. One manager also noted that court conditions make it challenging to plan program activities because funding for different priorities and geographic regions varies from year to year.

Internal processes were also frequently mentioned by managers, academics, potential applicants and recipients as posing challenges, particularly in terms of delays in receiving funding approval and the administrative burden related to proposal development and project reporting for proponents with limited capacity or for very small projects where the level of effort may not be commensurate to the size of the contribution. Some respondents also question the merits of current levels of project reporting and management. One academic and one potential applicant argue that project deliverables and indicators are rarely representative of a project’s true impact on society, while a second academic noted that a focus on project management issues can dilute the quality of the science funded through such programs by diverting energies away from the project itself.

Finally, several recipients and federal respondents noted difficulties related to engendering buy-in among key program partners, and in particular to promoting the EDF’s use among members of the judiciary. As a remedy, one federal respondent suggested the program could improve its reporting and promotion of program results and success stories to all program partners. Several EC respondents also remarked that it was often challenging to encourage Enforcement personnel in the Ontario region to promote the EDF while the regionally specific alternative ECCAP program was still in existence. Although the ECCAP no longer exists as an alternative, one EC respondent noted that there is still work to be done in terms of promoting buy-in to this “new” EDF program.


Top of Page

4.3 Success

Overall Findings:

The EDF is generally perceived by recipients and Environment Canada personnel to be successful in the achievement of most program outcomes, a finding which is confirmed by administrative files demonstrating a clear achievement of project deliverables and outcomes in a majority of cases. Evidence from key informants and project files suggest that the program is most successful in contributing to restoration objectives, and is somewhat successful in contributing to education and awareness and research and development outcomes as well. The primary factors underlying the degree to which different outcomes are achieved are thought to include the level of resources, number of projects and/or number of applications targeting the priority area.

The existence of clear documentation pertaining to the role recipients are required to play in the financial oversight of funded projects, as well as the high rates of projects conforming to audit and payment authority specifications, suggests the program is being administered with good financial oversight and accountability. While generally clear and well understood, the transparency of the funding process at the project level could be enhanced through more consistent staff support and better communications about court awards and funding decisions between Enforcement and EDF personnel.

Resources to support the administration of the program overall are not tracked in any systematic fashion, although regions are tracking the number and value of court awards and contribution agreements. Past research suggests that overall financial accountability and transparency would be enhanced through the development of an information management system, which is already under development.

The EDF program has made no apparent progress in increasing awareness of the program among the legal community and potential applicants to the program, or in increasing the court’s use of the EDF as a sentencing tool. Although the Atlantic region has made some effort to make information related to EDF projects public, this information was incomplete and out-of-date and no similar information was found for other regions.

Efforts have been made to expand the use of EDF in the context of federal legislation controlled by other departments. Although implicated departments (i.e., DFO and Transport Canada) are involved in the technical review of project applications, information transfer between federal partners could be improved by sharing more information pertaining to final funding decisions.

The EDF program employs a number of controls and processes to ensure the quality and feasibility of project proposals. Evidence suggests also that program materials on the application process and staff support are contributing to the acquisition of proposal and planning skills by community groups, as evidenced by the high rates with which projects achieve intended deliverables, the consistency of proposals with EDF objectives, the use of partnerships, and the ability of projects to leverage partner resources.

Evaluation Issue: To what extent have intended outcomes been achieved as a result of the program?

Indicator(s)

Methods

Rating

When asked to rate the extent to which the intended outcomes of funded projects have been achieved, both Environment Canada personnel and recipients in particular tend to provide positive assessments. Three-quarters of Environment Canada staff feel the outcomes of EDF projects have been achieved to a large (50%) or moderate extent (26%), while fully nine in ten recipients indicate large (83%) or moderate (11%) achievement of project outcomes (see Figure 4.3). It is noteworthy that one in five EC respondents report that they do not know whether outcomes had been achieved, likely reflecting their lack of direct involvement in providing support to funded projects (e.g., enforcement personnel) 52.

Environment Canada personnel most often cited personal experience with projects to explain their positive ratings of outcome achievement. Several also note that the program is administered according to clear terms and conditions and that project proposals are closely reviewed and monitored once implemented, while others note that the capacity and qualifications of project proponents, and the support and resourcing provided to projects may underlie the achievement of project outcomes. For several more, assessing project success was more difficult either because the program is not fully implemented in their region, a longer timeframe is needed to observe results, there is no standard approach to evaluating projects or the results of projects aren’t widely promoted.

Several project proponents explained that their project’s success was the result of strong support from partners, other funders and the program (i.e., flexibility), as well as the achievement of objectives that were beyond the scope of the original project proposal. A few respondents noted less than 100% success resulting from unforeseen circumstances that prompted design adjustments, while several others mentioned that it was difficult to assess the achievement of outcomes because the project was either not yet complete or ongoing, with results expected only in the longer term.

Figure 4.3 Perceived Achievement of Project Outcomes

Results of the review of project files are consistent with the perceptions of key informants. For nearly nine in ten (86%) project files reviewed, clear evidence of implementation of activities and the attainment of deliverables was found. A further one in ten (9%) files showed “partial evidence” of the attainment of activities and deliverables, while only 6% found no such evidence. Some funding recipients reported that the requirement for performance information on deliverables is a strength of the program.

Administrative files revealed less performance information pertaining to outcomes than to deliverables. Projects were deemed to have achieved intended outcomes if the final report specified that intended outcomes had been observed, such as by using an “indicators of [impact] success” checklist. Roughly three-quarters of project files reviewed contained either a very (46%) or moderately (31%) detailed 53 description of outcomes achieved, while one in four project files described achieved outcomes with only minimal detail (6%) or provided no such evidence (17%). Careful review of the administrative files demonstrated that awareness and education activities were less likely than others to have outcome data. This is likely explained by the complexity and/or high cost of collecting performance information for these types of activities (e.g., attitude surveys).

In terms of the specific outcomes to which EDF projects contribute the most, federal respondents were asked to indicate the extent to which funded projects contributed to each of four priority areas related to restoration, environmental quality improvement, research and development, and education and awareness. Federal respondents were most likely to feel the EDF has contributed to environmental restoration, with more than half reporting that the program has contributed somewhat or a lot to this outcome. Just fewer than half felt the same about the program’s contribution to education and awareness while roughly one in four felt the program contributed at least somewhat to research and development. Relatively few felt the program contributed to environmental quality improvement. This may not come as a surprise considering that the focus of the EDF is typically on restoring the “same type, quality and value of the natural resource lost” rather than on environmental quality improvements targeting “different natural resources in the locale.” 54

In general, respondents explained the program’s impact on one or another outcome as due to the number of projects associated with a given outcome, as well as the amount of funding for projects in that area. In particular, they note that the focus on one or another priority is often linked to court conditions, as well as to the types of projects being proposed. Some key informants also attribute the frequency of environmental restoration projects to a preference for these projects based on their highly visible and measureable results and high success rates. The program’s contribution to education and awareness and to research and development was often credited to the ease with which these outcomes can be incorporated into activities targeting other outcomes. As well, the relatively low costs of including the public in projects aimed at improvements in the communities affected by the environmental damage also contributed to the achievement of education and awareness objectives.

Although the EDF’s overall contribution to one or another of these four priorities may be explained by the number and/or value of contributions in the priority area, the review of project files shows very little difference in the degree to which individual EDF projects are successful, regardless of their focus. With few exceptions in any priority area, project files yielded at least partial evidence of the achievement of intended activities and outcomes, with the vast majority of these project files containing comprehensive evidence of their achievement. Evidence of outcome achievement, however, was slightly less likely to be found for projects targeting environmental quality improvement and education and awareness.

Summary: The EDF is generally perceived by key informants to be successful in the achievement of most program outcomes, a finding which is confirmed by administrative files demonstrating a clear achievement of project deliverables and outcomes in a majority of cases. Evidence from key informants and project files suggests that the program is most successful in contributing to restoration objectives, and is somewhat successful in contributing to education and awareness and research and development outcomes as well. The primary factors underlying the degree to which different outcomes are achieved are thought to include the level of resources, number of projects and/or number of applications targeting the priority area.


Top of Page

4.3.1 Project Delivery

Evaluation Issue: To what extent has the program achieved intended project delivery outcomes?

Indicator(s)

Methods

Rating

Outcomes related to the delivery of funded projects concern the extent to which the program demonstrates financial accountability and the degree to which the funding process is transparent. At the project level, the review of project files suggests that EDF-funded initiatives are managed in a financially accountable fashion. For over nine in ten project files reviewed there existed an explicit statement concerning the Government’s right to audit the project (91%) and a payment authority was identified (94%).

The EDF Recipient Handbook outlines the roles and responsibilities of funding recipients, including details related to project accounting, reporting, communications activities and project closure 56. It is not surprising, then, that most recipients expressed a good understanding of the funding process and in particular of the source of EDF funds and the specific restrictions for their use. Several recipients remarked that support received from EDF staff ensured that the funding process was clear and straightforward, although this transparency may be dampened somewhat by the uneven accessibility and program expertise of EDF-assigned personnel.

The detailed accounting and reporting requirements for funded projects serve to enhance the overall transparency of the funding process, but as noted previously, many recipients remarked that the financial and other reporting for the program seemed excessive in light of the amount of funding received. As well, program documentation is not clear about whether leveraging financial or other resources through partnerships is a requirement for EDF-funded projects (see Section 4.2.2.2) and a few recipients provided conflicting feedback concerning whether partners were or were not a requirement for their projects.

At the program level, the absence of dedicated resources for the EDF’s administration means that no systematic tracking of human or financial resources to administer the program is taking place. Nonetheless, the program has maintained detailed records by region and calendar year of the overall number and value of court awards directed to the EDF, as well as the number and value of contribution agreements supported through the program. This information is tracked separately by regional offices and aggregated by the program headquarters. While a 2007 process review concluded there was “no compelling reason to change the current system based on anticipated growth,” it was also suggested that the development of a centralized information management system (IMS) would improve the visibility of project information, the consistency of information available across community funding programs, and make information related to funding activity levels more readily available 57. Work has already begun to adapt the IMS used for the EcoAction program for use by the EDF program, but this new system is not yet operational.

The development of an IMS might also enhance transparency and information-sharing between the EDF program and its partners. In particular, a number of federal respondents felt there was a need for better communication between enforcement officers and Community Partnerships staff to track what funds are flowing to the program and seek input on funding decisions, training for program officers, and more information about project selection, approvals and funding processes.

Summary: The existence of clear documentation pertaining to the role recipients are required to play in the financial oversight of funded projects, as well as the high rates of project agreements containing audit and payment authority statements, suggests the program is being administered with good financial oversight and accountability. While generally clear and well understood, the transparency of the funding process at the project level could be enhanced through more consistent staff support.

Resources to support the administration of the program overall are not tracked in any systematic fashion, although regions are tracking the number and value of court awards and contribution agreements. Past research suggests that overall financial accountability and transparency would be enhanced through the development of an information management system, already under development.


Top of Page

4.3.2 Communications and Program Promotion

Evaluation Issue: To what extent has the program achieved intended communications and promotion outcomes?

Indicator(s)

Methods

Rating

In order for the EDF to function properly, it is important for parties involved in the prosecution of environmental infractions (i.e., prosecutors, investigators and judges) to be aware of the EDF program and how it operates 58. Research suggests that enhancing stakeholders’ familiarity with and interest in using the EDF requires a significant investment, but that understanding and judicial precedents are also necessary for the program to be accepted as a vehicle to be used in prosecuting environmental offences 59. Regional growth of the program similar to what has been observed in Atlantic Canada is contingent on greater promotion and use of the fund 60.

Previous studies have also found that “...counsel for the polluter and even Crown counsel and government officials were not familiar with the EDF” 61 and feedback from key informants for the current evaluation suggests that awareness of the EDF has not increased appreciably among external stakeholders in the interim. A large majority of potential recipient organizations reported being unaware of the existence of the EDF, while more than half of the academics interviewed acknowledged a lack of awareness. Although all key informants in the judiciary reported being aware of the EDF, they also emphasized that most judges and many crown attorneys are unaware of the EDF program. According to court/judicial key informants, it appears that the courts become aware of the EDF on a case-by-case basis mainly from the information provided by the crown attorneys.

These results may not be surprising if we consider that promotion of regional EDF awareness and use by the legal community, recipient groups, and other government and nongovernment stakeholders is the responsibility of EDF regional managers 62. The absence of dedicated resources for program implementation and evidence of uneven regional implementation of the program would suggest that little promotional activity is taking place in many EC regions. It should be noted, however, that in Atlantic Canada, where the program has a longer history and is far better developed, EC key informants report that efforts have been systematically undertaken to educate the judiciary.

The apparent lack of awareness of the program among the courts is consistent with administrative evidence which suggests there has been little increase in the court’s use of the EDF as a sentencing tool. Since 2000, the number of court awards to the EDF has remained steady at between 10 and 13 awards per year, with a larger number of awards occurring in 2004 (n = 18), 2007 (n = 20) and 2008 (n = 18). While the larger number of awards in 2007 and 2008 may be indicative of the initial stages of an increasing trend in the number of court-awarded contributions to the EDF, the continued absence of dedicated support to the program, uneven regional implementation and low awareness of the program in the courts suggest that this is unlikely the case.

Finally, there is little evidence to suggest that the EDF has made progress in making information on EDF-funded projects and their geographic location available to the public. The only evidence of efforts to publicize this information is a reference to completed EDF projects on Environment Canada’s Green Lane Web site 63. The information posted on the EC Web site, however, only included links to projects completed in Atlantic Region and only until 2005. As well, not all links for a given year were found to be active.

Summary: The EDF program has made no apparent progress in increasing awareness of the program among the legal community and potential applicants to the program, or in increasing the court’s use of the EDF as a sentencing tool. Although Atlantic Region has made some effort to publicize information related to EDF projects, this information was incomplete and out-of-date and no similar information was found for other EC regions.


Top of Page

4.3.3 Partnerships

Evaluation Issue: To what extent has the program achieved intended partnerships outcomes?

Indicator(s)

Methods

Rating

The EDF has been reasonably successful in engendering interdepartmental cooperation and information transfer. To begin, use of the fund is not restricted to Environment Canada legislation 65 but can also be applied to the management of court awards and penalties meted out through legislation governed by other federal departments, including the Department of Fisheries and Oceans (DFO) 66 and Transport Canada 67. Furthermore, the Government has signalled the expanded use of the EDF by amending three Parks Canada statutes 68 to enable awards and fines to be directed to the EDF. In regions where the program is being consistently implemented, implicated departments (i.e., DFO, Transport Canada, and Agriculture and AgriFood Canada) are also invited, as appropriate according to the nature of the project to be reviewed, to be members of technical review teams for EDF funding applications.

In terms of interactions between departments, EC respondents generally commented that these interactions were positive. Non-EC federal respondents were divided on whether these interactions could be improved, however, noting only that they would like more follow-up information on final funding decisions and that it is challenging to discuss and review proposals when the technical review team members are not all situated in the same area.

Similarly, although most Environment Canada and non-EC federal respondents report that the roles and responsibilities of various players involved in the EDF are clear (see Section 4.2.4), many also note the potential for improvement in this regard. Specifically, several remarked on the need for more consistency and oversight in the way the program is delivered across regions and better communication between federal stakeholders to share information on awards directed to the program and project selection and funding.

Summary: Efforts have been made to expand the use of EDF in the context of federal legislation controlled by other departments. Although implicated departments (i.e., DFO and Transport Canada) are involved in the technical review of project applications, information transfer between federal partners could be improved by sharing more information pertaining to final funding decisions.


Top of Page

4.3.4 Proposal Planning and Development

Evaluation Issue: To what extent has the program achieved intended proposal planning and development outcomes?

Indicator(s)

Methods

Rating

As discussed previously in Section 4.2.2.3, the available evidence suggests the EDF has been successful in encouraging scientific, technically sound and feasible restoration projects. The review of project files revealed a majority of funding recipients are eligible to receive program funds (97%) and possess the necessary skills to deliver the project (74%), both of which speak to the feasibility of the proposed initiative. As well, the file review found firm evidence of internal reviews of project proposals to ensure their scientific and technical merit for a large majority of proposals, as more than four in five project files were evaluated by program managers (83%) and signed off by the RDG (83%) 69. Much less evidence, however, was found to demonstrate the cost-effectiveness of individual project proposals, with only one in four (26%) project files containing sufficient information to determine that the project was cost-effective. Seventy-one per cent of project files contained too little information to assess the cost-effectiveness of the project.

Some program documentation exists to support applicants to the programs in acquiring knowledge and skills for proposal development. The EDF Applicant’s Guide defines eligible and ineligible projects and outlines required information for applying to the program, including details relating to required applicant information, the proposed project team, a project summary, the rationale or need for the project, evaluation plans, communications activities, a funding summary, and budget.

Although recipients’ frequent mentions of a reliance on program staff to assist them with their applications and proposal development suggests many do not yet possess expertise in the development of proposals, this close contact with program personnel likely reflects a great deal of information sharing and tutelage. A review of administrative data would seem to confirm that EDF applicants are acquiring these skills, as nearly four in five proposals to the EDF are approved (78%), and over nine in ten (92%) meet eligibility requirements for the program. While the high project approval rates are likely due at least in part to low awareness of the program and a corresponding lack of competition between eligible projects, it is reasonable to conclude also that the high rate of project eligibility is indicative of the quality of the applications received.

The review of administrative data also demonstrates that funding recipients have developed a certain expertise in the planning of restoration and other EDF projects. Of 91 approved projects supported through the EDF since its inception, none of the final project budgets was found to have exceeded the approved value. Further, the review of administrative project files demonstrates that the projects are well administered in terms of:

Summary: The EDF program employs a number of controls and processes to ensure the quality and feasibility of project proposals. Evidence suggests also that program materials on the application process and staff support are contributing to the acquisition of proposal and planning skills by community groups, as evidenced by the high rates with which projects achieve intended deliverables, the consistency of proposals with EDF objectives, the use of partnerships, and the ability of projects to leverage partner resources.


Top of Page

4.3.5 Unintended Outcomes

Evaluation Issue: Have there been any unintended (positive or negative) outcomes? Were any actions taken as a result?

Indicator(s)

Methods

Rating

In addition to the intended outcomes of EDF-funded projects, approximately half of EC key informants identify unintended outcomes. For the most part, these respondents report positive outcomes involving stronger intradepartmental, interdepartmental, and intergovernmental relationships. Roughly half of the funding recipients interviewed highlighted generally positive unintended benefits resulting from their projects. These include unexpected sources of partnerships and financial support at the local level, unforeseen positive impacts on community attitudes and awareness, and unanticipated discoveries in the realm of science and technology.


Top of Page

4.4 Cost-effectiveness

Overall Findings:

Overall, the evidence suggests that the EDF is cost-effective. Projects have been successful at leveraging funds from other sources and some limited file review evidence suggests that individual projects are more likely than not to be evaluated as cost-effective. At the program level, no other federal programs exist to manage specified purpose funds to carry out environmental restoration activities and overall administrative spending is low, particularly in light of the national scope and specificity of court conditions related to funded projects.

The evidence is inconclusive concerning the advantages of introducing a minimum threshold value for project funding in order to increase overall program efficiency. While cost-efficiencies might be realized by imposing a minimum value for EDF-funded projects, doing so could impair small projects whose requirements for restoration or other environment contributions could be lower than a potential award threshold.

Evaluation Issue: Are there alternative ways of achieving the objectives of the program?

Indicator(s)

Methods

Rating

The cost-effectiveness paradigm focuses on the following four analyses, two of which explore the issue of cost-effectiveness at the project level and two of which examine this issue at the program level 72.

Project-level analyses of cost-effectiveness consider evidence related to:

Program-level cost-effectiveness analyses consider evidence related to:

At the project level, evidence of leveraging is strong. In the sample of projects (n = 35) whose files were examined, recipients reported average contributions of approximately $36,000 as compared to average total project costs of approximately $77,000. This roughly equates to a one-to-one ratio of EDF funding to leveraged funds and suggests that project contributions benefited from significant leveraging. With the exception of three projects, project costs exceeded the value of EDF contributions.

In a minority of instances, the project files also contained evidence that project officers themselves had reached a conclusion that the individual project was cost-effective. In 23% of cases, “program officers or auditors assessed cost-effectiveness and reached a positive conclusion” (20%) or there was sufficient evidence to suggest the project was cost-effective (3%). In contrast, in only 6% of project files reviewed (n = 2) did the evidence (3%) or a direct assessment 73 (3%) suggest the project was not cost-effective. In the vast majority of cases (71%), however, the project files contained no evidence of efforts independent of the funding recipients to verify the cost-effectiveness of individual projects. Although the paucity of evidence related to the cost-effectiveness of individual projects does not allow firm conclusions to be drawn, it is nonetheless important to note that projects were much more likely to be assessed as cost-effective than not.

At the program level, cost-effectiveness involves a consideration of redundancy risk. If a significant redundancy risk existed that either projects could have been supported or court awards managed by an alternative program, it would be theoretically possible that the alternative program might have been more cost-effective. Although several G&C programs within Environment Canada exist which might in theory have supported EDF-funded projects 74, there is no evidence that projects could have been supported with greater cost-effectiveness. Furthermore, and as discussed previously in Section 4.1, the EDF is a unique federal mechanism for managing court awards stemming from prosecutions under various legislative authorities and for supporting creative sentencing by the courts in environmental damages cases. Consequently, it is reasonable to conclude that no redundancy risks exist in the context of the EDF program.

The absence of any dedicated funding to support the EDF program and consequent lack of any system to track administrative costs for program delivery means that it is not possible to evaluate the program’s overall cost-effectiveness with any certainty (see Section 4.2 for a more complete discussion). Nonetheless, Community Partnership program estimates of fewer than four person-years per annum for the administration of EDF projects would appear to be cost-effective in light of the program’s commitment to reflecting regional realities, the diversity of environmental subjects addressed by the awards, and the administrative costs that flow from very specific court-award conditions.

EC personnel almost universally agree that the program provides a great value for federal dollars spent. Respondents generally note that this is because the program costs almost nothing for the Department to deliver, while at the same time the recipients leverage additional dollars, resources and expertise. In Atlantic Region, other federal departments are also now contributing fines to the EDF. When the views of respondents are considered together with evidence suggesting that the program has been successful in achieving outcomes (i.e., majorities of project files contain clear evidence of the achievement of deliverables [86%] and outcomes [60%] – see Section 4.3), and the apparently low administrative costs for delivery of the program, the evaluation evidence suggests that the EDF provides good value for federal dollars spent.

EC personnel also made several suggestions to improve the efficiency of program delivery. The two most prominent were a national communications strategy to promote the program and its successes, and less paperwork for the funding recipients. Other suggestions included the development of national guidelines for administering the EDF and the development of better indicators to measure the program outcomes.


Top of Page

4.4.1 Minimum Project Size

The terms and conditions for EDF contribution agreements require that contributions have a ceiling of $5 million and no minimum or threshold value. The principle of internal economies of scale would call for a minimum award value, however, as the proportional costs of administrative oversight can safely be assumed to be higher for small projects 75. Ongoing pressure for efficiency in Government of Canada programs, the modest administrative capacity of the program, and the administrative burdens imposed by stringent court conditions all call for attention to be devoted to protecting and improving efficiency.

The issue of a possible minimum or threshold size for project funding elicits mixed reaction. EC personnel tend to oppose a threshold while academics and potential recipients support it. Existing funding recipients are divided on the matter 76. Opponents of a threshold tend to perceive a potential minimum figure as a threat to small organizations, whose restoration or other projects do not require large sums since the amount of environmental damage is small. The size of court awards and their potential conditions also weigh against having a threshold minimum. In the presence of a threshold, it would be difficult for the EDF to administer court awards that have conditions imposed on their use and that fall below the threshold size unless they can be combined with other awards that have identical or compatible geographic and court conditions.

Supporters of a threshold favour a minimum figure because they perceive the burden (i.e., application, reporting) on both applicants and assessors as disproportionate to the amount of the award. In practice, the average contribution is $22,000 over the life of the program, and fully one-third of all awards have been under $10,000. EC personnel who favour a threshold call for a low minimum amount of $8,000, while funding recipients recommend $14,643 on average. Potential recipients and academics both call for high minimum threshold amounts of $43,000 and $ 56,000 respectively.


Top of Page

5.0 Conclusions

The evaluation of Environment Canada’s Environmental Damages Fund (EDF) has led to the following broad conclusions about the program’s relevance, design and delivery, success, and cost-effectiveness:

  1. The EDF is an important and unique mechanism for the management of court awards from the prosecution of environmental infractions. It is relevant to federal and departmental priorities for the enforcement and conservation of Canada’s natural capital, and in the future is expected to play an expanded role in the management of specified-purpose awards stemming from prosecutions under federal legislation.
  2. To date, program delivery has been consistent with the original purpose and design of the EDF. In the absence of dedicated funding to support the EDF’s administration, however, delivery and implementation of the program have been significantly attenuated in different regions, such that the pace with which funds are being diverted to the program through the courts outstrips the program’s ability to allocate funds to EDF projects.
  3. Low awareness and promotion of the program as a result of a lack of necessary resources has likely impeded the use of the EDF by the courts and consequently the diversion of court awards and penalties to the program. As well, the number and possibly the quality of proposals received by the program from eligible recipients are affected by low awareness and promotion.
  4. A number of standard tools and processes exist to ensure consistent delivery of the program in all regions, although there is a need to update certain tools to ensure the program terms of reference are clear (e.g., the need for partnerships) and to streamline certain processes (e.g., more focused application forms). The consistency of delivery and implementation of the program across regions has been impeded to date as a result of low resourcing as well as a lack of national oversight.
  5. The program exhibits a strong tradition of identifying and collecting performance measures for funded projects, but no consistent approach to reporting this information to senior managers is in evidence for most regions.
  6. The roles and responsibilities of different EDF stakeholders are well documented but not clearly understood by a sizeable minority of departmental personnel and recipients.
  7. Environmental Protection Alternative Measures (EPAMs) are poorly understood. Those who report familiarity with EPAMs note that they should be applied only in limited circumstances (i.e., for accidental as opposed to intentional offences) and there is some question as to their value (e.g., perception of “paying to pollute,” cumbersome to administer).
  8. To the extent that the EDF has been administered to date, evidence suggests that the program has been successful in achieving most intended outcomes and that funded projects for the most part are producing their desired deliverables and results. While the program demonstrates good progress toward outcomes related to financial oversight, interdepartmental cooperation and the development of community capacity and skills, little progress is observed in increasing awareness of the program among the legal community and potential applicants.
  9. The EDF represents good value for federal dollars spent and is a cost-effective means of managing court awards and supporting projects in the pursuit of activities related to the program’s intended outcomes. Projects have successfully leveraged funds from partners, and to a lesser extent are reviewed for their cost-effectiveness. As well, no other federal programs exist to manage specified purpose funds stemming from federal prosecution of environmental offences and overall administrative spending is low.
  10. The merits of introducing a minimum threshold value for project funding in order to increase overall program efficiency are unclear in light of the small size of some funded projects and the challenges this would create in meeting court conditions.

Top of Page

6.0 Recommendations

  1. An allocation of dedicated staff and budget to support the ongoing management and administration of the EDF program should be considered. Uneven regional implementation of the EDF program can be traced, at least in part, to the absence of dedicated resources to support its implementation, and this in turn may affect the consistency of support to recipients. Further, resource constraints are felt to represent a key barrier to the proper promotion of the program among potential applicants, thus limiting the pool of potential funding recipients able to carry out court requests through the program.

  2. Standardized tools and program documentation for managers and funding recipients should be updated to ensure consistent program implementation. In updating these tools, consideration should be given to:
  3. Means of streamlining the program application process and forms should be explored to reduce the burden on recipients and enhance consistency with other community partnership programs. To this end, the EDF could consider research (surveys, focus groups, etc.) with recipients to explore ways in which the application form could be improved. Alternatively, a comparison with the application forms for similar funding programs (e.g., EcoAction) may yield information to guide such changes (e.g., word limits and question simplification).

  4. A communications plan should be developed and implemented to promote the EDF among potential recipients, the judiciary and enforcement officers. The communications plan should consider the development of standardized communications materials that can be modified, as appropriate, to address regional circumstances, as well as the development of national communication activities to enhance efficiency and consistency of messaging nationally. Efforts to promote the program among potential recipients would likely yield higher quality and a greater variety of project proposals. As well, communications efforts targeted toward judges and, to a lesser extent, Enforcement personnel may increase the number of awards and other contributions to the EDF and help to reduce the number of court-prescribed funding restrictions.

  5. A centralized national information management system should be developed to track financial and performance information for EDF contribution agreements across regions. The development and implementation of such a system is essential for proper measurement, evaluation and reporting of program performance.

  6. A formal process for annual EDF performance measurement and reporting should be implemented. No consistent national process exists for reporting on the program’s impacts and effectiveness. In particular, such a reporting process is necessary to monitor progress toward full national implementation and the timely disbursement of funding in proportion to program awards. As a first step in developing this process, the program logic model should be revised to better reflect intended outcomes and more clearly distinguish outcomes from activities and standard corporate functions (e.g., financial oversight).

Top of Page

7.0 Management Response

This section outlines the management response to the evaluation recommendations. The ES Board accepts the evaluation and its recommendations and the EDF program has provided a plan to implement the following management actions in response to the evaluation recommendations within the context the EDF program renewal.

Note to Reader

Bill C-16, the Environmental Enforcement Act, received Royal Assent on June 18th, 2009. Among other things, this Bill amends six EC statutes and three Parks Canada statutes to direct all fines to the EDF. Once these amendments come into force new minimum fines will start at $5,000 for individuals and $25,000 for corporations and maximum fines will increase to $1 million for individuals and $6 million for corporations. The amendments also allow for a portion of the fines to be used for administering the EDF Program.

It is anticipated that these amendments could significantly increase the monetary value and number of awards directed to the EDF and these changes will be factored into the renewal of the program.

1. An allocation of dedicated staff and budget to support the ongoing management and administration of the EDF program should be considered. Uneven regional implementation of the EDF program can be traced, at least in part, to the absence of dedicated resources to support program implementation, and this in turn may affect the consistency of support to recipients. Further, resource constraints are felt to represent a key barrier to the proper promotion of the program among potential applicants, thus limiting the pool of potential funding recipients able to carry out court requests through the program.

The ES Board agrees with this recommendation.

Under the Environmental Enforcement Act (Bill C-16), which is expected to come into force in 2011–2012, the program will have the option to use funds directed to the EDF for administrative purposes.

The business case to support program renewal will be requesting administrative resources, including dedicated staff and operational and management costs. The request for program renewal will be presented in September 2009.

Securing resources for the EDF will ensure that each region has dedicated staff to deliver the program. This will in turn help to bring national consistency in the delivery of the program and ensure that proper support is available to clients. These resources will enable staff to effectively promote the EDF program among targeted audiences and diversify the pool of potential funding recipients.

Timeline

Deliverable

Responsible Party

September 2009

Present the EDF business case and documentation related to program renewal.

Director, Outreach

2. Standardized tools and program documentation for managers and funding recipients should be updated to ensure consistent program implementation. In updating these tools, consideration should be given to:

The ES Board agrees with this recommendation.

As part of the program renewal package, the program is recommending the approval of the EDF Management Framework and the Funding Agreement Terms and Conditions for the Environmental Damages Fund Program. The management framework provides an overview of how the program works, the program logic model, performance measurement, and reporting and risk management strategies. The funding agreement terms and conditions set forth the terms and conditions under which the EDF program will provide funding for projects.

The program renewal package outlines a number of commitments to streamline administrative processes to ensure consistency across all regions:

  1. Development of operational guidelines to outline program governance, roles and responsibilities of program staff and clarify partnership requirements for the EDF program, as partnerships are considered an advantage but not essential.
  2. Development of regional management plans that define top priorities for potential EDF projects and enable the program to direct funding to local environmental priorities that represent the best use of funds in response to environmental damage or damage-to-wildlife incidents. Regional management plans will be prepared in consultation with EC regional management teams and include other government departments as required. The plans will be approved by the regional director general.
  3. Development of standard project indicators to guide managers and project proponents in measuring project success. These indicators will contribute to the evaluation of program results.

Timeline

Deliverable

Responsible Party

September 2009

Present the EDF business case and documentation related to program renewal

Director, Outreach

November 2009

Develop program operational guidelines

Director, Outreach

Fall 2009

Develop regional management plans

Regional Directors General

Fall 2009

Conduct project indicators workshop to develop standard project indicators

Director, Outreach

3. Means of streamlining the program application process and forms should be explored to reduce the burden on recipients and enhance consistency with other Community Partnership programs. To this end, the EDF could consider research (surveys, focus groups, etc.) with recipients to explore ways in which the application form could be improved. Alternatively, a comparison with the application forms for similar funding programs (e.g., EcoAction) may yield information to guide such changes (e.g., word limits and question simplification).

The ES Board agrees with this recommendation.

The EDF program will take a more directed approach to soliciting applications for projects. As outlined in the response to the second recommendation, the program will develop regional management plans that define top priorities for potential EDF projects and enable the program to direct funding to local environmental priorities that represent the best use of funds in response to environmental damage or damage to wildlife. Project proposals will be solicited on an on-going basis as EDF awards are received.

As a part of the EDF program renewal, the program has committed to streamlining EDF administrative processes, including reducing the burden of application and reporting processes on funding applicants.

The EDF program will participate in the Departmental Action Plan to Reform the Administration of Grants and Contributions, which has the goal of simplifying and streamlining application processes and improving forms for grants and contributions (G&Cs), while strengthening accountability. It is important to note that the scope and timelines of the commitments in relation to the Departmental Action Plan to Reform the Administration of Grants and Contributions are beyond the program’s control.

Timeline

Deliverable

Responsible Party

Fall 2009

Develop Regional Management Plans

Regional Directors General

Fall/winter 2009–2010

Review application and reporting processes

Director, Outreach

2011–2012

Adopt applications and reporting processes resulting from G&C reform

Director, Outreach

4. A communications plan should be developed and implemented to promote the EDF among potential recipients, the judiciary and enforcement officers. The communications plan should consider the development of standardized communications materials that can be modified, as appropriate, to address regional circumstances, as well as the development of national communication activities to enhance efficiency and consistency of messaging nationally. Efforts to promote the program among potential recipients would likely yield higher quality and a greater variety of project proposals. As well, communications efforts targeted toward judges and, to a lesser extent, Enforcement personnel may increase the number of awards and other contributions to the EDF and help to reduce the number of court-prescribed funding restrictions.

The ES Board agrees with this recommendation.

As part of the program renewal strategy, the program has committed to developing a promotional strategy that targets judges and prosecutors, enforcement officials from other federal government departments and potential funding recipients. In addition, a national communications plan will be created to ensure national consistency in efforts to promote the EDF program.

Overview of the EDF Promotional Strategy

Judges and Prosecutors

Enforcement

Potential Recipients

Timeline

Deliverable

Responsible Party

Fall 2009

Implement national EDF communications plan

Director, Outreach

Summer 2009

Develop EDF promotional strategy

Director, Outreach

July 2009

Participate in the CAPE Web portal

Director, Outreach

August 2009

Develop and distribute EDF fact sheet

Director, Outreach

Fall 2009

Disseminate EDF letter to prosecutors by the Department of Justice National Environmental Prosecutions Coordinator

Director, Outreach

Fall 2009

Complete Web site improvements

Director, Outreach

Winter/spring 2010

Develop enforcement training materials and tools

Director, Outreach

Spring/summer 2010

Develop information in conjunction with the Judicial Outreach Program for the National Judicial Institute

Director, Outreach

Fall/winter 2009–2010

Develop EDF brochure

Director, Outreach

5. A centralized national information management system should be developed to track financial and performance information for EDF contribution agreements across the regions. The development and implementation of such a system is essential for proper measurement, evaluation and reporting of program performance.

The ES Board agrees with this recommendation.

As a part of the departmental G&C reform initiative, an online application and information management system is being developed for its G&C programs. This system will improve program efficiency, enhance alignment with departmental priorities and improve the ability to report collectively on the results of departmental funding programs. This system is scheduled to be in operation by 2011–2012, after which the EDF program will adopt it to collect program data.

In the meantime, the program acknowledges that a consistent and effective mechanism for collecting and tracking program information and results is required. In April 2009, the program implemented a Management Information System (MIS), cloned from the existing EcoAction Community Funding Program MIS. The MIS allows program staff to track project information, including recipient information, funding agreements, project funding, and project results. Planned modifications and improvements will allow the program to track the source of monetary contributions, the legislation used and whether EC enforcement officials were involved. The system will facilitate reporting and allow project managers to track communications with funding recipients. The use of this tool will ensure consistency of data collection across the regions and significantly improve the program’s ability to track and report results. This system has been launched in all EC regions and is currently being populated with historical as well as current project information. This system will be used until the new G&C online system is up and running.

Timeline

Deliverable

Responsible Party

Fall 2009

Complete implementation of MIS

Director, Outreach

2011–2012

Adopt the departmental G&C information management system

Director, Outreach

6. A formal process for annual EDF performance measurement and reporting should be implemented. No consistent national process exists for reporting on the program’s impacts and effectiveness. In particular, such a reporting process is necessary to monitor program progress toward full national implementation and the timely disbursement of program funding in proportion to program awards. As a first step in developing this process, the program logic model should be revised to better reflect intended program outcomes and more clearly distinguish outcomes from program activities and standard corporate functions (e.g., financial oversight).

The ES Board agrees with this recommendation.

The program recognizes that a formal reporting process is required to reflect the success and progress of the EDF program. In response to this need, a revised program logic model, performance measurement framework and reporting strategy have been developed and included in the Management Framework as part of the program renewal. The program has also committed to developing project-specific indicators to assist in the effective measurement of project results.

Program results will be collected by NCR program staff on an annual basis and presented to the ES Board. The information will capture the number of projects that were funded and the results achieved, including environmental impacts. The program will also demonstrate its administrative effectiveness with respect to national implementation of the program as well as the timely allocation of funding.

A departmental performance measurement framework has also been developed and the EDF has identified three expected results and three corresponding indicators against which it will report on an annual basis in the Departmental Performance Report.

The expected results are:

The indictors that will be used to measure these results are:

As previously mentioned the program has developed an electronic Management Information System that will be used to track program results and help facilitate reporting and ensure consistency in reporting practices across all regions. The system will be used to roll up annual results and for program evaluation purposes.

Timeline

Deliverable

Responsible Party

September 2009

Present the EDF business case and documentation related to program renewal

Director, Outreach

Fall 2009

Conduct project indicators workshop to develop standard project indicators

Director, Outreach

Fall 2009

Complete implementation of MIS

Director, Outreach


Top of Page

Annexes

Annex 1: EDF Program Logic Model

  Communications
Program Promotion
Partnerships Proposal Planning and Development Project Delivery
Source: Environment Canada. 2008. EDF Evaluation Plan - Draft 6. Ottawa.
Activities Preparation of EDF project communication plans
Preparation of EDF informational and promotional products
Development and maintenance of EDF Web site
Preparation of media alerts for project approvals
Coordination of departmental/ interdepartmental technical review teams Provision of knowledge, expertise, informational tools and advice Administration of project approval process
Financial tracking and management of fund contributions
Preparation of contribution agreements
Project auditing and in-field monitoring
Reach Recipient community: environmental community, Aboriginal groups, universities and provincial, territorial and municipal levels of government
Legal community: Judges, crown prosecutors, DOJ, EC Enforcement and other federal departments
General public
EC branches and divisions (EPB, CWS, MSC, CDR)
Other federal departments (Habitat Management DFO, Marine Safety TC, PWGSC)
Recipient community Recipient community
Outputs EDF communications and promotional products (brochures, fact sheets, EDF Web site, public presentations, workshops);
media briefing material for project approvals
Review and evaluation of project proposals.
Provision of interdepartmental agency expertise and knowledge.
Damage compensation increased through utilization of environmental legislation (CEPA 1999, FA, MBCA 1994, CSA).
Information and advice to recipient community related to fund specifications, proposal development strategy, and federal partnerships and financial planning Administrative and technical reviews
Project approvals
Contribution agreements signed
Restoration projects funded
Outcome 1 Increased awareness of EDF by the legal community and use as sentencing option by the courts resulting in monetary awards Scientific and technically sound, feasible and cost-effective restoration projects Scientific and technically sound, feasible and cost-effective restoration projects Financial accountability
Transparency of funding process
Outcome 2 Awareness of the Fund among recipient organizations Increased court-awarded contributions to the EDF Community groups acquire knowledge and skills for proposal development and restoration planning EDF restoration objectives are achieved
Outcome 3 Public information on EDF-funded projects and on their geographic location Interdepartmental cooperation and information transfer    
Long-term Outcome Biodiversity is conserved and protected
Sustainable use and management of natural capital is ensured
Risk to the environment posed by toxic and harmful substances is reduced
Key Results Canada’s natural capital is restored, conserved and enhanced
The environment is protected from the effects of pollution and waste

Top of Page

Annex 2: Summary of Terms and Conditions for EDF Contribution Agreements

  1. Recipients must be eligible to receive the monies
  2. The contribution is consistent with application and decision
  3. The project meets the court’s conditions
  4. The project is cost-effective
  5. The recipient possesses the skills necessary to deliver project
  6. Projects undertaken build on partnerships
  7. Proposals are evaluated by program managers and signed off by the RDG
  8. The maximum amount payable is under $5 million
  9. Awards cover allowable expenditures (not basic costs)
  10. Payment methods are on invoice and with statement of expenditures
  11. Equipment purchased belongs to the project
  12. There exists a right to audit each project
  13. There exists a payment authority

Top of Page

Annex 3: Evaluation Issues and Data Sources

Issue Evaluation Question Documents Interviews Observations
Relevance 1. Is there a legitimate and necessary role for government in this program area or activity? 2002 Evaluation;
2008 RMAF/RBAF;
2007 Process Audit;
Administrative Files
No Not contested; already well documented, however, it will be re-stated from existing documentation
2. Is the program connected with societal/environmental needs? No
3. Does the program theory (activities, instruments, objectives, design) address the societal need identified? No
Design and Delivery 4. Does the program identify clear deliverables and expected results? RMAF/RBAF
EDF data base and financials
Minutes of mgmt meetings using PM data
EDF WG summary reports since 2006
Yes  
5. Is the program delivered as designed? Yes  
6. Is performance data collected against program activities/outcomes? If so, is collected information used to inform senior management/decision-makers? Yes  
7. Who is accountable for the program? Are the roles and responsibilities of all groups involved clear? Is there an appropriate accountability framework (e.g., for multi stakeholder agreements)? Yes  
Success 8. What are the best practices and lessons learned from this program? Process Audit 2007
RMAF/RBAF
Program review workshop summary 2002
Yes  
9. To what extent have intended outcomes been achieved as a result of the program? Administrative file reviews Technical survey reviews
EDF WG reports
Yes  
10. Have there been any unintended (positive or negative) outcomes? Were any actions taken as a result? Technical survey reviews
EDF WG summary reports since 2006
Program review workshop summaries
Recipient surveys
Yes  
Cost-effectiveness 11. Are there alternative ways of achieving the objectives of the program?   Yes  
12. Are others involved in the same areas of activities and/or share similar objectives? How is duplication avoided and complementarity achieved?   Yes  
13. If the program or activity continues, how could its efficiency be improved? EDF data base and financials Yes Financial analysis
14. Has the program provided value for federal dollars spent? EDF data base and financials Yes Financial analysis

Top of Page

Annex 4: Bibliography

Department of Finance. 2007. The Budget Plan 2007: Aspire to a Stronger, Safer, Better Canada. Accessed at: http://www.budget.gc.ca/2007/pdf/bp2007e.pdf

Department of Finance. 2008. The Budget Plan 2008, Responsible Leadership. Accessed at: http://www.budget.gc.ca/2008/pdf/plan-eng.pdf

Environment Canada. 2000. Introducing the Environmental Damages Fund. Ottawa.

Environment Canada. 2002. Environmental Damages Fund Recommendations from EDF National Workshop. Ottawa.

Environment Canada. 2004. Results-based Management and Accountability Framework for the EcoAction Community Funding Program. Ottawa

Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework. Ottawa.

Environment Canada. 2006. Environmental Damages Fund Framework. Ottawa.

Environment Canada. 2006. Environmental Damages Fund Grants and Contributions Approval Form. Ottawa.

Environment Canada. 2007. Communications Plan for the National Promotion of the Environmental Damages Fund (EDF) Program 2007–2009. Ottawa.

Environment Canada. 2007. Environmental Damages Fund Recipient’s Handbook. Ottawa.

Environment Canada. 2008. Environmental Damages Fund: A Program Overview. Ottawa.

Environment Canada. 2008. Environmental Damages Fund Results-based Management and Accountability Framework. Ottawa.

Environment Canada. 2009. 2008 Request for Update on Management Response to Address Evaluation Recommendations. Ottawa.

Environment Canada. 2009. Environmental Damages Fund Administrative Data. Ottawa.

Environment Canada. 2009. Environmental Damages Fund Application Form. Accessed at: http://www.ec.gc.ca/edf-fde/3901F8D5-B32D-4929-9FE5-AA1BE9180A11/EDF_Applicants_Guide.doc

Environment Canada. 2009. Environmental Damages Fund Final Narrative Evaluation Report. Access at: http://www.ec.gc.ca/edf-fde/BC9910A4-5735-43EF-AFA3-59D0770857CE/EDF_Final_Narrative_Report.doc

Environment Canada. 2009. Environmental Damages Fund In-Kind Contribution Report. Access at: http://www.ec.gc.ca/edf-fde/BC9910A4-5735-43EF-AFA3-59D0770857CE/EDF_In-Kind_Contribution_Record.doc

Environment Canada. 2009. Environmental Damages Fund Quarterly Narrative Evaluation Report. Access at: http://www.ec.gc.ca/edf-fde/BC9910A4-5735-43EF-AFA3-59D0770857CE/EDF_Quarterly_Narrative_Report.doc

Environment Canada. 2009. Environmental Damages Fund Sample Financial Audit. Access at: http://www.ec.gc.ca/edf-fde/BC9910A4-5735-43EF-AFA3-59D0770857CE/EDF_Example_of_a_Financial_Audit.doc

Environment Canada. 2009. Number of Spill/Releases, by Fiscal year, by Region, National Emergencies and Enforcement Management Information System and Intelligence System. Administrative data.

Environment Canada. Undated. Environmental Damages Fund Applicant’s Guide. Access at: http://www.ec.gc.ca/edf-fde/3901F8D5-B32D-4929-9FE5-AA1BE9180A11/EDF_Applicants_Guide.doc

Environment Canada. Undated. Environmental Damages Fund Framework for Environmental Damages Assessment and Restoration. Ottawa.

Environmental Law Centre. 2003. News Brief, Creative Sentencing Part I – Overview. Volume 18, No. 2. Accessed at: http://www.elc.ab.ca/Content_Files/Files/NewsBriefs/CreativeSentencing-V18-2.pdf

Government of Canada. 2009. Office of the Administrator of the Ship-source Oil Pollution Fund, Frequently Asked Questions. Accessed at:

(http://www.ssopfund.gc.ca/english/faq.asp

HDP Group Inc. 2002. Environmental Damages Fund Evaluation. Ottawa.

Morrison, S. 2007. Environment Canada Courts Awards (Pilot) Project: Progress Report.

PWGSC – Public Works and Government Services Canada. 2007. Environmental Damages Fund Process Analysis. Ottawa.

Treasury Board of Canada Secretariat. 1995. Environmental Damages Fund Guidelines, Appendix A. Ottawa.

Treasury Board of Canada Secretariat. 1995. Policy on Specified Purpose Accounts. Accessed at: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?evttoo=X&id=12249&section=text

Wruck, J. 2004. The Federal Environmental Damages Fund. The Advocate. Vol. 62-2.


Top of Page

Annex 5: Administrative (Project) File Assessment Guide

Project Attributes and Evaluation Criteria Scale
Project Characteristics  
Agreement number As shown
Project title As shown
Region As shown
Whether the project is single or multi-year one 1 = single year
2 = multi-year
9 = insufficient information to judge
Effective date (start date) (dd/mm/yy) (dd/mm/yy)
End date (dd/mm/yy) (dd/mm/yy)
Amount of EC agreement funding In thousands of dollars
Total project cost (final financial reporting) In thousands of dollars
Evaluation Issue  
Does the project identify clear deliverables and expected results? 1 = no mention of deliverables/outputs
2 = description of deliverables/outputs or their relations to promised results/impacts/outcomes are not entirely clear
3 = clear description of deliverables/outputs and their relationship to the promised results/impacts/outcomes
Is performance data collected against program activities/outcomes? 1 = no evidence of measuring or gauging activities/deliverables
2 = partial evidence of implementation of project activities and the attainment of expected deliverables
3 = clear evidence of implementation of project activities and the attainment of expected deliverables
1 = no evidence of measuring or gauging results/impacts/outcomes
2 = partial evidence that results/impacts/outcomes were measured or gauged
3 = clear evidence that results/impacts/outcomes were measured or gauged
Is there appropriate accountability framework (e.g. for multistakeholder agreements)? 1 = no mention of partners
2 = poor – stakeholders listed, with minimal detail on roles and responsibilities
3 = moderate – stakeholders listed, with moderate detail on roles and responsibilities
4 = very clear – stakeholders’ roles and responsibilities fully explained
Is the project delivered in partnership with any other individuals, groups or organizations? Record number
999 = insufficient information to judge
1 = municipality
2 = community organization (excl. environmental organization)
3 = environmental organization
4 = post-secondary educational institution
5 = business
8 = other
9 = insufficient information
TREAT AS MULTIPLE RESPONSE, i.e. SEVERAL ANSWERS ARE TO BE ALLOWED
1 = no statement of their obligations
2 = confused description of stakeholders’ obligations
3 = research
4 = assistance in service delivery
5 = funding
Have intended project outcomes been achieved? 1 = poor – Outcomes are difficult to discern, minimal detail
2 = moderate – Outcomes described/mentioned, moderate detail
3 = very clear – Outcomes successes described in detail
9 = no outcomes listed
Have there been any unintended (positive or negative) outcomes? 1 = mention of negative unintended outcomes
2 = mention of unintended outcomes that are neutral or un-assessed
3 = mention of positive unintended outcomes
9 = no mention of unintended outcomes
EDF Contribution Guidelines  
Recipients must be eligible to receive the monies 1 = clear demonstration of recipients’ ineligibility relative to program eligibility criteria
2 = clear demonstration of recipients’ eligibility relative to program eligibility criteria
9 = insufficient information provided
The contribution is consistent with application and decision 1 = little or no consistency with application and court decision
2 = consistent with some but not most aspects of application and court decision
3 = consistent with most aspects of application and court decision
4 = totally consistent with court decision and intended use or application of funds
9 = insufficient or no information provided
The project meets the court’s conditions 1 = the project does not meet the court’s conditions
2 = the project meets some conditions
3 = the project meets most conditions
4 = the project meets all conditions
9 = insufficient or no information provided on court conditions
The project is cost-effective 1 = program officers or auditors assessed cost-effectiveness and reached a NEGATIVE conclusion
2 = no direct reference to cost-effectiveness but files contain enough information to reach a general conclusion that project plausibly NOT cost-effective
3 = no direct reference to cost-effectiveness but files contain enough information to reach general conclusion that project was plausibly cost-effective
4 = program officers or auditors assessed cost-effectiveness and reached positive conclusion
9 = setting aside the issue that administrative and staff costs are not accounted for across all projects, this particular project provides too little information to determine its cost-effectiveness.
Recipient possesses the skills necessary to deliver the project 1 = file contains little evidence that the recipient has adequate formal skills and adequate experience for project delivery
2 = recipient provides moderate evidence of formal skills or experience relevant to project delivery
3 = recipient demonstrates formal skills and experience relevant to project delivery
9 = insufficient information to reach a conclusion
Proposals are evaluated by program managers and signed off by RDG 1 = no evidence of proposal evaluation
2 = program managers appear to have evaluated the proposal but evidence is informal
3 = program managers have formally evaluated the proposal
1 = no apparent RDG sign-off
2 = an apparent RDG sign-off
The maximum amount payable is under $5 million 1 = amount exceeded $5 million
2 = amount was under $5 million
9 = amount unclear
Awards cover allowable expenditures (not basic costs). The prohibited basic costs are those costs that are necessary for maintaining the operation of the recipient organization. 1 = amount covered basic costs as well
2 = amount covered allowable expenditures only
9 = insufficient information to judge
Payment methods are on invoice and with statement of expenditures 1 = most or all payments are lacking an invoice or statement of expenditures
2 = most payment methods are on invoice and with statement of expenditures
3 = all payment methods are on invoice and with statement of expenditures
9 = insufficient information
Equipment purchased belongs to the project 1 = does not belong to project
2 = belongs to project
9 = insufficient information
There exists a right to audit each project 1 = no statement of audit right in contract
2 = audit right stated in contract
There exists a payment authority 1 = a payment authority is not indicated
2 = a payment authority is indicated

Top of Page

Annex 6: Interview Guides

Federal Respondent’s Guide

Hello, my name is _____ _____ and I’m phoning from COMPAS Research. We’re currently conducting interviews with Environment Canada personnel as part of an evaluation of the Environmental Damages Fund. Is there a day and time when you might be available to take part in an interview?

A. IF THE KEY INFORMANT WANTS THE INTERVIEW NOW: Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

IF THE INTERVIEW IS SCHEDULED FOR ANOTHER TIME, (a) WE WILL TRANSMIT A COPY OF THE QUESTIONNAIRE AHEAD OF TIME TO THE PROSPECTIVE KEY INFORMANT AND (b) THE RESUMPTION OF THE CONVERSATION WOULD BEGIN HERE: Hello, it’s _____ _____ phoning from COMPAS Research for the interview with respect to an evaluation of the Environmental Damages Fund. Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

Background

1. Please briefly describe your responsibilities within the Environmental Damages Fund and how long you have been involved with it.

Design and Delivery

2. Using a seven-point scale where 1 means “not at all” and 7 means “to a great extent,” to what extent do you feel the EDF program is delivered as designed?

3. In what ways is the EDF’s delivery consistent or inconsistent with the program’s original design? PROBE: Please explain and provide examples.

4. Overall, how well is the program being delivered?

PROMPT AS NECESSARY: And how about:

5. Using a seven-point scale where 1 means “not at all clear” and 7 means “very clear,” to what extent do you feel the EDF program’s deliverables and expected results are clear?

6. For EDF to attract worthwhile applications, is there a minimum size contribution that it should consider? IF YES: What would that minimum amount be and why?

7. What means are used to promote the EDF to eligible recipient groups?

8. As you know, the program encourages recipients to work with partners. In your view, what is the effect of these partnerships on EDF-funded projects?

9. Is performance data being collected and reported against EDF activities/outputs and outcomes? Please provide examples.

10. Are the roles and responsibilities of the various parties involved in the delivery of the EDF program clear? IF NOT, PROBE: Which aspects of the EDF are not well understood? Please explain.

11. As you likely know, EPAMs are negotiated settlements as an alternative to court prosecution for a violation of the Canadian Environmental Protection Act. Are efforts being made to encourage EPAM awards to be directed to the EDF?

12. Although CEPA is the federal statute under which financial assignments to the EDF are generally made, it is also possible for the EDF to be used in relation to infractions under the Fisheries Act, the Canada Shipping Act, the Migratory Birds Convention Act, and the Species at Risk Act. In your view, are any efforts being made to encourage the courts to use these other laws to direct payments to the EDF? Please explain.

13. What have been the strengths and best practices in the design and delivery of the program? What has worked well?

Success

14. On a seven-point scale where 1 means “largely unachieved” and 7 means “largely achieved,” to what extent do you feel that the intended outcomes of funded projects have been achieved?

15. And to what extent has the EDF program overall contributed to:

16. What factors have had the greatest impact on the achievement of program outcomes? PROBE FOR EXAMPLES AND EXPLANATIONS

17. Has the EDF had any unintended or unexpected outcomes, either positive or negative? If yes, please describe.

Cost-effectiveness and Alternatives

18. In your view, is the EDF program’s current approach the most efficient and cost-effective way of accomplishing its objectives? Please explain.

19. Are you aware of any alternative programs or approaches in other federal departments/agencies or in other jurisdictions within or outside Canada that are comparable to the EDF?

IF YES:

20. Considering the program’s overall relevance, impacts and cost-effectiveness, to what degree do you feel the EDF provides value for dollars spent? Is the EDF able to achieve its outcomes at the lowest possible cost? Please explain.

21. As we near the conclusion, are there any other thoughts that you would like to share with respect to evaluating the Environmental Damages Fund?

THANK AND CONCLUDE

Funding Recipient’s Guide

Hello, my name is _____ _____ and I’m phoning from COMPAS Research. As part of an evaluation of the Government of Canada’s Environmental Damages Fund, we’re conducting interviews with individuals whose organizations have received support from the Environmental Damages Fund. Is there a day and time when you might be available to take part in an interview?

IF NECESSARY SAY: “Recently, an e-mail was sent out from Shelley Borys at Environment Canada advising you that COMPAS would be calling you to take part in an evaluation of the EDF program.”

A. IF THE KEY INFORMANT WANTS THE INTERVIEW NOW: Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

B. IF THE INTERVIEW IS SCHEDULED FOR ANOTHER TIME, (a) WE WILL TRANSMIT A COPY OF THE QUESTIONNAIRE AHEAD OF TIME TO THE PROSPECTIVE KEY INFORMANT AND (b) THE RESUMPTION OF THE CONVERSATION WOULD BEGIN HERE: Hello, it’s _____ _____ phoning from COMPAS Research for the interview with respect to an evaluation of the Environmental Damages Fund. Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

Background

1. Please briefly describe your role in the environmental project for which [NAME OF ORGANIZATION] received support from the Environmental Damages Fund.

2. Was your EDF-supported project mainly concerned with

Design and Delivery

3. Thinking of the information you received about the EDF program, what made the EDF an attractive source of funding for your organization?

4. Was there anything about the EDF that made it less attractive as a source of funding?

5. Based on what you know about how the EDF is meant to operate, to what extent do you feel the EDF program is delivered as designed?

6. In what ways is the EDF’s delivery consistent or inconsistent with the program’s original design? PROBE: Please explain and provide examples.

7. Overall, how well is the program being delivered?

PROMPT AS NECESSARY: And how about:

8. Are the roles and responsibilities of the various parties involved in the delivery of the EDF program clear? IF NOT, PROBE: Which aspects of the EDF are not well understood? Please explain.

9. As you know, the program encourages recipients to work with partners. In your experience, what is the effect of these partnerships on EDF-funded projects?

10. For EDF to attract worthwhile applications, is there a minimum level of financial support to applicants that it should consider? IF YES: What would that minimum amount be and why?

11. Thinking of your project, would you say that the amount of funding was

12. Using a seven-point scale where 1 means not at all clear” and 7 “very clear,” to what extent do you feel that the EDF program’s deliverables and expected results are clear?

13. Was performance data collected for your EDF-funded project? Was this information reported against EDF activities/outputs and outcomes? Please provide examples.

14. What have been the strengths and best practices in the design and delivery of the program? What has worked well?

Success

15. On a seven-point scale where 1 means “largely unachieved” and 7 “largely achieved,” to what extent do you feel that the intended outcomes of your funded projects were achieved?

16. Did your project have any unintended or unexpected outcomes, either positive or negative? If yes, please describe.

Cost-effectiveness and Alternatives

17. Are you aware of any alternative programs or approaches in other federal departments/agencies or in other jurisdictions within or outside Canada that are comparable to the EDF?

IF YES

18. As we near the conclusion, are there any other thoughts that you would like to share with respect to evaluating the Environmental Damages Fund?

THANK AND CONCLUDE

Potential Recipient’s Guide

Hello, my name is _____ _____ and I’m phoning from COMPAS Research. We’re doing an evaluation of the Government of Canada’s Environmental Damages Fund. As part of the evaluation, we’re conducting interviews with municipalities, community organizations, environmental organizations, universities, and First Nations’s organizations because these are the groups that are entitled to apply for project support from the Environmental Damages Fund. Is there a day and time when you might be available to take part in an interview?

A. IF THE KEY INFORMANT WANTS THE INTERVIEW NOW: Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

B. IF THE INTERVIEW IS SCHEDULED FOR ANOTHER TIME, (a) WE WILL TRANSMIT A COPY OF THE QUESTIONNAIRE AHEAD OF TIME TO THE PROSPECTIVE KEY INFORMANT AND (b) THE RESUMPTION OF THE CONVERSATION WOULD BEGIN HERE: Hello, it’s _____ _____ phoning from COMPAS Research for the interview with respect to an evaluation of the Environmental Damages Fund. Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

Background

1. Please briefly describe your role in any environmental project that [NAME OF ORGANIZATION] has undertaken or might undertake in the future.

Design and Delivery

2. The Environmental Damages Fund, administered by Environment Canada, makes contributions to local government, community organizations, environmental organizations, universities and First Nations for environmental restoration, environmental quality improvement, research and development, and education and awareness projects using court awards raised from penalties on polluters. Have you heard of the fund before?

3. Have you or your organization ever applied to the EDF for support? OPTIONAL PROMPT

4. ASK ONLY IF HEARD OF EDF IN Q2: Considering what you know about the EDF, what might make the EDF an attractive source of funding for your organization?

5. ASK ONLY IF HEARD OF EDF IN Q2: Is there anything about the EDF that might make it a less attractive source of funding?

6. To ensure the highest quality of funded projects, the program needs to be promoted among [SPECIFY LOCAL GOVERNMENT, COMMUNITY ORGANIZATIONS, ENVIRONMENTAL ORGANIZATIONS, UNIVERSITIES, AND FIRST NATIONS ACCORDING TO THE TYPE OF ORGANIZATION OF THE RESPONDENT]. What would be the most effective way of doing so?

7. For EDF to attract worthwhile applications, is there a minimum level of financial support to applicants that it should consider? IF YES: What would that minimum amount be and why?

8. The EDF program encourages applicants to have partners. In your view, what effect would the requirement for partners have on organizations like your own if you were to apply for funding through this program?

9. Based on your own or your organization’s experience applying for grants or contributions of any kind, what have been the strengths and best practices in the design and delivery of this type of program?

Cost-effectiveness and Alternatives

10. Are you aware of any alternative programs or approaches in other federal departments/agencies or in other jurisdictions within or outside Canada that are comparable to the Environmental Damages Fund?

11. Based on what you know of the EDF, what aspects of these other programs are most similar to the EDF? What is most unique about them? Please describe.

12. Are there any gaps in the activities and reach of these programs? IF YES: Is there anything EC’s EDF could do to help address these gaps?

13. As we near the conclusion, are there any other thoughts that you would like to share with respect to the Environmental Damages Fund?

THANK AND CONCLUDE

Academic’s Guide

Hello, my name is _____ _____ and I’m phoning from COMPAS Research. We’re currently conducting brief interviews with environmental experts as part of an evaluation of the Environmental Damages Fund. The environmental experts we're consulting do not need specific knowledge of the Environmental Damages Fund. Is there a day and time when you might be available to take part in an interview?

A. IF THE KEY INFORMANT WANTS THE INTERVIEW NOW: Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

B. IF THE INTERVIEW IS SCHEDULED FOR ANOTHER TIME, (a) WE WILL TRANSMIT A COPY OF THE QUESTIONNAIRE AHEAD OF TIME TO THE PROSPECTIVE KEY INFORMANT AND (b) THE RESUMPTION OF THE CONVERSATION WOULD BEGIN HERE: Hello, it’s _____ _____ phoning from COMPAS Research for the interview with respect to an evaluation of the Environmental Damages Fund. Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

Background

1. Please briefly describe your role within any environmental project that [NAME OF RESPONDENT’S ORGANIZATION] has undertaken or might undertake in the future.

Design and Delivery

2. The Environmental Damages Fund, administered by Environment Canada, makes grants to local government, community organizations, environmental organizations, universities and First Nations for environmental restoration, research and development, and education and awareness projects using court awards raised from penalties on polluters. Have you heard of the fund before?

3. Have you or your organization ever applied to the EDF?

4. To ensure the highest quality of funded projects, the program needs to be promoted among [SPECIFY LOCAL GOVERNMENT, COMMUNITY ORGANIZATIONS, ENVIRONMENTAL ORGANIZATIONS, UNIVERSITIES, AND FIRST NATIONS ACCORDING TO THE TYPE OF ORGANIZATION OF THE RESPONDENT]. What would be the most effective way of doing so?

5. For EDF to attract worthwhile applications, is there a minimum level of financial support to applicants that it should consider? IF YES: What would that minimum amount be and why?

6. The EDF program encourages recipients to have partners. In your view, what effect is the requirement for partners likely to have on projects? Is this a desirable requirement?"

7. Based on your own or your organization’s experience applying for grants or contributions of any kind, what have been the strengths and best practices in the design and delivery of this type of program?

Success

8. EDF project grants contribute to environmental restoration, research and development, and education and awareness. On the basis of your broad knowledge, would you expect a fund like the Environmental Damages Fund to be more effective than otherwise or less effective than otherwise in respect of one or other of these three purposes? OPTIONAL PROMPT: As mentioned, these three purposes are environmental restoration, research and development, and education and awareness.

Cost-effectiveness and Alternatives

9. Are you aware of any alternative programs or approaches in other federal departments/agencies or in other jurisdictions within or outside Canada that are comparable to the EDF?

10. As we near the conclusion, are there any other thoughts that you would like to share with respect to the Environmental Damages Fund?

THANK AND CONCLUDE

Judicial Segment’s Guide

Hello, my name is _____ _____ and I’m phoning from COMPAS Research. We’re currently conducting interviews on behalf of Environment Canada with individuals in the judicial or court systems as part of an evaluation of the Environmental Damages Fund. Is there a day and time when you might be available to take part in an interview?

[IF NECESSARY SAY] “Recently an e-mail was sent out from Shelley Borys at Environment Canada advising you that COMPAS would be calling you to take part in evaluation of the EDF program.”

A. IF THE KEY INFORMANT WANTS THE INTERVIEW NOW: Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

B. IF THE INTERVIEW IS SCHEDULED FOR ANOTHER TIME, (a) WE WILL TRANSMIT A COPY OF THE QUESTIONNAIRE AHEAD OF TIME TO THE PROSPECTIVE KEY INFORMANT AND (b) THE RESUMPTION OF THE CONVERSATION WOULD BEGIN HERE: Hello, it’s _____ _____ phoning from COMPAS Research for the interview with respect to an evaluation of the Environmental Damages Fund. Thank you for agreeing to the interview. We generally audio record our interviews to ensure our discussion is documented accurately. Would this be ok? In either instance, the interview would be confidential and no comment would be linked to you. UNPROMPTED

Background

1. Please briefly describe any direct or indirect involvement you have had with legal activity in environmental areas in general and also with the Environmental Damages Fund.

Design and Delivery

2. Have you ever heard of the Environmental Damages Fund?

3. The Environmental Damages Fund, administered by Environment Canada, makes contributions to local government, community organizations, environmental organizations, universities and First Nations for environmental restoration, environmental quality improvement, research and development, and education and awareness projects using court awards raised from penalties on polluters. How familiar are the courts with the fund insofar as you can tell?

4. How might Environment Canada increase awareness of the EDF in the courts?

5. As you likely know, EPAMs are negotiated settlements as an alternative to court prosecution for a violation of the Canadian Environmental Protection Act. Are efforts being made to encourage EPAM awards to be directed to the EDF?

6. Although CEPA is the federal statute under which financial assignments to the EDF are generally made, it is also possible for the EDF to be used in relation to infractions under the Fisheries Act, the Canada Shipping Act, the Migratory Birds Convention Act, and the Species at Risk Act. In your view, are any efforts being made to encourage the courts to use these other laws to direct payments to the EDF? Please explain.

7. Using a seven-point scale where 1 means “not at all” and 7 means “to a great extent,” to what extent do you feel the courts are using the EDF program as it was intended to be used?

8. What have been the strengths and best practices in the design and delivery of the EDF program through the courts?

Success

9. On a seven-point scale where 1 means “largely unachieved” and 7 “largely achieved,” to what extent do you feel that the Environmental Damages Fund contributes to environmental restoration, environmental quality improvement, research and development, and education and awareness?

Cost-effectiveness and Alternatives

10. Are you aware of any alternative programs or approaches in other federal departments/agencies or in other jurisdictions within or outside Canada that are comparable to the EDF?

11. As we near the conclusion, are there any other thoughts that you would like to share with respect to evaluating the Environmental Damages Fund?

12. Do you know of any individuals in the judicial or court systems who could provide feedback on the Environmental Damages Fund and its used by the courts? PLEASE ELICIT NAMES, TITLES, AND TELEPHONE NUMBERS TO THE EXTENT THAT THE KEY INFORMANT FEELS COMFORTABLE MAKING SUGGESTIONS.

THANK AND CONCLUDE


Top of Page

Annex 7: Summary of Findings

Evaluation Question (EQ) Achieved Progress Made, Attention Needed Little Progress, Priority for Attention Not Applicable
Relevance:
EQ1 Is there a legitimate and necessary role for government in this program area or activity?

EQ2 Is the program connected with societal/environmental needs?

EQ3 Does the program theory (activities, instruments, objectives, design) address the societal need identified?

Design and Delivery:
EQ4 Identification of deliverables and results

EQ5 Delivered as designed

EQ6 Performance measurement and reporting

EQ7 Accountability

EQ8 Best practices and lessons learned
Success:
EQ9 Achievement of program outcomes

a) project delivery outcomes

b) communications and promotion

c) partnerships
~
d) proposal planning and development

EQ10 Unintended outcomes

Cost-Effectiveness:
EQ11 Alternatives

EQ12 Duplication/ complementarity

EQ13 Improved efficiency

EQ14 Value for federal dollars


Top of Page

1 This report was co-written by COMPAS Inc. and Environment Canada’s Evaluation Division, Audit and Evaluation Branch.

2 The EDF is a specified-purpose account, which is an account that is opened in the general ledger to ensure that funds are used only for the purposes for which they were received and that allows managers to better control and manage these funds (Treasury Board of Canada Secretariat. 1995. Policy on Specified Purpose Accounts. Ottawa.)

3 Environment Canada. Undated. Environmental Damages Fund (EDF) Applicant’s Guide. Ottawa.

4 PWGSC Government Consulting Services. 2007. Environmental Damages Fund Process Analysis. Ottawa.

5 “The SSOP Fund is the Canadian insurance fund established for the purpose of ensuring the payment of claims for marine oil pollution that originates from ships. The system is designed to cover the risk of non-payment by the ship owner who is responsible for pollution. In addition, it covers claims for damage and clean-up costs where the identity of the ship that caused the discharge of oil cannot be established - i.e., mystery spill.”(http://www.ssopfund.gc.ca/english/faq.asp, accessed June 2009).

6 Environment Canada. 2008. The Environmental Damages Fund: A Program Overview. Presentation deck. Ottawa.

7 Ibid.

8 Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework. Ottawa.

9 Ibid.

10 The EDF Applicant’s Guide also stipulates a number of types of activities that are not eligible for EDF funding, such as existing activities; mandates by law or other levels of government; recreation, tourism, or beautification; formal curriculum material preparation; annual or regular events/campaigns; conferences and workshops; core funding; infrastructure, particularly as it relates to municipal, provincial, or federal government program areas; incentive or rebate programs; lobbying or advocacy activities; containment and clean-up of environmental spills; restoration of contaminated sites; and efforts outside of Canada (Environment Canada. Undated. Environmental Damages Fund (EDF) Applicant’s Guide. page 3. Ottawa.).

11 Environment Canada. Undated. Environmental Damages Fund (EDF) Applicant’s Guide. page 4. Ottawa.

12 Environment Canada. 2009. 2008 Request for Update on Management Response to Address Evaluation Recommendations. Internal document. Ottawa.

13 The Community Funding program also had responsibility for delivering the One-Tonne Challenge program within the timeframe investigated in the current evaluation. This program was cancelled in 2005–2006.

14 Morrison, S. 2007. Environment Canada Courts Awards (Pilot) Project: Progress Report. Powerpoint presentation.

15 Treasury Board. 1995. Policy on Specified Purpose Accounts, Section 6, sub-section (o). Ottawa.

16 The design and fieldwork for the EDF evaluation was carried out in the 2008–2009 fiscal year, prior to implementation of the new Treasury Board Policy on Evaluation (http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024). The current evaluation reflects those issues outlined in the 2001 evaluation policy that was in effect at the time this evaluation was conducted.

17 Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework, Appendix A. Ottawa.

18 PWGSC Government Consulting Services. 2007. Environmental Damages Fund Process Analysis. Ottawa.

19 Oversampling provides a more precise and reliable understanding of the characteristics of smaller sub-groups in instances where their occurrence in the population would not yield an adequate number of cases employing a stratified random sampling approach.

20 Respondents in this segment consist of four prosecutors, one judge, and one legal counsel to a federal department.

21 Potential applicants were identified as eligible for program funding according to the program’s terms and conditions.

22 Department of Finance. 2007. The Budget Plan 2007: Aspire to a Stronger, Safer, Better Canada. Ottawa.

23 Department of Finance. 2008. The Budget Plan 2008: Responsible Leadership. Ottawa.

24 The Environment Canada statutes include the Canadian Environmental Protection Act, the Canada Wildlife Act, the Migratory Birds Convention Act, the Wild Animal and Plant Protection and Regulation of International and Interprovincial Trade Act, the Antarctic Environmental Protection Act, and the International River Improvements Act.

25 The Parks Canada statutes include the Canada National Parks Act, the Canada National Marine Conservation Areas Act, and the Saguenay–St. Lawrence Marine Park Act.

26 HDP Group Inc. 2002. Environmental Damages Fund Evaluation. Ottawa.

27 The Environmental Law Centre defines creative sentencing as the option for judges “...to take an innovative approach to sentencing by ordering that funds be dedicated to certain projects like research, education or improvements in industry standards”. (http://www.elc.ab.ca/Content_Files/Files/NewsBriefs/CreativeSentencing-V18-2.pdf, accessed June 2009).

28 Creative sentencing might also include requiring polluters to undergo environmental audits or perform community service (Wruck, J. 2004. The Federal Environmental Damages Fund. The Advocate, Vol. 62-2.)

29 Wruck, J. 2004. The Federal Environmental Damages Fund. The Advocate, Vol. 62-2, page 228.

30 HDP Group Inc. 2002. Environmental Damages Fund Evaluation. Ottawa.

31 These points appear in the interviews with EC key informants and in Wruck, J. 2004. The Federal Environmental Damages Fund. The Advocate, Vol. 62-2., passim and Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework, page 4. Ottawa.

32 The one exception to this largely uniform viewpoint is the few key informants, primarily EC personnel, who advocate assigning the EDF’s entire role to either the judiciary or the provinces. These key informants, however, provided no insights as to how the courts or provinces would acquire the authority or administrative capacity to deliver such a program, nor how the assumption of these responsibilities would impact on the courts’ neutral adjudicative role.

33 Morrison, Stacey. 2007. Environment Canada Courts Awards (Pilot) Project: Progress Report Powerpoint presentation.

34 Environment Canada. 2009. Number of Spill/Releases, by Fiscal year, by Region, National Emergencies and Enforcement Management Information System and Intelligence System. Administrative data.

35 HDP Group Inc. 2002. Environmental Damages Fund Evaluation. Ottawa.

36 Presentations at meetings and retreats attended by judges are already given in Alberta.

37 In the ‘Project Details’ section (page 3) of the application form, the EDF asks applicants to “Identify what your project will accomplish and how the environment and community will benefit.” In the ‘Evaluation’ section (page 4) of the application form, the EDF asks applicants, “How will the environment benefit from this project?”

38 The EDF Applicant’s Guide says that “funding is available for projects which...demonstrate broad community support (e.g. both cash and in-kind partnerships)” and calls for “details on other funding sources which include cash and in-kind partnership support.”

39 For reasons of inherent overlap, Aboriginal organizations appear within the other categories in the classification displayed in Table 4.1.

40 HDP Group Inc. 2002. Environmental Damages Fund Evaluation. Ottawa.

41 While a number of documents make passing reference to the need for projects to be cost-effective (Environment Canada. Undated. Environmental Damages Fund Applicant’s Guide. Ottawa; Environment Canada. 2006. Environmental Damages Fund Grants and Contributions Approval Form. Ottawa; Environment Canada. 2000. Introducing the Environmental Damages Fund, program brochure.), there is no formal requirement for project proponents to demonstrate the project’s cost-effectiveness when applying for EDF funding. The closest any program documentation makes to an assessment of cost-effectiveness is the EDF project Technical Review Evaluation Form, where an aggregate rating (out of 5) of the project’s budget considers cost-effectiveness in combination with the budget detail and reasonableness of costs (Environment Canada. 2005. Results-based Management Accountability Framework, Appendix C. Ottawa).

42 Environment Canada. 2007. Environmental Damages Fund Recipient’s Handbook. Ottawa.

43 Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework - Draft 7, Appendix A. Ottawa.

44 PWGSC Government Consulting Services. 2007. Environmental Damages Fund Process Analysis. Ottawa.

45 Not all EC respondents were asked for their views on EPAMs, as this question was included in the questionnaire only after fieldwork had begun.

46 Environment Canada. 2006. Environmental Damages Fund Framework. Ottawa.

47 Environment Canada. 2008. Environmental Damages Fund Results–Based Management Accountability Framework. Ottawa.

48 Environment Canada. undated. Environmental Damages Fund Applicant’s Guide. Ottawa.

49 Environment Canada. 2007. Environmental Damages Fund Recipient Handbook. Ottawa.

50 Environment Canada. 2007. Communications Plan for the National Promotion of the Environmental Damages Fund (EDF) Program 2007 – 2009. Ottawa.

51 For example, a 2007 process review of the EDF recommended that the program establish a clear separation of duties when it was found that certain tasks, particularly those related to the diversion of funds from court awards to the EDF, could be performed by a number of different organizations and varied from region to region. Specifically, while offenders mail cheques to the courts in some regions (Pacific and Atlantic), enforcement officers take receipt of cheques in Quebec and in Ontario, it is sent to the Regional Director General (RDG) (PWGSC Government Consulting Services. 2007. Environmental Damages Fund Process Analysis. Ottawa).

52 Given the small sample size for non-EDF Environment Canada personnel, results for this sub-group could not be reported separately.

53 Evidence was deemed partial in cases where the final project report acknowledged that only some impacts had been achieved or where interim reports contained evidence of progress toward achieving intended outcomes but the final report was missing.

54 Environment Canada. Undated. Environmental Damages Fund (EDF) Applicant’s Guide, page 4. Ottawa.

55 The program logic model (Annex 1) also lists “EDF restoration objectives” as an outcome under the Project Delivery activity stream. This issue is addressed in the preceding section in the context of findings related to the achievement of overall program outcomes and the program’s contribution to environmental restoration, environmental quality improvement, research and development, and education and awareness.

56 Environment Canada. 2007. Environmental Damages Fund Recipient Handbook. Ottawa.

57 PWGSC Government Consulting Services. 2007. Environmental Damages Fund Process Analysis. Ottawa

58 HDP Group Inc. 2002. Environmental Damages Fund Evaluation, pages 3, 5 and 6.

59 Ibid, page 8.

60 PWGSC Government Consulting Services. 2007. Environmental Damages Fund Process Analysis, page 11. Ottawa.

61 Wruck, J. 2004. The Federal Environmental Damages Fund. The Advocate. Vol. 62-2, page 8.

62 Environment Canada. 2005. Environmental Damages Fund Results-based Management and Accountability Framework, Table 2. Ottawa.

63 Environment Canada. 2006. Environmental Damages Fund Funded Projects. Accessed at: http://www.ec.gc.ca/edf-fde/default.asp?lang=En&n=24600456-1

64 The program logic model lists “increased court-awarded contributions to the EDF” as both a partnership outcome and a communication outcome. This result is addressed in the discussion of communications and program promotion outcomes in Section 4.3.1.

65 Migratory Birds Convention Act (MBCA 1994), Species at Risk Act, and the Canadian Environmental Protection Act, 1999 (CEPA 1999).

66 Fisheries Act.

67 Canada Shipping Act, 2001

68 The Parks Canada statutes include the Canada National Parks Act, the Canada National Marine Conservation Areas Act, and the Saguenay–St. Lawrence Marine Park Act.

69 There was insufficient information to draw a conclusion about the manager’s review and RDG sign-off in 17% of project files.

70 Insufficient information was available to make a determination about the remaining project files.

71 Ibid.

72 For practical reasons, the evaluation approach did not include individual analyses of project files to assess their cost-effectiveness, and so relied on existing information. It would be prohibitive to undertake such an assessment in the context of the current evaluation. However, several funding recipients did note unexpected requirements for financial resources and the resulting necessity of stretching their funding.

73 In the one case where project officers reached a conclusion that the individual project was not cost-effective, the conclusion was not based on cost-effectiveness per se but rather on the absence of potential partners capable of supplementing the EDF contribution with additional financial or non-financial support.

74 For example, one EcoAction priority related to nature is to support “projects focusing on protecting wildlife and plants, and protecting and improving the habitat where they live (e.g., grasslands, rivers, forests, etc.).” (Environment Canada. 2004. Results-based Management and Accountability Framework for the EcoAction Community Funding Program. Ottawa).

75 Every project, however small, entails inherent assessment, processing, oversight, and audit costs. The proportional costs for these functions are necessarily higher for small projects.

76 The respective “yes/no/do not know" percentages for EC key informants are 28/61/11; academics, 78/0/22; potential recipients, 60/27/13; and funding recipients, 43/48/10.