Evaluation of the Canada-U.S. Clean Energy Dialogue

1.

Is the Clean Energy Dialogue connected with key international climate change and air quality needs?

What should be observed

• Activities under the CED are connected with key international climate change and air quality needs

Indicators

• Evidence that CED objectives agree with relevant scientific evidence on climate change and air quality
• Evidence that the CED adapts to or influences changes in government policy and priorities

Sources/Methods

• Document or file review
• Key informant interviews

2.

Is the Clean Energy Dialogue aligned with federal government priorities?

What should be observed

• Alignment with federal government priorities

Indicators

• Extent to which strategic directions within the CED match federal government priorities

Sources/Methods

• Document or file review
• Key informant interviews

3.

Is the Clean Energy Dialogue aligned with the priorities of participating departments?

What should be observed

• Alignment with the departmental priorities of EC, DFAIT and NRCan

Indicators

• Extent to which strategic directions within the CED match priorities of each of the participating departments

Sources/Methods

• Document or file review
• Key informant interviews

4.

Is there a legitimate and necessary role and responsibility for the government of Canada to deliver the Clean Energy Dialogue?

What should be observed

• Activities under the Clean Energy Dialogue support demonstrated federal role and responsibilities

Indicators

• Demonstrated evidence of federal role and responsibility in the area of international actions for climate change and air quality related to the CED
• Absence/presence of similar programs within the federal and/or provincial governments
• Extent to which similar programs complement/duplicate one another

Sources/Methods

• Document or file review
• Key informant interviews

5.

To what extent have intended outputs been achieved as a result of the Clean Energy Dialogue?

What should be observed

• Demonstrated achievement of the following outputs:

1. Policy and research papers, negotiating positions and advice

   • Indicators

   • Working Groups’ reports address all stated expectations
   • The Action Plan on clean energy addresses all stated expectations
   • Percentage of total joint Canada-U.S. reports to ministers or leaders where deadlines are met
   • Opinions of program staff, partners and stakeholders that the output has been achieved or is making progress towards achievement
2. Partnerships and processes outside the UNFCCC

   • Indicators

   • Number of high-level bilateral meetings, tele- & videoconferences in support of Working Group activities and deliverables
   • Number of high-level meetings with influential energy players in the U.S. by the Special Advisor to the CED
   • Opinions of program staff, partners and stakeholders that the output has been achieved or that progress is being made towards its achievement
3. Formal and informal consultations with domestic stakeholders

   • Indicators

   • Number of domestic stakeholders consulted on CED activities
   • Number of consultations held with domestic stakeholders on CED activities
   • Opinions of program staff, partners and stakeholders that the output has been achieved or that progress is being made towards its achievement

Sources/Methods

• Document or file review
• Key informant interviews

6.

To what extent have immediate outcomes been achieved as a result of the Clean Energy Dialogue?

What should be observed

• Evidence of progress towards the achievement of the following intended immediate outcome:

1. Implementation of recommendations in the Clean Energy Dialogue Action Plan

   • Indicators

   • Percentage of Action Plan projects and initiatives implemented through the CED process
   • Opinions of program staff, partners and stakeholders that the outcome has been achieved or that progress is being made towards its achievement

Sources/Methods

• Document or file review
• Key informant interviews

7.

To what extent have immediate outcomes been achieved as a result of the Clean Energy Dialogue?

What should be observed

• Evidence of progress towards the achievement of the following intended immediate outcome:

1. Canada’s profile as a provider of climate-friendly technologies is enhanced.

   • Indicators

   • Opinions of program staff, partners and stakeholders that Canada provides climate-friendly technologies
   • Opinions of program staff, partners and stakeholders that the CED is an effective vehicle to advance clean energy technology
2. Increased collaboration between Canada and the U.S. on clean energy research, development and demonstration and broader engagement with the U.S. on developing compatible policy/regulatory frameworks consistent with Canada’s interests

   • Indicators

   • Number of new or expanded joint projects and initiatives undertaken through the CED process in the following areas:
     • Clean energy research and development
     • Deployment of clean energy technology, particularly CCS
     • Building a more efficient North American grid
   • Evidence of agreement(s) to work towards compatible policy/regulatory standards that promote the development and transfer of clean energy technologies
   • Opinions of program staff, partners and stakeholders that the outcome has been achieved or that progress is being made towards its achievement

Sources/Methods

• Document or file review
• Key informant interviews

8.

Have there been any unintended (positive or negative) outcomes that can be attributed to the Clean Energy Dialogue? Were any actions taken as a result of these?

What should be observed

• Observed unintended outcomes can be attributed to activities within CED
• Where appropriate, actions were taken to address unintended impacts

Indicators

• Presence/absence of unintended outcomes
• Where appropriate, documented management actions and/or lessons learned from unintended outcomes

Sources/Methods

• Document or file review
• Key informant interviews

9.

Are there any external factors outside the Clean Energy Dialogue that influence the success of its programs?

What should be observed

• Factors external to the CED that have influenced the achievement of results

Indicators

• Evidence of factors outside the CED that have influenced the achievement of outputs and early outcomes
• Where appropriate, documented management actions to address the influence of external factors

Sources/Methods

• Document or file review
• Key informant interviews

10.

Are there more economic and efficient means of achieving objectives under the Clean Energy Dialogue?

What should be observed

• Alternative design/delivery methods that are less costly or effort-intensive, enable quicker achievement of results, and /or enable achievement of better results

Indicators

• Opinions of program staff and senior management, partners and stakeholders on the ability of program elements to achieve intended results, compared to alternative design/delivery approaches
• Opinions of program managers and senior management, partners and stakeholders on whether  investments under CED are a good use of public funds and whether the cost of producing outputs is as low as possible

Sources/Methods

• Document or file review
• Key informant interviews

11.

How could the efficiency of the Clean Energy Dialogue be improved?

What should be observed

• CED activities show opportunities for increased efficiency

Indicators

• Opinions of program managers and senior management, partners and stakeholders on how the efficiency of the CED could be improved

Sources/Methods

• Document or file review
• Key informant interviews

12.

To what extent has the Clean Energy Dialogue been implemented, or is on track to being implemented, as planned and on time?

What should be observed

• Program implementation is consistent with defined approach

Indicators

• Evidence that the CED is being implemented as designed
• Evidence of meeting (or progress towards meeting) program objectives within anticipated timeframe and/or budget
• Barriers to program implementation are identified
• Barriers to program implementation are resolved

Sources/Methods

• Document or file review
• Key informant interviews

13.

Is the management and accountability structure for the Clean Energy Dialogue in place and functioning adequately to achieve expected results?

What should be observed

• Management structure for activities is defined, implemented and functioning effectively, including accountabilities, responsibilities and roles of all participating departments
• Expected outputs and outcomes are clear and understood internally and externally
• Overlap/duplication of effort is minimized and no gaps in programming exist

Indicators

• Opinions of program managers, deliverers and partners
• Evidence of decisions being made
• Evidence that information, results and decisions are being communicated to the appropriate parties

Sources/Methods

• Document or file review
• Key informant interviews

14.

Are appropriate performance data being collected, captured and safeguarded? If so, is this information used to inform senior management and decision makers?

What should be observed

• Required performance data collection systems are in place, where appropriate
• Performance data are collected against outputs and are used to inform decision making

Indicators

• Presence of populated performance data systems, where appropriate
• Evidence of collection of performance data, as appropriate
• Evidence of decisions being made based on performance information

Sources/Methods

• Document or file review
• Key informant interviews