Skip booklet index and go to page content

Evaluation of the Hydrological Service and Water Survey

Annex 1 Evaluation Questions, Indicators and Data Sources

Relevance
Does the program remain consistent with and contribute to the federal government priorities and address actual needs?
QuestionIndicatorsSources/MethodsTB Policy Issue Addressed
1. Is there a continued need for the program?1.1 Evidence of/views on the existence of societal/environmental need
1.2 Presence/absence of other programs that complement or duplicate the objectives of the program
1.3 Degree to which gaps would exist in addressing societal/environmental need in absence of the program
  • Document review
  • Key informant interviews
  • Literature review
Issue #1:
Continued Need for Program
2. Is the program aligned with federal government priorities?2.1 Evidence of/views on the degree to which program’s objectives correspond to recent/current federal government priorities
2.2 Evidence of/views on the degree to which the program’s objectives are aligned with current departmental strategic outcomes
  • Document review
  • Key informant interviews
Issue #2:
Alignment with Federal Government Priorities
3. Is the program consistent with federal roles and responsibilities?3.1 Program mandate aligned with federal government jurisdiction
  • Document review
Issue #3:
Consistency with Federal Roles and Responsibilities
3.2 Views on the appropriateness of federal involvement
  • Key informant interviews
Performance – Efficiency and Economy
Are the most appropriate, efficient and economic means being used to achieve outcomes?
QuestionIndicatorsSources/MethodsTB Policy Issue Addressed
4. Is the program design appropriate for achieving its intended outcomes?4.1 Logical link between program activities, outputs, and intended outcomes
4.2 Comparison of program activities and products with those delivered by other similar programs 4.3 Evidence of/views on whether there are alternative program models that would achieve the same expected outcomes at a lower cost
  • Document review
  • Key informant interviews
  • Financial analysis
Issue #5:
Demonstration of Efficiency and Economy
5. To what extent is the governance structure clear, appropriate and efficient for achieving expected results?5.1 Clearly defined and understood governance structure, including roles, responsibilities, accountabilities, and processes for prioritization and decision-making
5.2 Evidence of/views on the appropriateness and efficiency of governance structure
  • Document review
  • Key informant interviews
Issue #5:
Demonstration of Efficiency and Economy
6. Is the program undertaking specific activities and delivering products at the lowest possible cost?
How could the program’s activities be more efficient?
Are there alternative, more economical ways of delivering program outputs?
6.1 Degree to which the program is delivered as designed and intended
6.2 Evidence of /views on appropriateness and efficiency of administrative and operational processes
6.3 Evidence of/views on effectiveness of communications and collaboration with partners and stakeholders
6.4 Evidence of use of technologies that may impact efficiency
6.5 Evidence of/views on the existence of program design elements that facilitate efficient delivery (e.g., benchmarking, service standards)
6.6 Degree of progress in implementing management responses to recommendations from previous evaluations and audits
  • Key informant interviews
  • Document review
Issue #5:
Demonstration of Efficiency and Economy
6.7 Evidence of/views on reasonableness of program resources/capacity in light of intended outcomes
6.8 Evidence of/views on whether the program utilized the least amount of resources needed to produce its outputs
6.9 Evidence of/views on whether opportunities for cost recovery are being pursued
  • Key informant interviews
  • Document review
  • Financial analysis
6.10 Evidence of/views on how the program’s activities could be more efficient
6.11 Evidence of/views on whether there are alternative, more economical ways of delivering program activities and outputs
  • Key informant interviews
  • Document review
7. Are performance data being collected and reported?
If so, is this information being used to inform senior management/ decision-makers?
7.1 Evidence that outputs and outcomes are well-articulated, and a performance measurement strategy/plan has been developed and implemented
7.2 Extent to which performance data are reliable, timely and valid
7.3 Extent to which performance data are reported
7.4 Extent to which performance data inform/support EC’s decision-making processes
  • Document review
  • Performance data analysis
  • Key informant interviews
Issue #5:
Demonstration of Efficiency and Economy
Performance – Effectiveness
Has the program achieved its intended outcomes?
QuestionIndicatorsSources/MethodsTB Policy Issue Addressed
8. To what extent have intended outcomes been achieved as a result of the program?8.1 Evidence of/views on the extent to which intended outcomes have been achieved as a result of the program (PMF and QMS outcomes)
8.2 Evidence of/views on factors outside the program which have influenced the achievement of intended outcomes
  • Key informant interviews
  • Document review
  • Performance data analysis
Issue #4:
Achievement of Intended Outcomes (Effectiveness)
9. Have there been any unintended (positive or negative) outcomes?9.1 Evidence of/views on the presence/absence of unintended outcomes
  • Key informant interviews
  • Document review
Issue #4:
Achievement of Intended Outcomes (Effectiveness)
Description of Annex 1

Annex 1 consists of a table presenting the nine evaluation questions examined in this evaluation and, for each question, the associated indicators, data sources and methods used to address the question. It also indicates for each evaluation question the related evaluation issue being addressed from the Treasury Board Policy on Evaluation.

 
Final Report
Date modified: