Third national assessment

2.0 Methods

2.1 Studies Summarized in the Third National Assessment Report

This report focuses on the biological components of environmental effects monitoring (EEM) studies in order to provide a national assessment of the effects of metal mining effluents on fish populations, fish habitat and the use of fisheries resources. The results of all completed biological monitoring studies undertaken to assess the presence or absence of effects that were submitted to Environment and Climate Change Canada (ECCC) before October 1, 2013 are summarized. In addition, biological monitoring studies submitted before June 2014 that investigated the magnitude, geographic extent and cause of observed effects are summarized. Sublethal toxicity (SLT) and effluent characterization study results from 2003 to 2012 are also summarized in order to provide supporting information.

Before October 1, 2013, 121 metal mines in Canada had conducted biological monitoring studies, with the number of studies dependent on when the mine became subject to the MMER. Among the 121 mines that carried out studies,

• 36 mines conducted one study;
• 25 mines conducted two studies;
• 43 mines conducted three studies;
• 17 mines conducted four studies.

Mines that are now recognized closed mines are included among the 121 mines that conducted studies. Six of the 121 mines were not able to obtain the necessary data to assess effects on either the fish population or fish habitat components and some of the remaining 115 mines were not able to obtain the necessary data to assess effects on both components. Insufficient number of fish captured was the most common reason for unsuccessful fish population studies, while differences in habitat between reference and exposure areas was the most common reason for unsuccessful fish habitat studies. Table B4 (Appendix B) provides an overview of the number and type of biological studies conducted by metal mines since the coming into force of the Regulations. Studies attempted but not successfully completed are included in Table B4 but are not included in the results section of this report.

For two mines that had changed the location of the final discharge point after completing one or two EEM studies, the results of the EEM studies conducted prior to this change in location were not included in this report. Similarly, for one mine that had made an important change in the water treatment system, the results from the EEM study conducted prior to this change were not included.

Top of page

2.2 General Methods Used to Compile Fish and Fish Habitat Study Results

The EEM biological monitoring study reports submitted by mines to ECCC were reviewed by department experts to ensure that all regulatory requirements had been fulfilled and that the studies had been conducted according to generally accepted standards of good scientific practice. The study reports presented the raw data, the analysis and interpretation of these data and the conclusions. A portion of the raw data was also submitted to ECCC in electronic format. To provide additional verification of results, ECCC experts analyzed the raw data relating to fish and benthic indicators using a statistical assessment tool (SAT)^{Footnote 2} developed by ECCC (Booty et al. 2009). The SAT was used to calculate the magnitude and statistical significance of differences between exposure and reference data for the five fish and four benthic invertebrate community effect indicators in cases where the biological monitoring study used the control/impact design^{Footnote 3} and the fish survey used lethal methods to collect the data. To take into account site-specific factors not discernible from the raw data, results from the SAT analysis were compared to results in the mine’s study report. When discrepancies occurred that could not be explained by calculation error, the results from the mine’s study report were used in the compilation process. Between 50 and 70% of studies  were conducted using lethal methods with a control/impact sampling design and were therefore analyzed using the SAT. For studies using other types of sampling designs,^{Footnote 4} results from the mine’s study report were used in the compilation process.

Observed effects (or absence of effects) were categorized on the basis of occurrence, magnitude and type. The effect categories, from highest potential risk to the environment to lowest potential risk, are:

• effect or confirmed^{Footnote 5} effect equal to or greater than the critical effect size (CES)
• effect or confirmed^{Footnote 6} effect less than the CES
• unconfirmed^{Footnote 7} effect
• absence or confirmed^{Footnote 8} absence of effects

The effect category for an entire component (fish or fish habitat) was determined by the effect with the highest potential risk for that component. The overall effect category for each mine was determined by the component with the highest potential risk. Results are provided on a mine-by-mine basis in Table C1 of Appendix C.

Top of page

2.3 Comparative Analysis of Bray-Curtis Index Significance Testing Methodologies

The mines used the Bray-Curtis Index (BCI) to assess the similarity of the structure of benthic invertebrate communities. A review carried out by Borcard and Legendre in 2013 for ECCC determined that the methodology described in the EEM technical guidance document (Environment Canada 2012a) for calculating the statistical significance of differences observed in BCI data between exposure and reference areas has a higher-than-recommended probability of returning a false positive result (i.e., identifying an effect when no effect is present). Borcard and Legendre suggested a revised methodology for testing the significance of differences in BCI data. ECCC will recommend the use of this revised methodology in EEM technical guidance going forward.

The information relating to the presence or absence of effects on benthic invertebrate community structure (Bray-Curtis Index, BCI) in this third assessment report was obtained using the existing methodology. To quantify the occurrence of false positives, ECCC conducted a comparative analysis of the results obtained with the two methodologies for the same data. Mines with a confirmed effect on the BCI and for which a false positive would affect decision making regarding the type of biological monitoring study required in the future were chosen for the comparative study. Two groups of mines were selected. The first group included seven mines that had confirmed an effect on the BCI and had not confirmed effects on any other fish habitat or fish indicators. The second group included 12 mines that had confirmed an effect on the BCI and had not confirmed effects on any other fish habitat indicators, but had confirmed effects on at least one fish indicator.

Data from 38 studies were used to compare results for these 19 mines. For each of the 38 studies, the significance of the difference between exposure and reference BCI data was re-calculated using the revised methodology. In all 38 studies, the revised methodology indicated that there was still a statistically significant difference (an effect) between exposure and reference BCI data. The agreement in results between the existing and the revised methodologies is consistent with the prediction made by Borcard and Legendre (2013) that there would be only a few differences in findings for the two methodologies. For the mines used in the comparative analysis, this means there were no observations of false positive BCI effects resulting from the use of the existing methodology. The BCI results from the 38 studies could therefore be used with confidence in the overall compilation of observed effects. Comparative analyses of other BCI results were not conducted since, for all other studies, effects were observed on other fish habitat indicators in addition to the BCI. The elimination of a false BCI effect in these studies would not have impacted decisions made regarding the type of biological monitoring studies required in the future.

Top of page