Issue 
EPJ Nuclear Sci. Technol.
Volume 4, 2018
Special Issue on 4th International Workshop on Nuclear Data Covariances, October 2–6, 2017, Aix en Provence, France – CW2017



Article Number  40  
Number of page(s)  9  
Section  Applied Covariances  
DOI  https://doi.org/10.1051/epjn/2018021  
Published online  14 November 2018 
https://doi.org/10.1051/epjn/2018021
Regular Article
Cross section covariances: a user perspective
Idaho National Laboratory,
Idaho Falls, USA
^{*} email: Giuseppe.Palmiotti@inl.gov
Received:
16
November
2017
Received in final form:
1
March
2018
Accepted:
14
May
2018
Published online: 14 November 2018
This paper makes a brief historical review, from the user point of view, of the impact of the different covariance data made available during this period, then look at today current situation in the different evaluated files, using practical examples. In particular, the performance, in terms of C/E, of two of the most recent and reliable evaluate files, CIELO and CIELO2, is presented for a set of selected integral experiments. Moreover, this performance is compared for consistency against the uncertainty computed with two of the current most used covariance data (COMMARA2.0 and COMACV1). A limited number of comparisons are also shown relative to the JENDL4 data. Of specific interest is the observed difference due separately to the diagonal and offdiagonal (correlations) contributions. Finally, some perspective and future needs are discussed.
© G. Palmiotti and M. Salvatores, published by EDP Sciences, 2018
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1 Introduction
In the last two decades there has been a significant effort by the neutron cross section evaluators in generating new high quality data and in parallel a large effort has been made for producing covariance matrices to be used mainly in uncertainty analyses and data assimilations. The need for good quality covariance data was expressed, motivated, and quantified by the users especially in view of reductions of safety margins and economical optimization of advanced reactor designs.
This paper intends to offer an assessment of the current situation on cross section covariance matrices from the point of view of a user. First a brief historical review of the use of covariance data for uncertainty quantification purposes will be provided, and then some background information, both theoretical and practical, will be illustrated. Next, a look will be taken at what is the performance (i.e. impact on measurement/calculations discrepancy) of three of the most recent and reliable evaluate files: CIELO (aka ENDF/BVIIIβ4) [1] and CIELO2 (aka JEFF3.3T3) [2], and JENDL4 [3].
CIELO (Collaborative International Evaluated Library Organization) has provided a new working frame to facilitate evaluated nuclear reaction data advances. It has brought together experts from across the international nuclear reaction data community to identify and document discrepancies among existing evaluated data libraries, measured data, and model calculation interpretations, and has aimed to make progress in reconciling these discrepancies to create more accurate ENDFformatted files. The focus has initially been on a small number of the highestpriority isotopes, namely ^{1}H, ^{16}O, ^{56}Fe, ^{235,238}U, and ^{239}Pu. The performance in terms of discrepancies between calculated and measured values will be compared for consistency against the uncertainty computed with two of the current most used covariance data. In particular, effect for specific isotope and reactions will be shown, limited for space constraints to the two CIELO data sets.
Finally, some conclusions, perspectives and future needs will be provided.
2 Brief historical review
The first library that provided “serious” covariance data was ENDF/BIV distributed in the second half of the 70'. This library was not open for use outside the United States. In parallel at the same period codes were developed for calculating sensitivity coefficients for integral parameters and for processing covariance data (e.g. PUFF [4] at ORNL). This would allow performing uncertainty quantification evaluations.
After a first burst of interest, however, the use of covariance data fell in “disgrace” with few exceptions, for instance applications to fast reactor design. In fact, the authors of this paper published in the mid 80' an article [5] where they used an in house very crude covariance matrix that computed an uncertainty for the critical mass of a large sodium cooled fast reactor of ∼1000 pcm.
A long time went by with no major activities until JENDL3 came toward the end of the 90' with a quite comprehensive set of isotopes and reactions with covariance data. In 2005 [6] again the authors of this paper issued, “provocatively”, a in house covariance data set for several isotopes, which included physically based correlations (e.g. by energy range and cross section characteristics: resolved and unresolved resonance ranges, heavy isotopes inelastic cross section threshold, fission cross section thresholds, etc.), and was derived through an educated guess based on nuclear data performance in the analysis of selected clean integral experiments. This in turn induced a feedback from the nuclear data evaluators of the western world that lead to produce more reliable and useful covariance data sets.
The in house covariance data was used for several works of uncertainty quantifications including ADS applications [7,8]. Interestingly enough, the uncertainty for the critical mass of a large sodium cooled fast reactor was ∼1000 pcm. At this point the nuclear data evaluators had received the message and the first, still “low quality” (i.e. a limited effort was put in producing this set, giving priority to a first uncertainty analysis and its requirements), covariance data matrix, BOLNA (collaboration among BNL, ORNL, LANL, NRG, ANL) [9], was produced and extensively used for the seminal work of WPEC SG26 [10]. The uncertainty for the critical mass of a large sodium cooled fast reactor was still ∼1000 pcm.
The rest is history of our days with “high quality” covariance data in all major data libraries (ENDF, JEFF, JENDL); however, the uncertainty for the critical mass of a large sodium cooled fast reactor is still ∼1000 pcm.
This value using the different previously mentioned covariance matrices was the results of different components, but, in general, is dominated by the large contribution of the uncertainty of ^{238}U inelastic reaction. This already indicated the struggle, during all these years, with improving such an important cross section for the design of advanced fast reactors.
3 Background and premises of computed values
In order to calculate the C/E performances corresponding to two sets of CIELO isotopes, the linearity hypothesis was used and the calculated values related to the CIELO isotopes was derived by using sensitivity coefficients. The following formula was used for deriving the new calculated value C': (1) where C is the reference calculated value, S the sensitivity coefficients for the measured quantity and the corresponding σ cross sections. The reference value C was obtained using the ENDF/BVII.0 cross section library [11] and the best available computational tool (Monte Carlo).
In order to derive the Δσ/σ relative variations the infinite dilute cross sections were used. In fact, all the experiments shown later are related to fast spectrum systems and in this case the use of the infinite dilute cross sections is justified as illustrated in [8]. They were computed for the three cross section sets (ENDF/BVII.0, CIELO, and CIELO2) using the latest version of (.84) of NJOY2012 [12].
An initial large set of 158 experiments was used [13]. This set includes not only criticality and reaction rate (spectral indices) measurements but also sample irradiations, reactivity measurements and neutron propagation experiments. Results shown later in this paper are limited only to the most significant ones and will not include the energy breakdown, even though this information is available.
Regarding covariance matrices, the two covariance matrices more extensively used in the analysis are COMMARA2.0 [14] and COMACV1 [15]. Even if the covariance matrix is available for CIELO2, this one has not been used because it includes an adjustment that uses an integral experiment (JEZEBEL). The fact of using only one experiment could completely skew the covariance data. In general, using only one integral experiment will severely limit the amount of information that is necessary to correctly perform the data assimilation and could be at the origin of some compensating effects. In general the issue of including adjusted data in the original evaluations is a controversial issue, still under discussion in international bodies, such as the NEA WPEC Expert Groups. Finally, as indicated above, some JENDL4 data were also included for a preliminary and still partial analysis.
Both covariance matrices originally do not include the P_{1} elastic data. The matrices were, therefore, completed using the corresponding JENDL4 [4] P_{1} covariance data. This is a significant missing in view of the large impact in many types of experiments. The users expect that also the secondary energy distribution for inelastic scattering covariance data would have a significant impact on uncertainty analysis. Moreover, a significant impact can be expected when cross correlations among isotopes will be included (today only one exists related to ^{235}U fission).
As illustration for the differences among the covariance matrices we show in Figures 1–6 those relative to ^{239}Pu fission and ^{238}U inelastic for three covariance matrices: COMMARA2.0, COMACV1, and JENDl4. As it can be seen, the differences are quite remarkable for both diagonal uncertainties and correlations:

In the case of Pu239 fission, the diagonal values are rather small in the three files (slightly higher in COMACV1), but the energy correlations above ∼10 keV are much smaller in COMMARA with respect to the other two data files.
In the case of U238 inelastic, both the diagonal values and the energy correlations are rather different in the three files, with, e.g., “longer” correlations in energy below few MeV in the case of COMACV1The analysis looks at two main consistencies:

the consistency between the individual (EC)/E of the two evaluated files and the associated uncertainties calculated with the two covariance matrices;

the consistency between the differences (impact on the C/E) between the two evaluated files and the corresponding uncertainties.
For this latter case also we look at the differences between the uncertainties obtained with the two covariance matrices of individual isotopes and, among others, at the impact of correlations.
Fig. 1
^{239}Pu fission uncertainty and energy correlation: COMMARA2.0 
Fig. 2
^{239}Pu fission uncertainty and energy correlation: COMACV1. 
Fig. 3
^{239}Pu fission uncertainty and energy correlation: JENDL4. 
Fig. 4
^{238}U inelastic uncertainty and energy correlation: COMMARA2.0. 
Fig. 5
^{238}U inelastic uncertainty and energy correlation: COMACV1. 
Fig. 6
^{238}U inelastic uncertainty and energy correlation: JENDL4. 
4 Analysis of consistency between covariance matrices and C/E
The first set of experiments considered is that of the LANL small criticals. In Table 1 we show, for some integral measured parameters of this set of experiments, the (CE)/E for both CIELO and CIELO2 and the corresponding uncertainties computed with COMMARA 2.0 and COMACV1 covariance matrices. We also report for illustration the (CE)/E for JENDL4, without uncertainties.
As for the CE/E values, the performances of the three files, even if not always comparable, are not radically different. Some significant differences among the files are found, e.g., for the ZPR353 keff and theZPPR15 keff. This last case is of particular interest, since that experiment is rather representative of fast cores presently under study in different laboratories.
When a more detailed investigation is performed, it can be observed that in general there is, e.g., a good agreement among the (CE)/E of CIELO and CIELO2 for this set of experiments; however, we will soon see that this can be misleading, because, in some cases, it is the result of huge compensations. There is a good agreement also on the uncertainties calculated with the two covariance matrices except for the case of the BIGTEN ^{238}U fission spectral index. Also noticeable is the fact that the COMACV1 uncertainty is lower than the calculation/experiment discrepancy for the same integral parameter as well as for the BIGTEN ^{237}Np fission spectral index.
In Table 2 we take a deeper look at the curious case of the JEZEBEL critical mass integral. In Table 2 (No. 1) we show the differences associated to the ^{239}Pu isotope by reaction on the calculated values using CIELO and CIELO2 (again linear perturbation theory is used), while in Table 2 (Nos. 2 and 3) we report the uncertainty by component obtained using the COMMARA 2.0 covariance matrix, where Table 2 (No. 2) has no correlation taken into account whereas in Table 2 (No. 3) the uncertainty computed values correlation. Similarly, Table 2 (No. 4 and 5) show the corresponding values using COMACV1 covariance matrix.
The first striking point is that the small total difference (less than 100 pcm) is the result of large compensation. In effect, there is ∼−800 pcm difference on the inelastic term that is compensated by large (few hundred pcm each) positive differences on the elastic, P_{1}, and fission components.
When summed independently from the signs, the total is 1760 pcm that indicates the high degree of compensation. Concerning the total uncertainty values there is a factor ∼two between the two covariance matrices, and, interestingly enough, the correlation effect goes on the opposite side: lowering the total for COMMARA 2.0 and increasing it in COMACV1. Looking at the component again we have contradicting correlation effects among the two covariance sets for the elastic and inelastic components, while for the fission one the increasing effect of the correlation is a lot more pronounced for the COMACV1 values. We will remark that the correlations contributions to the uncertainty is mostly coming from the correlations in energy, as, in fact, correlations among reactions are scarce in the two matrices (as in other major uncertainty data files), while those among isotopes are, practically, not existent.
In Table 3 there is a summary for selected ZPRs assemblies and different integral parameters (critical masses and spectral indices). As for the following cases, sensitivity analysis results were extensively discussed in previous publications (see e.g. ref. [13]). The discrepancies between calculations and experiments are particularly remarkable for the k_{eff} of the ZPR934 and ZPR354 for both sets CIELO and CIELO2. In the case of ZPR934 the main discrepancies come from ^{56}Fe and ^{235}U, while for ZPR354, which has an iron reflector, the major impact is related to ^{56}Fe. In both cases also the two covariance matrices fail to cover the discrepancies. As for the JEZEBEL case, in Table 4 we present a detailed analysis relative to the ZPR934 k_{eff} case and ^{235}U.
In Table 4 (No. 1), where the differences between CIELO2 and CIELO are shown, the largest impact is associated to the capture term. Regarding uncertainties (Table 4) (Nos. 2 to 5) COMACV1 produces significant greater value for the total uncertainty, but this is still not enough for covering the discrepancy between calculation and experiments. Again, the capture term dominates the uncertainty, while the effect of correlations looks quite similar in the two covariance matrices.
In Table 5 we summarize the results for selected ZPPR assemblies. Besides critical masses and spectral indices, we report also results for the sodium void reactivity measurements, and a central control rod worth. For the sodium void reactivity very large discrepancies are observed for both files and the two covariance matrices fail to compute an adequate uncertainty consistent with the observed discrepancies. It is interesting to note that CIELO2 has a notable discrepancy for the ZPPR15, which is the reference assembly used for validating metal fuel, k_{eff} while CIELO is right on the spot.
Table 6 shows the results for several reaction rates slopes of the ASPIS88 campaign, a neutron propagation experiment in a block of iron. Very large discrepancies are found for the slopes of the reactions very sensitive in the hard part of the spectrum and both covariance matrices seem to be inadequate to explain this poor behavior. Of course, the major impact is related to ^{56}Fe for which we do a more detailed analysis in the case of the S(n, p) reaction rate ratio measured at two different penetrations in the iron block (see Tab. 7).
Regarding differences between CIELO2 and CIELO the largest impact is related to the inelastic term, but also the elastic and P_{1} anisotropy give significant discrepancy. Looking at the total uncertainties they look consistent; however, the correlation effect of the inelastic term has opposite behavior between the two covariance matrices. This is the case also for the elastic term. For illustration we show in Figures 7 and 8 the uncertainty and correlation values for the two covariance matrices of the ^{56}Fe inelastic cross sections. As it can be observed, the differences are quite significant for both diagonal uncertainties and correlations. This is a further indication to the evaluator communities of potential inconsistencies that will have an impact on applications and their credibility.
We now summarize the major finding of this investigation.
In assessing the differences between the CIELO and CIELO2 cross sections we can make the following remarks:

–
As general observation from the point of view of a user, one can say that we are far away from reaching a consensus both on nominal values and on covariance data.

–
The case of the JEZEBEL critical mass is emblematic. The large compensations among the different reactions (elastic, inelastic, P_{1}, and fission) yield the same critical mass. The user is disoriented: where is the truth?

–
Regarding the 5 CIELO isotopes, the major impacts are related to:

•
^{ }^{235}U, ^{238}U capture

•
^{ }^{238}U, ^{56}Fe, ^{239}Pu inelastic

•
^{ 239}Pu nubar

•
Regarding the consistency between the two covariance matrices, we can make the following observations:

–
In many instances the calculated uncertainties would not cover the C/E spread of the experiments. This is true at one sigma level, while at two sigma most, but not all, the spreads would be covered.

–
Of specific interest is the effect of the correlations. In some cases the difference in correlations leads to a different sign in the contribution to the total uncertainty: what can explain this completely different behavior?

–
Regarding specific differences between the two covariance matrices related to the 5 isotopes, the major impacts are associated to:

•
^{16}O: elastic

•
^{56}Fe: elastic, inelastic

•
^{235}U: capture.

•
^{238}U: fission

•
^{239}Pu: inelastic, fission.

•
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: LANL Small Criticals (%).
(CE)/E of CIELO and CIELO2 ^{239}Pu and consistency with uncertainty of COMMARA 2.0 and COMACV1: JEZEBEL.
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: ZPRs. (%).
(CE)/E of CIELO and CIELO2 ^{235}U and consistency with uncertainty of COMMARA 2.0 and COMACV1: ZPR934.
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: ZPPRs. (%).
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: ASPIS FE88. (%).
(CE)/E of CIELO and CIELO2 ^{56}Fe and consistency with uncertainty of COMMARA 2.0 and COMACV1: ASPIS FE88.
Fig. 7
^{56}Fe inelastic uncertainty and energy correlation: COMMARA2.0. 
Fig. 8
^{56}Fe inelastic uncertainty and energy correlation: COMACV1. 
5 Conclusions
Despite significant efforts in generating new high quality neutron cross section data and in producing associated covariance matrices, the state of affairs is not yet fully satisfactory. The user is puzzled by many inconsistencies among evaluated cross sections and corresponding covariance data that in many cases fail to explain discrepancies between measurements and calculations for integral experiments. Moreover, the observed differences on correlation effects between two covariance matrices more specifically used in this paper, are quite noticeable.
Among recommendations to evaluators that can be done from a user point of view, we can list:

Provide the missing data in covariance matrix: P_{1} elastic, secondary energy distribution for inelastic cross sections (multigroup transfer matrix), cross correlations (reactions and isotopes), delayed data (nubar and fission spectra).

Finer energy grid and eigenvalue decomposition of the covariance matrix will be welcome for use in Monte Carlo assimilation [16].

In integral testing look also if the observed C/E discrepancy is consistent with covariance data.
When covariance will be available for CIELO isotopes, more relevant feedback could be provided through data assimilation using PIA strategy [17], which is intended for avoiding compensations, and careful choice among available experiments.
Acknowledgments
This submitted manuscript was authored by a contractor of the U.S. Government under DOE Contract No. DEAC0705ID14517. Accordingly, the U.S. Government retains and the publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a nonexclusive, paidup, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for U.S. Government purposes.
References
 M.B. Chadwick et al., The CIELO collaboration: progress in international evaluations of neutron reactions on Oxygen, Iron, Uranium and Plutonium, EPJ Web Conf. 146, 02001 (2017) [CrossRef] [Google Scholar]
 https://www.oecdnea.org/dbdata/jeffbeta/JEFF33T3/neutrons/ [Google Scholar]
 K. Shibata et al., JENDL4.0: a new library for nuclear science engineering, J. Nucl. Sci. Technol. 48, 1 (2011) [Google Scholar]
 D. Wiarda, M.E. Dunn, PUFFVI: a Code for Processing ENDF Uncertainty Data Into Multigroup Covariance Matrices, ORNL/TM2006/147, Oak Ridge National Laboratory, 2006 [Google Scholar]
 G. Palmiotti, M. Salvatores, Use of integral experiments in the assessment of large liquidmetal fast breeder reactor basic design parameters, Nuclear Sci. Eng. 87, 333 (1984). [Google Scholar]
 G. Palmiotti, M. Salvatores, Proposal for Nuclear Data Covariance Matrix, JEFDOC 1063 Rev.1, January 2005 [Google Scholar]
 G. Aliberti, G. Palmiotti, M. Salvatores, C.G. Stenberg, Transmutation dedicated systems: an assessment of nuclear data uncertainty impact, Nucl. Sci. Eng. 146, 13 (2004). [CrossRef] [Google Scholar]
 G. Aliberti, G. Palmiotti, M. Salvatores, T.K. Kim, T.A. Taiwo, M. Anitescu, I. Kodeli, E. Sartori, J.C. Bosq, J. Tommasi, Nuclear data sensitivity, uncertainty and target accuracy assessment for future nuclear systems, Ann. Nucl. Energy 33, 700 (2006) [Google Scholar]
 D. Rochman et al., Preliminary CrossSection Covariances for WPEC Subgroup 26, Tech. Rep. BNL774072007IR, Brookhaven National Laboratory, 2007 [Google Scholar]
 M. Salvatores, OECD/NEA WPEC subgroup 26 final report: Uncertainty and target accuracy assessment for innovative systems using recent covariance data evaluations. NEA/WPEC26. OECD–NEA No. 6410, Paris, France, 2008 [Google Scholar]
 M.B. Chadwick et al., ENDF/BVII.0: next generation evaluated nuclear data library for nuclear science and technology, Nuclear Data Sheets 107, 2931 (2006) [Google Scholar]
 E. MacFarlane, D.W. Muir, R.M. Boicourt, A.C. Kahler, Tech. Rep. LAUR1227079, LANL, 2015 [Google Scholar]
 G. Palmiotti, M. Salvatores, et al., New approaches to provide feedback from nuclear and covariance data adjustment for effective improvement of evaluated nuclear data files, EPJ Web Conf. 146, 06003 (2017) [CrossRef] [Google Scholar]
 M. Herman et al., COMMARA2.0 Neutron Cross Section Covariance Library, BNL948302011, U.S. Department of Energy, 2011 [Google Scholar]
 P. Archier et al., COMAC: Nuclear Data Covariance Matrices Library for Reactor Applications, in PHYSOR 2014 − The Role of Reactor Physics toward a Sustainable Future, Kyoto, Japan, 2014. [Google Scholar]
 M. Aufiero et al., Continuous Energy Cross Section Adjustment: a New Method to Generalize Nuclear Data Assimilation for a Wider Range of Applications, M&C 2017 − Jeju, Korea, 2017 [Google Scholar]
 G. Palmiotti, M. Salvatores, PIA and REWIND: Two New Methodologies for Cross Section Adjustment, M&C 2017 − Jeju, Korea, 2017 [Google Scholar]
Cite this article as: Giuseppe Palmiotti, Massimo Salvatores, Cross section covariances: a user perspective, EPJ Nuclear Sci. Technol. 4, 40 (2018)
All Tables
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: LANL Small Criticals (%).
(CE)/E of CIELO and CIELO2 ^{239}Pu and consistency with uncertainty of COMMARA 2.0 and COMACV1: JEZEBEL.
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: ZPRs. (%).
(CE)/E of CIELO and CIELO2 ^{235}U and consistency with uncertainty of COMMARA 2.0 and COMACV1: ZPR934.
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: ZPPRs. (%).
(CE)/E of CIELO and CIELO2 isotopes and consistency with uncertainty of COMMARA 2.0 and COMACV1: ASPIS FE88. (%).
(CE)/E of CIELO and CIELO2 ^{56}Fe and consistency with uncertainty of COMMARA 2.0 and COMACV1: ASPIS FE88.
All Figures
Fig. 1
^{239}Pu fission uncertainty and energy correlation: COMMARA2.0 

In the text 
Fig. 2
^{239}Pu fission uncertainty and energy correlation: COMACV1. 

In the text 
Fig. 3
^{239}Pu fission uncertainty and energy correlation: JENDL4. 

In the text 
Fig. 4
^{238}U inelastic uncertainty and energy correlation: COMMARA2.0. 

In the text 
Fig. 5
^{238}U inelastic uncertainty and energy correlation: COMACV1. 

In the text 
Fig. 6
^{238}U inelastic uncertainty and energy correlation: JENDL4. 

In the text 
Fig. 7
^{56}Fe inelastic uncertainty and energy correlation: COMMARA2.0. 

In the text 
Fig. 8
^{56}Fe inelastic uncertainty and energy correlation: COMACV1. 

In the text 
Current usage metrics show cumulative count of Article Views (fulltext article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 4896 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.