Update: the following does not explain the Caramilk secret of MBH99 confidence intervals, which remains unexplained and mysterious. End Update.
OK, Mann starts with a sigma obtained from the standard errors in the calibration period from his hugely overfitted model. He uses this in MBH98. In MBH99, recognizing the autocorrelation in the residuals, he adjusts the confidence intervals through a mysterious procedure. The adjustments as shown this morning in the figure below are done in bulk.
Top: black- MBH99 sigma; red – MBH98 sigma; bottom ratio of MBH99 sigma to MBH98 sigma (using the ignore2 to extend to 1000-1399).
The adjustment that we’re looking for is about 1.6 derived somehow. Now look at the following figure from MBH99 posted up previously here:
Original Caption. Figure 2. Spectrum of NH series calibration residuals from 1902-1980 for post-AD 1820 (solid) and AD 1000 (dotted) reconstructions (scaled by their mean white noise levels). Median and 90%,95%,and 99% significance levels (dashed lines) are shown.
This is derived from a spectrum but it is, as noted in the caption, "scaled by mean white noise levels". If you squint at the y-axis, you can perhaps persuade yourself that the value at the y-axis is about 1.6. which is the value in the "adjustment". So maybe what they do is use this y-axis value as an "adjustment" to the standard deviation and thus to the confidence interval. [Update: You cannot reasonably persuade yourself that it is 1.4. I was trying too hard and this possibility – which doesn’t make any sense anyway – is ruled out.]
Jean S sent me a note saying that he thinks he’s figured it out but was too busy to write it down till the end of the week. Jean S:
I kind of figured out the procedure from Mann and Lees. I don’t know what to say anymore, I’m kind of sad this type of papers do get published… Mann&Lees is sad.
[Update: I guess we’ll have to wait for Jean S. ]
Jean S has suggested a look at Stoica and Moses, Intro to Spectral Analysis, p. 37, section 2.4, "Properties of the Periodogram Method", so I’ll check that out.
I wonder what an econometrician working with spectral methods would say about this. – someone like Granger or Peter Robinson. For that matter, Bloomfield and Nychka of the NAS panel or both specialists in frequency domain. Now however odd Mann’s result is, bear in mind that Esper, Briffa, Jones, D’Arrigo are all even worse. At least Mann has indirectly considered the possibility of autocorrelated residuals and tried to allow for them – even if his method was weird. The other folks just ignore the problem and use the same approach as MBH98 for estimating residuals. Fit a model in a calibration period and then use the standard errors to calculate confidence intervals with no allowance for autocorrelation. But hey, they’re the Hockey Team.
[Update: The relevant section of Stoica says that the estimates of the Fourier coefficients through the periodogram are inconsistent and do not converge with N, but behave as a random variable. This doesn’t sound promising as a method of estimating an adjustment, that’s for sure. Now Mann estimates his spectrum using Thomson’s multitaper method – Stoica did not discuss this methodology; the purpose of the multitaper method is to reduce the variance of the estimate, so probably it’s less bad than the periodogram, but still the whole procedure, whatever it is, doesn’t sound promising. Having said that, doing nothing is not an alternative either. ]
