Climategate Code Analysis Part 2

Filed Under: Environmental News on December 8, 2009

Aaron’s note: This assessment, along with his first (linked below), is one of the most fair go-overs of the data and computer programming in the Climategate files I’ve seen so far.  Read it, digest it, consider it, and think for yourself.

by Robert Greiner, Cube Antics

There are three common issues that have been raised in my previous post that I would like to officially address concerning the CRU’s source code.

If you only get one thing from this post, please get this. I am only making a statement about the research methods of the CRU and trying to show proof that they had the means and intent to falsify data. And, until the CRU’s research results can be verified by a 3rd party, they cannot be trusted.

Here are the four most frequent concerns dealing with the CRU’s source code:

  1. The source code that actually printed the graph was commented out and, therefore, is not valid proof.
  2. No proof exists that shows this code was used in publishing results.
  3. Interpolation is a normal part of dealing with large data sets, this is no different.
  4. You need the raw climate data to prove that foul play occurred.

If anyone can think of something I missed, please let me know.

THE SOURCE CODE THAT ACTUALLY PRINTED THE GRAPH WAS COMMENTED OUT AND, THEREFORE, IS NOT VALID PROOF.

Had I done a better job with my source analysis, I would have found a later revision of the briffa_sep98_d.pro source file (linked to in my previous post) contained in a different working tree which shows the fudge-factor array playing a direct result in the (uncommented) plotting of the data.

Snippit from: harris-tree/briffa_sep98_e.pro (see the end of the post for the full source listing)

  1. ;
  2. ; APPLY ARTIFICIAL CORRECTION
  3. ;
  4. yearlyadj=interpol(valadj,yrloc,x)
  5. densall=densall+yearlyadj
  6. ;
  7. ; Now plot them
  8. ;
  9. filter_cru,20,tsin=densall,tslow=tslow,/nan
  10. cpl_barts,x,densall,title=‘Age-banded MXD from all sites’,$
  11. xrange=[1399.5,1994.5],xtitle=‘Year’,/xstyle,$
  12. zeroline=tslow,yrange=[-7,3]
  13. oplot,x,tslow,thick=3
  14. oplot,!x.crange,[0.,0.],linestyle=1
  15. ;

Now, we can finally put this concern to rest.

INTERPOLATION IS A NORMAL PART OF DEALING WITH LARGE DATA SETS, THIS IS NO DIFFERENT.

This is partially true, the issue doesn’t lie in the fact that the CRU researchers used interpolation. The issue is the weight of the valadj array with respect to the raw data. valadj simply introduces too large of an influence to the original data to do anything productive with it.

Here is the graph I plotted of the valadj array. When we’re talking about trying to interpret temperature data that grows on the scale of one-tenths of a degree over a period of time, “fudging” a value by 2.5 is going to have a significant impact on the data set.

NO PROOF EXISTS THAT SHOWS THIS CODE WAS USED IN PUBLISHING RESULTS.

Correct! That’s why I am (and always have) taken the following stand: Enough proof exists that the CRU had both the means and intent to intentionally falsify data. This means that all of their research results cannot be trusted until they are verified. Period.

The fact that the “fudge-factor” source code exists in the first place is reason enough for alarm. Hopefully, they didn’t use fudged results in the CRU research results, but the truth is, we just don’t know.

YOU NEED THE RAW CLIMATE DATA TO PROVE THAT FOUL PLAY OCCURRED.

This is assuming the raw data are valid, which I maintain that it probably is. Several people question the validity of the climate data gathering methods used by the different climate research institutions, but I am not enough of a climate expert to have an opinion one way or the other. Furthermore, It simply doesn’t matter if the raw climate data are correct or not to demonstrate the extreme bias the valadj array forces on the raw data.

So, the raw data could actually be temperature data or corporate sales figures, the result is the same; a severe manipulation of data.

FULL SOURCE LISTING

As promised, here is the entire source listing for: harris-tree/briffa_sep98_e.pro

  1. ;
  2. ; PLOTS ‘ALL’ REGION MXD timeseries from age banded and from hugershoff
  3. ; standardised datasets.
  4. ; Reads Harry’s regional timeseries and outputs the 1600-1992 portion
  5. ; with missing values set appropriately.  Uses mxd, and just the
  6. ; “all band” timeseries
  7. ;****** APPLIES A VERY ARTIFICIAL CORRECTION FOR DECLINE*********
  8. ;
  9. yrloc=[1400,findgen(19)*5.+1904]
  10. valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,$
  11. 2.6,2.6,2.6]*0.75 ; fudge factor
  12. if n_elements(yrloc) ne n_elements(valadj) then message,‘Oooops!’
  13. ;
  14. loadct,39
  15. def_1color,20,color=‘red’
  16. plot,[0,1]
  17. multi_plot,nrow=4,layout=‘large’
  18. if !d.name eq ‘X’ then begin
  19. window, ysize=800
  20. !p.font=-1
  21. endif else begin
  22. !p.font=0
  23. device,/helvetica,/bold,font_size=18
  24. endelse
  25. ;
  26. ; Get regional tree lists and rbar
  27. ;
  28. restore,filename=‘reglists.idlsave’
  29. harryfn=['nwcan','wnam','cecan','nweur','sweur','nsib','csib','tib',$
  30. 'esib','allsites']
  31. ;
  32. rawdat=fltarr(4,2000)
  33. for i = nreg-1 , nreg-1 do begin
  34. fn=‘mxd.’+harryfn(i)+‘.pa.mean.dat’
  35. print,fn
  36. openr,1,fn
  37. readf,1,rawdat
  38. close,1
  39. ;
  40. densadj=reform(rawdat(2:3,*))
  41. ml=where(densadj eq -99.999,nmiss)
  42. densadj(ml)=!values.f_nan
  43. ;
  44. x=reform(rawdat(0,*))
  45. kl=where((x ge 1400) and (x le 1992))
  46. x=x(kl)
  47. densall=densadj(1,kl)     ; all bands
  48. densadj=densadj(0,kl)     ; 2-6 bands
  49. ;
  50. ; Now normalise w.r.t. 1881-1960
  51. ;
  52. mknormal,densadj,x,refperiod=[1881,1960],refmean=refmean,refsd=refsd
  53. mknormal,densall,x,refperiod=[1881,1960],refmean=refmean,refsd=refsd
  54. ;
  55. ; APPLY ARTIFICIAL CORRECTION
  56. ;
  57. yearlyadj=interpol(valadj,yrloc,x)
  58. densall=densall+yearlyadj
  59. ;
  60. ; Now plot them
  61. ;
  62. filter_cru,20,tsin=densall,tslow=tslow,/nan
  63. cpl_barts,x,densall,title=‘Age-banded MXD from all sites’,$
  64. xrange=[1399.5,1994.5],xtitle=‘Year’,/xstyle,$
  65. zeroline=tslow,yrange=[-7,3]
  66. oplot,x,tslow,thick=3
  67. oplot,!x.crange,[0.,0.],linestyle=1
  68. ;
  69. endfor
  70. ;
  71. ; Restore the Hugershoff NHD1 (see Nature paper 2)
  72. ;
  73. xband=x
  74. restore,filename=‘../tree5/densadj_MEAN.idlsave’
  75. ; gets: x,densadj,n,neff
  76. ;
  77. ; Extract the post 1600 part
  78. ;
  79. kl=where(x ge 1400)
  80. x=x(kl)
  81. densadj=densadj(kl)
  82. ;
  83. ; APPLY ARTIFICIAL CORRECTION
  84. ;
  85. yearlyadj=interpol(valadj,yrloc,x)
  86. densadj=densadj+yearlyadj
  87. ;
  88. ; Now plot it too
  89. ;
  90. filter_cru,20,tsin=densadj,tslow=tshug,/nan
  91. cpl_barts,x,densadj,title=‘Hugershoff-standardised MXD from all sites’,$
  92. xrange=[1399.5,1994.5],xtitle=‘Year’,/xstyle,$
  93. zeroline=tshug,yrange=[-7,3],bar_color=20
  94. oplot,x,tshug,thick=3,color=20
  95. oplot,!x.crange,[0.,0.],linestyle=1
  96. ;
  97. ; Now overplot their bidecadal components
  98. ;
  99. plot,xband,tslow,$
  100. xrange=[1399.5,1994.5],xtitle=‘Year’,/xstyle,$
  101. yrange=[-6,2],thick=3,title=‘Low-pass (20-yr) filtered comparison’
  102. oplot,x,tshug,thick=3,color=20
  103. oplot,!x.crange,[0.,0.],linestyle=1
  104. ;
  105. ; Now overplot their 50-yr components
  106. ;
  107. filter_cru,50,tsin=densadj,tslow=tshug,/nan
  108. filter_cru,50,tsin=densall,tslow=tslow,/nan
  109. plot,xband,tslow,$
  110. xrange=[1399.5,1994.5],xtitle=‘Year’,/xstyle,$
  111. yrange=[-6,2],thick=3,title=‘Low-pass (50-yr) filtered comparison’
  112. oplot,x,tshug,thick=3,color=20
  113. oplot,!x.crange,[0.,0.],linestyle=1
  114. ;
  115. ; Now compute the full, high and low pass correlations between the two
  116. ; series
  117. ;
  118. perst=1400.
  119. peren=1992.
  120. ;
  121. openw,1,‘corr_age2hug.out’
  122. thalf=[10.,30.,50.,100.]
  123. ntry=n_elements(thalf)
  124. printf,1,‘Correlations between timeseries’
  125. printf,1,‘Age-banded vs. Hugershoff-standardised’
  126. printf,1,‘     Region    Full   <10   >10   >30   >50  >100′
  127. ;
  128. kla=where((xband ge perst) and (xband le peren))
  129. klh=where((x ge perst) and (x le peren))
  130. ts1=densadj(klh)
  131. ts2=densall(kla)
  132. ;
  133. r1=correlate(ts1,ts2)
  134. rall=fltarr(ntry)
  135. for i = 0 , ntry-1 do begin
  136. filter_cru,thalf(i),tsin=ts1,tslow=tslow1,tshigh=tshi1,/nan
  137. filter_cru,thalf(i),tsin=ts2,tslow=tslow2,tshigh=tshi2,/nan
  138. if i eq 0 then r2=correlate(tshi1,tshi2)
  139. rall(i)=correlate(tslow1,tslow2)
  140. endfor
  141. ;
  142. printf,1,‘ALL SITES’,r1,r2,rall,$
  143. format=‘(A11,2X,6F6.2)’
  144. ;
  145. printf,1,‘ ’
  146. printf,1,‘Correlations carried out over the period ’,perst,peren
  147. ;
  148. close,1
  149. ;
  150. end

WHAT DO YOU THINK?

Join the conversation! Don’t just accept what one person says, gather the evidence and make an informed decision.

Share

Related Posts (automated):

Leave a Reply

You must be logged in to post a comment.

Subscribe without commenting