Project Description

The Berkeley Earth Surface Temperature Study has been organized under the auspices of the non‐profit Novim Group (http://www.novim.org). The project has the following goals:

1) To merge existing surface station temperature data sets into a new comprehensive raw data set with a common format that could be used for weather and climate research
2) To review existing temperature processing algorithms for averaging, homogenization, and error analysis to understand both their advantages and their limitations
3) To develop new approaches and alternative statistical methods that may be able to effectively remove some of the limitations present in existing algorithms
4) To create and publish a new global surface temperature record and associated uncertainty analysis
5) To provide an open platform for further analysis by publishing our complete data and software code as well as tools to aid both professional and amateur exploration of the data

Progress

Preliminary Data Set

The Berkeley Earth Surface Temperature Study has created a preliminary merged data set by combining 1.6 billion temperature reports from 10 preexisting data archives (4 daily and 6 monthly). Whenever possible, we have used raw data rather than previously homogenized or edited data. After eliminating duplicate records, the current archive contains 39,390 unique stations. This is more than five times the 7,280 stations found in the GHCN (Global Historical Climatology Network) Monthly data set (GHCN‐M) that has served as the focus of many climate studies. The GHCN‐M is limited by strong requirements for record length, completeness, and the need for nearly complete reference intervals used to define baselines.

We believe it is possible to design new algorithms that can greatly reduce all of these requirements and as such we have intentionally created a more expansive data set.

Enlarged.

See much more detail on the plan here.

Dr Singer was hopeful about the project as he wrote here. So is E.M. (Chiefio) Smith:

Their method has a pretty good chance of working.

1) Fixes the “baseline” problem.  Right now there is a ‘cherry picked’ baseline in the cold 1/2 phase PDO.  The move to a ‘present baseline’ in some suggested methods is usually immediately followed by comparing it to the just past cold period and saying “Look, still warming!”.  By having each data series UP FRONT converted to an anomaly based on the data FROM THAT SITE ONLY and prior to homogenizing et. al. is a great improvement and eleminates many of the ‘games’.  It’s very similar to what I did with the dT/dt method, but adds the use of segments of data for each change of thermometer regime.  Nice.

2) Adds in more of the thermometers.  This reduces the impact of airports ASOS and helps to eliminate the ‘cherry pick’ of warming stations in the present GHCN airport bias.  Issues:  A hugh number of thermometers are still at airports.  Adding them in does not fix the airport problem.  The metadata are still limited to whatever the provider has, so “pristing rural” may still mean “runway tarmac"… Needs validation / checking.

3) Least Squares method sounds like a good way to bias the data against bad stations, but I worry about things that in an automated way select to choose one thing over another.  So, for example, might not those same “Major Airport ASOS” stations that are now deemed “best” for USHCN “QA” by NCDC continue to be seen as “best” and so given higher weighting?  Impact depends on implementation.  Needs watching.  Could be good or bad depending.

4) Avoidence of Gridding:  Yay!  One of the things I did.  It also, then, avoids the “make up data for empty grids” problem....  A key FUBAR in NCDC / GISS IMHO.

5) Overall, I’d give it an A- out the gate.  Could end up a B or an A+ depending on execution.  Beware of The Team trying to influence them doing it “the approved way"…

ICECAP NOTE: I am pleased this effort is ongoing. I expect they will find a reduced trend of warming because of being more inclusive and for all hte reasons EM Smith notes. The contamination by urban heat island and land use changes are likely though to still be present and unaccounted for.