News
Benchmarking – From Gold Medals to Major Projects
We often use benchmarking to provide a reference point of success, a way to gauge our performance versus that of our competitors.
The recent Tokyo Olympics have provided ample opportunities for media and various Olympic committees and fans to compare medal performance per insert parameter here, usually to show the best outcome for their efforts.
Australia has a ‘weighted medal’ for every US$12.7b of GDP[1] compared to the average of a ‘weighted medal’ of every US$34.7b.
By comparison, the USA, the most successful country in the gold medal tally, has a ‘weighted medal’ per US$71.9b of GDP. By this metric, Australia has over-performed compared to the average country using a GDP metric. Aussie, Aussie, Aussie!
However, our friends across the ditch will apply a different benchmark, comparing medals earned to population, to show how New Zealand finished fifth in the medal table, as that benchmark is preferable to their narrative. Australia finished 14th by this benchmark and the USA 59th.
When conducting high-level estimating in construction and infrastructure, benchmarking is often called upon to fill in the gaps in a cost estimate.
For high-level overviews, we call on metrics such as $/km or $/m2 and provide benchmark rates to develop a rough order of magnitude estimate with appropriate high-levels of risk contingency applied. For a deeper dive, benchmarking of IJCs (click here for more information), traffic management, environmental and landscaping costs are also common.
Where estimates are perceived as being above or below benchmark rates, an explanation is usually required, outlining the various factors that cause the cost to deviate from the average. For example, suppose the price of a km of highway is vastly above the average cost. In that case, there could be many factors from location to material supply, to local geotechnical conditions that impacted the price.
Critical to this process is an understanding of the universe of data from which the benchmark is created to form a view.
The Bureau of Infrastructure and Transport Research Economics provides a report Road construction cost and infrastructure procurement benchmarking: 2017 update[2]. While we applaud the intent to share data (more to come on this in our upcoming construction data series), unfortunately, the dataset is not extensive enough to be relied upon. This means we are already facing challenges as we don’t have enough benchmarking data to take a high level of confidence in a benchmark based purely on the numbers.
Therefore a benchmark that uses limited data for comparison runs the risk of being significantly different to the average of the small data set. A parallel would be flipping a coin twice and recording the heads vs tails outcome. With two flips, the coin could:
- Come up heads twice
- Come up tails twice
- Come up heads once and tails once.
In our logical heads, we know that outcome three sounds like it should be correct because there are two options and a 50/50 chance that the coin lands on a head or tail. But we may run the test and receive two tails, recording that data as the base benchmark. However, two flips of the coin aren’t enough data to make a fully formed view. Flipping the coin 100 times would be better, a million times even more so. But even then, the data could be skewed. Where all the coin flips in the same location, at the same time, under the same conditions? Was the coin flipped with the head facing up? Was the coin flipper able to replicate their technique across each flip?
Therefore, the context of the data used for the benchmark is critical because more data provides the opportunity for greater accuracy. Greater accuracy avoids results that sit way outside the average, which is great as people feel comfortable when outcomes perform as the herd expects, and human nature is to train the output back to the mean -> Central Limit Theorem (CLT),
Applying Benchmarking In Construction
Benchmarking is often part of the check process for cost estimating, but we must ensure that we understand the context of the benchmark data.
Some projects, by their nature, sit at the ends of the spectrum. For example, a highway project in a major urban environment requiring tunnelling, complex PUP management, numerous traffic switches, resumptions, noise restrictions and more will benchmark on the high side compared to a highway on a greenfield site with great geotechnical data, limited PUP and no flooding risk.
At Civil Project Partners, we price first principles and can drill into the reasons behind why a project may not conform to the average. We peel back the layers to get the level of detail down to the most granular level possible such as the number of labour and plant hours. This approach enables us to outline clear and rational data-driven reasoning as to why a project’s cost sits where it does against expectations and averages.
Our starting point is to determine the appropriate benchmarks via a thorough understanding of the project scope and risks. We compare them against the available data and factor in a critical element; human experience. Our team is an invaluable resource and will always be part of a robust approach to benchmarking because, through a combination of data and experience, we can provide the most accurate estimate possible and articulate why we have reached our outcome.
For our clients, we provide confidence. They understand the estimate, see how it compares to the average, and crucially, understand why it may sit above or below the mean.
Benchmarking Brisbane
Returning to the Olympics, there will always be countries that sit at the top of a benchmark on GDP, San Marino and down at the bottom, Saudi Arabia, and in the middle, France.
Looking forward to Brisbane’s Olympics in 2032, we have to be careful about the narrative used as the games approach and costs of construction projects are scrutinised by media and politicians. We will no doubt see the price of the games compared to previous games in London, Beijing, Athens or Los Angeles, and the narrative will swing wildly around and will no doubt include headlines like the ‘Most Expensive Games Ever!’
Therefore, when trying to control the narrative, it will be critical for all levels of government, organising and delivery committees and authorities, to get their benchmarks right and to clearly explain variations to the average in a way that people understand and can support.
As our city goes for gold and delivers the greatest games ever, we need to make sure that we know what we are benchmarked against in and outside of the sporting arenas.
[1] https://www.medalspercapita.com/#medals-by-gdp:2020
[2] https://www.bitre.gov.au/publications/2018/rr_148