P95 Probability or Implausibility

07/07/2018

For the past 18 months or so I have heard a lot about the P95.

What is a P95?
Good question!

Homer Simpson Doh — Waimea Community Dam quote = $120 million

I was told that it was a 95% likelihood that the Waimea dam would be built on or under budget.

Given the current state of press releases coming out of the Tasman Council, I thought it would be timely to delve a little deeper into this magical assurance while we await the next announcement. Incidentally, press releases are the only way anyone gets any information regarding the dam (not as big an exaggeration as you might imagine, councillors are just privileged to receive their copy 15 minutes ahead of the general ratepayer).

Back in January 2016, we read in the Nelson Mail the following:

The dam’s estimated cost rose to $83m last year after costs were revised by the council’s consulting engineers…

The council’s $83m figure is based on a risk factor which assumes there is a 95 per cent probability the dam can be built for that amount of money or less. It’s known as P95.

WCD project manager Nick Patterson said the dam can be built for $73m, including the $6.5m already spent. This was using a P50 risk factor that assumed the dam could be built for that amount of money or less.

There was a lot of confidence around what price the dam could be built for and the level of certainty once we added a few extra million and called it a P95. These confident assertions have continued right up until the press release on July 5 2018.

The Mayor was an early adopter of this confident boast as is recorded in articles such as this Stuff article on September 2017:

Richard Kempthorne said the dam costing was based on a “P95” confidence level, which meant the council could have 95 per cent confidence the dam would be constructed at or below the estimated cost.

“At or below the estimated cost” was the catchphrase that echoed the Council chambers every time anyone questioned the possibility of cost overruns. We have used a P95 to arrive at the budgeted figure.

On the 10th of August 2017 Natasha Berkett is quoted in the Farmers Weekly as saying:

However, there was also a $13 million contingency allowance in the $80m project for further increases.

“We are working to a P95 standard that is where we are 95% certain about the cost of the project.”

Waimea Irrigators Ltd advocates John Palmer and Murray King scoffed at any hint that there would be any budget blowout. The Waimea Water site boasts the following:

Are the cost estimates for the Dam accurate?

A great deal of work has gone into the current cost estimates for the dam construction and project leads are currently undertaking another robust analysis.

The base cost estimate for the Dam construction is $50m. On top of that there is $13.5m included in the project estimate for changes in scope and unexpected cost increases. This raises the level of confidence in the estimate to a very high level – there is only a 5% probability that the costs will exceed the $63.6m budgeted and a 95% probability the project will come in on or below budget. That level of confidence is far higher than the typical estimates you might see in other construction projects (emphasis added)

I hope you are feeling convinced in the level of confidence that we have that the dam will come in on or under budget. That is the completed dam, including construction cost blow-outs. We are 95% certain in our facts.

Just in case you still had a few doubts we put out a Dam Straight fact campaign at the ratepayer’s expense of almost fourteen and half thousand dollars (which I realise is chicken feed compared the ten million, in round figures, lost on the project to date).

To prevent large cost overruns, Council is doing all it can to keep risks to a minimum including:

using a p95 level for construction costs, meaning that there is a 95% probability that the project will come in at or under budget;

using modern engineering and risk management approaches and external professional advice; and

$13.5 million contingency for changes in scope and other unforeseen eventualities.

Can there be any doubt as to the robustness of the budget after we used a P95 level, modern engineering and risk management approaches, external professional advice, and even a whopping $13.5m contingency figure for good measure?

Well, some people thought so. One such person is Paul MacLennan who indicated:

TDC’s own recent admission they have never priced the civil cost of the 87 Hectare Reservoir and sealing of the lake bed to hold the 13.4 million cubic meters of water is appalling and makes the TDC claim $76.9 million at P95 a gross misrepresentation.(emphasis added)

A rather strong assertion. He claimed that “clearing 87 Hectares of steep, sloping, and fractured terrain will cost $30 million plus.” I tried to follow up on his concerns with staff only to receive repeated replies attesting to the robustness of our P95. A P95 that it seems, from reading between press release lines, was so grossly inadequate that it didn’t even cover the quote, never-mind the inevitable construction overruns.

In his submission, during the Waimea Dam consultation process, Dr. Roland Toder questioned the P95 claim.

A so-called ‘P95’, as mentioned by TDC is misleading. In project management, percentiles are expressed by P followed by the percentile. A P95 estimate of $82.5million means that the project has a 95% chance of being completed with this or less amount of money. Normally one uses data one has from similar activities (like executed several dams of similar size before). Ideally, if you have a number of similar activities (many dams of similar size) that would be best, because from this data, you need to find the mean (expected value) and the standard deviation (to determine the range of error) for each activity. THEN you can calculate cost and timing estimates for the entire project.(emphasis added)

People from the Water Information Network Inc also protested the validity of the P95 figure sold by WIL and Council staff.

Tasman District Council (TDC) admit they have never priced the clearing cost of the 87 hectare reservoir.

With steep, sloping and fractured terrain clearance of the area is likely to cost over $30 million.

Similarly, no costing has been made for the sealing of the lake bed to hold 13.4 million cubic metres of water.

Removed trees, bushes, shrubs, detritus, topsoil, and stumps will add up to a metre-high pile.

This will amount to around one million cubic metres to shift from the Dam’s footprint area to a stockpile area where run-off can be managed until final disposal.

Silting up risks and decommissioning costs have not been taken into account.

Suitability of rock fill ‘borrowed’ for the Dam wall has not been accurately assessed. So it may not be suitable.

All of this indicates that TDC’s claim of $76.9 million at p95 (95% ‘accurately’ costed) is a gross misrepresentation. (emphasis added)

I repeat that when I questioned the validity of these claims I was told that we had sufficiently budgeted all aspects of the dam, and early indications on pricing confirmed the validity of the budget.

When I questioned the age of the original estimate given that all our current contracts are experiencing a premium in a high demand marketplace I was again assured that the estimated budget for the Dam was adequate.

At the councillor 15-minute-advance-notice-of-the-press-release meeting, I asked TDC engineering department head Richard Kirby which aspects of the dam quote were currently running over budget he said he didn’t know. When asked what was the range of budget blowout on the figures that had come in to-date he responded that he didn’t know. When I questioned that they had called an extraordinary briefing and are about to send out a press release announcing that there are significant challenges with the contract price meeting budget and yet you are telling me that you have no idea what the overrun gap is? He confirmed that was correct.

The head of Engineering doesn’t know the figures.

The Mayor doesn’t want to know the figures.

I want to know whose head is going to roll for misleading the Council over the 95% certainty of the price for building the Waimea Community Dam. Information that resulted in the Council spending the best part of ten million dollars of ratepayer’s money progressing the project to this point. Whether deliberately misleading councillors or doing so through incompetence someone has to be held accountable.

Grandstand Dam Long Term Plan

04/07/2018

The 28^th of June full Council meeting should have been about Council adopting the Long Term Plan. However, it was overshadowed by a couple of other agenda items. View the agenda here.

A surprising turn of events saw those on council who wanted to raze the Takaka grandstand to the ground do an about face. One wonders what persuaded the change of heart when a petition of over a thousand signatures, a demonstration outside council, a multitude speaking in the public forum, and numerous emails wasn’t enough last time we voted. Maybe the TV cameras filming people in chains was a little harder to ignore?

While the Grandstand execution order has been lifted there is a long way to go before the old lady of the Bay can rest in peace. There are a lot of issues to come out of the woodwork to satisfy the Council compliance team. The success of the in-situ restoration bid will largely depend on the ability of all the stakeholder groups to put the past and egos behind them and find enough common ground to work together for the good of the grandstand.

Other agenda items were no less surprising. Councillors were recommended by staff to sign a waiver of councillor privilege and take on personal liability if we wanted to view the Term Sheets in relation to the dam. They stopped short of insisting on a strip search but required that we book a time to view the documents. We would then be allowed access to them in a secure controlled room where we were not to take any copies, electronic or otherwise, of the individually identifiable documents. If our document was proven to be leaked then we would be personally liable for the breach.

dodgy salesman picture — Sign Here … trust me, its the best deal for us.

What staff didn’t explain was how our personally liable documents would be securely stored, who would have access to them, what supervision would be in place for anyone that had access to them. They also admitted that our staff that had access to the current documents did not have similar requirements, nor did the members of WIL that had access to the documents – even the Term Sheets between CIIL and TDC (rather bizarre I would have thought).

This overreaction in it self leaves one wondering what they are hiding. An earlier version of the Term Sheets was circulated with councillors without such extraordinary security precautions, so what has changed? According to the staff report councillors have “previously been made aware of the key conditions in the Term Sheets” so this is rather extreme to be protecting the non-key conditions surely?

Of course, there were those around the table that did not want to see the Term Sheets. Although, at least one of those councillors wanted “all the information” before making a decision on the Grandstand. A cynical person might say that this selective interest in getting all the information on topics seems to be related to the outcome desired by the individual councillors rather than a reasoned and fair-minded approach.

Another example of this would be a referendum on the Waimea Dam, twice denied by Mayoral casting vote. Yet on the topic of the Public Water Supply Bylaw Review the Mayor is recorded in the latest Engineering Committee minutes as saying “… that without the Waimea Dam there will be significant changes to the District’s water supply during dry summer months. He said that the proposed bylaw clarifies those changes and that it was important to consult with the community.”

It is very important that we have all the information, and the community have a voice, when it comes to water restrictions applying particularly in a no dam scenario. I can hear that cynical ratepayer making a snide comment about this being an attempt to scare them into paying for the dam. However, the current Tasman Resource Management Plan provisions in the event of a no dam scenario will require drastic water cuts should we face dry weather.

It is a pity the community doesn’t have a voice when it comes to expressing an opinion over water restrictions verses unaffordable rate increases when the dam runs over budget don’t you think? The Mayor doesn’t think so.

WIL Chairman Murray King Accuses TDC of High Appetite for Risk

20/04/2018

Waimea Irrigators reveal some interesting information in their submissions to the Tasman District Council’s Long Term Plan. WIL chairman Murray King states that:

The allocation of cost Overruns beyond the first $3 million was reached between the parties on the basis of the following:

The Council’s higher appetite for risk than WIL’s around the construction contract – with WIL’s preference being a fixed price contract, whilst TDC’s preference was to proceed with an Early Contractor Involvement contract.

The inability of WIL’s shareholders to absorb higher water charges in the event project over-runs exceed $3m.

TDC receiving 100% of project under-runs between $3 and $6m – i.e. considerable ‘upside’ benefit to TDC in the event that the project comes in under budget.

Firstly, I would like to offer WIL an exchange of Council’s 100% of cost under-runs between $3 and $6m for WIL’s capped investment. I am sure the majority of ratepayers would be happy to switch out their “considerable upside benefit” on a project that is already running over budget on the work completed to-date for the risk of the highly-unlikely-event that the project will overrun more than $3 million.

Secondly, WIL Chairman Murray King has accused the Council of having a high appetite for risk which I find interesting given that our consultation document clearly states that our number one financial objective is to operate in a financially prudent manner. As Murray King was sitting beside Natasha Berkett during her WIL submission I thought I would ask him:

Does Murray King believe that council was being dishonest stating that their number one financial strategy is to act in a financially prudent manner?

He refused to respond.

WIL advocate John Palmer questioned the integrity of TDC. His submission states:

The suggestion that possible target rates can be used to negate the effects of legal agreements will cause any TDC counterparty to question whether it can have an agreement it can rely on. That [the ability to target rate irrigators in the event of dam over-spend] must be removed from the plan and the council thinking.

I find this a little ironic given that (according to staff) we had an understanding with WIL that all costs were going to be shared 50/50 and yet for the past 18 months council has been told we need to pick up more and more costs that WIL did not feel it should contribute towards.

I took the opportunity to ask John Palmer the same question that I asked of Murray King, did he feel that council was being dishonest stating that their number one financial strategy was to act in a financially prudent manner – given the WIL Chair’s accusation? Mr. Palmer responded by launching into a further tirade on Council’s dishonesty in trying to recoup costs beyond those stated in the TDC/WIL Terms Sheet without actually addressing the question put to him.

It is interesting to note that John Palmer is privy to more TDC information, and receiving information earlier, than are we as councillors. For me, this high-level access and influence that Mr. Palmer has begs the question of who is actually running council?

However, it would appear that Mr. Palmer is not happy with all Council decisions.

Bruno Simpson, another WIL advocate, was also unhappy with Council’s decision to rate irrigators in the event of a dam over-spend. His submission states:

It is our view that this is a breach of the agreement between Council and WIL in the term sheets between the parties, which were included (in summary) in the WIL PDS. Many individuals and organisations have supported the purchase of these shares, based on a reasonable limit to the exposure of cost overruns, and inclusion represents a material change in the risks associated. It contradicts the earlier consultation carried out by Council. Further, it suggests that all the benefits of completing the project fall to the irrigators rather than being shared by all water users, including urban and commercial, who benefit from the security of supply and increased capacity.

Ironically, Mr. Simpson is upset that this target rating system is unfair because it suggests that all the costs of dam over-runs will be unfairly attributed to irrigators rather than fairly shared around. Incidentally, it does not propose that all over-run expenses will be target-rated to irrigators. It does reserve the right to attribute some of the over-run expenses to irrigators which at the moment are unfairly being attributed to rate-payers only; a point that Mr. Simpson appears to have no problem within his world-view of what is fair.

Mr Simpson also makes mention that by including this provision in our LTP we are in contradiction with the WIL PDS. Given the amount of “errors” in the WIL PDS I think it laughable he should be accusing TDC of breaching the agreement between TDC and WIL. As councillors we tried to address the over-statements and out-right misinformation in the WIL PDS but we were told that it is a WIL document and that there was nothing that we could do.

The fact is that target rating always was, and remains, an option to recover costs. If WIL omitted to mention that to their potential share-holders it is hardly the fault of the Council. By including this information in the TDC LTP it is our way of addressing the omission of WIL to fully disclose the full position to their shareholders. I am not sure Council is the one that should be accused of dishonesty in this instance.

Speaking of WIL shareholders, John Palmer protested that WIL wasn’t just a few big shareholders, he stated that there were also many small shareholders who would find this target rating system unaffordable. Given that councillors have been asking for a breakdown of who exactly makes up WIL for the past 18 months, and been denied access every time, it will be interesting when the Companies Website is updated with a list of shareholders to see how accurate honest John’s statements are.

Bruno Simpson stated that if the change to the LTP in reference to the provision for targeted rating was not made “we believe there could be significant unintended consequences.” When I questioned him as to the meaning of that statement he said it referred to the fact that some high capital value properties with low water subscription would be unfairly rated in this system. He advocated that any target rate should be tied to water subscription rather than capital value.

I believe that the point of capital value rating is that those who can afford to pay more are paying more. It is, however, a blunt instrument and like all generalizations not without exception. In terms of legality, we tax property not individuals, and I don’t believe we can target rate based on the volume of water consumption. We can charge for water use, but that is different to our ability to rate. I do stand to be corrected on that assumption though.

The fact that Waimea Irrigators are so concerned about this target rating of dam over-spend beyond the $3million over-spend that they are happy sharing is confusing. They, like council staff, have sat before us assuring us that this dam has been costed so well that we are almost 100% certain (95%) that this dam will come in on or under budget. Therefore, in their world there is almost no risk of this target rating ever being acted upon. Why so upset then?

I am left wondering about all this monkey business.

Perhaps Murray King is correct that the Council does have a high appetite for risk (that is not being disclosed) and, therefore, Council is lying that our number one financial strategy is to act in a financially prudent manner?

DEAN MCNAMARA – YOUR VOICE IN COUNCIL

P95 Probability or Implausibility

Are the cost estimates for the Dam accurate?

Grandstand Dam Long Term Plan

WIL Chairman Murray King Accuses TDC of High Appetite for Risk

Councillor McNamara: As Reported In The News

Archives

Share the joy

Why Vote McNamara?

Are the cost estimates for the Dam accurate?

Councillor McNamara: As Reported In The News

Archives

Share the joy

Why Vote McNamara?

Tags