Developing an assessment profile

Prior to applying the IAF, Waka Kotahi will undertake an assessment of the business case to ensure it is fit for purpose and includes the information required to include the activity in the NLTP or for support or funding approval, taking into account the development stage of the proposal. Provided the business case passes this assessment, it will progress to IAF assessment.

Process:

1. Before a funding proposal with business case is submitted for inclusion in an RLTP and for funding approval, the approved organisation or Waka Kotahi (state highways) is required to provide an assessment of its proposed activity under the IAF.
2. Waka Kotahi will first assess the business case to provide assurance that a robust case has been developed under BCA principles. 
3. Provided the business case is robust, Waka Kotahi will then assess it under the IAF.

Waka Kotahi needs to be confident a proposal satisfies a number of criteria before including it in the NLTP for funding approval.

Inclusion of an activity in the NLTP requires that:

• a fit-for-purpose business case has been developed to the level appropriate to the stage of development of the activity
• assessment of the business case has been completed by Waka Kotahi and rated as Pass or Rework
• sufficient evidence has been used to justify the activity, depending on its stage of development
• the activity is included in a regional land transport plan (RLTP), or is a proposed Road Policing Programme, or is part of a Waka Kotahi nationally delivered programme
• the problem/issue/opportunity is clearly identified in a strategic business case
• the requirements of the Land Transport Management Act 2003 have been met, that the activity:
  • is assessed by the organisation that proposes it
  • identifies the objective or policy to which it will contribute
  • provides an estimate of the total cost and the cost for each year
  • shows its expected duration (automatically generated from the cash-flow plan in TIO),
  • shows any proposed sources of funding other than the National Land Transport Fund (NLTF)
  • provides any other relevant information.

 Seeking approval for funding from the NLTF requires all the above, plus:

• the proposed activity or programme is eligible for NLTF funding (consistent with the GPS activity class definition and the Waka Kotahi work category definition)
• all required information is provided in TIO and has been checked for completeness
• the proposal is ready to proceed. 

Activity class groupings for developing assessment profiles are:

• investment management activities, including transport planning
• Road Policing Programme
• promotion of road safety and demand management programmes
• road maintenance programmes – programmes of road and other network maintenance and renewal activities
• existing public transport programmes
• low cost, low risk road and public transport improvements up to $1 million total implementation cost
• walking and cycling improvement activities
• public transport improvement activities, including rapid transit and transitional rail – public transport infrastructure and service improvements
• road improvement activities – local road and state highway improvements over $1,000,000 implementation cost, ie projects other than low cost, low risk improvements.

A rating greater than low for results alignment or a rating above low for cost–benefit appraisal does not guarantee funding. The combined ratings for results alignment and cost–benefit appraisal are required to get an overall ranking.

While a cost–benefit appraisal rating of low will be taken into account in the ranking, Waka Kotahi also looks at other factors in the proposal, such as relevance to government strategy through results alignment. It may also, as an exception, consider a proposal with a benefit-cost ratio of less than 1, where evidence demonstrates the proposal has wider value when assessed against GPS results.

Waka Kotahi has introduced a streamlined approach to investment decision making and assessing business cases for low cost, low risk activities  by extending the model that was historically used for ‘minor improvements’ to activities with an implementation cost of up to $1 million.

The assessment considers how the problem/issue/opportunity is significant:

• in relation to the desired GPS result(s)
• in relation to the scale of the gap to the appropriate customer level of service or performance measure
• as part of an end-to-end journey
• from a national perspective (given local, regional, national perspectives)
• from a community perspective in regard to access to social and economic opportunities
• in relation to GPS timeframes ie a significant issue/opportunity within 3/10/10+ years.

Any potential need for investment comes from either addressing a level of service gap and/or system performance gap or opportunity in delivering the appropriate level of service, or from maintaining the appropriate level of service.

Refer to customer level of service below.

The results alignment criteria are defined in each assessment of the activity class page and follow the approach in the table below:

Waka Kotahi has combined the assessment criteria for activity classes where this makes sense. In some cases, the criteria in each table apply only to certain components of the activity classes: eg public transport improvements and continuous programmes are assessed using different criteria, even though they are part of the same public transport activity class.

The One Network Road Classification (ONRC) defines nationally consistent customer levels of service. Over time, all roads in a particular category should offer an increasingly consistent fit-for-purpose customer level of service for users. Identifying a gap to the ONRC customer levels of service is a key input into the assessment of results alignment for road maintenance and improvements.

No nationally consistent level of service or classification yet exists for public transport or walking and cycling. In these activity classes we use proxies to determine the significance of the problem, issue or opportunity including the geographical classification such as main urban areas or primary routes. We have provided reference frameworks at the bottom of this web page.

The level of service gaps are defined as follows:

• Significant gap: 
  Evidence shows that there is significant under performance in at least one key aspect resulting in performance lower than its classification and the gap between this and the appropriate service levels or system performance significantly impacts on the customer experience. This will have been robustly defined against appropriate levels of service (or other proxy) relevant to the issue.
  
  When using the ONRC if the level of service fails to meet the expectations for the road classification below the classification of the road in question, then there is a significant gap.

• Identified gap:
  Evidence shows there is a gap in service level or system performance that does not meaningfully meet at least one or more aspect described in its classification, intended use or function.
  
  When using the ONRC if the level of service fails to meet the expectations for the road classification of the road in question, then it is an identified gap.

• Addresses an issue:
  Evidence shows that the level of service or system performance materially achieves all aspects described in its classification. The value of benefits and change to levels of service or system performance are not significant or robustly defined.

During preparation of a business case, there may be situations where critical thinking through the business case approach uncovers no strong evidence that a gap in levels of service or system performance exists. This is a perfectly acceptable outcome to Waka Kotahi and could, in fact, suggest the system is running optimally.

Also during the preparation of the business case, service levels may be shown to be above expectations. This amounts to a system performance gap and further effort may be needed to assess the significance of this gap and consider system performance and optimisation.

Evidence is needed to support the significance of the gap.

The problem and benefits should have been described in the strategic case with reference to appropriate levels of service. Evidence collected about the current performance compared with the expected performance will help define a gap, if any.

Supporting evidence will show that:

• the right level of service framework is being used to define the gap
• the framework is being used appropriately for the activity
• current system performance expectations or customer levels of service are not being met
• the benefits will deliver an improvement in levels of service or system performance
• the benefits are significant in terms of GPS priorities.

The GPS emphasises value for money to maximise the impact of money spent to achieve the government’s outcomes.

Achievement of value for money will be supported through decision makers taking account of:

1. the full range of benefits and costs over the whole life of investments
2. alignment to the GPS priorities.

The GPS expects that achieving value for money will place greater emphasis on transparent investment decision making, and seeks to ensure that investments are made at the best cost for the best outputs and outcomes. It also identifies a need to investigate the appropriateness of current evaluation practices to ensure they are fit for purpose in giving effect to GPS strategic priorities.

Waka Kotahi is currently undertaking an investigation of the effectiveness of its current evaluation practices and provide the outcomes for consideration in the second stage GPS.

Incremental cost–benefit appraisal is required for selecting the best option(s) optimising business cases (other than between projects within packages) and evaluating proposed scope changes.

The BCR is the primary tool used to measure the efficiency of improvement activities.

All improvement activities other than low cost, low risk improvements and specified exceptions (see below) should be supported by the provision of a robust BCR that takes all reasonably quantifiable benefits into account.

Waka Kotahi requires that approved organisations and Waka Kotahi (state highways) use the Waka Kotahi Monetised benefits and costs manual (from August 2020) and Economic evaluation manual (superseded August2020) procedures and templates to determine the BCR for improvement activities.

Waka Kotahi may, at its discretion, consider any non-monetised benefits to determine whether the total of monetary and non-monetary benefits outweigh costs. It may also consider the incidence and relevance of benefits and costs in the context of individual proposals.

The cost-benefit appraisal rating for different BCR ranges is shown below:

An activity can be included in the NLTP when no cost–benefit appraisal has been made or when robust evidence is lacking to support the assessment. In such cases the rating for a cost–benefit appraisal will default to low*. Waka Kotahi represents these activities as  L* to indicate that more information is required to achieve a robust assessment profile.

An activity will not be considered for funding approval with an L* status.

Cost–benefit appraisal is not required for some activities.

Activities which are not required to calculate a cost–benefit appraisal include:

• individual low cost, low risk improvements activities, when included in the low cost, low risk improvements allocation
• transport planning activities, under work categories 001, 002, 003 and 004
• road safety promotion activities with a total cost of $1 million or less
• Total Mobility activities, under work categories 517, 519 and 521
• SuperGold Card concessions.

In specific cases generic or default BCRs may be used in lieu of a calculated BCR for the activity. These are:

• Stock effluent facilities – a generic BCR of 12 may be applied to a stock effluent facility identified in the 2008 National Stock Effluent Strategy, instead of a calculated BCR.
• Projects which are evaluated on a whole of life, net cost present value basis, eg bridge renewals have a default present value end of life (PVEOL). This is applied to an activity instead of a calculated BCR and the activity will be ranked as very high for cost-benefit appraisal.  However, where an improvement component exists in such a project, it must be supported by a calculated BCR. The present value evidence must be attached in TIO to support the PVEOL rating.
• Low cost, low risk improvement programmes –a generic rating of medium may be applied provided the Waka Kotahi is satisfied:
  • the activities proposed in the programme target medium and/or high results alignment
  •  the programme will be targeted to GPS outcomes
  • the activities represent reasonable value for money.

No other placeholder, generic or default BCRs should be used.

A streamlined investment pathway is available for applying for funding approval for standard safety interventions on projects valued between $1 million and $50 million.

For each intervention, a full BCR range has been calculated based on the Safe and Appropriate Speed as ‘Do Minimum’ approach, with inclusion of travel time disbenefit and vehicle operating costs.

 Please refer to the Standard Safety Intervention Toolkit for details.

 Other benefits and additional factors that may be considered:

• Additional benefits:
  Additional monetised benefits are benefits that are not specifically covered by the Waka Kotahi Monetised benefits and costs manual (from August 2020) and Economic evaluation manual (superseded August 2020). Waka Kotahi may consider additional monetised benefits as reasonable and may determine a higher rating as a result. Alternatively, it may consider the additional benefits should be presented as part of sensitivity analysis that will not impact on the rating.
• Discounting of benefits or dis-benefits:
  In certain situations some benefits, or dis-benefits, may be discounted from cost-benefit appraisal at Waka Kotahi discretion, where, in the context of the affected transport network, they are of minimal impact or significance.
• Additional factors:
  Waka Kotahi may consider any exceptional additional factors not otherwise captured by the two assessment factors, including additional economic and community benefits. These will be specific to the activity or combination of activities being assessed and relevant to determining the overall priority and funding source in the programme. Evidence will be required if additional factors are to be considered.  

The cost-benefit ratings for continuous programmes are:

• Low – when cost effectiveness or performance shows below-average efficiency
• Medium – when cost effectiveness or performance shows average efficiency
• High – when cost effectiveness or performance shows above-average efficiency.

Cost efficiency benchmarking is the main cost–benefit appraisal methodology for assessing continuous programmes as a whole, and the activities within them.

Levels of service performance comparisons are set against required customer levels of service and trend analysis (past and future costs and demands).

The  methods are:

For specific guidance relevant to activity class groupings see Assessment of activities by activity class. 

The IAF assessment methodology enables programmes of safety projects to be included in the 2018-21 NLTP to deliver the change in safety outcomes sought in the GPS by:

1 - Allowing for assessment and investment decisions to be made at a programme rather than individual project level:

• Currently packages of inter-related and inter-dependent activities submitted for funding consideration are able to include activities with BCR <1.0 provided the overall package demonstrates delivery of net positive benefits that exceed whole of life costs and all the components of the package have been completed.
• The new policy enables the same approach as for packages to be taken for programmes of work when the components of the programme are seeking to deliver a common outcome but they are not necessarily inter-related or inter-dependent.
• The policy also establishes the mechanism to allow inclusion of activities that are highly effective in achieving GPS priorities but have a BCR <1.0 into a programme, provided the overall programme demonstrates delivery of net positive benefits that exceed whole of life costs.

2 - Providing clarification of the ‘do nothing’ and ‘do minimum’ option for economic appraisal of speed management activities.

In this situation, the ‘do minimum’ option will be set at the safe and appropriate speed under the Speed management guide(external link). The ‘do nothing’ option is the existing baseline conditions of the network, based on the existing speed limit and existing infrastructure and services. It forms the basis for assessing the effects of a safe and appropriate speed limit in the ‘do minimum’ option. For some situations the best safety improvement option may be to simply lower the operating speed to a safe and appropriate level through the use of speed limit signs and minor infrastructure improvements that go with the new speed limits – this will be the do minimum option and can potentially be the preferred option to enable safety improvements.

3 - Travel time changes sensitivity testing

For the treatment of travel time changes, Waka Kotahi expects the evaluation of all activities and/or programmes to include the calculations of all positive and negative benefits. Where the travel time changes are neither significant nor relevant, it may be appropriate to provide sensitivity testing excluding travel time changes.

Guidance for the 2018-21 NLTP will be subject to change once the results are available from research that is planned during 2018/19 for implementation in the 2021-24 NLTP.

In all instances where any proposal (standalone project, or an activity within a programme) has a BCR <1.0, funding for these will only be approved by exception at the appropriate level of delegation.

For selecting the best safety investment option for a project, incremental economic cost-benefit appraisal continues to be required. 

Waka Kotahi reserves the right to require a peer review of an investment proposal.

Currently, Waka Kotahi requires a peer review of improvement activity business cases with a likely implementation cost of over $5 million or that involves a significant level of risk.

Approved organisations and Waka Kotahi (state highways) are encouraged to have small projects (between $1m and $5m implementation cost) peer reviewed in certain circumstances. Refer to the peer review guidelines for further information.

Safer Journeys(external link) is the government’s strategy to guide improvements in road safety during the period 2010-2020. The strategy’s vision is a safe road system increasingly free of death and serious injury and introduces the safe system approach to New Zealand. In May 2016 the third and final Safer Journeys action plan 2016–2020(external link) was released.

The safe system approach recognises that people make mistakes and are vulnerable in a crash. It reduces the price paid for a mistake so crashes do not result in loss of life or limb. Mistakes are inevitable – deaths and serious injuries from road crashes are not.

Significant progress has been made under the two previous action plans across all key areas of the safe system. This includes initiatives such as:

• raising public awareness through advertising campaigns
• lowering blood alcohol levels
• making our high-risk roads safer through rumble strips and median barriers
• mandating electronic stability control for light vehicles.

Many initiatives will continue as a core part of the policies and decision making of various agencies.

However, there are still areas where progress towards a safe road system needs more momentum. The third action plan will renew focus on areas of greatest risk and disproportionate harm, and present opportunities for the use of current and emerging technologies. In particular, this action plan’s focus (and specific objectives) is to:

• enable smart and safe choices on the road (particularly through technology enablement)
• make motorcycling safer (road environment, education and technology to address crash severity)
• ensure roads and roadsides support safer travel (highest risk roads including local urban arterials, head-on, run-off road and intersection crashes, and open road crashes)
• encourage safe vehicles (technology and vehicle safety standards).

Examples of investment in primary safe system treatments that reduce the risk of:

• head-on and run-off road crashes (such as through the installation of median and side barriers)
• urban and rural intersection crashes (such as through the installation of roundabouts or speed management devices)
• harm to vulnerable road users, including pedestrians, cyclists, mobility impaired (such as through segregated facilities, markings or speed management devices, including raised platforms at roundabouts, traffic signals and pedestrian facilities)
• lower cost safety interventions such as improved skid resistance, signs and markings (including rumble strips), safety targeted seal widening and speed management
• maintenance to ensure these safe system treatments remain fit for purpose
• rail safety, including to partner with KiwiRail to upgrade level crossings.

The One Network Road Classification (ONRC) has been adopted by the sector to ensure national consistency around the levels of service delivered by a network. Road Efficiency Group (REG), including Waka Kotahi, has developed a suite of 27 performance measures to support the next phase of implementing ONRC in the 2018–2021 NLTP.

All approved organisations and Waka Kotahi (state highways) are required to provide evidence of the customer levels of service they propose to deliver and how they relate to the ONRC measures.

Maintenance programme submissions from approved organisations and Waka Kotahi (state highways) may be supported by and take account of a wider set of performance measures than those currently mandated in support of the ONRC. For example an approved organisation may wish to support their submission with their long-term plan (LTP) measures in addition to the ONRC measures. Consideration of funding allocations for maintenance programmes in the 2018–21 NLTP will be based primarily on assessment of the transport network performance to be achieved as measured against the ONRC.

Waka Kotahi expects further development of the suite of performance measures will continue during the 2018–21 NLTP and lead to a broader set of specific performance targets for investment for the 2021–24 NLTP.

For public transport ‘levels of service’ refers to a range of factors from service frequency to quality and placement of public transport infrastructure such as bus stops. Determining the appropriate level of service for public transport, unlike roads, is not subject to a national standard. For the purposes of determining whether there is a gap in current levels of service factors, the following could be considered:

• Land use and density – do services link residential areas with commercial/social/employment centres (ie places where people want to go) and does the public transport network reflect future land use intentions?
• Existing services – does demand exceed supply of services? Is there a need to increase service levels to meet demand?
• Congestion – are there congested parts of the roading network that an increase in public transport service levels could help alleviate?
• Transport disadvantaged – do existing public transport service levels provide an adequate modal alternative to travelling by private vehicle?
• Hours of service – do the hours of operation enable travel at times people want to travel?
• Customer information – does the level of customer information provided meet customer expectation?
• Reliability – can users rely on the service to consistently arrive on time at a stop, and/or will the journey take longer than scheduled? For example, a reliable service means a person does not need to allocate extra time waiting at a bus stop for travel just in case a bus is early or late and can be confident they will arrive at their destination on time.

A draft Guidelines for public transport infrastructure [PDF, 2.2 MB] provides for bus stops and facilities at bus stops. Waka Kotahi recommends use of the draft document to guide appropriate levels of service at bus stops. We are actively engaged in considering the wider public transport infrastructure levels of service and guidelines.

Waka Kotahi continues to recommend use of the Cycle network guidance –  planning and design (CNG) to guide appropriate levels of service to determine what is fit for purpose on target corridors when considering new cycling facilities. A New Zealand level of service tool for cycling is under development.  In the interim the CNG recommends using the Austroads levels of service tool.

The CNG aims to promote a consistent, best-practice approach to cycling network and route planning throughout New Zealand. It sets out a principles-based process for deciding what cycling provision is desirable, and provides best-practice guidance for the design of cycleways.

Regional transport committees and approved organisations need to have a clear understanding of who the cycling programme is targeting, what level of service is required in delivering the activity or activities, and how different activities might complement each other to maximise the benefits of investment.

The main factors influencing levels of service relate to safety, comfort, delays which consider network characteristics such as traffic volumes and speed, degree of separation from motor traffic, facility width, etc.  

For walking, Waka Kotahi recommends using  the Pedestrian planning and design guide along with the Austroads levels of service tool. For existing situations the Community street review (external link)process is also recommended and for new designs the Pedestrian levels of service prediction tool in NZTA research report 452 Predicting walkability.

The full reference for the Austroads levels of service tool is: ‘Level of service metrics for network operations planning (AP-R475-15)’.

The GPS identifies the following as ‘major metropolitan areas’ for the purposes of land transport planning:

• Northern Auckland zone
• Western Auckland zone
• Central Auckland zone
• Southern Auckland zone
• Hamilton zone
• Tauranga
• Porirua zone
• Upper Hutt zone
• Lower Hutt zone
• Wellington zone
• Christchurch
• Dunedin (urban area only).

Waka Kotahi has identified Queenstown as a specific high-growth urban area given similarities in its transport requirements to those in the major metropolitan areas, and therefore the application of the IAF criteria is treated in the same way as for major metropolitan areas.

The combined group is referred to as 'major metros' in the IAF criteria.

‘High-growth urban area’ means any urban area (as defined by Statistics New Zealand in 2016) that has either:

• a resident population of over 30,000 people according to the most recent Statistics New Zealand urban area resident population estimates

or

• at any point in the year a combined resident population and visitor population of over 30,000 people, using the most recent Statistics New Zealand urban area resident population estimates

and:

• in which the resident population of that urban area is projected to grow by more than 10% between 2013 to 2023, according to the most recent Statistics New Zealand medium urban area population projections for 2013(base)-2023.

This definition is from the National policy statement on urban development capacity (NPS-UDC) 2016.

Areas currently listed as in the NPS-UDC high growth urban areas:

• Auckland
• Christchurch
• Hamilton
• New Plymouth
• Tauranga
• Queenstown
• Whangarei.

‘Medium-growth urban area’ means any urban area (as defined by Statistics New Zealand in 2016) that has either:

• has resident population of over 30,000 people according to the most recent Statistics New Zealand urban area resident population estimates

and:

• in which the resident population of that urban area is projected to grow by between 5% and 10% between 2013 to 2023, according to the most recent Statistics New Zealand medium urban area population projections for 2013(base)-2023.

This definition is from the National policy statement on urban development capacity 2016 (NPS-UDC).

Areas currently listed as in the NPS-UDC medium growth urban areas:

• Dunedin
• Gisborne
• Kapiti
• Marlborough
• Napier-Hastings
• Nelson
• Palmerston North
• Rotorua
• Wellington.