Archive - this information is for reference only and no longer maintained.

Introduction

The Transport Agency assesses investment proposals in the 2015-18 National Land Transport Programme (NLTP) under three factors, being strategic fit (the alignment of the proposal's key transport issues with Government Policy Statement on Land Transport (GPS) results), effectiveness (how well the proposed solution can address the issues) and benefit and cost appraisal (how efficient resources are used to deliver benefits from the proposed solution).

The information relating to developing assessment profiles is organised by activity class groupings, e.g. road maintenance programmes, with guidance provided on each of the three factors within the groupings.

Assessment factors

The Transport Agency assesses investment proposals under three factors, being:

  • strategic fit (the alignment of the proposal's key transport issues identified in the strategic case with results sought under the current Government Policy Statement on land Transport (GPS) ),
  • effectiveness (how well the proposed solution can address the issues), and
  • benefit and cost appraisal (how efficiently resources are used to deliver benefits from the proposed solution).

The specific guidance on how each of these apply to activity class groupings is provided in relevant sections listed at the bottom of this page. 

Strategic fit

Assessment of strategic fit considers how the problem, issue or opportunity identified in the strategic case aligns with the results sought under the Government Policy Statement on Land Transport (GPS).

Strategic fit focuses on the problem, issue or opportunity being considered without regard to the possible solution.

The strategic fit assessment considers how the problem/issue/opportunity:

  • matches the desired GPS result(s)
  • is significant in relation to the desired GPS result(s)
  • is significant in relation to the scale of the gap to the appropriate customer level of service or performance measure
  • is significant as part of an end to end journey
  • is significant from a national perspective (given local, regional, national perspectives)
  • is significant in relation to GPS timeframes, i.e. a significant issue/opportunity within 3/6/6+ years

The need for investment comes from addressing a level of service gap. Historically, the gap has not been explicitly or consistently defined.

The One Network Road Classification(external link) (ONRC) defines nationally consistent customer levels of service. Over time, all roads in a particular category should offer an increasingly consistent fit for purpose customer level of service for users. Identifying the gap to the ONRC customer levels of service will become a key input into the assessment of strategic fit for road maintenance and improvements.

No nationally consistent level of service or classification yet exists for public transport or walking and cycling. In these activity classes we use proxies to determine the significance of the problem, issue or opportunity including the geographical classification such as main urban areas or primary routes.

Effectiveness

Effectiveness considers the contribution that the proposed solution makes to achieving the potential benefit identified in the strategic fit assessment and to the purpose of the Land Transport Management Act 2003(external link).

Effectiveness assesses how well the proposed activity will deliver the desired results, while considering integration, and whether the solution has been correctly scoped, is affordable, timely and manages risk.

Fit for purpose

The assessment of effectiveness should be fit for purpose, taking into account the scale and complexity of the project or programme and its stage of development.

Examples:

  1. For a small, straightforward rural road bridge replacement, the focus of assessment should be on option selection, scope and timing of the project. The assessment of outcomes, integration, affordability and confidence should be a simple and brief exercise. If these latter criteria are satisfied as well as they can be, they should rate as 'High'. The simple nature of the project does not limit the rating of effectiveness.
  2. A large project that originates from a multi-modal, multi-organisation programme business case would require a focus across all criteria to assess effectiveness. A significant proportion of the evidence around outcomes, integration, affordability and timing should come from the programme business case, with the assessment considering the fit with the business case. The scale, complexity and riskiness of the project, and how it fits against the programme business case, necessitates a detailed assessment of effectiveness, which should be peer reviewed.
  3. For a programme, e.g. existing public transport services or road maintenance, where there is limited change requested from previous expenditure and key trends are in line with the Transport Agency's expectations, the assessment of outcomes, integration, affordability, scope and confidence should be a simple and brief exercise. If these criteria are satisfied as well as they can be, they should rate as 'High'.
  4. Where significant change is proposed for a programme and there is considerable cost and risk exposure for the Transport Agency, the programme business case, e.g. the activity management plan, should provide robust evidence around outcomes, integration, affordability, scope and confidence. A detailed assessment of effectiveness is required that considers the scale, complexity and risk associated with the change and how it fits with the business case. The assessment should be peer reviewed.

Judgement required

Assessment of effectiveness as a whole is based on quantitative and qualitative evidence, although for each of the six criteria it may comprise solely one or the other. It requires the use of judgement rather than being a simplistic "box ticking" exercise. For large, complex programmes and projects the assessment should be moderated amongst experienced peer reviewers, to ensure different perspectives are debated to arrive at a consensus.

Sufficient evidence must be provided to support the effectiveness rating. This includes as appropriate:

  • a strategic business case
  • a programme business case, progressing to detailed business case depending on stage of development
  • an Activity Management Plan (AMP)
  • a Regional Public Transport Plan (RPTP)
  • a Road Safety Action Plan (RSAP) or equivalent planning tool
  • a recognised regional or local lifelines transport network study
  • a strategy or study, project feasibility report (PFR), or scheme assessment report (SAR) (for activities still moving through the old life cycle prior to the Business Case Approach)

Under development

On occasion, early in the development of some complex activities, e.g. before completion of the programme business case, there may be insufficient evidence to make a confident assessment of effectiveness. An interim rating of M* at the time of inclusion in the National Land Transport Programme (NLTP) indicates there is insufficient information and that further development is required.

The M* rating enables an activity to be included in the NLTP, but funding approval will not be considered until the information is developed and a firm rating of effectiveness is provided.

Low and medium ratings

A low rating means that there is insufficient evidence to show that a solution can successfully address the problem, issue or opportunity. If the effectiveness is assessed as low, the proposed solution should remain in the programme business case stage until the effectiveness criteria assessed as low have been addressed through further development. 

Medium effectiveness means that an activity has not achieved the full potential identified in the strategic fit assessment. This may be due to a deficiency in the development process, e.g. sensitivity analysis should have considered more scenarios to provide a fuller view of the risks, or an acknowledged shortfall in the chosen intervention, e.g. a safety intervention that addresses only part of the crash risk. The deficiency or shortfall is not considered by the assessor as significant to the extent that the activity should not be progressed, more that its effectiveness is not as good as it could be.

Assessing effectiveness

The assessment considers all criteria. The overall effectiveness assessment is reported as the lowest rating for any criterion, i.e. an overall M rating will be given when all criteria and parts have either an M or H rating. An overall L rating will be given if any criterion is rated L.

The effectiveness assessment criteria look at how well the proposed activity or programme:

  • is outcomes focused
  • is integrated
  • is correctly scoped
  • is affordable
  • is timely
  • provides confidence

Benefit and cost appraisal

The benefit and cost appraisal considers how well the proposed solution maximises the value of what is produced from the resources used, and the timeliness of intervention.

Assessment of improvement activities uses the benefit–cost ratio as the default approach.

Cost-effectiveness and performance comparisons are used for road maintenance and public transport programmes.

Benefit Cost Ratio

The Benefit Cost Ratio (BCR) is the primary tool used to measure the efficiency of improvement activities.

All improvement activities other than Minor Improvements and specified exceptions (see below) should be supported by the provision of a robust BCR.

The Transport Agency requires that Approved Organisations and the Transport Agency (state highways) use the Transport Agency Economic Evaluation Manual procedures and templates to determine the BCR for improvement activities.

Alternatives to benefit cost ratio

For assessment of road maintenance, existing public transport programmes and road safety promotion programmes, alternative methods may be used in place of the BCR.

A brief description of these methods is provided below:

Present value method (PV)

 

The present value of future costs of options are determined and compared to identify the long term least cost option.

The method is recommended to determine if replacement/renewal is more cost effective than on-going maintenance.

In certain cases the use of benefit streams rather than future costs may be more appropriate.

Cost effectiveness method

Cost effectiveness analysis is used where the objective is to compare the cost of different ways of achieving a given effect (e.g. level of service), or comparing the relative cost of different strategies with different effects.

This approach is used to evaluate the benefit and cost appraisal of public transport, maintenance, or road safety promotion programmes by comparing a programme with similar programmes for other organisations.

Benchmarking method

This approach involves making comparisons of programme benchmark and key performance measures against similar regions and the national average. A lack of evidence to support differences from peer group, regional and national averages may result in a lower benefit and cost appraisal rating for the programme. Alternatively it may result in a requirement for a review of the supporting business case as a condition of investment approval.

Trends in benchmarking measures over time are used rather than just annual values.

Marginal contribution method

The marginal contribution of programme expansions and incremental new services will be considered through benchmark and key performance measures. The cost-effectiveness of the changes will be considered as well.

Guidance is provided to help determine benefit and cost appraisal ratings for each activity class grouping.

Non-monetised benefits and additional benefits

If a proposed solution has demonstrable non-monetised benefits, then these should be taken into account and may, if the Transport Agency considers these benefits to be significant, result in a higher rating.

Additional benefits are usually in the form of wider economic benefits that are not specifically covered by the Transport Agency’s Economic Evaluation Manual. The Transport Agency may consider additional benefits as reasonable and may determine a higher rating as a result. Alternatively, it may consider that the additional benefits should be presented as part of sensitivity analysis that will not impact the rating.

Exceptions

Benefit and cost appraisal is not required for some activities.

Activities which are not required to calculate a benefit and cost appraisal include:

  • individual minor improvements activities, when included in the minor improvements allocation
  • Transport planning activities, under work categories 001, 002, 003 and 004
  • Road safety promotion activities with a total cost of $300,000 or less
  • Total mobility activities, under work categories 517, 519 and 521
  • SuperGold Card concessions (which are funded outside of the NLTF).

Use of generic or default BCR

In specific cases generic or default BCRs may be used in lieu of a calculated BCR for the activity. These are:

  • Stock Effluent Facilities - a generic BCR of 12 may be applied to a stock effluent facility, as identified in the National Stock Effluent Strategy, instead of a calculated BCR
  • Projects which are evaluated on a whole of life, net cost present value basis, e.g. bridge renewals. A default BCR of 99 will be applied to an activity instead of a calculated BCR. However, where an improvement component exists in such a project, it must be supported by a calculated BCR. The present value evidence must be attached in TIO to support the 99 rating.
  • Minor improvement programmes - a generic rating of Medium may be applied provided the Transport Agency(planning and investment) assessor is satisfied that the activities proposed in the programme target medium and/or high strategic fit GPS results, that the programme will be effective and that the activities represent reasonable value for money.

No other placeholder, generic or default BCRs should be used.

Peer review

The Transport Agency reserves the right to require a peer review of benefit and cost appraisal determinations and measures, including any non-monetised/additional benefits and adverse impacts, regardless of the scope, prior to an investment decision.

Insufficient information (1* or Low*)

An activity can be included in the National Land Transport Programme (NLTP) when no benefit and cost appraisal has been made or when robust evidence is lacking to support the assessment. In such cases the rating for a benefit and cost appraisal will default to 1* for improvement activities and Low* for programmed activities. The Transport Agency represents these activities as 1* or Low* to indicate that more information is required to achieve a robust assessment profile.

An activity will not be considered for funding approval with a 1* or Low* status.

No rating

If the calculated BCR is below 1.0, then the activity is considered to be economically inefficient. In this case, no rating for benefit and cost appraisal will be given. The Transport Agency may, at its discretion, make an assessment of any non-monetised benefits to determine whether the total of monetary and non-monetary benefits outweigh costs.

Application to improvements

Assuming that the BCR is 1.0 or higher, the benefit and cost appraisal for improvements to local roads, state highways, public transport, and walking and cycling will fall into one of three bands:

BCR range 1- 3

All activities with BCR greater than or equal to 1 and below 3 are prioritised in this band.

BCR range 3 - 5

All activities with BCR greater than or equal to 3 and below 5 are prioritised in this band.

BCR range > 5

All activities with BCR greater than or equal to 5 are prioritised in this band.

Application to continuous programmes

Continuous programmes, e.g. road maintenance, existing public transport services, road safety promotion and minor improvements,  are assessed with a benefit and cost appraisal rating based on their relative cost effectiveness or performance comparisons.

The ratings are:

  • Low - when cost effectiveness or performance shows below-average efficiency
  • Medium - when cost effectiveness or performance shows average efficiency
  • High - when cost effectiveness or performance shows above-average efficiency

Incremental benefit and cost appraisal

Assessment of incremental benefit and cost appraisal is required for option selection, optimisation (other than between projects within packages) and scope change proposals.

Pre-assessment requirements

A number of criteria need to be satisfied before assessing an activity for inclusion in the NLTP for funding approval.

Requirements for NLTP inclusion

Inclusion of an activity in the NLTP requires that:

  • sufficient evidence has been used to justify the activity, dependent on its stage of development;
  • the activity is included in a Regional Land Transport Plan (RLTP), or is a proposed Road Policing Programme, or is part of a Transport Agency nationally delivered programme;
  • the problem/issue/opportunity is clearly identified in a strategic business case;
  • the requirements of the Land Transport Management Act have been met, that the activity:
    • is assessed by the organisation that proposes it;
    • identifies the objective or policy to which it will contribute;
    • provides an estimate of the total cost and the cost for each year;
    • shows its expected duration (automatically generated from the cash-flow plan in Transport Investment Online);
    • shows any proposed sources of funding other than the National Land Transport Fund (NLTF); and
    • provides any other relevant information.

Requirements for NLTF funding

Seeking approval for funding from the National Land Transport Fund requires all of the above, plus:

  • the proposed activity or programme is eligible for NLTF funding (consistent with the GPS activity class definition and the Transport Agency's work category definition);
  • all required information is provided in Transport Investment Online (TIO) and has been checked for completeness;
  • the proposal is ready to proceed. 

Additional factors

The Transport Agency may consider any exceptional additional factors not otherwise captured by the 3 assessment factors, including wider economic and community benefits.

These will be specific to the activity or combination of activities being assessed and relevant to determining its overall priority and funding source in the programme.

Evidence will be required if additional factors are to be considered.

Activity class groupings

Links to guidance on developing assessment profiles for each of the activity class groupings are provided below.