Portfolio, program and project assurance framework

CDSB is currently reviewing the layout of this document to better align with the Queensland Government design system.

Introduction

Purpose

This Queensland Government Enterprise Architecture (QGEA) framework is mandated by the Portfolio, program and project management and assurance policy, under the Financial and Performance Management Standard (2019). It provides information and defines best practice processes for Queensland Government departments to undertake assurance on all digital and ICT-enabled portfolios, programs and projects.

The framework applies to all Queensland Government employees (permanent, temporary and casual) working in a Queensland Government department and Queensland Government owned corporations including all organisations and individuals acting as its agents (including contractors and consultants).

Scope

Assurance is the objective examination of information for the purpose of providing an independent assessment on governance, risk management and control processes for an organisation.

The Portfolio, program and project assurance framework (‘the framework’) includes all types of project, program and portfolio assurance such as:

• point-in-time assurance activities such as gated reviews, health checks, portfolio assurance and Portfolio, Program and Project Management Maturity Model (P3M3®) assessments
• one-off assurance (such as deep dives)
• continuous assurance (such as project assurance and assurance partnering).

This framework is not a substitute for a rigorous portfolio, program and project governance framework that manages a department’s key processes. These include:

• business and strategic planning
• investment appraisal and business case management (including benefits management under the Financial Accountability Handbook: Volume 1 – Introduction)
• project, program or portfolio management
• risk management
• procurement and acquisition
• service and contract management.

Role of the Office of Assurance and Investment

The Queensland Government requires visibility across the government’s investment in digital and ICT-enabled initiatives and assurance that the expected benefits will be delivered on time, to budget and in line with Queensland Government legislation and QGEA policy requirements. The Queensland Government expects initiative risks and issues to be transparent with departments acting on and mitigating problems before there is an impact on community and consumer outcomes and benefits.

The role of the Office of Assurance and Investment (OAI) is to monitor and report on all assurance activities undertaken by digital and ICT-enabled initiatives, on behalf of the Digital Economy Leaders Sub-Group (DELSG).

OAI receives and registers the following documents for profile level 2, 3 and 4 initiatives:

• the initial and subsequent versions of the initiative assurance profile
• the initial and subsequent versions of the initiative assurance plan
• the initiative’s gated assurance reports, health check reports and post-implementation review report.

Assurance principles

This framework adopts a principles-based approach to assurance. Principles are foundational and ‘universal truths’, which are:

• self-validating, because they have been proven in practice
• empowering, because they enable practitioners to apply effective and pragmatic assurance to different portfolios, programs and projects.

The five assurance principles are:

Assurance is planned and appropriate

Each initiative has an approved fit-for-purpose assurance plan which:

• aligns to the baselined schedule and critical path
• includes a budget for assurance in the initiative business case or the portfolio annual budget
• is periodically updated to reflect any changes in risk profile or lessons learned
• avoids duplication and overlap with other initiative information.
• defines and agrees the roles and responsibilities for assurance, including that of the Accountable Officer, investing department and those organising the assurance.

Assurance drives and informs good decision making

• Assurance provides a timely, reliable assessment to inform key investment decisions.
• Assurance is timed to occur before key decision points such as funding allocation, business case approval and operational go-live.
• Assurance information is clear and uses consistent definitions and standards to support comparisons over time (for example, using the standard delivery confidence ratings and priority ratings for recommendations in gated assurance).
• Accountable Officers and investing departments have full and unimpeded access to the assurance review team and their report and use assurance information to focus support, attention and resources where it is most needed.

Assurance is objective and independent

• The independence of the assurance review team members is determined in accordance with the principles of the Three Lines Model (see Appendix A), where an assurer cannot assure their own work.
• Those undertaking assurance are not involved in the delivery of initiative products, nor do they make initiative management decisions.
• Assurance is provided by suitably qualified reviewers with the right skills and experience to assure the scale and complexity of the investment.
• Conflicts of interest for the review team are identified and managed.

Assurance is supported by senior management and a culture of continuous improvement

• The Accountable Officer is open to external scrutiny and constructive challenge, and this behaviour is expected of their teams.
• The Accountable Officer and key stakeholders actively engage in assurance activities, ensuring any planned reviews remain fit for purpose throughout the initiative lifecycle.
• Assurance is actively promoted as a valuable process in securing the successful delivery of expected benefits.
• Implementation of assurance report recommendations are actively planned and reported.
• The review team is supported by the Accountable Officer to access the people and resources they require.

Assurance is forward-looking and focused on risks to benefits realisation

• Assurance supports the Accountable Officer and investing department to identify and communicate key risks and opportunities likely to impact realisation of expected benefits.
• The assurance activities outlined in this framework are not conducted as audit activities.
• Assurance reviews use available information from a range of sources that reflects the delivery approach, scale and complexity of the initiative.
• Assurance looks for, and considers, information from a range of sources including documents, interviews, reports and visual displays.

Assurance processes for initiatives

The assurance process consists of four activities – starting with assurance profiling, assurance planning, assurance reviews and lastly, reviewing the action plan implementation and reporting.

1. Assurance profiling

The assurance profiling step is used to determine the minimum assurance requirements and the degree of independence of the review team members. Assurance profiling occurs at the commencement of an initiative. The assurance profile should then be periodically re-assessed when there are significant changes to the initiative such as changes to project scope, complexity, impact and/or risks. The assurance profile is independently re-validated as part of any gated assurance review or health check review.

All Queensland Government digital and ICT-enabled initiatives must undertake assurance profiling to determine their assurance profile level, which ranges from level 1 (being the lowest rating) to a level 4 (being the highest rating).

The assurance profile is used by OAI to determine those initiatives that will be subject to the Queensland Government investment review process.

All assurance profiles are registered with OAI prior to Gate 1 – Preliminary evaluation and again at Gate 3 – Contract award for projects, and prior to the Gate 0 – Initial for a program. Once the profile is noted by OAI, assurance planning for the next project stage or program tranche can commence.

Assurance profiling tool

Assurance profiling involves answering a series of questions to determine initiative complexity and impact. This activity should be undertaken with the Accountable Officer, board members, and key stakeholders with a good knowledge of the initiative and the technical and business challenges.

Assurance profile levels

Four assurance profile levels are defined. Each progressive assurance level supports an increasing level of assurance activity, scrutiny, and independence of the review team members.

For example, profile level 1 is applied to less complex initiatives that may, for example, be undertaken by existing departmental resources and funded by capital or operational budgets. A profile level 4 initiative represents highly complex or transformational change initiatives that may, for example, involve several agencies, be the subject of Ministerial interest, deliver on an election commitment and/or require tied funding such as Cabinet Budget Review Committee (CBRC) funding or grants.

Profile level

Level 1 (lowest)

These projects involve ‘known knowns’. Projects take a ‘sense-categorise-respond’ approach to establish the facts (‘sense’), categorise the plausible solutions, then respond by applying project best practice.

In these projects the relationship between the business problem and solution is relatively clear. A similar project may have been undertaken previously that can inform the delivery approach and there is high confidence in estimates for duration and cost.

The timeframe may be relatively short – for example within a financial year. They may be funded either from capital and/or existing operational budgets.

Level 1 projects are not ‘business-as-usual’; they are still a temporary endeavour, and there are perceived advantages to applying project management to govern, direct and report them. As a minimum, they will have a project manager and an Accountable Officer appointed

Level 2

These projects represent the ‘known unknowns’.

The relationship between business problem and solution requires analysis or expertise from subject matter experts; and there are a range of plausible solutions. There is some degree of solution complexity.

The initiative takes a ‘sense-analyse-respond’ approach to assess the facts, analyse the solutions, and apply tailored project management best practice.

Level 2 projects may have some very specific delivery dates to be delivered within one to a few years. Project estimate confidence remains high with a small plus/minus range based on a well-defined scope.

Level 3

These initiatives represent ‘unknown unknowns’.

The relationship between business problem and solution is ambiguous and there may be considerable complexity in the system architecture and business process changes required.

Initiatives probe first, then sense, and then respond to determine a demonstrable value for money solution to the business problem.

The impact and complexity of these initiatives is high.

Level 4 (highest)

These initiatives represent high value and high-risk strategic investments.

The relationship between business problem and solution is unclear.

Initiatives take an ‘act-sense-respond’ approach to determine a demonstrable value for money solution to the business problem.

These initiatives may involve challenging timeframes with firm political, legislative or technical deadlines, and a high level of public interest. Estimate confidence may be low because of a large number of assumptions or unknowns. The initiative may involve a number of departments in delivery and benefit realisation.

2. Assurance planning

Assurance planning is the second activity in the assurance process.

Assurance for an initiative should be planned to reflect:

• the initiative assurance profile level
• the baselined schedule, critical path and timing of key decision points
• the initiative risks (threats and opportunities).

The first version of the assurance plan should be prepared when starting up an initiative, after the approval of the assurance profile. It should be completed in consultation with the Accountable Officer and include a budget for assurance. The assurance plan will be influenced by who is able to undertake the reviews and the number of reviews that are planned in line with the minimum assurance requirements.

The Accountable Officer approves the final draft version of the plan and any subsequent updates. The plan will need to be maintained with current dates and any changes to planned reviews.

Guidance for projects

Profile level

Level 1

Level 1 projects require minimal assurance through health checks.

A Gate 3 assurance review will be required if the project is planning to enter into a new contract or extend an existing contract for the provision of digital or ICT systems and software.

Ongoing project assurance occurs as part of project board roles and their respective business, user and supplier assurance responsibilities to ensure compliance with all required Queensland Government and departmental policy requirements.

Level 2

Level 2 projects undertake gated assurance and health checks if it is more than six months between gated assurance reviews.

Level 2 projects may require a Gate 3 review to be repeated, once before pilot implementation and again to confirm the investment decision is appropriate prior to contract execution and/or full funding commitment and implementation.

Level 2 projects may require a Gate 4 review to be repeated if there are multiple significant go-live releases or different site implementations.

The Gate 5 assurance review is undertaken if the project has claimed any financial, economic and/or productivity benefit types.

Ongoing project assurance occurs as part of project board roles and their respective business, user and supplier assurance responsibilities to ensure compliance with all required Queensland Government and departmental policy requirements.

Level 3

Level 3 projects must undertake gated assurance reviews and health checks if it is more than six months between gated assurance reviews.

Level 3 projects may require a Gate 3 review to be repeated, once before pilot implementation and again to confirm the investment decision is appropriate prior to contract execution and/or full funding commitment and implementation.

Level 3 projects may require a Gate 4 review to be repeated if there are multiple significant go-live releases or different site implementations.

Ongoing project assurance occurs as part of project board roles and their respective business, user and supplier assurance responsibilities to ensure compliance with all required Queensland Government and departmental policy requirements.

Level 4

Level 4 projects must undertake gated assurance reviews and health checks if it is more than six months between gated assurance reviews.

Level 4 projects may require a Gate 3 review to be repeated, once before pilot implementation and again to confirm the investment decision is appropriate prior to contract execution and/or full funding commitment and implementation.

Level 4 projects may require a Gate 4 review to be repeated if there are multiple significant go-live releases or different site implementations.

Ongoing projects assurance occurs as part of project and program board roles and their respective business, user and supplier assurance responsibilities to ensure compliance with all required Queensland Government and departmental policy requirements.

Minimum assurance requirements

The minimum assurance requirements for both programs and projects based on the approved profile level are shown in the table below. A health check must be undertaken for profile level 2 or above initiatives if it has been more than six months between gated assurance reviews. Internal program and project assurance undertaken within the initiative representing the business, supplier and user interests will be ongoing as part of program and project board roles.

Projects

1. A Gate 1 - Preliminary Evaluation and Gate 2 – Business Case review can be undertaken only once. In addition, Gates 1 and 2 cannot be combined as each serves a specific purpose.
2. The early stage health check is undertaken at the end of the initiation stage to inform the decision to move to the delivery stage. More details on health checks are provided in Appendix B.
3. A Gate 3 – Contract Award can be repeated, once before pilot implementation and again to confirm the investment decision is appropriate prior to full funding allocation and implementation.
4. A Gate 3 – Contract Award will need to be undertaken if the initiative is planning to enter into a new contract or extend an existing contract for the provision of digital or ICT systems and software.
5. A Gate 4 – Readiness for Service once the investment decision is made and can be repeated per site implementation depending on complexity and risk.
6. The post implementation review is undertaken in the project closure stage. More details on PIRs are in Appendix B.
7. Gate 5 – Benefits Realisation will apply when there are claimed financial, economic and/or productivity benefit types identified at Gate 3 – Contract Award.
8. More details on the integrated/continuous/ assurance partner assurance activity is provided in Appendix B.

Programs

1. The initial Gate 0 review occurs as the program nears the end of the Identify process and/or the initial Plan Progressive Delivery process using Managing Successful Programs (MSP® 5th edition).
2. The mid-stage review can be repeated multiple times during the life of the program. This review should be completed at the end of each tranche, incorporating each project within the tranche.

Minimum independence requirements

This framework applies the principles of the Institute of Internal Auditors’ Three Lines Model to determine if the chosen reviewer has the appropriate level of independence for the type of assurance activity. The Three Lines Model distinguishes between:

• continuous controls and assurance undertaken by the initiative team and within the initiative to manage risks (first line)
• assurance undertaken by subject matter experts outside the initiative such as the portfolio management office (PMO) and other departmental teams to manage risks (second line)
• the independent assurance undertaken at points in the initiative lifecycle such as gated assurance and health checks (third line).

The Three Lines Model must be considered when planning the initiative’s assurance as it will guide who can undertake which assurance activities. More details on the Three Lines Model are provided in Appendix A.

Level 1

Projects may be assured by certified Gateway reviewers internal to the department and external to the initiative. They may, for example, be within the department’s portfolio, program or project management office however they should not be involved in any way with project delivery, or advising on the delivery, of the initiative. An external assurance provider may be used where required, for example, due to capacity or capability constraints.

Level 2

Projects can be assured by certified Gateway reviewers internal to the department and independent to the initiative. However, some level 2 projects can be quite large in their scope and have considerable complexity. When planning the assurance for these projects consider the experience and credentials of the internal assurance reviewers. Once again, the review team may comprise of reviewers from the department’s portfolio, program or project management office however they should not be involved in any way with delivery, or advising on the delivery, of the initiative. The Cross Agency Assurance Working Group is able to undertake assurance reviews for Level 2 projects (subject to availability). An external assurance provider may be used where required.

Level 3 and 4

Profile level 3 and 4 initiatives require greater independence and scrutiny by the review team members. This requirement applies to both gated assurance reviews, health checks and post implementation reviews. Profile level 4 initiatives also require a larger review team with four review members to allow for the necessary subject matter expertise on the review team (also see Appendix C for details).

These initiatives should engage an appropriate external vendor for their assurance reviews by using the Standing Offer Arrangement.

External assurance providers (vendors)

Given assurance requires a minimum degree of independence, consider whether the chosen vendor is planned to be engaged or contracted in another capacity for the initiative.

Program and project schedule alignment with key decision points

The baselined initiative schedule showing the critical path should inform when key decision points are required and dictate the timing for assurance activities. Gated assurance reviews must occur prior to the Accountable Officer’s decision to continue, discontinue, vary the scope for implementation or enter into a contract.

Individual gate reviews are timed to coincide with a specific point in the initiative lifecycle. As such, areas of focus for each of the reviews changes as the initiative progresses through its lifecycle.

For a program, the Gate 0 review should be timed to occur at major investment decision points, for example, at the end of a program tranche or just prior to the drawdown of capital and/or operational funding. A Gate 0 review may be combined with its component project gate reviews if they are planned to be undertaken at the same time. This allows the review team to undertake assurance at both the program, for example, Mid Gate 0 and project levels (for example, Gate 4) concurrently allowing some leverage of both the review team and stakeholder’s time and avoid doubling up of review activity just prior to key decision points.

Applying risk management across assurance

Management of Risk (M_o_R®) is unique in explicitly considering the similarities and differences of applying risk management across the strategic, portfolio, program, project, product and operational perspectives. Depending on the complexity of the program and project, each of these perspectives may be relevant or a sub-set from an assurance perspective.

For example, an initiative may have specific risks regarding resourcing availability, technical risks regarding the chosen solution or testing plan, security, or data migration. There may be other risks surrounding procurement or vendor contracts, or risks regarding the identification and quantification of benefits or their realisation.

Figure 1: Typical integration of risk information across the M_o_R® 4 perspectives (M_o_R®4 Management of Risk: Creating and protecting value, AXELOS 2022). Arrows represent indicative risk information flows between perspectives. These will vary depending on which perspectives are relevant to the organisation

Agile

The QGEA Portfolio, program and project management and assurance policy states “Departments must use project, program and portfolio management methodologies endorsed by the Department of Customer Services, Open Data and Small and Family Business (CDSB), to direct, manage and report all digital and ICT-enabled programs and projects.”

CDSB currently endorse the AXELOS Global Best Practice guidance, namely Managing Successful Programmes 5th edition, Managing Successful Projects with PRINCE2® 7th edition and PRINCE2 Agile®.

As such, the minimum assurance requirements will still apply following the Queensland Treasury Gateway Review process where PRINCE2 Agile® is used.

Gated assurance reviews

The assurance review is the third activity in the assurance process. This section focuses on gated assurance reviews; guidance regarding other initiative assurance activities is at Appendix B.

Gated assurance reviews are an assurance methodology that examines programs and projects at key decision points in their lifecycle to provide assurance that they can progress to the next phase or stage. It provides independent guidance and recommendations to Accountable Officers and investing departments on how best to ensure that initiatives are successful.

Gated reviews are an independent peer review of the initiative’s viability and feasibility. Gated assurance reviews are not an audit; whilst they consider progress to date they are forward looking and provide recommendations based on findings on what is needed to help take the initiative forward. A gate review will not stop an initiative but informs Accountable Officers. It is a high-level review at a point in time.

The gated review process is a third line assurance activity and increases confidence in:

• alignment with Queensland Government and departmental strategic objectives
• investment decisions
• delivery of initiatives to time and budget with the realisation of benefits.

Triggers for gated reviews in projects

Review team size and qualifications

Appendix C provides details on the required review team size and qualifications.

Review timelines

A gated assurance review involves four main activities:

• planning the review
• conducting the document and information review
• undertaking interviews
• preparing the draft and final report.

Further details on these activities and timeframe requirements are at Appendix C.

Requirements for the review report

Details on the requirements for the review report are at Appendix D.

3. Review action plan

This activity consists of developing the review action plan based on the review report recommendations including the timely implementation and reporting of progress to the Accountable Officer.

Drafting the action plan

Following receipt of the final report the Accountable Officer distributes the report recommendations to the Project/Program Manager to develop an appropriate course of action. The action plan includes details on how the prioritised recommendations will be redressed, by whom and by when based on the target resolution timeframes.

4. Implementation and reporting

The implementation and reporting of progress against the recommendations is the final activity in the assurance process. The value from undertaking assurance is realised when the assurance review recommendations are fully implemented and resolved. Progress against the action plan should be tracked and included in the initiative reporting.

Progress on assurance review recommendations is considered as part of the next following gated assurance review or health check.

Portfolio assurance

A portfolio is the totality of the department’s investment in the changes required to achieve its strategic objectives (Management of Portfolios, AXELOS 2014). Business change can be delivered by a range of activities, including everything from a simple change in a business process step, through to complex initiatives. For the purposes of this framework though, a portfolio is taken to include the initiatives delivered as programs and projects.

Portfolios are different to initiatives in that they are perpetual. The portfolio composition is determined by investment decisions that enable the optimal mix of organisational change and business as usual, whilst working with available funding. Managing the portfolio includes the strategic decisions and processes to select the initiatives, control the portfolio and monitor progress.

Portfolio assurance is an optional assurance activity for departments.

Areas of focus for the review

An assurance review of a portfolio gives independent guidance to Accountable Officers, senior leaders and portfolio teams on how best to ensure the portfolio will enable the realisation of government policy, strategy and objectives and achieve value for money. A portfolio assurance review does not focus on the likelihood for successful delivery of the whole portfolio or any individual initiative within the portfolio. While the process for completing assurance reviews for portfolios, programs and projects is similar, the content and what they examine as part of the review, and the skills needed on the review team are different. Rather, a portfolio assurance review is about the department’s ability to manage a portfolio of investments and focuses on providing a realistic view on the portfolio’s ability to:

• manage the portfolio’s strategic alignment with Queensland Government objectives and appropriate delivery in line with funding and people constraints
• monitor and respond to performance and risk (threats and opportunities) across the portfolio
• build the right culture and processes to manage the portfolio; and
• understand and respond to the capability and capacity requirements for the portfolio.

Review rating

A portfolio assurance review report will have four ratings as outlined below – one for each assurance topic and one for the portfolio overall. All four ratings use the same ratings scale and are a maturity assessment combined with an assessment of how the purpose of that practice area is being achieved.

Review specifics

A portfolio review follows the same process as that for initiative reviews and involves:

• engaging a review team
• agreeing a terms of reference and holding a planning meeting
• interviewing stakeholders and reviewing documentation
• completing a review report with four ratings (see further information above) and recommendations marked as critical, essential and recommended.

A portfolio review can be undertaken at any time. There can be benefit from repeating a portfolio review at two- or three-year intervals to understand any movements in maturity.

The independent review team should consist of three to four individuals with experience in portfolio management, organisational strategy development, planning and financial management with at least one review team member holding a Practitioner certification in AXELOS Management of Portfolios (MoP®).

Further guidance

Further guidance on undertaking portfolio assurance reviews can be found in the UK government’s Infrastructure and Project Authority Portfolios and Portfolio Management Assurance Workbook.

Assurance Workbook: Portfolios and Portfolio Management - GOV.UK (www.gov.uk)

Portfolio, Program and Project Management Maturity Model (P3M3®) review

P3M3® is a framework for assessing and benchmarking your department’s current performance and developing plans for individual capability and organisational maturity improvement. The whole organisation, or specific areas of the organisation, can be assessed to determine the capability of the portfolio, program and project teams. P3M3® is unique in that it considers the whole system and not just the processes. It analyses the balance between the process, the competencies of the people who operate it, the tools that are deployed to support it and the management information used to manage delivery and improvements.

P3M3® comprises of 3 maturity sub-models - the portfolio (PfM3®), program (PgM3®) or project (PjM3®) – and you can choose to assess one, two or all three of these.

Figure 2: The P3M3^® model

Areas of focus for the review

The P3M3® model can best be thought of as a three-dimensional cube built on perspectives, threads and levels.

There are 7 perspectives. The perspectives are the main process groups and they form the basis for reporting.

• Benefits management – ensures that the desired business change outcomes have been clearly defined, are measurable and are ultimately delivered.
• Finance management – ensures that the likely costs, both capital and operational, of delivering programs and projects are captured and evaluated within a formal business case and that costs are categorised and managed.
• Management control – covers how the direction of travel is maintained throughout the project, program or portfolio management lifecycle.
• Organisational governance – looks at how the delivery of projects, programs and portfolios is aligned to the strategic direction of the organisation.
• Resource management – covers management of all types of resources required for delivery.
• Risk management – the process to systematically identify and manage opportunities and threats.
• Stakeholder management – includes communications planning, the effective identification and use of different communications channels, and techniques.

The 13 threads describe common themes across the perspectives and help to identify process strengths and weaknesses. The 13 threads are:

• asset management
• assurance
• behaviours
• commercial commissioner
• commercial deliverer
• information and knowledge management
• infrastructure and tools
• model integration
• organisation
• planning
• process
• standards
• techniques.

Review rating

A P3M3® assessment will determine a maturity assessment for each of the three sub-models. A maturity level enables the identification of an improvement pathway – this will be a long-term strategic commitment to continuous improvement. The maturity levels can be briefly summarised as follows (AXELOS P3M3^® Overview):

Review specifics

A certified assessment is undertaken by an AXELOS Consulting Partner. The AXELOS Consulting Partners can be located on the AXELOS website. Before undertaking a P3M3® assessment a few decisions will need to be made.

• Which P3M3® maturity model will be used – portfolio, program or project maturity? If the department is focused on wanting to improve project management the PjM3 model will provide all of the necessary information needed.
• Scoping the assessment – for example – projects can be delivered within different divisions of the department. Will the assessment be undertaken within one main project delivery area, or across the department?