This paper describes a proposed enhanced framework for the
management of information technology projects in the federal
government. This enhanced framework, the detailed components of
which will be developed and implemented over the next eighteen
months, is designed to ensure that government information
technology projects fully meet the needs of the business
functions they are intended to support, deliver all expected
benefits and are completed within their approved time, cost and
functionality.
The enhanced framework comprises best practices, principles,
methodologies, tools and standards that were identified in an
inter-departmental initiative and which provide solutions to
project management concerns experienced in the federal
government. The framework will continue to evolve as new ideas
and tools become available. Opportunities to incorporate new best
practices and community knowledge and experience will be sought
continuously in the future.
The enhanced framework is so-called because it is based on the
existing project management framework defined in Treasury Board
policies. Most of the proposed enhancements can be implemented
without changes to these policies while the remainder, if
adopted, will require only minor changes that will not affect the
overall thrust of these policies.
Departments are expected to adopt the enhanced framework as it
evolves. The Treasury Board has directed departments to apply
this framework to existing information technology projects as
applicable, as well as all future information technology projects
and to so attest when seeking Treasury Board project approvals.
Departments are expected to apply the elements of the framework
in an intelligent fashion appropriate to the nature, size and
risk of each project. Some elements of the enhanced framework
will require some effort and financial investment and may take
three to five years to be fully implemented in large
departments.
The government is committed to deliver its programs and
services more efficiently and effectively through the use of
information technology. However, recent cancellations of major
information technology projects together with indications that
other projects may be experiencing difficulty have raised
concerns for the achievement of this goal.
In recognition of these concerns and as part of its ongoing
responsibility for advising Ministers on major projects, the
Treasury Board Secretariat carried out a review of a sample of 25
of the government's information technology projects, with a total
estimated cost of two billion dollars, to identify business,
project management, risk management, and human resource issues
influencing their outcome.
The Auditor General also reviewed four large government
information technology systems under development as described in
the Report of the Auditor General, Chapter 12 - Systems Under
Development - Managing the Risks, released on October 5, 1995.
The Auditor General's findings concurred with those of the
Treasury Board Secretariat's review.
In addition, in 1992 Public Works and Government Services
Canada initiated a series of Common Purpose Procurement pilot
projects in an attempt to improve project success through
procurement reform. In January 1995, the department published a
summary of findings from audit reviews of this method of
procurement and its relation to the success of information
technology projects. In addition to procurement issues, these
reviews also identified a number of deficiencies in departmental
project planning, e.g., risk analysis and business-case.
While problems with information technology projects are not
new, their impact seems more pronounced. Their underlying cause
seems to be related to significant changes in the project
environment that have occurred in the last few years. These
changes include: the decentralization of the management of
information technology; the transformation of the way the
government does its business (business process re-engineering);
and rapid technological evolution from mainframe-based systems to
more complex distributed client-server systems.
The Canadian government is not alone in this regard. A recent
study of 8,380 projects in government jurisdictions and the
private sector in the United States indicated that a staggering
31% of all information technology projects are cancelled before
they are completed; 53% of those that are completed cost an
average of 189% of their original estimates and average only 42%
of the originally-proposed features and functions. Only 9% of
projects are completed on time and on budget. The lost
opportunity costs from project delays and cancellations could not
be measured but could easily be in the trillions of dollars,
e.g., the delay of the baggage-handling software for the Denver
airport cost the city $1.1 million per day.
Applying these Standish findings to the sample of twenty
government information technology projects reviewed by the
Treasury Board Secretariat would predict that seven would be
cancelled and project overruns would total over one billion
dollars. It is clear that the government must change to ensure
that this prediction does not come true.
The challenge for the government, therefore, is to enhance its
framework for managing information technology projects to resolve
this situation and ensure greater success. A Treasury Board
Secretariat Project Management Office has conducted research in
government and private sector organizations that are successfully
implementing information technology projects and, with
departments, has identified enhancements to the current framework
based on the successful practices discovered.
The enhanced framework for the management of information
technology projects is founded on the following broad
directions:
- all components of project management will be addressed;
- the correct balance will be struck between corporate
objectives and departmental empowerment and accountability;
- departments will be involved in its development and be
committed to its implementation;
- the enhanced framework will be flexible and dynamic to allow
optimum application in different situations and to permit
evolution of solutions;
- tried and true solutions from other organizations will be
adopted as much as possible; and
- departments will deploy ideas as soon as appropriate (rather
than waiting until the development of enhanced framework is
complete).
The enhanced framework will reflect the following conclusions
of the Treasury Board Secretariat's review.
- initial project definitions are inadequate;
- senior management's understanding, involvement and support
are lacking;
- client focus and involvement are insufficient; and
- Treasury Board Secretariat's submission review process is
incomplete.
- project management discipline is inconsistent;
- the project manager's experience often does not reflect the
project's size and risk; and
- changes are not rigorously managed.
- regular "sanity" checks are not performed; and
- project performance and progress is poorly measured and
reported.
- the current procurement approach is too inflexible and not
conducive to co-operation; and
- private sector consultants lack knowledge of government
organizations, responsibilities and policies.
- the present culture discourages open discussion of problems
and their resolution; and
- while project failures are well advertised, successful
projects are generally not.
The government has responded to the above findings as
follows:
- the Treasury Board Secretariat has established a Project
Management Office to develop and implement an enhanced framework
for the management of information technology projects in
government;
- an Implementation Team comprising information technology
project managers meets regularly to provide advice and review
proposals and documents from the Project Management Office;
- an Interdepartmental Committee consisting of assistant deputy
ministers responsible for information technology projects meets
regularly to provide general direction to the Project Management
Office; and
- the Project Management Office has consulted with departments,
other governments and the private sector to identify successful
practices that could be adapted for use in the federal
government.
The enhanced framework for the management of information
technology projects is described in seven key areas or "pillars"
(see Figure 1).
- Governance: an appropriate management framework that
balances the government's corporate requirements with
departmental empowerment needs.
- Review: the establishment of effective mechanisms for
monitoring the status and assessing the performance of
projects.
- Facilitation: the creation and maintenance of a set of
best practices, processes and guides that will support and assist
departments.
- Professional development: building a cadre of
information technology project managers within government who are
trained in the application of the enhanced framework and
designated common methodologies and tools.
- Pathfinder projects: exploiting opportunities to use
and assess new ideas and solutions in an operational
situation.
- Communications: initiating and supporting a
communication program on the enhanced framework across
government.
- Management of Change: introducing and acceptance of changes
in attitudes and cultures.
Four overall principles have been defined for the enhanced
framework to set the broad parameters within which information
technology projects are managed. Specific practices will be
developed to support the achievement of these principles.
Principle: Projects are aligned with, and support, the
business directions and priorities
Information technology projects are undertaken to achieve
successful and economical support of a business function. The
project sponsor, project leader and project manager, together
with the management and major users of the business function,
must ensure that the system achieves these goals and delivers the
expected benefits . The support of business needs must be
especially foremost when the system is first conceived and then
re-evaluated throughout the project.
There is a danger of this principle becoming mere rhetoric
unless there is definite commitment by management and the project
team to take it seriously.
Principle: Clear accountabilities are established
Information technology projects can be a huge, complex
undertakings. Unless the responsibilities of all parties are
clearly defined and their delegated authority clearly specified,
the project's accountabilities will be blurred and it will be
less likely that problems are solved in a timely manner to
prevent them from threatening the success of the project and the
achievement of the benefits expected from it. Departments must
establish appropriate accountability frameworks to ensure that if
problems cannot be resolved promptly they are elevated to the
appropriate higher level for resolution.
Principle: Project managers are developed in, and work within,
a corporate discipline
The project manager is the key for the successful completion
of the project and the achievement of the expected benefits. The
project manager, therefore, should have the appropriate training,
skills and experience required to manage the project's scope and
risks. The department must be able to rely on this person to know
the project's status, i.e., its progress, cost, and problems, at
all times.
Only a few departments have enough projects for them to
implement a comprehensive project manager development program.
Most have little choice but to appoint an internal manager who
may lack some of the desired training and experience or to
purchase project management expertise from the private sector.
Both of these choices increase the risk that the department will
not be able to exercise adequate planning and control over the
project. A solution to this problem is a corporate approach to
the development or acquisition of project managers based on
government-wide practices and common tools and methodologies.
Principle: Project management decisions are based on risk
management
The objective of a system development project is to develop an
information technology system that successfully supports the
business function and complete it within its planned cost and
time parameters. Typically the government has emphasized meeting
the target date with the result departments have pressed ahead to
meet commitments even when there are indications the project is
in difficulty. More attention must be given to the project's risk
at the outset and at periodic reviews during the course of the
project.
The project management framework is expected to change this.
Risk management, the determination of and resolution of all
threats to the successful achievement of project objectives
including the benefits identified in the business case, is a
primary concern of the enhanced project management framework.
Projects must be planned, organized and structured to ensure
success from their initial organization and planning through
design, development and implementation. Project management
training should place more focus on risk management.
This section describes practices to support each of the above
four principles for managing projects. Where considered
appropriate, common methodologies and tools are proposed that
will ensure a consistent, quality project management discipline
government-wide.
4.2.1.1. Projects Compatible with Business Plans and
Information Management Plans
Projects must be fully consistent with their departmental
context. The department's annual Business Plan describes the
department's mission, goals and priorities and the changes to be
made to implement these. Departments are currently required to
have an Information Management Strategy and an Information
Management Plan that describe the department's information and
technology strategies, architectures, infrastructures, and
projects. Projects must be consistent with these strategies and
plans and, for those departments that have one, their Long Term
Capital Plan. Projects must also be consistent and compatible
with ongoing or planned re-engineering initiatives designed to
change fundamental business processes.
4.2.1.2. IMPs Aligned with Blueprint and Common
Infrastructure
Departments' strategies and plans in their Information
Management Plans must be consistent with government-wide
strategies for the delivery of services, e.g., the TBS
Blueprint for Renewing Government Services Using Information
Technology. Departments must also conform with approved
common information technology architectures or infrastructures,
e.g., the public key infrastructure and infrastructure elements
provided by the Government Technology and Information Service,
unless there are over-riding operational requirements.
4.2.1.3. Full Business-case Analysis Performed
Departments should identify and select the most beneficial
investments using an institution-wide strategic planning process
based on the business-case approach. Project approval must be
based on a business-case analysis that relates each investment
directly to the business function and demonstrates the benefits
of the investment to the department or government as a whole.
The business case should be based on the full cost of the
system from initiation through development, implementation and
estimated annual cost of operation. It should be reviewed and
revalidated at each scheduled gate and whenever there is a
significant change to the project or the business function. If
the business-case of a project changes it should be re-approved
through the departmental planning and approval process.
4.2.1.4. Clients are Integral to all Project Phases
Clients of systems should be fully involved in all phases of
projects to ensure that systems meet their business requirements
in the best manner possible and the expected benefits are
obtained. The project manager must ensure that all types of
clients are represented in the project so that each client group
can influence the system's design and implementation. They should
be given a clear picture of how they will interact with the
system and what it will do for them while there is opportunity to
recommend improvements.
Clients should also be involved in the reviews at the
project's gates and subsequent decisions on the project's
future.
4.2.1.5. Client Responsibilities Defined and Committed
To
The project leader must ensure that representatives of client
groups commit to the level of effort required to meet their
project responsibilities. Their names and explicit
responsibilities should be included in the project charter. They
should not be seen merely as people to bounce ideas off or to
"approve" decisions of the project team but as full team members
who take part in all design and implementation decisions of the
project. They should be involved in the reviews at project gates
and have a major say in the release of funds to continue the
project.
4.2.2.1. Departments are Accountable for Projects
Departments are accountable for the successful completion of
projects. This accountability should be clearly re-affirmed to
Parliament, the Auditor General and the media. This will
encourage the senior management of departments to take a more
active role over their projects and the department's policies and
practices for their management.
4.2.2.2. TBS Responsible for the Project Framework
The Treasury Board Secretariat's central policy role over
projects is to establish and maintain the government's framework
for the management of projects and to monitor the departmental
implementation of this framework. The Treasury Board Secretariat
will verify that departments have implemented the framework in
order to advise Ministers on project submissions.
4.2.2.3. PWGSC Role in Contracting
Public Works and Government Services is the government's
contracting agent for goods and services. Their role with respect
to information technology projects is to devise a contracting
plan and contract terms and conditions that best meet the
requirements of the project and the application of the enhanced
framework. In order for PWGSC to achieve this without incurring
delay, departments must involve PWGSC at the beginning of the
projects. This will allow PWGSC to understand the projects needs
and the project manager to take government contracting
regulations, trade agreements and economic benefit requirements
in consideration when planning the project.
4.2.2.4. Accountabilities of Multiple Stakeholders are
Defined
Some projects are not the sole responsibility of one
department or a department's project may have requirements
attached to it by another department (e.g., economic
benefits).
All departments involved in a multi-departmental project
should understand and agree with all of the other departments'
objectives, roles and levels of participation. These should be
documented in memoranda of understanding (MOU), or a project
charter signed by a senior official of each department. All of
the departments should establish a project management structure
for their part of the project. The designated lead department
should establish an overall project office, separate from its
project team, with an overall project sponsor, project leader and
project manager (see 4.2.2.8.) staffed at a sufficiently senior
level to be able to intercede at the appropriate level in all
participating departments.
For projects to which another department has attached
requirements such as regional economic benefits, there is an onus
on both departments to ensure that these requirements are
achievable without compromising the successful completion of the
project. Where the department setting the requirements has
operational responsibilities in the area of the requirements,
that department should commit through a memorandum of
understanding to help meet them. There should be a process
whereby a department can obtain relief from the imposed
obligations on presentation of evidence that the obligations
cannot be met without compromising the success of the
project.
4.2.2.5. Organizational Readiness Determined
Departments have different experiences with regard to managing
projects. Some departments have many projects underway
simultaneously while others may only occasionally have a major
project. While the former departments can be expected to maintain
project support functions that develop departmental policy and
practices for project management, the latter may not.
Furthermore, departments may vary in the degree to which they
have implemented the management framework for projects. Thus,
departmental "readiness" to commission a major project, which is
directly related to project risk, varies widely and so a
department's authority to commence projects should take the
department's readiness into consideration.
4.2.2.6. Higher Project Authority Levels Based on
Readiness
Departments that have fully implemented the enhanced
management framework as indicated by their certification at a
level of readiness can request higher project authority levels
commensurate with the level of readiness they have achieved. In
the interim, departmental project submissions will include an
assessment of the extent to which they have implemented the
enhanced framework.
4.2.2.7. TBS Reporting Based on Readiness
Departments that have fully implemented the enhanced
management framework as indicated by their certification at a
level of readiness will have a commensurate reduction in the
central verification of their projects (i.e., projects above
their authority level). These departments will be allowed to
submit abbreviated reports to the Treasury Board Secretariat for
projects for which they have been directed to provide progress
reports, e.g., major crown projects or projects with high
risk.
4.2.2.8. Project Sponsor, Leader and Manager Roles
Defined
Three key officials should be identified for each project (for
small projects these roles could be assumed by two or even one
person; however, all of the responsibilities for the three role
are to be specifically assigned). The Project Leader's and
Project Manager's roles and responsibilities are defined in, and
required by, the Treasury Board policy, Project
Management, while the Project Sponsor role has been
recommended by the Auditor General. In summary, these
responsibilities are:
- project sponsor: is responsible for ensuring that the
department understands the value and importance of the project
and, ultimately, for realizing the benefits predicted for the
project. The project sponsor is typically a senior official in
the organization responsible for the business function that the
project will support. The project sponsor should have a major say
in the release of funds after a review gate.
- project leader: has overall responsibility for the
project and is accountable for all external and internal aspects
of it. The project leader is typically a senior departmental
official.
- project manager: has specific accountability for
achieving all of the defined project objectives within the time
and resources allocated. The project manager performs the day to
day management of the project. Without diluting the
responsibility of the overall project manager, he or she may be
supported by one or more assistant project managers who have the
same responsibilities over specific portions (sub-projects) of
the overall project. Project managers should have demonstrated
knowledge, skills and experience commensurate with the size,
complexity and risk of the project.
The specific responsibilities and obligations of these key
project officials should be documented in a formal project
charter.
4.2.2.9. Project Team has the Necessary Competencies
The project manager must ensure that the project team has all
the necessary competencies required by the project. This requires
careful determination of all the different factors in the project
that demand particular skills or expertise and then staffing the
team to include them.
4.2.2.10. Core Responsibilities and Functions not
Outsourced
Organizations that are successful in implementing information
technology projects report that they do not outsource the core
project management responsibilities and functions. They consider
that these are so vital to the project's success that they want
to ensure that these people have the requisite training, skills
and experience, and that they are under the complete control of
the organization's management. They also question the objectivity
and conflict of interest of a project manager who is an employee
of the company responsible for the project's implementation.
Currently, the government does not have enough employees with
the necessary project management training, skills and experience
to make it a policy not to outsource core project management
responsibilities and functions; however, this is a goal to work
towards as such a cadre is developed. In the meantime,
departments that must contract for core project management
responsibilities should consider acquiring them from a different
supplier than that awarded the primary development contract. When
these core project management responsibilities are outsourced,
the department must structure the contract and its internal
project management to ensure full departmental accountability is
preserved.
4.2.3.1. Project Managers Selected Against Project
Parameters
The project manager is key to the successful completion of
projects. The project manager must, therefore, have the
knowledge, skills and experience required to manage the project's
scope, complexity and risk profile. Project management should be
recognized as requiring specialized knowledge, skills and
experience and not considered a function that any manager can
perform without first acquiring the requisite knowledge, skills
and experience. Departments should perform a preliminary
assessment of the project's scope, complexity and risk before
finding and assigning a project manager who is capable to manage
it.
4.2.3.2. Consistent Project Management Discipline
Only a few departments are large enough to have a critical
mass of information technology project managers to have their own
development program, and even they could benefit from
cross-fertilisation of experiences from other departments. To
ensure that all departments employ similar standards for
information technology project management and to facilitate the
movement of project managers among departments, the government is
considering a information technology project management
discipline that is consistent across all departments.
4.2.3.3. Government-wide Project Manager Development
A single information technology project management discipline
will allow the development of government-wide training courses,
workshops and forums that should have better quality and higher
standards than an individual department could implement. This
common program will be implemented through existing government
training organizations and shared departmental programs.
In addition to formal development initiatives, an
apprenticeship program will be considered whereby beginner
project managers and project managers ready to move up to a
higher level could obtain a few months experience on existing
projects before being appointed as project managers of new
projects.
4.2.3.4. Continuous Learning Culture
An individual project manager can never experience all of the
things that can happen to projects. A project management culture
will be encouraged whereby information technology project
managers share their experiences and pass on lessons learned.
Forums, workshops and information dissemination vehicles will be
organized to support the exchange of practical project management
knowledge.
4.2.3.5. Project Manager Support Network
In addition to the general acquisition of knowledge, an
information technology project manager support network will be
established whereby project managers encountering specific
problems could get advice from other knowledgeable, skilled and
experienced project managers. This network will take the form of
peer group forums specifically set up to discuss actual problems
and propose potential solutions, and a mentor system whereby a
project manager with more experience will act as a mentor to a
project manager with less.
4.2.4.1. Existing Solutions Adopted
Adopting a proven solution that can meet the needs of the
business function is less risky than developing a custom
solution. An off-the-shelf product, or a solution that performs
similar functions and services, has been implemented in an
organization, and can be adapted to the department's needs,
offers lower cost and risk than a custom system. A custom
solution should be selected only as a last resort.
4.2.4.2. Projects Structured for Minimum Risk
The size and duration of a project are directly related to
risk. Large projects are inherently more complex and risky, and
long duration increases the risk of the business needs and
technology changing. Successful organizations implement project
structures in which large projects are reconstituted into a set
of more manageable and less risky sub-projects, each of which is
typically less than 12 months in duration and costs less than
$1 million.
For this solution to work, a management approach is required
that can effectively co-ordinate all of the individual
sub-projects, ensure communication among the different
sub-project teams, and address shared or horizontal issues.
Typically departments must:
- re-organise a large project into a number of smaller
sub-projects with sub-project teams of 10 people or fewer -
probably more suitable for a major business (re)engineering
initiative; or
- establish continuous improvement to the business process
through smaller sub-projects providing frequent upgrades to the
system - likely more suitable for re-engineering an existing
business process that does not require immediate fundamental
change.
4.2.4.3. Project Off-ramps Linked to a Gating Process
Projects, particularly long ones, must have scheduled
checkpoints or gates when the project is reviewed and management
can decide on the future of the project and any appropriate
corrective action. Note that a project "gate" is not synonymous
with the project milestones, e.g., PPA, EPA, or initial planning.
definition, implementation, and close-out. Rather gates should be
chosen to coincide with logical project review points, e.g.
completion of an activities or sub-projects, that also correspond
to relatively constant cash expenditure increments.
Gating also allows the department to control the cost of
projects and minimize the financial loss on problem projects. In
this approach, a designated senior departmental official, e.g.,
the project sponsor, manages the funds allocated to the project
and releases only the funds needed to reach the next gate to the
project leader. The performance of the project is reviewed at
each gate, or when the released funds run out (to avoid delays,
the designated official could release sufficient funds to permit
work to continue on the project for a short time while this
review and decision took place). After the review, departmental
management can decide to proceed with the project as planned,
modify the project and/or its funding, or even terminate the
project limiting the loss to the amount previously allocated.
Projects and contracts will have to be structured to avoid
incurring major penalties from the application of gating. By
requiring the contractor to provide complete information on
project performance and progress and also specifying in the
contract when the scheduled reviews are to take place, the
reviews could be conducted in a reasonable time without the need
to stop work. By specifying the option to cancel the contract at
the scheduled gates including the criteria on which such a
decision would be made, in the contract, gating can be
implemented without incurring major penalties.
After performing a risk assessment to identify the risks
involved and establishing the project's scheduled gates and
decision points, departments can develop contingency plans for
potential problems at these points. By developing contingency
plans in advance, they can design the project to reduce the
impact of switching to another contingency. These contingency
plans can be built into the contract to avoid major penalties or
having to re-tender the contract.
4.2.4.4. Methodologies and Tools
The starting point for managing risk is the determination of
threats to project success and their level of risk. With this
knowledge, departments can develop strategies and plans to deal
with the identified risks and make informed decisions on the
project. Some potential solutions are as follows.
Risk Assessment: Successfully managing risk requires a
thorough understanding of all of the risks in the project,
including: scope, complexity, extent of change in the business
function, skills and experience of the project team, the
employment of new technology, and the number of organizations
involved. A project risk assessment, such as the non-proprietary
methodology developed by the Software Engineering Institute, can
identify and quantify the risks of the project so that strategies
can be developed to control them. Since risk factors can change
over the course of the project, the risk assessment should be
performed whenever there is significant change to the project and
at scheduled times during a long project.
Determination of project complexity: project complexity
derives from the number of business rules, the technology
employed and the project's size, and is a major component of
project risk . Complexity should be determined at the start of
all large projects, and when changes occur so that appropriate
action to minimize its risk can be taken. One way to quantify
project size is function point analysis. Experience has shown
that risk increases rapidly with increasing numbers of function
points. In addition, this method also provides an estimate of
project cost early in the life-cycle. Projects should be
structured so that each sub-project is less than 1500 function
points.
Early product: when the project has a long duration,
changes may occur to the project environment, e.g., changes in
the organization of the department, changes to the business
processes, technological change, changes in project team members,
and changes affecting the contractor. These risks can be
mitigated with a project implementation strategy that produces
results in successive implementable pieces - each piece designed
to be completed in a relatively short time to provide immediate
benefit to the business process. Any environmental change would
affect only the parts already completed or in progress. There are
a number of methodologies for early product delivery, e. g.,
prototyping and rapid (architected) applications delivery (RAD or
RAAD).
Change Management: unforeseen changes to projects are
inevitable; therefore, every project needs a process to manage
functional changes. This process should ensure that changes are
analyzed quickly to determine their impact (risk, cost and time)
and that this information is brought to the attention of the
appropriate level of management as soon as possible. Contracts
should bind contractors to the department's change management
process and provide for third party intervention to resolve any
disputes over the cost to implement changes.
Performance measurement: the risk of exceeding time and
cost targets is inversely related to the amount and accuracy of
the information received by the project manager on the
performance of the project team. To know the current project
status, the project manager needs data on the time and money
expended and on the work completed at frequent intervals.
Information on time and money expended is easily obtained for
projects done in-house and should be required to be provided as
part of the contract terms and conditions. To get complete,
up-to-date data on work completed, the work needs to be broken
into distinct modules that can be completed in short time
periods. A work breakdown structure based on large items forces
the project manager to assume that work completed equals time
expended and he or she will not know, for some time, if the work
on a work item is exceeding the time forecast. With complete
information for all three parameters, the project manager knows
the status of the project against the project schedule. A
suggested performance measurement tool is one based on the
national standard, C/SPMS.
Adoption of common methodologies and tools: the
methodologies for performing risk assessment and control are so
crucial that common methodologies and tools should be considered
for use across government. Project managers trained in these
methodologies and tools could then use their skills, knowledge
and experience in any assignment in any department. In addition,
central agency staff and managers new to a particular department
will be familiar with the methodologies and tools used and will
have more confidence in the risk assessment.
4.2.4.5. Procurement Managers Involved at an Early Stage
Government procurement is complex because it is governed by
regulations designed to give equal access to government contracts
to all eligible companies, regional development policies designed
to ensure that all regions get their fair share, and
international agreements such as GATT and NAFTA. When PWGSC
procurement officers are involved early in the project planning,
they can develop a procurement process that reduces delays, and
they can design a procurement plan that best aligns the
contracting plan with the project plan while meeting legislative,
policy and treaty requirements, and which aligns the contract
with the stated benefits of the project. The project's gating
plan can be incorporated in the procurement plan and thus in the
contracts in such a way that the gating process and subsequent
decisions taken will not incur major additional costs or
penalties, or result in unnecessary contract splitting.
Departments find that if they delay involving PWGSC until the
time they want to contract the requirement, they may face long
delays to resolve issues and may not be able to structure the
contracting as they would like.
4.2.4.6. Project Review
To contain the cost and time wasted because of problems,
departments should implement activities that will help to
identify and resolve problems as early as possible. The project
manager can then investigate the problem and develop a strategy
to resolve it before it goes too far. The additional cost of
these reviews should be offset by the saving that will accrue
from early warning of project problems allowing more timely
resolution. The following types of reviews are available.
Independent reviews: These are conducted by an
independent party at scheduled gates to identify environmental
changes, overrun of time and cost targets, or other problems.
Funding should also be set aside for unscheduled independent
reviews to be undertaken whenever there are significant changes
in the project environment or serious concerns about the
project.
Internal Peer reviews: Departments engaged in several
projects simultaneously will have several project managers and
other managers in the systems development, maintenance and
operations groups. This body of expertise can be drawn on to
conduct periodic peer reviews of projects. These semi-formal
reviews allow the project manager to present performance and
progress data, to discuss upcoming challenges and to identify any
horizontal issues. The object of the peer review is for the group
to verify that the project is still on course and to provide
expert advice, council and assistance to the project manager.
Thus the combined skills and experience of all of these managers
is applied to the project.
External Peer Reviews: Departments may also draw upon
similar people in other departments or organizations to provide a
different perspective and to bring a wide range of expertise to
bear on project strategies, plans and issues.
Project team sanity checks: Another source of early
warning for project problems is the project team members. These
people are the most intimately aware of difficulties or planned
activities that may pose particular challenge. The project
manager should plan regular sessions where team members can
review the continued relevance of the project, project
performance and concerns about actual or potential problems in a
non-incriminating way.
Oversight reviews: Oversight reviews under a senior
steering committee should be planned to take place at each gate
to reconfirm that the project is aligned with departmental
priorities and directions and to advise senior management on
project progress.
Internal audit: The departmental internal audit group
can also review the performance of projects. As part of this
enhanced framework for managing projects, kits will be developed
to assist auditors to check that the framework is being applied
to the department's projects and thereby measure the health of
projects, e.g., by determining that an appropriate performance
measurement tool is being used and from the current status of the
project as shown by that tool.
Following consultation with departments, the Management of
Information Technology and Project Management policies will be
reviewed and amended, as necessary, to include recommended
changes to support the requirements of the enhanced framework.
Potential new policy directives could include:
- new projects are to be subject to a rigorous risk assessment
and appropriate measures taken to reduce and manage risk;
- a gating process is to be implemented for each project to
ensure projects are reviewed at prescheduled points with funding
for subsequent phases of projects subject to this review;
- methodologies and tools approved for common use are used to
manage projects;
- all types of users of a system are to be represented in its
planning, design and development;
- departments are to participate in a corporate approach for
project manager development;
- project managers will have the requisite skills and
experience to match the project's size, complexity and risk;
and
- departments that can attest they have fully implemented the
enhanced framework and have been certified at a high level of
readiness can request a higher project authority level. In the
interim, departmental project submissions will include an
assessment of the extent to which they have implemented the
enhanced framework.
4.3.2.1. Information Technology Procurement Review
Purpose: Building on the review of the Common Purpose
Procurement approach, this review of information technology
procurement will identify and resolve procurement policy and
process problems impeding the timely and cost-effective execution
of large information technology acquisition projects.
Scope: The review will identify and resolve issues that
can be dealt with within current legislation governing federal
acquisition and the constraints imposed by international
agreements; however, issues that require legislative changes to
resolve will be identified.
Topics: The findings of the Treasury Board
Secretariat's review of a sample of projects identified the
following procurement-related issues:
- requirements imposed by the Procurement Review Committee,
Treasury Board Secretariat and other government departments may
add significantly to project risk. Means by which departments
responsible for these requirements can be held accountable should
be examined;
- in order to reduce risk, the evaluation of bid responses
should emphasize the best overall solution;
- the approval, contracting and, in particular, contract
amendment processes can absorb excessive amounts of the project
team members' time. These processes should be reviewed and
streamlined;
- although flaws were identified in the Common Purpose
Procurement approach and methodology, its principles are
recognized internationally and it provides valuable improvement
over traditional acquisition approaches. The optimization of this
approach, now referred to as Benefits-Driven Procurement
including the examination and incorporation of enhanced
partnering arrangements with industry, will be pursued; and
- competitive project definition for requirements
specifications is a valid option but requires certain refinements
to reduce potential risks.
More recent research indicates that:
- other innovative approaches to acquisition, including
Evolutionary Acquisition, New Wave (or Best Value) Procurement
should be examined;
- project managers need instruction in current procurement
practices and policy, and the implications of international
agreements (NAFTA, GATT) to help them to plan for dealing with
them; and
- procurement personnel need to acquire more knowledge and
exposure to project activities in order to better understand the
special issues of systems development contracts.
4.3.2.2. Risk Assessment
As stated earlier, the key to successfully managing risk is a
thorough understanding of all of the risks in the project,
including: scope, complexity, the extent of change in the
business function, skills and experience of the project team,
employment of new technology, the number of organizations
involved, etc. In addition, the methodologies for performing risk
assessment are sufficiently crucial that common methodologies and
tools should be adopted across government.
It is proposed that the government phase in standard
approaches to risk assessment and risk management in the
government and for government contractors. Starting with the
implementation of the Applied Software Engineering Centre's
(ASEC) comprehensive S:PRIME approach as a base, departments
would migrate, as appropriate, to the Software Engineering
Institute's (SEI) Team Risk Management process followed, also as
appropriate, by SEI's Capability Maturity Model (CMM) and Project
Risk Assessment Model. It should be noted that the ASEC and SEI
approaches are fully compatible since ASEC's approach is a
streamlined version of SEI's and is based directly on the SEI
models.
The Applied Software Engineering Centre's approach is basic,
concise and cheaper and is well suited to organizations with
between 10 and 50 professionals directly or indirectly involved
in software development and maintenance. Due to its easier
implementation, it can be used to "jump start" the adoption of a
proactive risk management regime throughout government. The SEI's
robust and complex models are well suited to very large systems
organizations, and offer a natural, straightforward evolutionary
path from ASEC's S:PRIME base as and when appropriate.
Industry representatives will be consulted with regard to
their implementation of these assessment tools. They will be
advised that, as the government implements them, they will be
expected to evolve towards them as well so that both use the same
assessment standards. Eventually the achievement of a specified
maturity level could be mandatory for both government departments
and potential contractors.
4.3.2.3. Professional Development of Information Technology
Project Managers
Recognising that the experience and skills of the project
manager are key to the successful completion of projects, a
government-wide approach to the training and development of
information technology project managers is proposed with the
following objectives:
- to build and maintain a cadre of project managers trained in
the application of the enhanced framework for the management of
information technology projects;
- to encourage a continuous learning culture;
- to facilitate an exchange and sharing of information with
industry; and
- to provide a forum to discuss project management issues with
peers.
Content: the proposed approach will be based on common
government practices, methodologies and tools. It will consist of
two interrelated components:
- a training program consisting of workshops, forums and
accredited training courses divided into five competency areas:
project, technology, procurement, interpersonal skills and
business. This training will cover: aligning projects with
business needs, directions and priorities; assessing and managing
project risks; selecting and awarding contracts; resolving
project management issues and problems; employing best project
management practices; sharing information on government and
industry strategic directions and trends; and enhancing
interpersonal skills; and
- a career development program providing development
opportunities through on-the-job training, public and private
sector rotational assignments and secondments.
Participants: program participants will be divided into
four groups in order to address their specific training and
development needs:
- (Intern) project managers who are at the start of their
project management career path;
- (Professional) project managers with some experience managing
medium-sized information technology projects and are responsible
for project deliverables;
- (Master) project managers with more extensive experience who
have had responsibility for the management of larger, more
complex information technology projects; and
- project leaders at all experience levels.
4.3.2.4. Pathfinders
Pathfinder projects provide opportunities to verify and extend
the conceptual management framework for information technology
projects. Two existing departmental initiatives have been
selected as initial pathfinder projects:
- the Immigration Modernization Program in Citizenship &
Immigration is using a number of innovative procurement
approaches for the acquisition of development and integration
services, for infrastructure consolidation, and for key human
resources. The evaluation of these approaches will provide
information on the possible need for process change, improved
training, and further application in government. The department
also intends to use a peer review approach at key milestones to
ensure the necessary independent perspective is brought to bear
on the progress of the project. This aspect of the project will
provide a vehicle for determining the peer review process and
assessing the relevance of such an approach in a complex project;
and
- the implementation of the PeopleSoft human resources
software package involves some sixteen departments and
consequently will present a major challenge to the government to
orchestrate its implementation and on-going maintenance. This
initiative will be used to assess the issues involved in the
conduct of independent reviews, e.g., positioning of a review,
scoping of the work and the methodology used, and to address
problems associated with the accountabilities of stakeholders,
e.g., departmental users, application managers and service
providers.
4.3.2.5. Common Tools
The enhanced framework for the management of information
technology projects is expected to include a number of best
practices, guidelines and common methodologies and tools.
Initially, departments will be asked to provide examples that
they use in the management of their projects that can be adapted
for government-wide use. The remaining tools will have to be
developed.
The enhanced framework for the management of information
technology projects is being applied as it is being developed.
The Treasury Board has directed departments to apply this
framework to their existing information technology projects as
applicable, and to all future information technology projects and
to so attest when seeking Treasury Board project approvals.
Departments are at present applying the concepts espoused in this
document; this application will continue to improve as supporting
products and services are developed.
The Treasury Board Secretariat Project Management Office is
facilitating this application of the evolving framework in two
ways:
Communication and Consulting: the Treasury Board
Secretariat Project Management Office meets regularly with
members of the Implementation Team and when needed with the
Interdepartmental Committee to advise on its findings from
meetings with officials of other government jurisdictions and the
private sector. Often members of these organizations are brought
to Ottawa to make presentations to federal officials.
Project Review and Advice: Treasury Board Secretariat
Project Management Office members are also working with
departments with large information technology projects and are
advising them on the concepts of the proposed framework as
applicable. These departments are starting to perform risk
assessments, to structure their projects into smaller more
manageable sub-projects and to implement gating processes.
1996/97 Development: In 1996/97 the Treasury Board
Secretariat will lead the development of policy changes and
government-wide practices making up the enhanced framework. The
following priority work items will be carried out:
- necessary policy changes will be formulated;
- a review of information technology procurement will be
performed;
- a common methodology for project risk assessment will be
adopted;
- best practices (methodologies and techniques) will be
developed, or adopted (project manager's tool kit);
- the development of a government-wide approach to training and
developing project leaders and project managers will begin;
- a management of change program will be developed to ensure
the effective implementation and use of the enhanced framework;
and
- an inventory of project managers and large information
technology projects will be established.
1996/97 Implementation: During 1996/97 the new
framework will be implemented in departments as components are
completed. The following schedule is anticipated:
- a common methodology for assessing project risk will be
selected and two departmental persons will be trained in its
use;
- a common tool for assessing project size (function point
analysis) will be selected and departments will have staff
trained in this technique and will begin to implement it in their
departments;
- a common tool for project performance and progress tracking
(based on the national standard, C/SPMS) has been adopted,
departments will train their staff on its use and continue to
implement it; and
- departments are expected to conduct three independent project
reviews.
1996/97 Costs: The estimated costs to the Treasury
Board Secretariat in 1996/97 to perform this work are:
- procurement
review |
$100K
|
- risk assessment
review |
$20K
|
- project manager's tool
kit |
$50K
|
- risk assessment
training |
$50K
|
- management of change
program |
$50K
|
- project manager course
development |
$125K
|
- salary and benefits
for 5 people (in addition to internal TBS resources) |
$360K
|
TOTAL |
$755K
|
In addition, departments will incur costs in 1996/97 which
will be absorbed within existing project budgets:
- risk assessment training |
$25K
|
- progress tracking training |
$25K
|
- conduct two risk assessments |
$50K
|
- conduct three independent
reviews |
$150K
|
TOTAL |
$250K
|
1997/98 Development The Treasury Board Secretariat will
continue the development (begun in 1996/97) of a government-wide
approach to training and developing project leaders and project
managers and will conduct 2 pilot courses.
1997/98 Implementation: Departments are expected to
undertake the following implementation activities in 1997/98:
- twenty-four departmental project managers are expected to
attend the above projects leader and project manager pilot
courses; and
- departments are expected to conduct ten risk assessments and
five independent reviews.
1997/98 Costs: The estimated costs to the Treasury
Board Secretariat in 1997/98 to perform this work are:
- project manager course
development |
$125K
|
- salary and benefits for 5 people
(in addition to internal TBS resources) |
$360K
|
TOTAL |
$485K
|
Departments will incur the following costs in 1997/98 which
will be absorbed within existing project budgets:
- professional development course
fees (24 persons) |
$100K
|
- ten risk assessments |
$250K
|
- five independent reviews |
$250K
|
TOTAL |
$600K
|
The Treasury Board Secretariat is not expected to incur any
additional costs to monitor and maintain the framework.
The ongoing annual costs to departments from 1998/99 on that
will be absorbed within project budgets are estimated to be:
- 25 risk assessments |
$625K
|
- five independent reviews |
$250K
|
- professional development course
fees (50 persons) |
$175K
|
TOTAL |
$1,050K
|
Stage
|
1996/1997
|
1997/1998
|
1998/1999 on
|
Develop
|
TBS:
Professional Development $125K
Procurement Review $100K
Risk Assessment $20K
Project Manager's Tool Kit $50K
Management of Change $50K
Depts: na
|
TBS:
Professional Development $125K
Depts: na
|
TBS: na
Depts: na
|
Implement
|
TBS:
3 Additional People $216K
Risk Assessment Training $50K
Depts:
C/SPMS Training $25K
Function Point Analysis Training $25K
|
TBS:
2 Additional People $144K
Depts:
PD Pilots (2x12) $100K
|
TBS: na
Depts: na
|
Operation
|
TBS:
2 Additional People $144K
Depts:
2 Risk Assessments $50K
3 Independent Reviews $150K
|
TBS:
3 Additional People $216K
Depts:
RAs (10) $250K
IRs (5) $250K
|
TBS: na
Depts:
RAs (25) $625K
IRs (5) $250K
PD (4x12) $175K
|
IRs - independent reviews
RAs - risk assessments
PD - project manager professional development (courses x
participants)
|
The findings of the Standish Group International Inc. were
presented at the beginning of this report. If these findings were
to apply to the sample of 25 government information technology
projects estimated to cost two billion dollars that were reviewed
by the Treasury Board Secretariat, as many as seven will be
cancelled and project overruns will exceed one billion dollars.
Given that, in total, there are about 80 large information
technology projects in government then, based on the Standish
prediction, the potential for loss could be over three billion
dollars.
Although the government's record so far does not support the
prediction of loss of this magnitude, it is clear that
significant funds are at risk. Even if the enhanced framework
only succeeds in saving a fraction of the funds at risk, the
potential return on its investment is hundreds of millions of
dollars! The expectation, of course is that, while nothing can
totally eliminate risk from large, complex undertakings, this
enhanced framework will significantly reduce project overruns,
ensure project cancellations are rare and, when they do occur,
they do so much sooner in the project life-cycle when less money
has been spent on them.
|