Guide to Financial Management Benchmarking

ACKNOWLEDGEMENTS

Benchmarking - a management tool that can help organizations improve the efficiency and effectiveness of their procedures and processes - has been in evidence for several years now, both in the private and public sectors.

When we set out to produce this Guide, we found an extensive body of research and literature that defines benchmarking. While the literature is varied, researchers and authors tend to use similar elements when describing the process of benchmarking. Rather than recreate existing benchmarking processes, we borrowed ideas from recognized works and adapted them to best fit Financial Management Benchmarking in the Canadian federal public service.

In particular, we wish to acknowledge the contribution of ideas from the following sources:

EXECUTIVE SUMMARY

The Report of the Independent Review Panel on Modernization of Comptrollership in the Government of Canada recommends that departments and agencies have standards for performance information and a decision-making framework that encourages continuous improvement.

Benchmarking, which can be used in the public sector, is an example of a continuous improvement tool that can:

• Provide meaningful performance information.

• Improve strategic planning and provide an assessment of the organization's strengths and weaknesses.

• Establish challenging performance goals and stimulate better financial management.

• Foster implementation of best practices and lead to increased efficiency in the use of resources.

This type of continuous improvement can be achieved in an environment that embraces the process of benchmarking and the adoption of best practices. The art of benchmarking in the public sector is in its infancy. While private industry has used benchmarking as an improvement tool for many years now, the public sector has been slower to do so. Nevertheless, as indicated in the references and Web sites listed at the end of the Guide, interest is mounting.

The Guide to Financial Management Benchmarking was developed following consultations with departments and agencies on the state of, and interest in, financial management benchmarking across government. The purpose of the Guide is to ensure a common understanding of the concept across government and to provide practical advice to those wishing to embark upon a benchmarking exercise. Roles and responsibilities are described, as are lessons learned by organizations that have already used benchmarking.

The core content of the Guide is its five-phase generic model and the practical case study, which applies and demonstrates the model phases. The planning phase is crucial to the success of a benchmarking exercise. This is where you define what is to be benchmarked, who the best performers are to provide good comparisons, and how the data will be collected. The establishment of a benchmarking team and identification of performance indicators also take place during the planning phase. The process then moves to the data gathering phase using questionnaires, surveys and site visits. The data are analyzed and presented in a form that helps in drawing conclusions and recommendations. Implementation of a performance improvement plan and regular monitoring are part of the fourth phase in the process. Finally, periodically revisiting the performance indicators and benchmarks ensures that the organization maintains superior performance in a changing environment.

Implementation of benchmarking in the public sector presents many challenges. Each department will need to decide when is the right time for them to proceed with financial management benchmarking. Most financial managers and officers will need training to be able to use benchmarking effectively. It is our intention that this Guide provides guidance and advice, while allowing the readers to formulate their own opinions about detailed implementation. It is hoped that this will encourage the use of benchmarking as an important tool for improving and modernizing financial management practices in departments and agencies.

1. INTRODUCTION

1.1 Background

The Report of the Independent Review Panel on Modernization of Comptrollership in the Government of Canada lists, as one of the critical prerequisites of modern comptrollership, the need to have standards for performance information that are adaptable to the requirements of departments. The Panel Report further emphasizes that part of good comptrollership involves a decision-making framework, which includes appropriate benchmarking and performance measurement. As well, it stipulates that Deputy Heads and their staff would welcome and expect advice in areas such as benchmarking techniques.

Benchmarking provides a useful tool that may be used to assist in the achievement of modern comptrollership by identifying ways to develop financial management capabilities and strengthen key financial management functions such as budgeting, forecasting, cost, financial and performance analyses.

Some work has been done in the past at the Treasury Board of Canada Secretariat (TBS) on benchmarking. For example, the Innovative and Quality Services Group published a series of Guides on Quality Services. Three of those guides contain, directly or indirectly, information regarding benchmarking.

This Guide is generic and could apply to all benchmarking, but our focus is on benchmarking financial management functions (as illustrated by the example in the case study). The Guide was developed following a series of studies conducted by the firm PricewaterhouseCoopers under the direction of the Financial Management Policy Division of TBS. Part of the studies included consultations with representatives from TBS, departments who are members of the Comptrollership Council and other departments who agreed to participate in the projects. The Guide would provide departments and agencies, wishing to undertake financial management benchmarking, with information on how benchmarking of financial functions works, how it helps and what the benefits are.

1.2 Why is a Guide necessary?

Benchmarking is not easy. Many initiatives yield inappropriate comparisons or inconclusive data, leading to ill-conceived or unsuccessful improvement measures. Others drain resources with poorly planned and managed benchmarking projects or by taking on projects that are too big in scope. How does the committed manager direct and guide his or her organization's resources to successfully benchmark financial management?

Benchmarking involves rigorous self-examination, careful quantification and qualification of important performance measurements, extensive data collection and analysis, and the development of a process for continuous improvement. Improvement opportunities often involve major cultural or operational changes for the organization.

However, the benefits of financial management benchmarking far outweigh the costs and risks involved. Identifying strengths and weaknesses, increasing client satisfaction, prioritizing improvement opportunities, setting goals and developing a climate of continuous change are all marks of the successful organization, the one against which other organizations benchmark.

The main purpose of this Guide is to ensure a common understanding of the concept of financial management benchmarking across government. It is intended to be a tool that provides guidance and advice. While this Guide is designed to provide a blueprint for success in this effort, it is not mandatory. The Guide includes a process designed to provide consistency across government while remaining flexible enough to ensure that it can be adapted and used by financial, functional and operational managers.

The model recommended in this Guide provides managers and employees with a description of the suggested financial management benchmarking process, as well as practical guidance on its application. Fifteen private and public sector models were combined and then refined to obtain a model best suited for the needs of the federal government. A case study illustrates, through a practical example, how to apply the recommended model.

2. OVERVIEW OF BENCHMARKING

2.1 Definitions

Benchmarking, best practices and related concepts mean different things to different people in different contexts. Definitions of each of the key concepts were developed, through the literature review process, to ensure a clear and common understanding of the subject matter. The definitions are provided below.

Table 1 - Definitions of Important Terms

2.2 Benefits of benchmarking

Benchmarking is a tool that provides goals for realistic improvement and helps you understand the changes required for improving performance. You may use benchmarking to identify and rectify problems, implement strategic change initiatives, or for continuous improvement. "In the private sector, the primary rationale for benchmarking is the desire to maintain or regain a competitive market position. While most public sector departments and agencies do not actively compete for market share, there are equally valid reasons to consider benchmarking as a public sector management improvement technique" (Bibliography #13, Chapter 1, p. 1). (1)

Provide meaningful performance information

"All levels of government need reliable ways of assessing the relative performance of public programs in order to be able to set overall priorities and strategies. Benchmarking can assist public sector managers improve the quality of their performance information. Such improvements can, in turn, help organizations better meet external and internal accountability requirements. Benchmarking information often adds an important comparative perspective to organizational outputs. Specifically, some data may only be valuable when compared through time or with other organizations..." (#13, ch. 1, p. 2).

Improve strategic planning and provide an assessment of the organization's strengths and weaknesses

The organization can learn how to plan for the long term more effectively by seeing how other organizations have reached better levels of performance through their own strategic planning. Benchmarking allows management to determine where major problems lie, and what can be done to strengthen weak areas. Areas of excellence will also surface, enabling the organization to continue with what it is doing well.

Establish challenging performance goals and stimulate better performance

"Benchmarking is all about comparison, and comparison can be a driving force to spur on organizational or individual performance" (#13, ch. 1, p. 1). Realizing what an organization is doing wrong, or could do better, leads to easier planning for future target performance levels. Management will know where it stands in terms of performance and what has to be done to get where it wants to be. This should result in more realistic goals being set.

Benchmarking of activities or functions can help senior managers and staff determine how organizations and programs are performing in relation to the leading organizations in their field. The technique can uncover new and creative ideas to assist in performance improvement. Benchmarking serves as a tool, among others, to assist managers in their mandate to modernize and improve financial management in the federal government.

Foster implementation of best practices and lead to significant savings

Benchmarking and other comparative information can be used to address pressures by identifying ways to streamline processes, or opportunities to improve the allocation of resources. The implementation of best practices found in other organizations through benchmarking will help the organization become more efficient and effective.

2.3 Practices that promote effective benchmarking in the public sector

Successful benchmarking can be achieved in the public sector if there is a conscious effort to create the right environment and build a culture of continuous learning and improvement. The following ideas provide a blueprint for fostering success.

Appropriate environment

• Establish and sustain an environment across government and within organizations that embraces the process of benchmarking and the sharing of best practices.

• Secure endorsement and resource commitments from all levels of management for a quality service strategy incorporating benchmarking and best practices.

• Promote government-wide and departmental sharing of benchmarking and best practice experience and information.

• Promote benchmarking and best practices partnerships and alliances across government, within federal departments, and with other public and private sector organizations.

Dissemination of information

• Disseminate and share best practice and benchmarking information and results in a timely, accessible, user-friendly and efficient manner.

• Publicize and support the sharing of benchmarking and best practices through various media.

• Identify high-profile benchmarking and best practice pilot projects to demonstrate process effectiveness.

• Build benchmarking and best practice sharing into training programs.

Ongoing improvement in benchmarking process

• Assess progress in implementing benchmarking and best practice sharing and continuously improve benchmarking and best practice processes.

• Assess the effectiveness and efficiency of the different means of benchmarking and best practice sharing, and of improvements to operational results.

• Conduct benchmarking and best practice user consultations to encourage managers, the professionals who support them and other employees to continuously improve the means for capturing, disseminating and sharing information on benchmarking and best practices.

Note: It should be noted that departments are currently involved in the implementation of the Financial Information Strategy (FIS). The FIS initiative seeks to fundamentally change the way in which government manages its financial information. Some departments may decide to wait until some of these changes are in place before embarking on extensive benchmarking.

3. ROLES AND RESPONSIBILITIES

This section describes the roles and responsibilities of various players in the government context as they relate to financial management benchmarking. It should be noted that financial management benchmarking is one of several tools available to assist managers in carrying out their functions.

3.1 Treasury Board of Canada Secretariat

• Develop guidance on financial management benchmarking, as required.

• Facilitate the conduct of benchmarking studies.

• Facilitate the sharing of information on benchmarks and best practices.

• Advise on the availability of training in benchmarking.

3.2 Senior departmental and operational managers

• Provide leadership and communicate effectively about benchmarking with all staff.

• Set corporate culture and ensure credibility, respect and trust for benchmarking activities.

• Provide necessary resources and commitment.

• Translate benchmarking objectives to departmental staff.

• Develop partnerships with organizations to benchmark with.

• Approve benchmarking measures and standards.

• Implement data gathering tools for benchmarking.

• Incorporate benchmarking as a key ingredient in the organization's Total Quality Management (TQM) and Continuous Process Improvement (CPI) programs, and in its strategic planning and budgeting process.

• Provide effective input to decision making and use benchmarking information to achieve better results.

• Implement agreed to changes following benchmarking studies.

3.3 Financial and audit managers

• Assist in setting the financial management benchmarking priorities.

• Provide leadership in initiating the financial management benchmarking projects, the agreement on organizations to be studied and team members' roles.

• Inform employees and other key players in the organization of the objectives and processes involved in conducting financial management benchmarking.

• Ensure organizational awareness of the existence of financial management benchmarks, performance gaps and opportunities to close the gaps.

• Identify the cost and benefits of changes based on the performance gaps.

• Develop financial and performance measures to monitor progress against the benchmark standards.

3.4 All employees

• Demonstrate expertise in an area of work.

• Balance independent judgement and effective teamwork.

• Show open-mindedness regarding exchanging information.

• Participate effectively in data gathering for benchmarking purposes.

• Participate in implementation of changes following benchmarking studies.

4. QUESTIONS TO ASK BEFORE YOU START

4.1 What, and when, should we benchmark?

Generally speaking, you should choose to benchmark financial management functions that are important to the effectiveness of your organization and to your ability to meet your organizational strategic objectives.

Benchmarking should be an integral part of your overall management strategy. The timing of benchmarking efforts is critical. Benchmarking during times of major changes may lead to inaccurate benchmark data.

4.2 Do we have the resources to do the benchmarking and implement the outcomes?

There need to be adequate resources (people, time, and funding) for the benchmarking project to be planned and carried out successfully. Once the scope of the project is defined, resources will be needed to gather the data. One creative approach may be to create an interdepartmental consortium to jointly fund the initiative, and share the effort needed to plan and launch a benchmarking study.

In addition to the cost of benchmarking, there are costs associated with implementing the results of a benchmarking study. If the results of the study are not acted upon due to lack of resources, there is risk that the process could be of no value.

4.3 What type of benchmarking is appropriate?

Benchmarking is a generic term that can take a number of different forms:

"Results benchmarking involves comparing two or more organizational outcomes against ... related performance indicators. These indicators are not usually general standards (i.e., goals to be achieved), but more often, a suite of measures or proxies used by organizations to ascertain organizational or program efficiency and/or effectiveness. The focus on outcomes does not mean that questions relating to the efficiency and effectiveness of internal processes are less important" (#13, ch. 1, p. 3).

Specific examples of financial results benchmarking include the cost of departmental financial services as a per cent of the overall departmental operating budget and percentage of overall finance staff effort devoted to transaction processing versus operational decision-making support.

Process benchmarking involves benchmarking results-oriented organizational processes. Processes are defined as particular methods or tasks undertaken by selected work units. This typically involves a number of discrete steps or operations. What is commonly referred to as formal work procedures and rules often influence a process. The goal of process benchmarking is to improve the quality or quantity of an organization's outputs. Improvements realized can have a direct bearing on organizational or program outcomes (#13, ch. 1, p. 2).

Examples of financial process benchmarking include, but are not limited to the expenditure cycle, revenue cycle, travel, financial planning and analysis, and general accounting and reporting.

In effect, results benchmarking could be a precursor to performing process benchmarking by first taking a macro followed by a micro view of an organization. Results benchmarking is the starting point for determining the underlying reasons for performance differences and may lead to process benchmarking to measure differences in processes.

Benchmarks are a form of standard, which can be defined as goals against which actual performance can be measured. Benchmarks are usually categorized into minimum, average and best practice standards defined as follows:

• Minimum standards imply a level of service or performance that will almost always be achievable.

• Average standards comprise the mathematical average of the level of service provided to all users as well as efficiency and quality of service factors such as error rates and/or number of transactions processed per full time equivalent (FTE).

• Best practice standards are used to define an expected level of service and to motivate staff to achieve a higher level of performance or productivity (#13, ch. 1, p. 3).

4.4 Who should we compare ourselves to?

There are a variety of ways to identify who to compare your organization to. The financial management process(es) you decide to benchmark determines, more than anything else, which organizations you should choose as benchmarking partners.

Organizations looking to conduct benchmarking studies can choose among external benchmarks, internal benchmarks or analogous benchmarks.

External benchmarking identifies the product, services and work processes of organizations providing similar products or services and compares their results. It is a useful lever to highlight areas in need of improvement within the organization (#12, p. 5).

Benchmarking against other federal government departments, other levels of government (such as provincial or municipal) as well as the private sector would constitute external benchmarking.

Internal benchmarking compares an organization's own similar processes or products. This is the easiest type of benchmarking to perform, but is limited to the organization's best internal practices. It should be considered as a means of establishing a baseline performance that will later be used for comparison to external performance and to identify the scope of improvement opportunities (#12, p. 5).

An example of internal benchmarking in the public sector would be a decentralized department, which compares financial management processes conducted in its regional offices.

Analogous benchmarking is considered the most difficult and most desirable type of benchmarking as comparison is made with a world-class organization which may be performing a similar process but in a different field. Such organizations are hard to identify and may require adjustment in accounting and other practices (#12, p. 5).

4.5 Should we use consultants?

Consultants can provide help and add credibility for benchmarking surveys in small- and medium-sized organizations that might find it difficult to conduct the activity in-house. For instance, consultants can provide the necessary benchmarking training. There may be greater confidence in the results if the benchmarking is carried out by a third party that is perceived as objective and independent.

The information should be provided in the form of a customized benchmarking report for your organization that compares your performance against similar or suitable organizations for benchmarking purposes. You benefit from accessing key data normally constrained by confidentiality issues, and no reference is made to either organizations or individuals in the presentation of results, thus maintaining anonymity.

5. THE BENCHMARKING PROCESS

"Benchmarking is a deliberate, time consuming process requiring organizational discipline, and a strong and active commitment from senior management. The desire to close any performance gap requires tough operational, organizational and resource allocation choices, which must be supported by upper management, and backed by a willingness to adapt and learn from others" (#12, p. 5).

This section is a useful guide for financial and program managers intent on effectively conducting a financial management benchmarking study and seeking to accelerate the introduction of benchmarking into their organizations.

Its aim is to help participants understand how to:

• identify benchmarking projects that are consistent with senior management's strategic plans;

• analyze a process flowing across functional areas;

• structure an approach to effectively gather "best-in-class" information;

• develop a framework for identifying, organizing and analyzing process performance; and

• analyze benchmarking data effectively to compare performance measures and underlying causes and enablers.

Leading practitioners have formalized the benchmarking process into five general phases:

• planning

• data gathering

• analysis and integration

• implementation/execution

• recalibration.

Each phase requires specific actions to be completed, and although an organization can modify them to suit its specific situation, they are recommended as guides for the successful implementation of the overall benchmarking process.

5.1 Planning

"This phase is the most crucial and its objectives are to identify what is to be benchmarked, who the best performers are, and how the data will be collected" (#12, p. 5).

Identify and prioritize the areas to be benchmarked

It is important to have an understanding of your organization's existing internal processes, products, and services before commencing a benchmarking study. A thorough knowledge and understanding of the internal environment is critical for several reasons:

• it positions the organization to readily determine the potential gaps between its outcomes or activities and those of best practices organizations;

• it may reveal important sources of information and assistance as well as benchmarking opportunities;

• it facilitates the selection of appropriate benchmarking partners and meaningful performance indicators; and

• the exchange of information is the cornerstone of benchmarking. Without an internal review, an organization may not be able to engage in the meaningful exchange of information and, as a result, may encounter difficulties in obtaining information from others.

In the case of process benchmarking, determining which processes should be benchmarked and in what priority, is accomplished by answering the following questions:

• Is the process key to the success of the organization?

• Is the process a significant drain on resources?

• Is the process perceived to be either overstaffed or underperforming? (#2, ch. 3, p. 11)

• Is the process stable (i.e., not being redesigned or changed)?

• Is the process high profile and viewed as a candidate for improvement?

It is important not to try to benchmark too many activities at once and to start with two or three of the highest priorities. In many cases, the number of functions to be benchmarked may be obvious or established by previous research. Critical success factors, products, services, and processes are examined and a decision criterion is used to select the processes requiring immediate benchmarking focus.

Identify the internal clients of benchmarking, their requirements, the outputs of the benchmarking effort and gain the endorsement of management benchmarking, their requirements, the outputs of the benchmarkingeffort and gain the endorsement of management

Managers of individual processes are typically focused on those processes that affect their operation. A clear understanding of what use will be made of benchmarking information is critical to the success of the project. An organization will need to set parameters around the resources established for the benchmarking project such as the available time, money and personnel devoted to the task. At this stage, it is also useful to consider how benchmarking results will be used as this may indicate what should be benchmarked and the level of detail required.

Gaining the support and "buy-in" of senior management is crucial at this point. While senior management commitment is critical to ensuring that the project results are implemented, the day-to-day efforts associated with the benchmarking initiative will need to be driven by the managers of the financial functions to be benchmarked.

Establish the benchmarking or process improvement team benchmarking or process improvement team

Forming a project team with the knowledge and capacity for planning, communicating the results and implementing the findings is critical for setting a clear and concise direction for the project. This also includes setting limits on the number of benchmarking team members and estimating the number of processes to be benchmarked. "The team should be made up of individuals who are most knowledgeable about internal operations, are more likely to be affected by the changes due to benchmarking, and who are flexible and open to change. Eventually, these team members will become the change agents in the organization" (#12, p. 6).

Manage the change

As with any new initiative or undertaking, unless skilfully managed and implemented, benchmarking will not necessarily produce the desired results. Benchmarking involves changes to processes and ways of doing business which may be met with scepticism and resistance on the part of staff. There are always choices about how to move through the change process. Sometimes it makes sense to move quickly; whereas at other times it is preferable to invest time in crafting the change process carefully to minimize disruption and cost.

To be effective and increase the likelihood of success of the financial management benchmarking exercise, the benchmarking team needs to formulate a change management plan and update this plan periodically. The key components of this plan should include:

• clear articulation of benchmarking objectives and their linkage to the overall departmental as well as finance organization vision;

• strong and visible senior management commitment throughout the process;

• identification of a critical mass of people from all levels of the finance organization who will push for the change and demonstrate ownership and commitment;

• honest, open and regular communications with staff; and

• a clear migration approach from the current to the future state based on benchmarking results.

Although included as part of the planning phase, change management is an activity that would need to occur throughout the benchmarking process.

Review and document processes

The analysis of processes chosen for benchmarking is a pivotal part of the benchmarking exercise. Inadequate collection of information on processes and sub-processes may ultimately limit successful analysis and the overall usefulness of the exercise. The objective of this step is to identify the primary characteristics of one or more processes, including enablers such as tools and systems associated with specific processes. This can be accomplished through process mapping, which involves charting the sequence of events or activities associated with the delivery of a service or product. Process mapping can be initially undertaken at a fairly high level with a focus on the main business processes. Subsequently, the high-level maps can be dissected into varying levels of detail until the desired level of detail is obtained for all activities related to the target processes. In some instances, the mapping process may point to immediate improvement opportunities.

Define the relevant benchmarking measurements benchmarking measurements

Selecting performance indicators includes defining what is to be measured and how it is to be measured in very clear terms. This step determines the criteria for how an organization's performance will be assessed against the performance of others. A balanced set of measures or family of indicators which reflects the perspectives of clients (both internal and external), stakeholders and employees is necessary to properly gauge performance.

Most processes are measurable but the challenge lies in identifying the "right" measures. Sometimes, it may not be possible to develop meaningful indicators for certain processes such as those related to strategic planning or the provision of financial advice. In these instances, an option is to adopt a case study approach to identify lessons learned. Measuring the quality of the outputs of a process should also be taken into consideration. For instance, the timeliness of producing certain types of financial information may be improved, but the cost may be prohibitive and the gains are not appropriate for the increase in cost.

When selecting performance indicators it is important to consider a range of factors such as alignment with organizational objectives and priorities, potential impacts on employee performance, stakeholder requirements and resource implications. The performance indicators most commonly used for benchmarking studies consist of ratios or percentages. Qualitative measures of performance (timeliness, client satisfaction, etc.) should be developed whenever possible.

Depending on the nature of the services provided and the objectives of departmental programs, it may be important to link indicators to client needs and expectations. When establishing the indicators, it is important to note that differences between organizations may diminish the validity of comparisons based on simple ratios. For example, differences in resources used to carry out transactions may limit the value of comparisons based on the cost per transaction. In such circumstances, it is preferable to define performance indicators more broadly by including a range of factors such as transaction complexity, the value of invoices and number of customers.

Research and choose benchmarking partners

A useful approach to identifying potential benchmarking partners is a two-step process that involves:

• Developing a set of possible selection criteria such as:

• Creating a partner profile based on the important evaluation criteria to assess the suitability of potential partner organizations.

In general, comparable organizations of similar size and structure should be chosen to benchmark with to ensure that relevant and applicable information is collected.

A final selection would be determined by the similarity of the business, program or function to produce valid results, the potential partners' willingness to share information, the value and importance of the information to the organization, and what information must be exchanged with the benchmarking partner.

Potential types of benchmarking partners that a public sector organization could consider include:

• other parts of one's own organization;

• other federal government departments and agencies;

• departments and agencies in provincial and municipal governments;

• departments and agencies of foreign governments; and

• private sector companies.

Establish code of conduct

Establish appropriate benchmarking protocols that define expected behaviours and outcomes towards benchmarking partners. Appendix B presents benchmarking ethics and a code of conduct.

Determine the data gathering methods

"Finding credible information with which management can be convinced to implement change is considered the heart of the benchmarking process" (#12, p. 6). There are essentially three main data sources: internal information sources, external information sources and original benchmarking research. Information could be obtained from one or several of these sources. The validity of benchmarking results is strengthened when more than one source of information is used and integrity of the data is verified through cross-checks.

The benchmarking team must decide on the best method for data gathering after taking into consideration the benchmarking objectives and information needs. The following criteria provide a rule of thumb for deciding on the best method of gathering information:

• extent and accuracy of the information required;

• cost of obtaining the information;

• the intended uses of the information;

• amount of time available to conduct the data gathering exercise;

• accessibility of the data source(s) (#6, p. 7).

Ideally, internal data should be collected first because it may reveal strengths and weaknesses that point to areas in the greatest need of benchmarking. External data sources include government reports/documents/publications, research papers, journals and magazines, international benchmarking clearing houses, professional associations, academic sources and seminars or conferences. Sources of original research encompass questionnaires, telephone interviews and site visits with benchmarking partners.

5.2 Data gathering

"The following provide some general guidelines for data gathering.

Questionnaire

"A questionnaire is the foundation for any good benchmarking study and provides a common communication link among the benchmarking participants. Prepared before initial contact, it ensures that the team has a good understanding of the processes being benchmarked, and is verified by those who do the actual work" (#12, p. 7).

Surveys

When conducting a mail or telephone survey, the appropriate target population is identified and asked to respond to a questionnaire. It is sometimes necessary to offer some incentives (such as free communication of results) to ensure a rate of response that will provide reliable results (#12, p. 7).

Personal visit and interview

"The initial contact should be made by the benchmarking team leader or by senior management, especially where sensitive data is to be considered. The data analysis methodology should be carefully considered to ensure that the data is in a format conducive to analysis" (#12, p. 7). A two-member team is considered ideal for a visit: one to ask the questions, another to take notes and observe the interview process.

5.3 Analysis and integration

Analysis of data is a precursor for the identification of performance gaps and the underlying causes of such gaps. The validity of benchmark data is affected by the degree of comparability between organizations. In some instances, such as where one organization is heavily automated while another is not, performance measures may not be readily comparable. In these cases, the benchmarking team would need to normalize the data in order to draw accurate conclusions.

Data is normalized on a ratio basis using factors such as:

• size (budget dollars/employee);

• age (infrastructure, IT systems); and

• working environment (fiscal pressures, regulatory requirements).

Comparing performance indicators between organizations may reveal differences and gaps that need to be addressed. Comparing results internally and among organizations leads to questions as to why differences occur. For results benchmarking, this means looking beyond the quantitative data and examining the environmental context including processes and other factors such as policy, legal or legislative differences in order to determine the causes of variances in performance.

A basic tool for analyzing performance gaps is a matrix chart listing performance measures for each of the benchmarking partners. This matrix format has been found to be useful in highlighting performance gaps. In evaluating performance gaps, the focus of the benchmarking team should be future oriented. Instead of targeting current levels of performance, the team should look beyond today's performance levels and targets to understand the level of performance that will be required in the future and the enablers required to achieve that level. A prospective approach is essential for achieving and maintaining superior performance.

The analysis of data should be related to the original purpose of the study. If the purpose of the study is to gather qualitative business practices and methods, organizations will need to synthesize the raw data using any of the following tools:

• charts;

• graphs; and

• descriptive anecdotes.

The analysis of data should lead to the determination of benchmark performance and to the understanding of the practices used to achieve them. The performance gap represents the difference between the internal performance and that of the best in the field, and could be either negative, zero or positive.

Where performance gaps exist, the next step would consist of designing and implementing a performance improvement plan and, subsequently, recalibrating the benchmarks.

5.4 Implementation and execution

"This is a crucial stage, as results must be communicated properly. The benchmarking team should document results in a manner appropriate to its organization, intended audience and the nature of the study... At this stage, the benchmarking team presents management with its findings to obtain acceptance of the analysis, conclusions and implementation actions necessary to close the performance gap. The presentation to management should include the goals or planned performance to narrow, close or exceed the benchmark standard, based on the organization's objectives. The successful implementation of a performance improvement plan requires senior management sponsorship and buy-in. Performance improvement plans may range from incremental improvement to the redesign of processes to reengineering. A business case outlining the cost/benefit of the chosen level of improvement is often necessary to secure senior management commitment.

"Once management acceptance is obtained, the organization should develop a set of action plans to achieve the new goals. It is critical that actions be well defined to ensure their successful implementation. For each action, a description of time frame, responsibility, resources requirements and its impact on the performance gap should be included. Action plans should also be reviewed with the staff in affected areas to obtain their commitment. After the implementation stage begins, progress should be monitored against milestones established in the action plans" (#12, p. 8).

5.5 Recalibration

"Organizations are aware that their environment is not static and in that regard, neither should benchmarking" (#12, p. 8). Periodically revisiting benchmarks is essential for maintaining superior performance in a rapidly changing environment. Targets and standards will evolve over time and decisions will need to be made whether to measure performance against existing or new standards/targets. These decisions will need to consider that benchmarking entails financial costs as well as human resource costs triggered by the impact of change on staff.

A balanced approach is required to prevent excessive change while avoiding complacency. Senior management will need to determine how often and how extensively the benchmarks need to be recalibrated.

6. LESSONS LEARNED

Where financial management benchmarking is undertaken, it is important to apply lessons already learned. A list of some of the lessons identified from the benchmarking experiences of others is provided below.

• Before embarking on a benchmarking exercise, a cost-benefit analysis should be undertaken. Benchmarking is very expensive and it is important to decide in advance how the information will be used and whether it is worth it.

• Make sure you have a good understanding of the underlying business processes.

• Benchmarking is a process that requires full commitment from top management as well as incentives for successful implementation. A benchmarking champion can help in fostering this commitment as well as monitoring progress in implementing changes.

• Choose your partners carefully. The processes and results to be benchmarked should be comparable and yield meaningful comparisons. Benchmarking with private sector organizations can be useful for government departments and agencies provided the comparisons are relevant. Ensure that you have an "apples to apples" comparison and that critical differences are identified and taken into account.

• Try to use existing systems to generate the data needed to support benchmarking. Do not initiate a separate data collection exercise unless absolutely necessary.

• Each government organization has a different operating structure and there are different types of funding. Similar organizations must be grouped together for comparison (i.e., they cannot all be compared against each other).

• Implementation begins at the project-planning stage. Seasoned benchmarking professionals will tell you that for a benchmarking project to deliver the goods, careful planning and preparation are essential.

• Benchmarking projects must be focused on highly specific comparisons and aimed at delivering broad business benefits. Don't rush in until you have precise questions for which you need answers.

• Keep senior management informed all the way through. Include the people who are currently managing the aspect of the business being benchmarked - and who will be responsible for implementing changes - on the benchmarking team.

• "Improve processes (in the case of process benchmarking) not individual metrics. Don't confuse exchanging performance measures with benchmarking. Statistics cannot be improved, but the operations or processes that those statistics purport to measure can" (#14, p. 3).

• In the hunt for that elusive best practice, consultants are certainly one place to start. The major benchmarking consultants claim to be able to tell you not only how you perform but also how that performance compares to the best-in-class.

• "Structured visits yield more data.... Some simple pointers will ensure that your benchmarking visits turn out to be jackpots, not junkets.... Teamwork has to start before stepping through the door of another [organization].... A structured agenda is vital.... Each member of the team should know exactly what his/her role is and understand how the information that he/she gathers will integrate with other information to form a coherent summary of the entire visit.... Asking for learning experiences as well as factual details is vital" (#14, pp. 6-7). Take copious notes.

• "Develop a targeted implementation.... 'The more complex the change, the greater the risk.' It's not metrics that count but processes.... So by carefully focusing on clearly defined processes to benchmark and ensuring executive buy-in at the beginning, benchmarking exercises can get off to a good start and come to a fruitful conclusion.... Don't try to tackle too much at once, set action-oriented milestones and monitor them regularly" (#14, pp. 7-8).

7. CONCLUSION

The implementation of benchmarking in the public sector presents many challenges. While many departments and agencies have been using some form of internal benchmarking for a number of years (e.g., comparing the practices, processes and performance of regions or service offices), there is not a common understanding of the concepts of benchmarking, benchmarks, best practices and standards. The terms are used differently and, sometimes, interchangeably. This, in itself, presents challenges in advancing the implementation of benchmarking as a tool in the federal government financial management community.

Benchmarking in general, and financial management benchmarking in particular, is in the embryonic stages in the federal public service. The tools and techniques are not yet fully understood and most financial managers and officers have not been trained in their use. The number one complaint voiced by those we interviewed is that it sounds good in theory but it is very difficult to apply in real life.

This Guide was developed to provide guidance and advice on the use of benchmarking for the financial management functions in particular. The Guide outlines a process and model that can be adapted to the need of the organization and the manager wishing to use benchmarking to measure and improve performance. The goal is to give the readers an outline of the steps necessary to implement a strong benchmarking and performance measurement program while allowing them to formulate their own opinions about detailed implementation. The desired outcome is that benchmarking become an important tool for improving and modernizing financial management practices in departments and agencies.

8. ENQUIRIES

For further information, guidance and advice on this Guide, please contact the following office:

Copies of Treasury Board of Canada Secretariat publications are generally only available in electronic format. The Guide to Financial Management Benchmarking can be accessed through the TBS Internet Site at the following address:

http://www.tbs-sct.gc.ca/

APPENDIX A - INTEGRATED MODEL FOR BENCHMARKING

Successful benchmarking must be based on a structured approach. Organizations employ benchmarking models with as few as four and as many as 33 steps. Each phase of the model below has specific activities as explained in Section 5 of the Guide. This integrated model was chosen to be a public service oriented model, which organizations can modify to suit their particular situation.

Display full size graphic

APPENDIX B - BENCHMARKING ETHICS AND CODE OF CONDUCT

Ethics

In actions between benchmarking partners, the emphasis is on openness and trust. The following guidelines apply to the partners in a benchmarking project.

Do

• Establish specific ground rules up front.

• Consult with legal counsel if any information gathering procedure is in doubt.

• Treat any information obtained from a benchmarking partner as internal, privileged information.

Do not

• Ask organizations for sensitive data or cause the benchmarking partner to feel that sensitive data must be provided to keep the process going.

• Disparage an organization's business or operation to a third party.

• Misrepresent yourself as working for another employer.

• Disclose or use any trade secret that may have been obtained through improper means or that was disclosed by another in violation of duty to maintain its secrecy or limit its use.

As the benchmarking process proceeds to the exchange of information, benchmarking team members are expected to:

• "Know and abide by The Benchmarking Code of Conduct.

• Have basic knowledge of benchmarking and follow a benchmarking process.

• ...Have determined what to benchmark, identified key performance variables, identified superior performing organizations, and completed a rigorous self-assessment."

• Have developed a questionnaire and interview guide and share these in advance if requested.

• Have the authority to share information (#1, p. 3).

Code of conduct

To contribute to efficient, effective and ethical benchmarking, individuals agree for themselves and their organization to abide by the following principles for benchmarking with other organizations. These are based on the Professional Code of Conduct jointly approved by the Strategic Planning Institute's Council on Benchmarking and the American Productivity & Quality Center's International Benchmarking Clearinghouse.

Principle of confidentiality

• Treat benchmarking interchange as something confidential to the individuals and organizations involved.

• Information obtained must not be communicated outside the participating organizations without prior consent of benchmarking participants (#1, p. 1).

• Do not extend benchmarking study findings to another organization without first ensuring that the data is appropriately blinded and anonymous so that the participants' identities are protected.

• Obtain an individual's permission before providing his or her name in response to a contact request.

• Avoid communicating a contact's name in an open forum without the contact's prior permission.

Principle of legality

When benchmarking with a private sector partner, government departments and agencies should:

• Consult with their legal counsel if there is any potential question on the legality of an activity.

• Avoid discussions or actions that could lead to or imply an interest in restraint of trade, price fixing, bid rigging, or bribery.

• Refrain from the acquisition of trade secrets from another by any means that could be interpreted as improper including the breach or inducement of a breach of any duty to maintain secrecy (#1, p. 1).

Principle of contact

• "Respect the corporate culture of partner organizations and work within mutually agreed procedures.

• Use benchmarking contacts, designated by the partner organization, if that is their preferred procedure.

• Obtain mutual agreement with the designated benchmarking contact on any hand-off of communication or responsibility to other parties" (#1, p. 2).

Principle of preparation and completion

• "Demonstrate commitment to the efficiency and effectiveness of benchmarking by being prepared before making an initial benchmarking contact.

• Make the most of your benchmarking partner's time by being fully prepared for each exchange.

• Help your benchmarking partners prepare by providing them with a questionnaire and agenda prior to benchmarking visits.

• Follow through with each commitment made to your benchmarking partner in a timely manner.

• Complete each benchmarking study to the satisfaction of all benchmarking partners, as mutually agreed" (#1, p. 2).

Principle of exchange

• Be willing to provide to your benchmarking partner the same type and level of information that you request from them.

• Communicate fully and early in the relationship to clarify expectations, avoid misunderstanding, and establish mutual interest in the benchmarking exchange.

• Be honest and provide information that is complete (#1, p. 1).

Principle of use

• "Use information obtained through benchmarking only for purposes stated to the benchmarking partners" (#1, p. 2).

• Agree how the benchmarking partners wish to have information treated and handled, and honour that agreement.

(1) Style for Parenthetical References: The first part of the reference is a number that relates to the corresponding number for the authors listed in this Guide's Bibliography in Appendix E (e.g., #13=Trosa, Sylvie and Suzanne Williams); the second part of the reference will contain, if appropriate, the applicable chapter (ch.) or Appendix number of the author's work being cited; and the third part includes the page number of either the document, the chapter or the Appendix being cited.[Return]