Audit of Management Group Performance Management
Corporate Audit and Evaluation Branch
April 2005
Executive Summary
Background: In April 2000, the Canada Customs and Revenue Agency (CCRA) Board of Management approved the Employee Performance Management Policy. The policy provided the authority for the Agency to proceed with the development of guidelines to administer performance management. Performance management is one of the key initiatives and change objectives intended to modernize the Agency's management and operations programs, systems and practices. The Commissioner has stated through the Corporate Business Plan that managers will be held accountable for human resource management through annual performance agreements.
The objective of the Agency's performance management policy is to support and strengthen organizational and individual performance through the management of human resources by establishing expectations, identifying measurement criteria, where applicable, and by providing performance feedback.
Specific Employee Performance Management Guidelines and the Performance Pay and Leave Guidelines (PP/PL) were established for the Management Group (MG) and equivalent positions in the Personnel Administration Group. These guidelines were to be used in conjunction with the Employee Performance Management policy.
The purpose of performance pay and leave is to acknowledge and reward managers for the critical role they play in leading and developing employees to achieve the goals of the Agency. The leadership role, played by managers in supporting and developing employees, is essential to the success of the Agency.
The audit focused on the fiscal year 2002-2003 review cycle for the Management Group. Human Resources Branch estimates that the cost of performance rewards for the 2002-2003 review cycle was approximately $6 million.
The examination phase was conducted from November 2003 to February 2004. Selected sites included the Compliance Programs Branch, in Headquarters, and selected Tax Services Offices (TSOs) and Tax Centres (TCs) in the Northern Ontario, Pacific, Prairie, Quebec, and Southern Ontario regions.
Objective: The objective of the audit was to provide assurance on the level of compliance with the Agency's Performance Management policy and guidelines, achievement of goals and objectives, and consistency in the application of the Performance Pay and Leave Guidelines applicable to the MG group.
Conclusion: In general, the audit found there was compliance with the Agency's performance management policy. Further clarification of the roles and responsibilities of review committees and Human Resources Branch (HRB) regarding monitoring and review activities would increase the level of understanding. Monitoring generally consisted of statistical analyses being conducted with limited monitoring of the quality of performance assessments. Human Resources Branch should clarify roles and responsibilities to ensure quality, consistency and monitoring of the performance management process.
Changes to performance management guidelines were not communicated in a timely manner. Human Resources Branch should ensure that future changes are issued prior to the commencement of the review cycle and include a communication strategy with rationale for the changes. Corporate Administrative System (CAS) data reviewed was not reliable. Human Resources Branch should improve data capture and correct previously captured data to ensure that reports are reliable and accurate.
Overall, in the selected offices, there was compliance with the performance management policy and guidelines. Performance assessments were completed and the performance management process was followed. However, based on a review of performance assessments and interviews with supervisors of MGs, improvements could be made to the performance assessment templates and the identification of protected information. Templates should be user friendly to complete and managers should identify any necessary training. Human Resources Branch should ensure that documents are properly identified and marked with the appropriate protection level.
It was difficult to assess if the performance pay and leave guidelines were being administered in a consistent manner since assessments were rarely reviewed at the regional level and only statistical analyses was conducted at the national level. In addition, there were limited mechanisms in place to ensure consistency in interpretation and application. Supervisors of MGs would make their own interpretation of terminology as no specific examples were provided to assist in evaluating MGs. Human Resources Branch should ensure that definitions are clarified and appropriate examples are provided in the guidelines.
Action Plan: The Human Resources Branch's Performance Management Working Group, chaired by HRB, has been expanded to include managers who are contributing their own experiences. The group has met twice. Concerns raised by this group have been documented and action plans developed. Specific changes to the Performance Management program have been identified. In the interim, HRB will provide advice to senior management regarding change implementation timelines and will support changes with an impact analysis. HRB will explore the possibility of using other software to facilitate data capture until the CAS functionality is further developed and more user friendly.
HRB will improve the performance assessment templates and will review the possibility of changing the templates to a direct data capture, into CAS, model. Templates to be used for the 2005-2006 cycle will include the appropriate protection level and be easier to complete.
HRB will work with managers and Performance Management Coordinators to determine the best way of ensuring clarity of terminology and will communicate to management.
Introduction
In April 2000, the CCRA Board of Management approved the Employee Performance Management Policy. The policy provided the authority for the CCRA to proceed with the development of guidelines to administer performance management. Performance management is one of the key initiatives and change objectives intended to modernize the Agency's management and operations programs, systems and practices.
The first phase of the implementation process, completed in August 2000, required that all managers and supervisors be provided with written performance expectations outlining their goals and objectives. During the second phase of the process, in June 2001, managers and supervisors were to be provided with their first formal assessment. In September 2001, all remaining employees were provided with written goals and objectives in accordance with the policy. The performance management process was continuous from that point forward.
The objective of the Agency's performance management policy is to support and strengthen organizational and individual performance through the management of human resources by establishing expectations, to identify measurement criteria, where applicable, and provide performance feedback.
Specific Employee Performance Management Guidelines and the Performance Pay and Leave Guidelines (PP/PL) were established for the newly created Management Group (MG) and for equivalent positions in the Personnel Administration Group. These guidelines were to be used in conjunction with the Employee Performance Management Policy. It should be noted that these guidelines do not apply to employees in the Executive (EX) or Senior Manager (SM) groups.
The purpose of performance pay and leave is to acknowledge and reward managers for the critical role they play in leading and developing employees to achieve the goals of the Agency. The leadership role played by managers in supporting and developing employees is essential to the success of the CRA.
Costs estimated by HRB for the 2002-2003 review period were approximately $6 million. This is a combination of approximately $4 million for performance leave and approximately $2 million for performance pay. The audit team was unable to verify these figures based on the CAS data retrieved.
Focus of the Audit
The objective of the audit was to provide assurance on achievement of goals and objectives, compliance with the Agency's Performance Management policy and guidelines, and consistency in the application of the PP/PL applicable to the MG group.
The scope of the audit included a review of the administration and processes of performance management for MGs in the 2002-2003 fiscal year and visits to the Compliance Programs Branch, in headquarters, and selected offices in the Northern Ontario, Prairie, Pacific, Quebec, and Southern Ontario Regions. The review included 14 sites and interviews with 162 MGs, 37 supervisors of MGs and approximately 60 other stakeholders. The audit team also reviewed approximately 168 Performance Assessments and related documents.
The audit excluded prior years except where they were linked to the 2002-2003 review cycle; Taxation Centre employees performing MG duties for tax seasonal programs (different review cycle); and MGs on Special Projects (performance pay guidelines for these individuals did not take effect until the next performance management cycle i.e. fiscal year 2003-2004).
Findings, Recommendations and Action Plans
Achievement of Goals and Objectives
The objective of the Agency's performance management policy is to support and strengthen organizational and individual performance through the management of human resources by establishing expectations, identifying measurement criteria, where applicable, and by providing performance feedback.
In general, the objective of the Agency's performance management policy was met. Further clarification of the roles and responsibilities of review committees and Human Resources Branch (HRB) regarding monitoring and review activities would increase the level of understanding. Monitoring generally consisted of statistical analyses being conducted with limited monitoring of the quality of performance assessments. Human Resources Branch should clarify roles and responsibilities to ensure quality, consistency and monitoring of the performance management process.
Changes to performance management guidelines were not communicated in a timely manner. Human Resources Branch should ensure that future changes are issued prior to the commencement of the review cycle and include a communication strategy with rationale for the changes. Corporate Administrative System (CAS) data reviewed was not reliable. Human Resources Branch should improve data capture and correct previously captured data to ensure that reports are reliable and accurate.
Roles, Responsibilities and Monitoring
The Employee Performance Management Guidelines stipulated that each Assistant Commissioner must establish a Branch or Regional review committee. The primary role of this committee was to monitor and review the performance pay and leave process to ensure it was being administered in a fair and consistent manner resulting in a quality performance management process.
Roles and responsibilities of most stakeholders, such as MGs and MG supervisors, were understood and implemented. However, the roles and responsibilities of HRB and the Regional/Branch review committees should be strengthened regarding monitoring and review of the performance management program.
In some cases the roles and responsibilities of review committee members were not clearly understood or followed to ensure consistency, fairness and transparency in the application of the performance management process across the Agency.
There were limitations in the determination of the final assessments due to on-going changes to rated requirements for EPM. Some Regional/Branch review committees used the distribution of ratings table, which was included in the communiqué, issued May 2003 to MGs, as a guide while others used it as a rule. Most Regional/Branch review committees looked only at the distribution of ratings and did not review actual assessments. HRB received the Quality Review reports and summarized the findings and action plans when reporting to AMC.
At the Regional/Branch level in five of the six Regions/Branches visited, control mechanisms were not in place to ensure consistency. Two regions did not have a Regional review committee in place. Regional finding sheets were issued to these regions resulting in Regional review committees being established.
Most areas visited had formal or informal review committees at the divisional and local levels that helped to ensure consistency.
There was little evidence of any review of MG performance management assessments conducted at the national level. The performance management policy states that, "The application of the performance management policy will be monitored and reviewed by the Agency". The performance management program falls under the responsibility of HRB, whereby they provide support and advice to managers. However, it is a managerial responsibility to assess employees. Some statistical analysis had been conducted by HRB. For example, if there were discrepancies between employees with expectations at the beginning of the cycle and employees appraised at the end of the cycle, concerns were discussed with the regions and additional information was requested. Although HRB stated that review committees are responsible to ensure quality of assessments, only one review committee reviewed performance assessments.
At the time of the examination phase of this audit, monitoring and review of the performance management process were minimal and differed from region to region. The lack of a monitoring framework at the Branch level made it difficult to assess the quality of assessments on a national basis. A management framework that details the governing procedures for review committees to follow did not exist and only statistical monitoring was conducted at the national level.
Subsequent to the examination phase of this audit, on June 4, 2004, a memorandum was sent from the Assistant Commissioner, HRB to the Agency Management Committee. This memorandum clarified the mandatory requirement to establish review committees to ensure consistency, quality and fairness; to identify best practices, issues and solutions; and to report results to HRB.
The memorandum did not provide any guidelines on the methods to be used by these committees in order to ensure a fair and consistent process resulting in a quality performance management process.
The memorandum did not comment on the methods that HRB would use to ensure that the application of the performance management policy is monitored and reviewed nationally. Though it is understood that the accountability for good performance rests with managers and is found within the EPM criteria.
Recommendation
HRB should clarify the roles and responsibilities of HRB and Regional/Branch review committees to ensure quality, consistency and monitoring of the performance management process.
Action Plan
HRB will discuss the role of the Review Committee with the working group and will include the need to report to HRB on results of activities in the guidelines (was done through a yearly call letter). The guidelines will also indicate what exactly HRB is monitoring to avoid duplication of monitoring efforts.
The Performance Management Working Group, (chaired by HRB), as of November 2004, has been expanded to include regional managers to bring forward regional concerns. These concerns are documented and action plans are developed.
The Working Group has met twice. At the November 2004 and February 2005 meetings, the Working Group discussed and reviewed the Performance Management Process and identified that some changes were required to the Performance Management process. The Working Group also discussed most of the Internal Audit recommendations, such as the roles and responsibilities of Review Committees, rating scales, forms and the definition of exemplary leadership. An action plan is being implemented to improve the forms for the 2005-2006 cycle, and to develop a tool to assist managers in understanding exemplary leadership. The topic of rating scales and roles has been scheduled for discussion during the 2005-2006 cycle. Discussions such as these will continue and relevant information will be passed on to all stakeholders to continuously improve the process, as well as to clarify roles and responsibilities in the performance management process.
Timeliness of Changes to Performance Management Guidelines
The performance management cycle for MGs is April to March. Significant changes to the performance management guidelines for the 2002-2003 review cycle were not communicated in a timely manner. This had a major impact on the 2002-2003 review cycle causing additional work and frustration in the completion of performance assessments.
At the outset of the 2002-2003 review cycle there were four possible ratings for Effective People Management (EPM). In April 2003, one month after the end of the cycle, the rating scale increased to six possible ratings to be applied retroactively for that cycle.
Human Resources Branch stated that the changes for the 2002-2003 cycle were in response to clients' requests to align the ratings to the EX and SM Groups. MGs agreed with the changes to the rating scales but not the timing of the implementation of these changes. MGs were working towards the achievement of one set of guidelines while they were rated on a different set of guidelines. These concerns would have been alleviated had the changes occurred at the beginning of the cycle rather than at the end of the cycle.
There was an improvement in the timeliness of communications for the 2004-2005 review period.
Recommendation
HRB should ensure that any future changes to policies and guidelines be issued prior to the commencement of the review cycle.
HRB should develop a communication strategy whenever changes are to be made to the performance management process to ensure the affected employees are notified in advance and understand the rationale for the changes.
Action Plan
The HR Directors will be informed of the importance of reminding their clients of any change. HRB will continue making recommendations to senior management about the timeliness of implementation on any change and will support it with an impact analysis on the timeliness of the change.
HRB will continue developing a communication strategy whenever a change is made and will, as applicable, provide the background and rationale for the need to make changes.
Corporate Administrative System (CAS) Data
Performance management data was keyed into CAS commencing in 2003. A CAS Template was developed to capture specific information fields concerning performance management.
The audit found that the CAS reporting templates did not include all key information, such as Responsibility Centre/Cost Centre, to ensure easy retrieval of data. In addition, not all information captured on the CAS template was actually input into CAS. For example, data fields captured in the CAS template distinguish between action plans prepared for Core and action plans prepared for EPM. Only an aggregate of action plans were captured in CAS with no distinction made between Core and EPM.
The audit team requested a number of CAS reports, for the 2002-2003 review cycle. A review of these reports found that the total number of MGs in the Agency varied from 3,944 to 4,187. The number of action plans prepared varied from 34 to 45.
In summary, information retrieved from the CAS Reports was not accurate or reliable. Based on the information retrieved, the audit team was unable to determine the total number of MGs in the Agency, the number of action plans prepared, or the number of MGs receiving Performance Pay or Leave with any degree of accuracy.
Recommendation
HRB should improve data capture wherever possible, in the CAS system to ensure that CAS reports, related to MG information, are reliable and accurate.
HRB should work with ITB to correct data previously captured in CAS to ensure that it is accurate and reliable.
HRB should work with ITB to address concerns and bring about efficient and timely improvements.
Action Plan
HRB is looking at improving the format so that when Manager Self Serve is available, the Y280 and Y280-1 can be downloaded directly into CAS. Data was captured on template January 2005. CAS upload is presently underway with anticipated completion date of mid-May 2005. The responsibility for data capturing starts with managers (i.e. They complete the template and they are the ones accountable for providing accurate data). HRB will remind every one about the importance of ensuring that they provide the right information. This will be part of the guidelines, revised version, to be ready for May 2005.
Compliance with the Agency's Performance Management Policy and Guidelines
Overall, in the selected offices, there was compliance with the performance management policy and guidelines. Performance assessments were completed and the performance management process was followed. However, based on a review of performance assessments and interviews with supervisors of MGs, improvements could be made to the performance assessment templates and the identification of protected information. Templates should be user friendly to complete and managers should identify any necessary training. Human Resources Branch should ensure that documents are properly identified and marked with the appropriate protection level.
Performance Assessment Templates (Y280 and Y280-1)
For the 2002-2003 review cycle, supervisors of MGs were required to use two forms/templates to record performance expectations and assessments of MGs. The Y280 entitled "Employee Performance Management Report" was used to record the Core Business (Core) expectations and assessments and the Y280-1 entitled "Addendum to the Employee Performance Management Report" was used to record the Effective People Management (EPM) expectations and assessments.
Most interviewees raised concerns that these templates were not user friendly to complete. Many used alternate customized forms to prepare the assessments because of frustrations experienced in trying to work with the forms.
Some concerns raised were: flipping back and forth was necessary because the "Goals", "Objectives" and "Measurement Criteria" areas were shown in a vertical format and the "Assessment" area was on a different page; and formatting difficulties were experienced.
The first page of the Y280 and Y280-1 templates did not indicate the appropriate level of protection. This was in contravention of the Financial Administration Manual Policy Chapter 5 - Security - Identifying Classified and Protected Information and Assets, Section 8 Protected Information and Appendix A - Section 6 "Marking Requirements".
Although most assessments indicated "Protected (when completed)" on the front page, in the majority of the assessments reviewed, "Protected" was not marked on every page. When an alternate Word document was used "Protected" was not indicated on these pages.
The Y280 and Y280-1 templates were revised April 2004 for the 2004-2005 review period; however, the revisions did not include identification of appropriate protection levels. There is a risk that personal information could be compromised.
Recommendation
HRB should ensure Y280 and Y280-1 templates are user friendly to complete and that training to utilize these forms (e.g., self-study, coaching and on-line instructions) is available to managers.
HRB should ensure that Performance Management documents are properly identified and marked with the appropriate protection level.
Action Plan
HRB will continue working with the regional Performance Management Working Group to ensure they improve the Y280 and Y280-1 to make them as user friendly as possible. HRB is also looking at changing the word format to another one that will allow direct capture into CAS.
The forms will be revised and the proper level of protection will be indicated. This will be done for the beginning of 2005 MG cycle to avoid potential impacts on the cycle. This is currently underway.
Consistency in the Application of the Performance Pay and Leave Guidelines
The costs, estimated by HRB, of performance rewards for the 2002-2003 review cycle were approximately $6 million. It was difficult to assess if the performance pay and leave guidelines were being administered in a consistent manner since assessments were rarely reviewed at the regional level and only statistical analyses was conducted at the national level. In addition, there were limited mechanisms in place to ensure consistency in interpretation and application. Supervisors of MGs would make their own interpretation of terminology as no specific examples were provided to assist in evaluating MGs. Human Resources Branch should ensure that definitions are clarified and appropriate examples are provided in the guidelines.
Terminology and Definitions
Based on interviews conducted in the 14 sites visited, MGs and Supervisors of MGs raised concerns about terminology and definitions and suggested that examples be provided to ensure clarification and consistency in the interpretation and application of definitions. Some supervising managers and review committee members suggested that examples should be balanced so that they do not become too prescriptive. Human Resources Branchrepresentatives stated that consistency is not as important as regional/branch flexibility. The goal should be in managing performance rather than uniformity.
The Employee Performance Management Guidelines and the PP/PL used terminology, such as "Additional Initiatives" and "Exemplary Leadership" without providing clear definitions or examples. How these terminologies differ from "initiatives" and "leadership" had to be interpreted by Supervisors of MGs when evaluating their MG employees.
In order to achieve an "Exceeds" rating on Core, an MG had to exceed the majority of performance expectations and undertake and complete "Additional Initiatives". An MG needed to demonstrate "Exemplary Leadership" under EPM in order to be eligible for Performance Pay or Leave. There were many interpretations of these terminologies, which increased the risk of inconsistencies in evaluating and rewarding performance.
Recommendation
HRB, in consultation with Regions and Branches, should ensure that senior management and review committees clarify definitions and provide examples. The guidelines for the beginning of the 2005-2006-review cycle should be updated to include this information.
Action Plan
HRB will work through the joint managers / Performance Management Coordinators committee to ensure clarification of definitions and communicate to management.
At the November 2004 meeting with the manager/coordinator group, members clearly expressed that they don't want examples because the performance management process is dealing with individuals rather than clichés. The group indicated that the process needs improvement and managers are working at it. Key messages around PP/PL and a tool to assist managers to understand exemplary leadership have been developed and completed at the February 2005 meeting. They will be in place for the 2005-2006 MG performance cycle.
The Observe and Attest initiative will be supported by a Professional Development Program that clarifies the setting of business expectations linked to competencies and how to complete the gap analysis between the employee behaviours and the expected behaviours as well as how the gap affects the employee performance.
Conclusion
In general, the audit found there was compliance with the Agency's performance management policy. Further clarification of the roles and responsibilities of review committees and Human Resources Branch (HRB) regarding monitoring and review activities would increase the level of understanding. Monitoring generally consisted of statistical analyses being conducted with limited monitoring of the quality of performance assessments. Human Resources Branch should clarify roles and responsibilities to ensure quality, consistency and monitoring of the performance management process.
Changes to performance management guidelines were not communicated in a timely manner. Human Resources Branch should ensure that future changes are issued prior to the commencement of the review cycle and include a communication strategy with rationale for the changes. Corporate Administrative System (CAS) data reviewed was not reliable.
Overall, in the selected offices, there was compliance with the performance management policy and guidelines. Performance assessments were completed and the performance management process was followed. However, based on a review of performance assessments and interviews with supervisors of MGs, improvements could be made to the performance assessment templates and the identification of protected information. Templates should be user friendly to complete and managers should identify any necessary training. Human Resources Branch should ensure that documents are properly identified and marked with the appropriate protection level.
It was difficult to assess if the performance pay and leave guidelines were being administered in a consistent manner since assessments were rarely reviewed at the regional level and only statistical analyses was conducted at the national level. In addition, there were limited mechanisms in place to ensure consistency in interpretation and application. Supervisors of MGs would make their own interpretation of terminology as no specific examples were provided to assist in evaluating MGs. Human Resources Branch should ensure that definitions are clarified and appropriate examples are provided in the guidelines.
|