Use of aggregated 360° leadership assessment results in the Federal Public Service: A strategic advantage to learning organizations.
François Chiocchio, Personnel Psychology Centre
Sunjeev Prakash, Personnel Psychology Centre
Greg J. Sears, McMaster University
62nd Canadian Psychological Association Conference
Québec City
June 2001
1. Abstract
The use and effectiveness of 360° feedback processes as a component of individualized leadership development is well established in the private and public sectors. In line with the Personnel Psychology Centre's mandate of providing Federal Public Servants with assessment tools that contribute to a professional and meritorious Public Service, 360° leadership assessments have been used for the past decade. This decade was pivotal in setting the stage for important modifications to work flow issues within and across organizations. Organizations now rely on flexible managers comfortable with flatter organizations and cross-functional teamwork. Work revolves around areas of responsibilities, creativity and proactiveness in a context of ambiguity rather than on clusters of specific tasks and linear process-driven thinking. In this context, a practical methodology was developed that aggregates and presents individualized 360°-type results in a way that addresses an organization's need to further develop leaders in this horizontal cross-functional teamwork context. The current paper discusses this methodology, it's impact on leadership competencies and on the alignment of individual goals on organizational strategic objectives.
2. Introduction
The Public Service Commission of Canada's role within the Federal Public Service is to uphold the merit principle. In short, the merit principle implies that public servants are selected through the use of reliable, valid and equitable measures. As part of the Public Service Commission, the Personnel Psychology Centre develops and implements assessment processes that help key players in the Canadian federal Public Service make efficient human resource management decisions.
While valid forms of assessment are a key component in upholding the merit principle during the personnel selection process, steps need to be taken to ensure a meritorious work force is maintained after individuals have entered the Public Service. Managers throughout the federal government contribute to this goal by maintaining a proactive stance in their efforts to provide learning opportunities for their employees. One method of gathering valuable developmental information is the 360° feedback assessment process. To this end, the Personnel Psychology Centre has been offering public servants a full range of 360° feedback instruments and services since 1992. These instruments were designed to provide all levels of management with detailed information relating to the 14 leadership competencies described in the following section. Although 360° feedback has been recognised as a useful tool to facilitate individual development, it's use as an aid in developmental initiatives at the organizational level is just beginning to be explored.
3. Nature and structure of the competency profile and associated 360° instruments
Because a psychometric instrument is only as good as it's underlying concepts and structure, a brief summary is in order. The most recent version of the Personnel Psychology Centre's 360° instruments are based on the Profile of Public Service Leadership Competencies. This profile was developed by the Treasury Board of Canada and the Public Service Commission, using a content driven validation method. The result was a set of 14 leadership competencies, each of which was further defined by specific behaviours that are indicative of superior performance for 5 levels of management within the federal government. By using this behavioural grid, behaviours that are important for each level or management are clearly stated. For example, superior performance for an Assistant Deputy Minister under Communication may include effectively responding on the spot to questions from the media or a Parliamentary committee. For a Supervisor, superior performance under Communication may exclude behaviours related to the media and have a greater emphasis on expressing clear goals to his or her employees. In both of these cases, good listening skills are necessary.
In accordance with the structure of the Profile of Public Service Leadership Competencies, the Personnel Psychology Centre has created five 360° feedback instruments (one for each management level). Figure 3.1 provides a synthesis of the nature and structure of the competency profile as it relates to the five 360° questionnaires in use. This structure has major advantages when applied to training and development. First, upon receiving the results from the 360° feedback process, a manager can focus his or her learning plan on key behaviours that will improve performance in his or her current position. Secondly, the manager can begin to assess his or her behaviours as they relate to superior performance at the next management level. This provides a behavioural guide that helps managers focus on performance while considering positions that carry greater responsibilities.
4. Organizational perspective.
By aggregating the results of all of the individual 360° feedback results at the same management level, common themes in behaviours may become evident. This can then set the stage for organization-wide development initiatives. An organizational view of aggregated 360° feedback results can help answer questions such as: What are the highest and lowest rated competencies at a given management level?
What do superiors have to say about their employees across all of the management levels involved in the 360° process? Do perceptions differ across geographic locations? How do individuals rate themselves compared to how, as superiors, they rate their employees, and compared to how they, as employees, rate their superiors? How do self-perceptions compare to peer perceptions? Do perceptions differ across functional teams?
The Personnel Psychology Centre has developed a means that not only provides valuable information at an individual level, but also examines the data acquired through the 360° process at a more global level. The following sections describe the steps involved in developing the organizational perspective from the 360° data.
Step 1. Setting the stage for the use of 360° results for organizational purposes
In order to make the 360° process useful for leaders that wish to promote organization-wide change and professional development, it is first necessary to define groups of participants (managers that will be the focus of the assessment). These groups can be based on management level, workflow within the organization, geographic location, or any other criterion that defines the organization. The next step is to describe a set of discreet categories of raters (people that will assess the participants). One may summarise by asking: "Who are the actors in this 360° process and with whom do they interact?"
Figures 4.1, 4.2 and 4.3 show three possible group definitions within an organization. Most organizational 360° processes will use either one of these three models or a combination thereof. Shaded shapes represent participants, who engage in a self-assessment. Arrows represent categories of raters and point to the persons being assessed. It is important to recognise that members of an organization play different roles when a 360° process is carried out for more than one level within an organization. For example, Fig. 4.1 shows that Directors can potentially rate their superiors as employees, assess themselves in their own 360° process, assess their fellow Directors as peers and, finally, assess their employees as superiors. We will see later that the interplay of results according to these different roles can provide a rich set developmental recommendations.
Step 2. Proceed with the individual 360° process
Once an organizational model is chosen, the organization must proceed with the usual steps of any good 360° feedback process. In short, this process includes: a strategic communication plan that aims at buy-in; distribution of questionnaires (electronic or otherwise) to participants and raters who, in turn, respond to those questionnaires anonymously; compilation of confidential individual reports, followed by distribution of the results to the participants and delivery of one-on-one feedback sessions. When individuals analyse their individual results and implement an individual learning plan, significant advances occur within the competencies as well as promoting the idea of continuously learning on the job. This meta-learning is very useful as it sets the stage for the more elaborate organizational strategies. Once these steps have been completed, work on the organizational perspective can begin.
Several factors must be considered when deciding on the proper time to make the transition from the individual 360° feedback process to the organizational perspective. First, conducting the one-on-one feedback sessions first gives the feedback provider the opportunity to gather important contextual information that, while maintaining confidentiality, will assist in the generation of meaningful organizational level recommendations. Secondly, it is unwise to start analysing aggregated organizational data before all of the completed questionnaires are in. On the other hand, if the period of data collection takes too long, the initiative loses it's momentum and motivation drops. Finally, participants can suffer from Ainformation overload@ when provided with individual and organizational 360°-type information at the same time. When this overload occurs, the participants' ability to focus on their individual feedback can be hindered, thereby undermining the benefits from the entire process.
Step 3. Create an organizational view of the 360° data
Using the model similar to Fig. 4.1, we will show the results from an organization where Directors and Middle Managers were the focus of a 360° process. As previously noted, in order to create an organizational view from the individual data, it is necessary to aggregate the information for each management level. This process is depicted in Figure 4.4.
Whether the organizational 360° report is presented orally or in writing, the description of the groups should proceed from the general to the specific, while always maintaining a focus on the 14 competencies. For example, overall averages for each competency could be displayed, followed by average ratings from each rater category. Numbers, charts and a fair amount of information are important at this stage but only as part of a process that will help the target audience to accept the results that pertain to key competencies and behaviours within their own organization. The final portion of the results should be as simple and clear as possible (examples are shown in Fig. 4.5 and 4.6), since they are intended to be a tool to foster discussions around developmental activities.
The results should culminate in an understanding of the interactions between clusters of competencies, functional responsibilities of participants and the perspectives of the different rater categories who were involved in the process. Continuing with the example of Directors and Middle Managers, the Directors' competencies which received the highest ratings seem to link Personality, Cognitive Capacity and a mix of Visioning and Organizational Awareness. The competencies which received the lowest ratings seem to link Stamina and Stress Resistance, Action Management and Behavioural Flexibility. In the case of Middle Managers, top ranked competencies seem to link Ethics and Values, Teamwork and Self-Confidence. Bottom ranked competencies link the same three competencies in which the Directors received the lowest ratings. These results are presented in Figures 4.5 and 4.6.
It is interesting to note from Fig. 4.5 that Directors are alone in perceiving that they are good at Teamwork. On the other hand, from the overall perceptions, they seem to be bright (Cognitive Capacity), driven (Personality) people who understand their organization's direction (Vision) and know how to work effectively within their organization (Organizational Awareness). Also, Directors display a high level of self-awareness for their behaviours relating to Stamina and Stress Resistance, Action Management and Behavioural Flexibility.
From Figure 4.6 we can note that Middle Managers, rated themselves high on Creativity and Teamwork. In contrast, their employees rated the Middle Managers lower on Creativity and their superior rated them lower on Teamwork. Middle Managers agree with the other raters on Stamina and Stress Resistance, but seem to display low self-awareness on Action Management and Behavioural Flexibility. Also from Figure 4.6, Directors, as superiors, do not seem to notice that Action Management, Behavioural Flexibility and Stamina and Stress Resistance seem to be areas in which Middle Managers could improve. Rather, they seem to view Communication, Organizational Awareness and Teamwork as the areas in which Middle Managers should improve.
Combining the results from Figures 4.5 and 4.6, we can see that the various raters' perceptions on strengths differed across the 2 management levels, while a consensus seems to prevail in terms of the lowest-rated competencies. Many combinations of results are possible and could be used to address issues that are relevant to the organization. For example, Figure 4.7 shows all of the lowest rated competencies as a function of all Middle Managers' roles in the 360° process. Taking this perspective, the lowest rated competencies from the Middle Managers' self-assessments and the lowest ratings the Middle Managers received from the peers coincide with the lowest ratings the Middle Managers gave their superiors. Although obvious from the aggregate results, this common theme may not have been recognised through an examination of all individual feedback reports. Through the aggregation of the 360° feedback data at each management level, potential areas of development from which entire organization can benefit are clearly identified.
Step 4. Generate organizational recommendations
Good recommendations stem from the use of conceptual and circumstantial links between competencies to generate cause and effect hypotheses that can serve as a basis for competency-based HR management decisions and for organizational developmental initiatives. The process that underlines the development of organizational developmental activities is a crucial process which the leaders of an organization should visibly facilitate and endorse. Participants should engage in the process with a clear understanding of the key strategic objectives of the organization. Individual and organizational development on valued competencies then become clearly linked to the organization's ability to advance it's mandate, giving the learning endeavour a sense of purpose.
Coming back to our example, because workload management issues (Stamina and Stress Resistance) seem to be perceived throughout the organization as one of the lowest rated competencies, it is conceivable that discussions around this issue could generate new criteria for accepting work assignments. Since there is an interplay between the 14 competencies, these discussions could also impact on the organization's ability to deal with ambiguity (Behavioural Flexibility), or provide a better sense of control and empowerment, which could also affect the manner in which work is completed (Action Management). Because the specific behaviours under each of the competencies differ for Directors and Middle Managers, level-specific discussions and implementation plans could fine tune the actions each group would take to address the organizational concerns. These initiatives would complement each other and, in concert address the identified competencies across the entire organization.
Discussions around the competencies never occur in a vacuum (as it may appear in this short paper). The process of describing and analysing organizational 360° results and arriving at recommendations is necessary to establish a clear language around the competencies and to establish the link between the competencies, individual behaviour and the organization's mandate. The interplay of competencies and cause-and-effect hypotheses would generate different recommendations depending on organizational variables such as the size of the organization, the nature of the mandate, the type of work involved, etc.
5. Conclusion
The Profile of Public Service Leadership Competencies can be discussed across many levels of management while maintaining level specificity. When individual 360° information is aggregated and properly presented, the added value is an increased awareness of key organizational issues that, in the past, may not have been clearly identified. The leaders who choose to have members across their organizations assessed in this manner demonstrate a strong commitment to competency-based HR management and a commitment to working through it's challenges. The method described here has been used extensively for the past three years in many organizations throughout the federal Public Service. Engaging in this process has provided organizations with a unique perspective on their ability to move forward and has supported movement towards becoming a learning organization.
