TSB Responses to Treasury Board Secretariat Assessments, 2009-2010

Introduction

In 2009-2010, the Transportation Safety Board of Canada (TSB) was assessed as part of Round VII Management Accountability Framework (MAF) assessments carried out by the Treasury Board Secretariat (TBS). As a small agency, the TSB is only assessed once every three years. The last time the department was assessed was in 2006-07, as part of Round IV.

Summary of Observations in Relation to the TSB's Performance against the MAF

The observations relating to TSB's management capacity are positive overall. In total, for the 15 Areas of Management (AoM) against which the organization was assessed, it received fourteen "acceptable" and one "opportunity for improvement" rating (AoM #2). The Transportation Safety Board was able to maintain its high standards of management performance.

The TSB has two overall concerns with the MAF assessment methodology used for this assessment. First, the methodology for some AoMs uses a one-size-fits-all approach; meaning that the MAF expectations of a small department are the same as a large department, which is unrealistic given the considerable difference in resource levels. Secondly, for the human resources elements of the MAF, the results of specific questions from the Public Service Employee Survey were the sole basis to arrive at a rating without considering any contextual information to explain the results. The TSB believes that this limited approach does not provide a fair assessment. The TSB has provided feedback to TBS on its concerns with the MAF methodology.

The following summarizes the TBS assessment for each AoM and details the TSB's response for any identified opportunities for improvement. Each of the AoM consists of sub-categories or elements that were individually assessed in order to arrive at an overall conclusion of management capacity. The TSB has provided a response for every element that was rated below acceptable within each AoM, even if the overall AoM was rated as acceptable.

AoM #1 - Values-based Leadership and Organizational Culture

The TSB was rated acceptable for Leadership and rated as opportunity for improvement for Infrastructure. Within the Infrastructure category, the following two opportunities for improvement were identified:

  • "TSB does not have a formal Values and Ethics Plan. It is suggested that a Values and Ethics plan be developed and implemented that includes a strategy championed by senior management with medium-term activities to raise awareness of the importance of public service values and ethics and that TSB identify opportunities for its leaders to champion public service values and ethics as part of an overall V & E strategy and encourages ongoing dialogue with employees related to V & E."

Response: TSB does not have a formal Values and Ethics plan but does engage in ongoing activities related to V & E (e.g., new employees are provided with values and ethics guidelines including the TSB Code of Professional Conduct; training on Respectful Workplace is incorporated as part of mandatory training). The Chief Operating Officer engages in a discussion on values as an integral part of the TSB employee orientation program for all new employees. The designated senior official sends periodic reminders on values and ethics to all employees. TSB is a small agency and as such, its leaders are able to communicate fairly closely with all its employees on a regular and informal basis on public service values and ethics. This is demonstrated by the TSB's positive results to the 2008 PSES where employees reported that essential information flows effectively from senior management to staff, as well as by the absence of problems, complaints or issues. However, the TSB will review its communication strategy for Values and Ethics in light of the government-wide TBS initiative to update the Values and Ethics Code for the Public Sector and assess whether any adjustments are required.

  • "The 2008 PSES results were below average for employee perception that senior management will resolve concerns raised in the survey was below average."

Response: The department takes the PSES results seriously. A working group composed of managers and employees was formed and it analyzed the 2008 survey results at the time of their release. Key areas of concern were identified. The below average result in this area is due in large part to a single labour relations issue that concluded just prior to the PSES and it was therefore not considered as a significant on-going concern. An action plan was developed by the working group and approved by senior management to address the key areas of concern raised in the PSES. This action plan was distributed to all employees in October 2009 and a status report on actions taken was distributed in August 2010. Significant progress has been made on the approved action items and it is expected that all items will be fully addressed by the end of the 2010-2011 fiscal year.

AoM #2 - Utility of the Corporate Performance Framework

This Area of Management was rated as having opportunity for improvement. The two rated elements of "Strategic Outcome Clarity and Measurability" and "Program Activity Completeness and Alignment" were both rated below acceptable. The following three opportunities were identified by TBS:

  • "The Strategic Outcome should be refined into an outcome statement that is expressed as an end-state. It should clearly specify the difference or benefit the organization is trying to achieve."

Response: The TSB's strategic outcome was approved by Treasury Board in 2007. The TSB believes its strategic outcome is clearly worded. However the TSB will review the wording as part of the process to develop its 2011-2016 Strategic Plan.

  • "Program descriptions need to be further refined to express how each program uniquely aligns with the Strategic Outcome, the descriptions are currently duplicative."

Response: The TSB's program activities are distinguished by the different modes of transportation (i.e. rail, pipeline, marine, air) and are similarly worded as the activities and business processes are similar in each mode. The TSB will endeavor to refine the description of its program activities to capture any unique aspects applicable to each transportation mode.

  • "The organization is encouraged to develop a PMF."

Response: As seen in the TSB's 2010-2011 Report on Plans and Priorities, the TSB does have a partial Performance Management Framework (PMF) in place which specifies some expected results, performance indicators and targets. However targets have not currently been set for all performance indicators. As part of the strategic planning process underway in 2010-2011 to develop a new 5 year Strategic Plan, the TSB is working to develop a fully articulated PMF.

AoM #3 - Effectiveness of the Corporate Management Structure

The TSB was rated acceptable on its Business Plan and strong on its Governance Structure. As an opportunity for improvement, the TBS encourages the TSB to continue with its plans to develop a refined integrated planning process for 2010-11.

Response: The TSB will continue its work on refining and improving its integrated planning process.

AoM #7 - Quality of Performance Reporting

The TSB was rated acceptable for its Management Resources and Results Structure Basis and for Credible Information. However, Context was identified as having the following opportunity for improvement:

  • "The Departmental Performance Report (DPR) could provide explanations for performance expectations included in the Report on Plans and Priorities (RPP), but not the DPR. The DPR could enhance the linkage between resources and results by providing information on resource changes and their impacts on plans and results. The performance story could be enhanced by providing substantiation and discussion of corrective actions for instances where plans were not fully met."

Response: The TSB agrees with the identified opportunity for improvement. The TSB has worked to provide greater detail on its annual performance in the 2009-2010 Departmental Performance Report. The TSB will continue to enhance its year-end performance reporting in the Departmental Performance Report once its new Performance Management Framework is fully articulated.

AoM #8 - Managing Organizational Change

The TSB was rated acceptable for all three elements of this Area of Management: Scanning or planning for change, Engagement and capacity, and Achieving Results.

AoM #9 - Effectiveness of Corporate Risk Management

The TSB was rated acceptable in three elements of this Area of Management: Senior management is accountable; Implementation; and Integration. The element of Continuous improvement was identified as having the following opportunity for improvement:

  • "The TSB could clarify the responsibilities of senior management with regard to risk mitigation strategies and specify the links between organizational risk and priorities."

Response: The TSB is not in agreement with this observation. The annual planning process clearly identifies senior management's responsibility for identifying risks and setting priorities in response to those risks. The Report on Plans and Priorities describes senior management's analysis of risk factors, the strategic priorities in response to these risks and the individual projects designed to contribute to the achievement of the priorities. The TSB's Business Plan provides additional detail with respect to each project including the project manager, goals, activities, deliverables, timelines and resources. Senior management's responsibilities for risk mitigation and the link between organizational risks and priorities are well understood and articulated. However, the Corporate Risk Profile is being updated in 2010-2011 and adjustments will be made to better document the risk management process currently in place.

AoM #10 - Excellence in People Management

The TSB was rated strong in the elements of Employee Engagement and Official Languages and acceptable in the elements of Leadership, Employment Equity, Employee Learning and Staffing. The following two elements were identified as having opportunities for improvement:

  • Performance Management: The assessment is based on two measures:
    • The measure of whether supervisors provide feedback on employee performance and assessment against identified goals and objectives. The TSB was assessed acceptable for this element.
    • The extent to which (% difference) the department's performance rating for executives diverges from the established bell curve. The TSB was rated as requiring improvement because the distribution of performance ratings across its six executives did not match that of the established government-wide bell curve (i.e. at least 5% are rated as "Did Not Meet", at least 10% are rated as met most, etc.).

Response: Given the very small size of the department's executive complement; it is not feasible to expect that the ratings of TSB executives will consistently match the bell curve set at the government-wide level. The TSB believes that its performance management process for executives is rigorous and satisfies objectives, and that the performance ratings accurately reflect the performance of its executives. No corrective action is planned.

  • Integrated HR and Business Planning: The assessment was based on the following two elements, which were both assessed as requiring improvement:
    • The measure of workload and planning effectiveness which is based on responses in the PSES survey to four questions and identified a need for improvement in planning effectiveness.


    Response: The TSB has implemented a new integrated HR and business planning process starting with the 2010-2011 budget cycle for a more comprehensive and effective planning process. The 2010-2011 Business Plan includes the TSB risks, challenges and priorities and major projects as well as the detailed plans of each Branch.

    • The measure of overtime (higher than average overtime hours per employee)and succession planning (lower than average employees who were promoted internally) were both identified as requiring improvement.

Response: Due to the nature of its activities, TSB employees are regularly required to work overtime as part of investigations into transportation occurrences. In 2010-2011, the TSB is reviewing its stand-by system and exploring options to identify the most efficient, effective and economic approach to meeting its operational needs. With respect to succession planning, the TSB has challenges increasing the percentage of employees who are promoted internally given the specialization and uniqueness of many of its positions. Additionally, many of the positions are "one deep", that is there is only one person responsible for a specific task or function like in many other small departments. This means when these people leave the organization their skill sets are not available internally and replacements must be sought externally.

AoM #12 - Effectiveness of Information Management

TSB was rated acceptable in the elements of Governance, Access to Information and Privacy Act. TSB's Strategic Planning and Implementation was identified as having the following opportunities for improvement:

  • "Ensure IM governance and strategic planning address, where possible, all activities described in the IM Internal Services Profile."

Response The TSB is reviewing its IM strategy in light of new TBS policies and directives relating to information management and will incorporate the above opportunity for improvement as part of its strategy.

  • "Continue with promulgation of IM accountabilities, roles and responsibilities throughout the organization."

Response: The TSB has provided IM awareness training in the department over the past year. It will consider expanding this training in light of new TBS policies and directives relating to information management.

  • "Develop and implement IM strategy and implement reporting and monitoring on the IM strategy, e.g. timelines, results to date, resourcing, etc."

Response: The TSB is reviewing its IM strategy in light of new TBS policies and directives relating to information management and will incorporate the above opportunity for improvement as part of its strategy.

  • "Ensure that the locations of all information holdings are documented and that retention plans are in place."

Response: By the 2010-2011 year-end, the TSB will have completed the implementation of its new File Classification Structure for all information holdings. The Multi-Institutional Disposition Authorities (MIDAs), issued by Library and Archives Canada, have been incorporated in the new file plan for all administrative records. A records inventory and clean-up project is now underway. The planned review and update of the IM strategy will address the updating of retention plans for all program records, as well as the updating of IM tools and systems.

  • "Develop consistent metadata that can be applied to all information holdings."

Response: A review and standardization of metadata structures existing in the department's modal databases is underway. A project to identify, describe and integrate metadata information in TSB systems is planned for 2011-2012 to ensure compliance with the new federal standard on metadata.

  • "Ensure that all institutional functions, programs, activities and related information holdings are identified and described in Info Source."

Response: A review of information holdings and personal information banks is currently underway. The changes will be published in the next annual update of Info Source.

AoM #13 - Effectiveness of Information Technology Management

TSB was rated acceptable in all three elements of Leadership, Planning and Value. TBS commended TSB for its progress and encouraged the department to continue to improve and share on its best practices.

AoM #14 - Effectiveness of Asset Management

The TSB was rated acceptable in this Area of Management. The following opportunities for improvement were identified:

  • "Finalize draft policy on capital assets and disseminate to staff."

Response: The TSB agrees and intends to have the policy finalized and distributed by December 31, 2010.

  • "Develop performance targets and indicators and measure performance of materiel management."

Response: The TSB does not agree that structured performance measurements are required in the area of materiel management given the small size of the inventory and the relatively low dollar value of materiel in the department. User feedback and performance issues relating to materiel are identified and addressed as they arise.

AoM #15 - Investment Planning and Management of Projects

The TSB was rated acceptable in the elements of Investment Planning and Management of Project Resources. The following opportunities for improvement were identified for the element of Management of Project Results:

  • "There is an opportunity for the Department to develop funding models that generate effective project cost estimates and support the achievement of expected project outcomes."

Response: The TSB agrees with the identified opportunity for improvement.  The department has identified the development and implementation of costing tools for major investigations and projects in its 2010-2011 Business Plan.

  • "There is an opportunity for the Department to develop a mechanism to ensure that lessons learned from past projects are used to support continuous project management improvement across the Department."

Response: The TSB does not agree that a formal mechanism to track lessons learned is required. While lessons learned are a very important part of how we do things at the TSB, the department reports on its projects results and consequential lessons learned through its existing governance structure over projects.

AoM #16 - Effective Procurement

The TSB was rated acceptable for the elements of "Governance and Oversight" and "Meeting Operational Requirements". Although no elements were assessed below acceptable, several opportunities for improvement were identified by TBS:

  • "TSB should conduct another comprehensive audit of its procurement and contracting practices similar to the one that was previously done in 2005."

Response: The TSB participated in the TBS Horizontal Audit of Contracting for Professional, Technical and Temporary Help Services in Small Departments and Agencies which was released by TBS in March 2010. No significant findings were noted for the TSB during the audit and as a result procurement and contracting are not considered high risk or priority for an internal audit.

  • "Unless written in its legislation, the TSB does not have full authority to enter into legal services contracts without the authority of the Minister of Justice as outlined in the Government Contracts Regulations, section 4."

Response: The TSB will review, validate and streamline its process for the procurement of legal service contracts.

  • "Surveys should be conducted to determine the client's level of satisfaction with regard to the procurement process."

Response: The TSB does not agree that surveys are required to determine client satisfaction. Given the small size of the department and the very limited number of employees involved in the procurement process, feedback on procurement activities is readily available through existing departmental committees (e.g. Management Committee, Executive Committee) and direct internal communications.

  • "TSB should create a guideline or policy on acquisition card usage considering that acquisition card transactions represent a large percentage of TSB's total procurement."

Response: The TSB has a policy on acquisition card usage. The policy will be reviewed, updated and communicated to all card users.

AoM #17 - Effectiveness of Financial Management and Control

The TSB was rated acceptable for the elements of Quality of Financial Reporting, Financial Systems and Internal Control Frameworks, and Independent Reviews. The following two elements were identified as having opportunities for improvement:

  • Authorities and Policies: "TSB needs to focus on improving its compliance with management policies and its costing practices."

Response: The TSB complies with all TB management policies. The TSB will review its procedures and tools to ensure business practices are properly documented as per TBS policies and directives. The TSB agrees it would benefit from better costing practices and, as identified in AoM #15, has identified the development and implementation of costing tools for major investigations and projects in its 2010-2011 Business Plan.

  • Management Capacity: "More attention is needed in increasing the number of staff who have approved learning plans. Succession planning processes need improvement and management team stability can be improved by decreasing the number of positions that are vacant and/or temporarily filled."

Response:The TSB disagrees with the assessment that its management team stability requires improvement. The assessment should be based on a review of vacancies over a period of time rather than simply at the date of the MAF assessment. The Chief Financial Officer (CFO) was in place for nine years and the Deputy CFO was in place for three years prior to the change in staff. The department experienced significant turnover during 2009-2010 in its finance operational positions; however, this was due to factors beyond management control. The department does ensure that learning plans are completed and approved for employees. Due to the high turnover, approved learning plans were not in place for certain employees at the time of the MAF assessment; however approved learning plans are established for all employees and are updated on an annual basis. Succession planning is done to the extent possible in a small department.

AoM #18 - Effectiveness of Internal Audit Function

The TSB was rated acceptable for this AoM and was recognized as demonstrating leadership towards strengthening internal audit across government.

AoM #19 - Effective Management of Security and Business Continuity

The TSB was rated acceptable for the elements of "Departmental Security Program" and "Management of IT Security (MITS)". Although no elements were assessed below acceptable, several opportunities for improvement were identified by TBS:

  • "Establish action plan to meet expectations of Policy on Government Security, including development of a departmental security plan and establishment of processes for on-going monitoring of policy compliance and security program effectiveness."

Response: The department is currently drafting its Departmental Security Plan. This will include a security awareness plan with an education element for current employees on regular security issues and monitoring of policies.

  • "Pursue initiatives to maintain and improve Management of Information Technology Security (MITS) compliance and consider sharing best practices and lessons learned with the Government of Canada ITS community."

Response: The department will continue to maintain and improve MITS compliance as part of its ongoing strategy to strengthen IT security. The department continues to share lessons and best practices with the GC IT community through participation and membership in interdepartmental committees. The TSB Manager of IT is currently the Vice-Chair of the Government of Canada Heads of IT committee.

  • "Develop statements of sensitivity for all IT systems and information holdings."

Response: The TSB disagrees with the requirement to develop statements of sensitivity for all IT systems and information holdings. Our limited resources are focused on sensitive information holdings and critical IT systems. The TSB maintains an inventory of sensitive information and critical IT system holdings. Statements of sensitivity are developed for IT systems that hold or process sensitive information, or that support the critical business functions. The TSB will review its inventory to see if any additional IT systems or info holdings require a statement of sensitivity.