Evaluating Change:
Below is the business case for evaluating a change effort. The evaluation will focus upon one programs executive leadership team’s communication and flow down within production because the company is large and complex.
Small and large changes in the production system transformation are concurrently in process and competing. Systematic changes do not happen quickly as well as affect thousands of people in cross functional areas which creates layers of complexity in both perceptions and desired realities.
Background:
As Boeing transitioned from the past enterprise CEO to Dennis Muilenburg the approach simplified to “One Boeing” with a consistent vision, mission, values and behaviors. One Boeing is an expression of the company’s purpose and behaviors designed to inspire and focus all employees on a shared future and to reaffirm that, together, we can meet the challenges that lie ahead. Individual programs align their goals and strategy to the One Boeing approach and specifically the Boeing Behaviors. My program is part of the commercial division of the larger enterprise organization.
Expected Outcome: Communication and Aligned Engagement
Through the lens of Employee Involvement, where I currently work; seek to understand how the flow down of the program’s executive leadership team communications flow down to production and how perceptions of these communication efforts affect the opportunity for factory production employees and multiple levels of leadership to be engaged and involved in a common vision of continuous improvement to support “One Boeing” specifically the Boeing Behaviors which are listed below.
Boeing Behaviors
- Lead with courage and passion
- Make customer priorities our own
- Invest in our team and empower each other
- Win with speed, agility and scale
- Collaborate with candor and honesty
- Reach higher, embrace change and learn from failure
- Deliver results with excellence – Live the Enduring Values
The inclusion of evaluation will assist the program in staying on track throughout the communication and engagement project, as well as help determine if the effort achieved its key objectives of improving communication flow and perceived impact supports employees being more aligned and engaged.
To best support and make this a meaningful effort there will be three stages:
- Preintervention evaluation
This includes seeking to understand the current state and what changes we hope to achieve; using this information to set the objectives for the rest of the intervention outcomes
- What is the root cause?
- What do we plan to change?
- How does the program’s organizational size, cross functional complexity, resources, and culture affect capability to effect a change?
- Assessing the potential influences on the change effort—who will be involved in this change effort?
- What is the level of readiness for change?
- What potential training is needed to support the efforts?
- Evaluation methods may include: 1:1 interviews, focus groups, surveys, communication artifacts, HR artifacts
- Intraimplementation evaluation
During the implementation it is important to make sure the identified initiative is on track to meet it objectives.
- Selecting which objectives and Key Performance Indicators (KPIs) will be used for evaluation during the intervention process
- Determining how often the intervention will be evaluated
- Deciding what actions will be taken if the intervention is off-track or ineffective
- Postintervention evaluation
Assessing the overall results at the end of the designated time frame as well as potentially develop an ongoing continuous improvement to sustain the efforts moving forward.
We will ask the following questions:
- How effective was the intervention?
- Did it meet the objectives / KPIs?
- Is the change effort sustainable?
- What are the unexpected impacts both short term and long term?
- Evaluation methods may include: 1:1 interviews, focus groups, surveys, communication artifacts
(Jones and Rothwell, 2017).
Key performance indicators (KPIs)
Key performance indicators (KPIs), both financial and non-financial, are an important component of the information needed to explain a company’s progress towards its stated goals (PriceWaterhouseCoopers, 2007). Additional conversation with leadership will help define which KPIs will be most meaningful.
Some potential KPIs to be used and further defined in evaluation of this project are listed below.
- Number of questions asked by employees at factory wide Leadership meetings vs Last year increase 10% in 2019.
- Personnel trust rate to our program (base line and follow up survey)
- Communication surveys to multiple level of managers to be tracked after each leadership program wide message (open rate, Likert scale on meaning and impact)
- Tracking communication messages impact (number sent, open rates, impact to be defined)
- Average employee absence time decreases 10%.
- Average manager absence time decreases 10%.
- Time to fill FLL manager openings decreases over LY.
- Number and attendance of training hours for emerging manufacturing leaders increases by 15% respectively
- Employee turnover rate leaving production positions in our program for other programs.
- Manager turnover rate leaving production positions in our program for other programs
- Employee motivation:
- Number of names submitted for Treasure chest recognition
- Number of names submitted for recognition
- Number of employee run projects increase by 10% over LY.
Defined Outcomes
The first outcome of the Preintervention evaluation is to better understand our current state and the baseline of production employee satisfaction and engagement which is connected to the current flow and perception of executive leadership communication flow down. It can be measured in multiple ways and is dependent upon artifacts I am able to gain access to such as the most recent Employee Engagement Survey, the Boeing Behaviors Survey, absentee rates, open manager requisitions, manager turnover, employee motivation (see recognition and projects in KPIs above), previous feedback from focus group and 1:1 interviews on communication. Permission is needed, and I will use these as a foundational starting point to better determine objectives of what we plan on changing.
The second outcome, is related to pre, intra and post evaluation. The specific executive leadership communication flow down and impact, will be measured in two ways. The first way will be through an anonymous manager and employee survey. A baseline survey must be taken to identify the starting point. Questions related to culture, leadership, communication and trust must be asked. It is recommended to do one prior to the change effort to gather baseline results, and one post-evaluation period, which is to be determined based upon the change effort.
The third outcome will be to repeat these evaluation measures after communication change has taken place and will be related to potentially different KPI’s that can be identified due to root cause and learning.
Hopefully the KPIs will provide us the foundation for ongoing evaluation and continuous improvement results as well as provide feedback that could lead to improving better aligned results, engagement and culture through better communication flow.
References
Jones, M. C., & Rothwell, W. J. (2017). Evaluating organization development: How to ensure and sustain the successful transformation. Boca Raton, FL: CRC Press.
Jones, M.C., Pennsylvania State University (n.d.) WFED 585, Lesson 3: The Business Case for Conducting OD Evaluation and Appraisal, Retrieved From https://courses.worldcampus.psu.edu/canvas/sp19/21911—7892/content/03_lesson/04_page.html
PriceWaterhouseCoopers. (2007). Guide to key performance indicators: Communicating the measures that matter. London, UK: Author. Retrieved from https://www.pwc.com/gx/en/audit-services/corporate-reporting/assets/pdfs/uk_kpi_guide.pdf