Table 4.1 Checklist for starting up the evaluation planning
What needs to be achieved through the OD intervention?
Ideally multiple levels of leadership will better understand that by using intentional and ongoing communication plan efforts they can create more alignment, impact and positive perception of the program’s executive leadership team communications flow down through multiple levels of leadership to production. Ultimately, they will positively affect the factory production employees and multiple levels of production leadership in our program to increase engagement and alignment in a common vision of continuous improvement to support organizational alignment of our mission and values (specifically operating as “one Boeing” supporting the Boeing Behaviors).
What would the desired change state look like?
Ideally more communication transparency and flow would help create a culture of increased organization alignment and increased individual engagement with production employees.
Production team members (leadership and manufacturing employees) would exhibit the following behaviors:
- collaborate with candor and honesty
- deliver results with excellence
- invest in our team and each other
- empower each other supporting our vision
- inspire and focus on a shared future
- trust that together we can meet the challenges ahead
- ultimately meet our production and performance KPI’s and goals
Increased engagement would mean that First Line Leaders and production employees would feel heard and input valued versus a “body” for production headcount.
I am hopeful that due to the sheer amount of engaged leaders that enough of them will recognize the benefits of more intentional and transparent communication flow and develop a better habit of creating more alignment and engagement. As their teams are more engaged, which can be measured by artifacts such as absenteeism and attrition rates, engagement survey scores and the number of individual team projects focusing upon continuous development which improves first time quality and overall production goals.
What metric will be used to measure success for the desired change after the intervention?
The first outcome of the evaluation is the big picture. To measure and evaluate current big picture state, an by using the artifacts we already have as a baseline. We will review production employee attrition, absenteeism, as well as satisfaction and engagement. Because this is a large and complex company, this information is connected but not necessarily causal to the current flow and perception of executive leadership communication flow down; so it is important to do our best to look for validity.
We will also want to use a criteria / scorecard to compare outgoing communication artifacts and well as a survey and 1:1 feedback from leadership. A baseline survey and interviews must be taken to identify the starting point. Questions related to culture, leadership, communication and trust must be asked. I would also want to use a pulse survey with communication recipients and the leaders to determine understanding and impact of the messages and larger plan.
Repeating these evaluation measures after communication change has taken place and finding continued positive response as compared to baseline evaluation results would be an indicator the efforts are successful.
This information will also be used to compare to ongoing evaluations since this type of behavior change requires ongoing efforts. Potentially different KPI’s will be identified due to root cause and learning.
Who/which are the targets of change (e.g., people, groups, units?) What level of change do we need?
Targets are:
- Program executive leaders: creating transparent and intentional communication that is flowed to program leaders/managers
- Program Senior Leaders: utilizing program message point to create intentional and aligned communication that is flowed to their respective teams of First Line Leaders / Managers
- First Line Leaders / Managers: flowing intentional program communication messages to production employees.
- Production employees: permission to be engaged and aligned in the larger program message, KPIs, choosing to show up to work and remain in this program.
Boeing Behaviors
- Lead with courage and passion
- Make customer priorities our own
- Invest in our team and empower each other
- Win with speed, agility and scale
- Collaborate with candor and honesty
- Reach higher, embrace change and learn from failure
- Deliver results with excellence – Live the Enduring Values
What are the specific evaluation expectations at each level?
Ultimately each level will have ownership of OD change when they are actively involved in decision-making processes that support increased engagement and alignment to the larger program and organization’s goals.
The inclusion of evaluation during multiple stages will assist the program in staying on track and aligned throughout the communication and engagement project, as well as help determine if the effort achieved its key objectives of improving communication flow and perceived impact supports employees being more aligned and engaged.
Are the evaluation expectations aligned with the aims and levels of the change intervention?
We are looking for behavior change which ultimately means culture change. This type of change will benefit both the Program and the larger enterprise organization in aligning a “One Boeing” approach. The results of the behaviors can be increased alignment to a common goal and increased continuous team engagement which improves first time quality and overall production goals. By empowering all individuals within executive leadership, leaders/managers at multiple levels and ultimately the production employees, we empower the organization as a whole to support it’s vision.
Table 4.2 Influential factors of evaluation strategy planning | Demands for evaluations
Accountability for results
Hopefully the KPIs will provide us the foundation for ongoing evaluation and continuous improvement results as well as provide feedback that could lead to improving better aligned results, engagement and culture through better communication flow. Any newly identified investments will need to be reviewed with the executive sponsors for approval.
Attracting and retaining talent is a strategy that is key to competitiveness in the marketplace, and the measures used for this strategy indicate alignment between the organization strategy and evaluation.
Revising the OD intervention strategy
Although the HR department is already responsible for ongoing analytics and metrics for attendance, attrition, the bi-annual engagement survey, bi-annual pulse survey and the new Boeing Behaviors survey, they are not analyzing / evaluating the combined data results to focus upon increased alignment to a common goal and increased continuous team engagement which improves first time quality and overall production goals.
Once the communication project’s value has been fully established that we gain agreement with executive leadership, the senior leadership team will also need to get involved in refining the metrics and criteria for ongoing evaluation.
Although not all of these data points would be measured at the same time for this specific project, some potential KPIs to be used and further defined in evaluation of this project are listed below.
- Number of questions asked by employees at factory wide Leadership meetings vs Last year increase 10% in 2019.
- Personnel trust rate to our program (base line and follow up survey)
- Communication surveys to multiple level of managers to be tracked after each leadership program wide message (open rate, Likert scale on meaning and impact)
- Tracking communication messages impact (number sent, open rates, impact to be defined)
- Average employee absence time decreases as engagement scores increase
- Average manager absence time decreases as engagement scores increase
- Time to fill FLL manager openings decreases over LY.
- Number and attendance of training hours for emerging manufacturing leaders increases by 15% respectively
- Employee turnover rate leaving production positions in our program for other programs.
- Manager turnover rate leaving production positions in our program for other programs
- Employee motivation:
- Number of names submitted for Treasure chest recognition
- Number of names submitted for recognition
- Number of employee run projects increase by 10% over LY.
Finding a success story
The larger company continues to share success stories in “plane and simple” and BNN to inspire others and to serve as a roadmap of how to live the Boeing Behaviors.
A program communication plan to share big and small “wins” and stories would be helpful in creating increased awareness and focus upon the sustaining implementation of the desired behaviors with the leaders and potentially create a pull from the first line leaders from more senior levels of leadership.
Success stories could focus on how the communication flow impacts them to create better alignment and engagement. These are the people who will be most impacted by the changes of better aligned communication and increasing continuous engagement to meet on time and first quality production goals.
Organizational power and political situations
A potential obstacle of strategic alignment is ensuring that in addition to the executive leaders that the senior leaders are aligned to the intervention because the senior leaders don’t always pay attention to the ‘throughput’ – because although strategic direction can easily be set, achieving the strategic priorities in a constantly change and demanding production environment that constantly focuses upon daily production numbers is much harder. The words “the plane is boss” is a reality for production. Also due to the size of the organziation there are multiple cross functional areas that affect production that have their own goals. Executive leaders are four levels of leadership from the floor, and may not receive all of the “facts” to how the larger plan is is really serving the desired outcome.
Significance of results for employees health and safety
First Line Leaders and manufacturing production employees are the people who will be most impacted by the changes of better aligned communication and increasing continuous engagement to meet on time and first quality production goals. Their health and safety is affected in each decision whether it be mixed communication signals that causes employees to need to work 10-12 hours days including weekends to meet production goals, and an employee experiences physical strains, sprains or injury on the job or emotional strains both on the job and at home. The company focuses upon Go for Zero, injury and incident free and still has a ways to go. For a manager, long hours can cause poor decisions to be made and or altercations with team members that have lasting impact.