The following two questions “What will you evaluate? How will you evaluate it?” are answered for my case study with a series of questions and answers.
What do we want to know?
We want to know if the culture changed and are the desired behaviors aligned to create increased organizational alignment and increased individual engagement with production employees.
Although this seems like it is a simple focus it is very complex and not easily answered. Multiple data pieces already exist, and many do not exist; more significantly the data is not aligned to specifically allow us to understand if the culture has changed allowing increased organizational alignment.
How will we know we have been successful?
Clarity of our objectives with specific metrics and timing, as well as built in evaluation of the overarching objective and result.
We would see the following behaviors:
- Collaboration with candor and honesty, investing in our people, empowering each other
- A shared future
- Aligned results with each shop
- Meet our production and performance KPI goals
What specific kinds of data will we need?
Both qualitative and quantitative data.
There would be a communication plan, a pulse survey for both leaders and recipients connected to evaluation would measure understanding and impact
The engagement survey would reveal satisfaction and engagement: increased engagement, increased use of desired behaviors, trust, communication, and increased team scores
HR records would show: decreased absenteeism, attrition rates, and internal promotions
EITMS and VSM reviews would show increased number of projects started and completed that are aligned to the KPI’s.
Tableau data analytics would give us both raw data and variety of dashboards that support both production and HR.
Does the data exist? If so, where? Who owns them? Can we get them?
No data exists for the following: communication plan and impact; pulse survey and evaluation, high level program KPI alignment on every KPI, EITMS and VSM projects. Individual shops have some shared KPIs (which have data) and some independent and relevant KPIs (data is only measured by the shop business team). There are many individual systems like EITMS (independent system that supports team problem solving and projects) and VSM (lead by engineering to support shops in manufacturing production needs) as well in shop servers that store local projects. It would be a huge under taking to bring the information together for evaluation, so most likely it would be asking each of these areas to contribute data.
Quantitative Data:
Specific KPI’s will have quantitative data built into the system but the information would need to be gathered and analyzed
Engagement survey data: owned by enterprise HR. Raw quantitative data is not available. Summarized results by managers is available through tableau HR data reports. It currently does not include specific questions on trust or communication
Qualitative Data:
Engagement survey data: owned by enterprise HR. Raw & summarized qualitative data is not available from text inputs.
How should we display the data?
A dashboard so that everything is in one place. Often data already exists in other dashboards; and when this happens using an image with a link to the pertinent data.
How often should data be collected? How often can they be collected? Who will collect them?
There are so many homes for the data. It is very complex and timely.
For the entire change event, initial baseline data should be collected before the projects starts, mid-point and post change event so that a story can be told as well as any corrections/learnings can be integrated into the change event.
Data for production KPI’s would be continuously collected and used by individual teams to understand the business and production needs, these relevant KPI’s are measured daily. It would be beneficial to collect the larger picture of KPI success on a monthly basis.
Data for HR related KPI’s that affect the daily business are also available daily e.g., absenteeism where ask impact areas that are more trend related like attrition are maintained monthly through data analytics; engagement data in generally done annually.
Do we need permission to access the data?
Permission is required
Anonymity of individual employees is maintained
Data analytics for production is available within manufacturing. Data analytics for HR is on a limited basis.
What resources do we need?
This in itself is a big question as the business is complex with multiple layers and cross functions affecting day to day results. I would recommend a beginning, middle and ending point for each phase that clearly outlines focus areas, objectives, desired results and evaluation; as well as plans for continued changes and sustainment.
In addition to a dedicated change event project manager with relevant staff support to support the stakeholder team and larger project, SME’s (subject matter experts)that have access to data and are able to effectively analyze the information and trends are needed to participate and support data from each of the areas: Manufacturing Analytics, HR Analytics, Communications. We would also want a stakeholder and sponsor group that includes: Exec Leadership, manufacturing champions from both Structures and Final Assembly, Employee Involvement – to support facilitation of the individual groups as well as throughout the levels of leadership and manufacturing.