If you cannot measure your business processes, you cannot improve them. I think everyone at least once heard this statement in you professional life. When it comes to product lifecycle management, service lifecycle management and change management, the question about measurement or metrics is becoming super critical. As a manager, picking the right metric is absolutely critical. It will help you to focus your product data management and overall quality of your business systems as well as your product development process.
Earlier this week, I came across an interesting article by Martijin Dullart – Using the right Metrics in Configuration Management. My absolute favorite was – “The road to hell is paved with good intentions and the wrong metrics!”. It is so true… imagine you pick the number of ECO your organization issuing. Both maximizing or minimizing the number of ECO leads to bad situations – too many won’t give you much. but not enough will not be good as well.
Measuring what truly matters is often a challenge. Metrics should illuminate progress toward desired outcomes, not obscure it with irrelevant or misleading data. While metrics like speed and quantity might appear appealing at first glance, they can lead to questionable outcome because they will trigger wrong behavior. Therefore, you need to think about good metrics that can trigger quality and alignment with business objectives. Good metrics give you a control over the entire product lifecycle, product quality, production process and not focusing on the narrow disciplines such as document management and product lifecycle management.
Speaking about metrics, you cannot avoid the conversation about how to get those metrics in your organization. Additionally, I will touch on the topic of AI and how it can refine metrics for your organization.
Good Metrics: Focusing on Quality and Business Outcomes
Good metrics connects a change effort to a measurable outcome – improvements in one of the business parameters – quality, user satisfaction, and business objectives. They are actionable, specific, and aligned with business goals (preferable long term).
Here are three examples of such metrics:
- Defect Reduction Rate
Measures the percentage decrease in defects reported X days after a change is implemented (usually I pick a month or a quarter, depending on the volume of the activity). It is a clear picture of the quality of improvement. If the defects rate goes down, you certainly do something right. - Customer Time Saved (or anything User Experience specific)
Measure what customers do with your products and how much faster users can complete the task after a change. This metric offers tangible evidence of enhanced user efficiency and satisfaction. If you make a product, which is user focused, such KPI is priceless. - First Time Success Rate
This is an interesting metric I found useful, because it makes an assessment of successful planning. Calculates the percentage of changes deployed without rollbacks or hotfixes – high first-time success rate demonstrates robust planning, testing, and deployment processes. It is an indicator of smooth operations.
What other “good metrics” you know? Please share.
Problematic Metrics: Quantity Over Quality
Productivity and efficiency although been used a lot of engineering organization and “easy to grasp” are actually not a good idea based on my experience. Here are a few examples include:
- Change Volume
Counting the total number of changes made can push teams to prioritize quantity over quality, leading to wrong implementations. The number of changes cannot be maxed or mined. More problems is bad, but little problems can be a problem too. - Deployment Frequency
To deploy too much or to many doesn’t really say something about quality metrics. Think you can maximize and you compromise testing. Saving on deployment (especially service pack development) can lead to another group of results. - Change Pipeline Velocity
This is one of the most typical. It is technically can give you an idea about how fast you can do your job, but can be confusion. This metric (although sometimes questionable the broader business impact and stability of those changes.
From my perspective, these metrics (3 above) fail to provide a holistic view of change effectiveness. .
Enhancing Metrics with AI
AI technologies can revolutionize how organizations track and leverage metrics. Therefore, when we speak about modern PLM software the importance of automatic assessment of various aspects of busienss process, but most importantly – configuration and change management as they represent the core of activities such as design and engineering phase. By automating data collection and analysis, AI eliminates manual errors and enhances decision-making. Examples of AI-driven enhancements include:
- Predictive Analysis: AI can identify patterns in historical defect data and predict potential risks, enabling proactive process adjustments.
- User Behavior Insights: AI-powered tools can measure time savings by analyzing user interactions, providing a clear view of efficiency gains.
- Real-Time Monitoring: AI can analyze deployment logs to track first-time success rates, flagging potential issues before they escalate.
- Dynamic Dashboards: Automated dashboards offer real-time insights, reducing reporting overhead and empowering teams with actionable data.
What is my conclusion?
Pick the right metrics for your organization. Effective change management relies on using metrics that matter. Prioritizing quality-focused metrics ensures that change initiatives drive meaningful results. By integrating AI tools, organizations can elevate these metrics’ precision and scalability, fostering a culture of continuous improvement. Check how modern PLM software allows to bring the right metrics to your organization using AI, graph data science and other modern technologies.
What metrics are you using to measure change in your organization? Share your thoughts and let’s discuss how to make them even more effective!
Just my thoughts…
Best, Oleg
Disclaimer: I’m the co-founder and CEO of OpenBOM, a digital-thread platform providing cloud-native PDM, PLM, and ERP capabilities. With extensive experience in federated CAD-PDM and PLM architecture, I’m advocates for agile, open product models and cloud technologies in manufacturing. My opinion can be unintentionally biased
