The other broader challenge is that many times, benefits cant directly be attributed to the data or AI project as data and AI are mostly a means to the end, not the end itself. This means you will need a benefits sharing agreement with the process owner that leverages your project to deliver benefits - this can be very difficult.
This is often compounded by any existing benefits framework which often don't factor in enablers like data.
100% - it’s not an issue in the earlier days of value estimation & demonstration, but over time as you need to formalise it splitting the credit becomes a point of friction.
My favourite approach on this has come from Benny Benford (JLR’s former CDO), who established a very simple heuristic approach instead of trying to go down a hyper-precise path
- 10% value for enabling activities (training, ingestion, data modelling etc.). i.e. if a business team will get £1m of benefit from a data set being made available, then the data team can claim £100k of value
- 50% value for data products where the data team works alongside a business team to deliver value
- 100% for data products that automate a decision-making process and deliver value with little business input
i find attribution to be one of the hardest things to do in data. I haven't been able to agree to % attributions at a company-wide level (maybe it works in small-mid sized companies), but what works is positioning data as an enabler on big strategic programs - this way you dont have to justify benefits specific to data and you can use the overall program benefits to cover data as well.
For smaller continous delivery projects, you can usually find productivity benefits, risk reduction and customer experience benefits and although these may be considered as non-financial in your company, CFO's are usually okay to let you bank these.
Also, i try to prioritise initiatives that align to the biggest and most visbile problems in the company e.g. risk reduction in financial services, improvement in safety metrics in mining, cost savings in utilities etc.
Most of all, i also find quantifying hard benefits alone is not sufficient, the best recognition of the value data teams deliver comes from the softer side e.g. better employee experience, increasing data literacy, improved productivity, and the buzz around the use of data in the company.
Working in finance I see similar problems everyday. Capex projects won’t get approved without a clear ROI but no one actually measures the ROI after completion.
You bring up a great point that we are incentivized on delivering outcomes not reporting if the outcome meets expectations.
Yup. We’re just lying to ourselves if the business case is just a little story we have to craft to get budget.
The worst part is it’s not like teams get away with it anyway - if you’re not delivering value, you’ll be a clear target next time there’s layoffs. But that expectation isn’t made clear upfront - the short-term incentives are tied around things like delivering against a roadmap on time & on budget, but the actual long-term assessment is done based on $$$
This is very good, thank you. I would say the number one challenge is that - for most D&AI projects - the KPIs that are specified are never thought through. No one ever asks HOW we are going to measure that thing. This was the biggest blocker in all the projects I worked on, and over the years I learned to reject projects that didn't clearly articulate what and how every KPI was to be measured, and by who. And as you say, ingriaing this in the project from day 1 is essential.
Thank you! 🙏 And yes, totally agree. But a lot of the time, folks think about it, then don’t speak up. (“If it was important someone would’ve said so already, no?”)
Slowly but surely I’m seeing more folks wake up to the need to speak up (or, if in leadership roles, to act up).
Good post Nick.
The other broader challenge is that many times, benefits cant directly be attributed to the data or AI project as data and AI are mostly a means to the end, not the end itself. This means you will need a benefits sharing agreement with the process owner that leverages your project to deliver benefits - this can be very difficult.
This is often compounded by any existing benefits framework which often don't factor in enablers like data.
100% - it’s not an issue in the earlier days of value estimation & demonstration, but over time as you need to formalise it splitting the credit becomes a point of friction.
My favourite approach on this has come from Benny Benford (JLR’s former CDO), who established a very simple heuristic approach instead of trying to go down a hyper-precise path
- 10% value for enabling activities (training, ingestion, data modelling etc.). i.e. if a business team will get £1m of benefit from a data set being made available, then the data team can claim £100k of value
- 50% value for data products where the data team works alongside a business team to deliver value
- 100% for data products that automate a decision-making process and deliver value with little business input
Substack won’t let me tag him, but his Substack is here: https://open.substack.com/pub/datent?r=3pv9x&utm_medium=ios
And he’s spoken about this heuristic approach in various places, e.g. around 11:15 on this podcast: https://driven-by-data-the-podcast.captivate.fm/episode/s3-ep-16-delivering-500m-in-value-with-benny-clive-benford-chief-data-officer-formerly-of-jlr
(Hit enter too soon)
How about you, what have you seen work well for benefit attribution? & what hasn’t?
i find attribution to be one of the hardest things to do in data. I haven't been able to agree to % attributions at a company-wide level (maybe it works in small-mid sized companies), but what works is positioning data as an enabler on big strategic programs - this way you dont have to justify benefits specific to data and you can use the overall program benefits to cover data as well.
For smaller continous delivery projects, you can usually find productivity benefits, risk reduction and customer experience benefits and although these may be considered as non-financial in your company, CFO's are usually okay to let you bank these.
Also, i try to prioritise initiatives that align to the biggest and most visbile problems in the company e.g. risk reduction in financial services, improvement in safety metrics in mining, cost savings in utilities etc.
Most of all, i also find quantifying hard benefits alone is not sufficient, the best recognition of the value data teams deliver comes from the softer side e.g. better employee experience, increasing data literacy, improved productivity, and the buzz around the use of data in the company.
Working in finance I see similar problems everyday. Capex projects won’t get approved without a clear ROI but no one actually measures the ROI after completion.
You bring up a great point that we are incentivized on delivering outcomes not reporting if the outcome meets expectations.
Yup. We’re just lying to ourselves if the business case is just a little story we have to craft to get budget.
The worst part is it’s not like teams get away with it anyway - if you’re not delivering value, you’ll be a clear target next time there’s layoffs. But that expectation isn’t made clear upfront - the short-term incentives are tied around things like delivering against a roadmap on time & on budget, but the actual long-term assessment is done based on $$$
This is very good, thank you. I would say the number one challenge is that - for most D&AI projects - the KPIs that are specified are never thought through. No one ever asks HOW we are going to measure that thing. This was the biggest blocker in all the projects I worked on, and over the years I learned to reject projects that didn't clearly articulate what and how every KPI was to be measured, and by who. And as you say, ingriaing this in the project from day 1 is essential.
Thank you! 🙏 And yes, totally agree. But a lot of the time, folks think about it, then don’t speak up. (“If it was important someone would’ve said so already, no?”)
Slowly but surely I’m seeing more folks wake up to the need to speak up (or, if in leadership roles, to act up).