Before putting your data products in production, make sure they're actually *products*

Is your data product actually three (or thirty) data projects hiding under a trenchcoat?

As a largely self-taught product manager (like most of us), I’ve had to try and figure things out as I go. 

Over the past few years, I’ve read lots of pieces around my domain (data, analytics, and ML) that talk about operationalisation and productionisation. I’m sure you’ve seen the headlines: How the vast majority of ML models never make it past POC, and how value from data & analytics projects can only really be derived when they’re operationalised and scaled across a business. And it’s certainly good advice! We shouldn’t rely on one-off recommendations, and of course if a business is to be data-driven (or at least data-informed) it should democratise data and insights to more than a handful of people. 

While this advice made sense, I’ve often found it largely irrelevant to the immediate challenge of scaling the data products I was responsible for. Our actual problem was a little more embarrassing: We didn't quite have a 'product' in our hands yet.

I wanted to share my thoughts on the challenge of going from data projects to data products. To that end, I’ve split this post into three broad parts:

1. Warning signs your product isn't a product (yet)

2. The likely root causes that got you here

3. My advice for getting unstuck

If this sounds like it might be true in your data teams or clients, have a read, and let me know what you think! I'd love to hear how common (or uncommon) this experience has been for folks.

Product? More like three ad hoc projects under a trenchcoat

Across the data teams I’ve worked in, we’ve often had to work in a very project-centric fashion. Sometimes this was a deliberate decision. Other times, there was a vision from the top to develop scalable products and reap the exponential returns they unlock. Despite that, we found ourselves delivering projects anyway - at least at first.

Had it been a deliberate decision, that would have been fine. But it wasn’t, at least not fully.

Warning signs your product isn't a product (yet)

I'm sure this isn't an exhaustive list - but it's a list of things I've seen come up several times.

• When you need to generate new outputs (a data file, a dashboard, a report), someone needs to modify code - for example, to change a hard-coded value like the date range you want to use, or generate an output for a different customer's domain.

• Developers don't work off a single, unified codebase.

• There are many versions of similar scripts or notebooks - probably because they're hardcoded to match different project requirements, or because they were made by different developers in the team.

• Developing new features, or adjusting existing ones, is a very cumbersome process that takes inexplicably long amounts of time to do.

• A broader sniff test: Every time you have a new customer/user, you need to invest time and effort to make your product work for them. I'm not talking about new features or use-cases here, or configuration, or about ingesting their data, but about what happens with that input data until it's been turned into the output your product is about.

How did we end here?

Why does this happen? Besides any incidental or individual drivers, I see three structural factors at play: The skills and know-how of the technical team, the business/product team, and the priorities of the business at large.

1. Lack of software engineering experience among the data/dev team

First off, many data teams consist of data scientists and analysts that haven’t come from a software background, and so aren’t experienced in the sorts of engineering practices common in software teams. This includes knowledge of git, parameterisation of inputs, automation of workflows, test-driven development, CI/CD, code optimisation, and so on. This means that the way things are built from the start may not be generalisable, refreshable, automatable, or otherwise scalable. The crucial note to make here is that I’m not talking about building production-grade systems instead of MVPs: It often takes the same amount of time (or at least not orders of magnitude more) to build something that’s relatively generalisable versus something completely bespoke.

The example I’ve seen over and over again (and been guilty of myself at times!) has been with hardcoded inputs: You specify the start and end date within your code, rather than treat it like a parameter that can be tweaked without having to edit various lines in the code to e.g. get data for July instead of June. Had the person or team doing the development had this in mind from the start, they might've had to slow down by 20% or even 50% for the first delivery but end up with something that can be reused at will.

2. Lack of software (or product) experience among leadership (and/or the product team)

Separate from the technical delivery team’s skills and background, there’s the profile of the data team's leadership. Who 'leadership' is depends on the org structure - it could be the PM, it could be a Head of Data/BI, or it could be the commercial director that's building a data or data science team under them.

When your entire professional career has been about delivering projects, you think in terms of, well, projects. So even if you’ve been given the mandate to develop an internal product for a certain recurring use case (e.g. pricing, sales, marketing), you might end up treating it as a series of projects: It might be the “X use case project” versus the “Y use case project”, or the “Country A project” versus the “Country B project”, or even the “Q1 recommendations” versus the “Q2 recommendations”. I've been guilty of this as the Product Manager, and I've seen my peers in data teams do the same.

This could happen in subtle but recurring ways, like the way you (de)prioritise tech debt, or in terms of bigger design decisions taken. It might be in the form of listening to those on your team that speak your language better (BAs, PMs, data scientists from a business analytics background) rather than those who ‘don't seem to get it’: Data engineers, more junior data scientists (who might be more technical), or otherwise anyone that's banging on about technical stuff that doesn't seem as important as your next big milestone or commercial priority.

My advice: Listen to those voices too, and try to understand them (and help them explain). It might not be easy - but if it was, wouldn't their concerns and suggestions have been heard already?

3. The higher-level priorities of the team, department, or company

Even if you have identified all the right bits of technical debt and understand the importance of productising your not-quite-a-product product, your hands might be tied, at least in the short term. It might be that the project-after-project approach was a commercial necessity, or a deliberate approach for the purpose of seeking out product-market fit. Perhaps the pursuit of bespoke solutions for different internal teams was a political decision, to appease other leaders who also only spoke the language of projects. Or, well, maybe your senior team had a broad vision of data products scaling across the business, but lacked either the specific know-how or the time to get stuck in the right level of detail to ensure that vision became a reality.

This might overlap heavily with the previous root cause: It depends how large and layered your organisation is. In a smaller company, leadership and product/business might be the same folks. In a larger enterprise, there will likely be many layers of decision-makers influencing your priorities. If you're in an empowered product team, I'm willing to bet the sort of challenge I discuss in this post isn't one you're dealing with, but I'd be happy (and curious!) to be corrected.

---

To recap, your product might not quite be a product yet for a variety of reasons, ranging from a lack of technical expertise, product leadership, or simply the fact that senior leadership’s priorities lie elsewhere. Whatever the reason or reasons, you've ended up stuck in a less-than-ideal situation - but also one that's solvable.

To get unstuck, make sure everyone knows you're stuck

So much of Product Management ultimately comes down to effective, high-impact communication: In order to get buy-in for your new productisation initiative, you'll have to get appropriate buy-in.

Here's my generic but effective template for what this looks like:

1. Articulate the problem and the pains it causes;

2. Paint a picture of a world with the problem solved;

3. Show what it takes to get there

How you go about gathering the ammunition for these three sections will depend on the situation you're in: Look back at the potential root causes of the state of your product to understand which levers are available to you.

• Tech skills gap: If the challenge is a skills gap among your development team, look at adjacent teams first - for example, is your team comprised of data scientists, who in turn work with a data engineering team further upstream? They might be able to help share best practices or even become the team who'd be responsible for taking over ownership of the data asset and helping turn it into a data product.

• Management skills gap: If the problem is that the technical team's calls for tech debt being prioritised have been ignored, then that team will help you get your technical roadmap/next steps nailed, as well as point to all the specific issues you're dealing with. Just make sure you're presenting the information the tech team gives you in a way your audience will understand. As an aside, they'll probably really appreciate you for it, and your working relationship will grow stronger, which will pay many dividends in the future.

• Leadership buy-in: If you've found yourself in a situation where those above you (or those above them, or those above them) are the blocker, it's time for some upwards management: Educate your superiors on the situation (without going into unnecessary levels of detail that will make them lose interest or the ability to follow), emphasising the outcomes you want to achieve. Make sure you've done your homework on what you're actually asking for: Is it to push back on some delivery dates to make room for productionisation? Is it to change what your sales team is offering to clients into something more standardised?

Some final notes for anyone embarking on a productisation initiative

• Sell the benefits: By standardising and automating your capability to produce data deliverables into a product, you are making the lives of everyone involved better: Your tech team can focus on developing new features (or improving existing ones, like data quality) instead of doing tedious, repetitive work, your customers can get their outputs faster and cheaper, and the business can benefit from getting exponential returns on their investment. Honestly, it's pretty neat.

• You can't do this alone: You'll need both the input and support from your team, peers, manager, users, and stakeholders: The plan for productisation will probably come mostly from the tech team, or other technical experts. Your manager or other leaders will likely have great advice on how to get wider buy-in from the organisation. Your users will help you understand what your product will need to evolve into. Other teams may have gone through a similar journey previously, or may also need to go through this now.

• Don’t fall for the Spotlight Effect: Your product may be your whole world (or at least your main professional focus!), but it's probably just one of many (or many, many) concerns of your managers, their managers, and anyone else you need to influence across your organisation. Be clear on why this matters to the things each of them cares about, tailor your comms approach for each one, and remember you may need to repeat your message a few times before it sinks in.

PS: If you’re wondering about the cover photo, I used Stable Diffusion’s AI art tool, DreamStudio - really fun to play around with, but definitely takes patience and specificity to get anywhere near something you want!