Today, there’s a growing misconception that data science is a silver bullet. It can solve any problem, no matter how big or small, and it will make things easier for everyone involved. How, exactly? Well, that’s not as important.
This gap between what the C-suite expects from data science and the results that data scientists can realistically provide is dangerous. It puts pressure on data scientists to deliver on impractical expectations, and it causes C-suite leaders to lose faith in data science, quickly writing off initiatives that could have led to much-needed results.
A recent study from Wakefield Research showed that 82% of data executives said their employers have no trouble pouring money into “splashy” investments that yield only short-term results while ignoring the long-term benefits of data science.
So, how can you close that gap and collaboratively develop a data science strategy that shows value from the get-go and demonstrates consistent results? Let’s find out!
Three Steps for a Successful Data Science Strategy
At RapidMiner, we believe there are five core elements of an AI strategy—mission, drivers, vision, strategic goals, and tactics. When you think of developing a data science strategy, the first thing that comes to mind is probably finding ways that AI can help you get a leg up over your competitors and develop a better, more profitable product for your customers.
But, how do you get there? Seriously, step-by-step, how do you do it?
This post is all about the “tactics” piece of an AI strategy—aligning with a vendor that supports you and your team, determining the right metrics to track, and setting use case-driven goals to show progress.
1. Align With a Vendor That Enables Your Team
The road to a successful AI strategy is a long one, and it’s not one you should embark on alone. But, when it comes to selecting a vendor to partner with, it’s important to choose one that not only helps you operationalize models more effectively, but also prioritizes every type of user in your organization.
One of the “strategic goals” many organizations have when developing their data science strategy is enabling employees to complete data science projects on their own, whether they’re machine learning engineers or citizen data scientists. Tactically speaking, aligning with a vendor that provides a collaborative data science environment allows coders and non-coders to participate in data science, developing a deeper understanding of the data they’re working with and facilitating meaningful collaboration.
But wait, there’s more! Working with a vendor who just provides visual workflows and coding environments isn’t enough. What your team really needs is a platform that supports true upskilling of both data scientists and business experts. Teaching data scientists the business context behind the models they’re building while teaching subject matter experts how data supports the business’s daily operations creates a significant, rapid impact on the organization.
In the spirit of upskilling your entire team, RapidMiner created RapidMiner Academy, an online learning platform that teaches beginner to advanced data science and machine learning topics. While we use RapidMiner in our training videos, the Academy teaches users data science concepts beyond our platform—including mapping real-world problems to AI use cases and creating replicable ML models.
As you develop your AI strategy, it’s important to choose a vendor that values your users’ continued growth just as much (if not more!) than providing essential capabilities and features to support your ML projects.
2. Gather & Track the Right Metrics
Tooling alone isn’t enough—you also need to track key metrics that prioritize long-term value and return on investment. Metrics not only give you a baseline to build off of, they also help data science initiatives gain leadership buy-in without needing to show that models are leading to revenue growth right away.
Unsure of where to start? From our experience working with customers, here are a few metrics we recommend keeping track of:
Component Reuse
One great way to show data science effectiveness beyond just one model is by tracking component reuse as a KPI. When DS teams create a widely used component, like a particular customer dataset or data diagnostic tool, it helps future models get produced and deployed faster.
When teams standardize templates, easily access and reuse high-value data, and preserve effective software configurations, they can work smarter and produce results in which they have more confidence. Component reuse indicates how productive a project is for future use cases and generates value even after its original use case is complete.
Team Productivity
Productivity is about more than just the number of models created and deployed. By setting targets to measure productivity in a multitude of ways, data scientists can prove that they’re not wasting time every day they don’t produce a new model.
Example tasks to report on could be collecting pertinent data from a user activity database, preparing a report on exploratory data analysis, or creating a base model with 65% accuracy.
Stakeholder Use and Feedback on End Product
Gathering direct feedback from customers is the best way to tell if models in production are having their intended impact. Have stakeholders been able to accomplish what they intended with the model? Has the model reduced customer churn by a certain percentage, or led to more cross-sell and upsell opportunities?
Feedback is also great to see how well a model is working, and if it needs maintenance. Is it performing better than the baseline, or has it flatlined (in which case, it might need to be retrained)?
A Few More Ideas
These examples aren’t the only metrics necessary to measure success. If they make sense for your organization, you can and should keep track of:
- Projects your team has in progress, upcoming, and in the backlog
- Completion rate
- Efficiency/time spent on value-added activities
- Number of competencies/new skillsets gained from executing a given project
3. Set Use-Case-Driven Goals & Demonstrate Valuable Results
While strategic goals deserve a discussion of their own, use-case-driven goals align closely with the tactical piece of your strategy.
Use-case-driven goals are developed entirely around a specific use case, and they can look like:
- Increase retail sales this month compared to last month
- Use a model to correctly identify credit card fraud 90% of the time
- Train citizen data scientists on one new type of ML algorithm this quarter
These are operational goals, rather than strategic—they’re the daily milestones your organization needs to reach to achieve that longer-term strategic vision. But, by setting (and reaching) short-term goals, your team can easily align the work they’re doing every day with target business outcomes, using the metrics you’ve defined to support them.
The fact is, with data science, 99% of the time you won’t see immediate results. So, how do you zoom out and show leadership that you’re still providing value to the organization day in and day out?
One strategy for data scientists to show ROI is by leveraging control groups. If you’re rolling out a new model for customer segmentation, you can use a small holdout group that doesn’t participate in the model. Then, after a predetermined amount of time, compare the average revenue attributed to customers whose data was analyzed by the predictive model with customers whose data wasn’t analyzed. The results should be crazy different, and in your favor, showing the direct impact of your models while also building up your team’s credibility.
Wrapping Up
Creating a sustainable, long-term data science strategy that can demonstrate value in the short term may seem like an impossible task. But, by selecting a platform that encourages collaboration and gets your team invested in meeting your goals, you can (and will!) get there—one step at a time.
Eager to learn more about turning “AI” from a buzzword into a business plan? Check out our webinar with Michael Gualtieri from Forrester Research, 5 Elements of an AI Strategy: Deliver Rapid Results & Long-Term Success, where we break AI down to its core elements so you can easily execute on it.
