Using the full RapidMiner Platform to Improve Sales & Marketing for the Customer Journey

00:03 All right. Thanks, everybody. I’m going to go ahead and get started. I am Elise Watson from Clarkston Consulting. I am one of our service leads for our insights to actions group. We focus on working with clients and partnering with clients to use their data to solve business challenges and to overcome some of their everyday issues. I’m going to talk to you guys a little bit today about a client I’ve been working on for the past about 10 months using RapidMiner to enhance their sales and marketing tactics. Some really interesting stuff on the sales and marketing side and ways that we can be more predictive and enhance that process. So the company I’m working for now is a pharmaceutical company. So get that life sciences hat on, right? We’re talking about working with doctors to sell more drugs, right? To increase those patients, to increase the way that we’re interacting with doctors to get that more engagement so that they can better adopt over time. So they brought us in to say, “Hey, we have a bunch of data. We have a few business rules that we have set in place. We know these particular doctors have a lot of patients, so let’s target them, right?” Some very basic business rules, but they had so much data that they had no way to comprehend it all at once, so they brought us in and they said, “How can we do that better?” So as Clarkston, one of the first things that we do when we get to clients is we do something called a business value workshop. So the goal of these business value workshops is really just to come in and better understand your business, better understand where we can provide value where data has value, and then we can really make things actionable.

01:31 So that’s where that insights to actions piece of our group name always comes in, is, how can we make things that are actionable? How can we generate insights that the business actually wants to do right now? Right? Because a lot of times we go to clients and we’ll say, “Here’s where you’re missing data,” or, “Here’s where you have gaps. Here’s where you can improve,” and they say, “Okay, great,” and then they don’t actually do anything. So it’s that gauge of the business side as well, to say, who’s ready to do this type of analytics? Who’s ready to take this action? So during that business value workshop, a few things were uncovered at this client. So a few goals that they were wanting to understand a little bit better, a few challenges that we needed to understand about their data, and an overarching goal was, how can we unify their sales and marketing tactics? They had these pockets of people analyzing data in different ways, some even doing the exact same thing, but just had different processes for how to do it. So one of the first things we did was we said, “Okay, we need to unify all of this together. We need to unify these processes,” and then we also just had a couple of goals come out of that. Once we have a unified process, we want to better understand each healthcare professional. Healthcare provider, sorry. So when you think about each HCP, there’s a lot we can know about them. We want to know how they’re prescribing. We want to know if they’re prescribing our drugs or are they prescribing other people’s drugs. Are they interacting with our material? Are they opening our emails? Are they talking to our sales reps? Are they looking us up online? Are they attending speaker programs? Right? All of this is around that one particular HCP.

02:58 So I’m not going to go too much into this, but we actually built out a graph database to help us with some influence scoring and some– between us, use a couple different graph algorithms to help enhance our machine learning models. So we’re taking the outputs from that graph as part of input to a lot of our predictive algorithms. What they also wanted to understand was to improve their customer experience. So their marketing strategy when we first started was this, like, “Let’s contact everyone about everything.” So it was this straight shot of just, “If you are on our email list, you’re getting every email we could possibly send you.” So not very effective, and they were seeing these really low engagement rates, and just that customer experience really high unsubscribe rates, because if you’re sending the same people the same information over and over again, eventually they’re going to get annoyed with you. Right? So it’s that idea of, how do we eliminate that annoyance factor? So they wanted to, in general, improve that overall customer experience. And then obviously, the – sorry – main goal was to increase drug adoption. How can we continue to get doctors to see our drug as a valuable drug, something that they want to prescribe to their patients? Right? A few challenges specific to this client. So they’re within rare disease and rare disease oncology. So think about an oncologist who sees maybe eight patients a day. Right? All of those patients could be liver cancer patients or a very specific type of cancer, whereas this one specific rare disease, maybe you see two of these patients a year. So how can we continue to stay top of mind for oncologists when we’re overshadowed by all these other disease states? How can we continue to make sure that they see our brand as important, that they continue to know our space?

04:39 Another interesting aspect to this client was that they have one main drug and one main product that launched about 10 years ago, and they had never had a competitor in the space. So this was a brand new- put it down. Brand new competitor in the space in the fall of last year, so it was a big freak-out of “oh my God we’ve never had a competitor. What does that mean for us? Are they going to take all our sales?” And without that historical view of, “We’ve had a competitor before. We know what space to get into here,” it was very concerning for them. So that was another big challenge. And then they also have some big data gaps, especially when it comes to visibility and to scripts. So they have two main channels that they see scripts on. One is through a specialty pharmacy, and another is through a hospital. So when you’re getting those prescriptions at a hospital, you’re seeing it at that organizational level versus at the specialty pharmacy you can actually see it at that HCP level. So understanding how to merge those gaps when we’re looking at overall patients and things like that. How can we kind of marry those all together? So all of this to say that during that first business value workshop, all of these things came to light. We were blown away with the excitement that came from them, and we came up with three main use cases. We’ve done a lot of other things with them in this time frame, but three big things that I wanted to share with you guys today, first being marketing channel optimization. I’m going to spend a lot of time on this, but this is an ongoing project for us. We’re continuing to improve this. It’s a very iterative process, whereas the other two, the competitor product launch as well as their new product launch, the analysis within there was more of a one-time thing, because data will continue to improve over time. So we’ll dive into that now.

06:23 First, thinking about channel optimization. So if I want to contact an HCP, what’s the best way to do that? What content should I be shooting them with? What channel should I be going after them? So is that an email? Is that an email from our client, or is it an email from a third party? Is it an email from a rep? Or is it that we actually need a rep to go out and talk to them in person? Or is it that we want to invite them to a speaker program? Right? These are all ways that we can contact an HCP and get them involved and better understand their space. But then it’s also the content aspect of it as well. So someone who’s experienced in this space, should we be contacting them with disease ed? Probably not. Right? They’ve already had a lot of patients. They’ve already been in the space for a long time. Why are we continuing to tell them about symptoms? Right? They probably already know. So some of the basic things that we just had to get out of the way first was some segmentation, right? This idea of creating that customer journey. So we used machine learning to find these four basic segments, to say– mostly based on experience. So say someone who’s not experienced versus someone who is experienced, and how can we increase that experience over time? So how can we get them to be more adopted? What can we send messaging-wise to these first people versus messaging to someone more advanced to increase that engagement over time? And then we also had this predicted values for engagement within each of the different channels. So we’re saying in an email channel, you are 20% likely to open an email. And it’s really difficult to think about, because 20% seems pretty low, right? But when your overall average for opens is like 4%, then 20% seems pretty good. Right?

07:53 So it’s that understanding of you have to see the whole picture of that HCP and understand that even though that prediction of 20% seems low, that still might be a business action that we want to take because it’s high in comparison. So these were some of the initial things that we had going for us, and we created this recommendation engine. So if you’re saying, “I want to contact this HCP,” here are all the factors that we want to consider. It’s a lot of affinity. It’s the alerts. So has a patient come in recently? Through claims data we can see if they’re seeing patients or not. And then we also did a lot of A/B testing. So around the segmentations that we created and around some additional analysis, we did a lot of A/B testing. Some really good A/B tests that we did, one was around content. So the more experienced people, our hypothesis was that they are engaging with branded content more than they’re engaging with unbranded content. So these are things that we’re able to see, specifically through A/B tests, that were successful. We had a couple that were less successful. So things like timing, which we thought was going to be a huge aspect of whether someone was opening an email or not, like were we sending that email at the right time of day or the right day of the week? And we had all this analysis done and we had all these hypotheses, and it kind of came out that it didn’t really matter. So it’s those types of things that A/B testing has been super helpful for us, and it’s really given us a lot of business input as well, because they’re able to see, like, okay, these results are actually working. So A/B testing, hugely successful for us. We are pushing it a lot on our clients as well, so.

09:25 So more work– just to keep talking about the business impact here, so. We had a dramatic change in the way that content was generated. So if you think about it previously, again, it was that straight shot, everybody needs the same content. So now we’re able to say, “These people need specific content.” It’s providing those recommendations back to the people that are generating those assets, and back to sales reps to say, “When you go see this doctor, make sure that you’re mentioning A, B, and C.” We can tell that they are interested in that information. We can see that when they go online they search for specific things, and so you want to continue to bring that up with them. It’s that idea of messaging across the board, across all the channels, being consistent. This is something we continue to have pushback on. It’s wasn’t an easy adoption here. So we’re continuing to work through that change management piece, but it’s providing step-by-step recommendations, like baby steps to get them into that thinking. So it started with much larger groups of saying, “Okay. There are people that we just want only branded content. It doesn’t make any sense to send them unbranded content. They’re not engaging with it. The disease education that comes with that is not relevant to these people.” And so okay, that’s something that they can bite on. And then eventually, we’ll get down to very specifics to say, “We know that this person is engaging with this content. We’ve seen previously that they don’t prescribe this particular drug to certain people because of the symptoms, or the symptoms aren’t strong enough and they don’t think it’s useful.” Right? So there’s very specific reasons as to why, and we’re working towards predicting that eventually. It’s just a slow-moving process to get them to adopt.

10:57 What we did see was an increase in the doctors that were transforming in that HCP journey. So getting people to adopt– it’s natural to see that some people are going to adopt over time, of course. As this disease is out there and you see more patients, you’re going to get higher adoption. But we were actually able to increase the number of people that moved from non-adopters to adopters by 10%, which in like a six-month time frame, was pretty good. We were feeling really good about that, because we’ve really only made these small incremental changes, and we’re working towards bigger and bigger changes. In general, lessons learned. Ingo said it again this morning, but it’s that change management piece, right, on the technology side as well as on the cultural side, right? So that cultural piece where we keep talking about getting them involved and getting the business involved, we are trying to do that more and more, and that’s where this iterative piece of the channel optimization comes into place, because we’re continuing to get feedback from them. We’re continuing to combine business rules with our predictions to give that business impact. So that piece of change management has been pretty successful, but it’s just about getting their buy-in earlier and earlier. On the technology side, we’re realizing that we are limited based on some of the technology choices that they’ve made, and that we’re not able to do everything that we want to do, and so how can we cope with that? So for example, we’re able to see when an HCP goes online and sees one of our banner ads. So if you’re reading a particular article about a specific subject, you can see the ad for that particular drug. And that data only comes in monthly. So we did all these association rules to determine that you get a lot of lift after a pull engagement. We keep calling them pull because it’s the HCP that goes out and pulls that information.

12:41 So we were able to– makes a lot of sense, right? You see pull engagement online, then they’re seeing more engagement in their email channels as well as in those face-to-face. You’re more likely to get a face-to-face interaction after they’ve pulled something online. But that no longer takes effect if it’s a month later. Right? So at the beginning of the month, you’re pulling something online and I don’t find out about for another four weeks, we can’t actually take that action on it, which is pretty– we’re working through that now. We’re working on getting daily feedback from them now, but that’s something that we have to work with that vendor on. It’s something that they weren’t prepared technologically to be that quick. So over time, we’re continuing to get better here. Again, I can’t stress it enough. A/B testing. We’ve been doing so much testing here to find out that true impact. And then the overall impact takes time. So as I’ve mentioned, we have several different channels, and we have engagement scores within each of those channels, but just because I’m increasing my online presence doesn’t mean that my overall engagement has really changed, right? Because maybe I’m just interacting more online and not interacting in person. So we have these overall engagement scores to kind of pull that all together, and it takes time for those to increase, for people to realize that you’re shooting them with the same messages across the board, that you are focusing their content towards you. So it’s not something that happens overnight. Even though we see these great results from A/B testing, when you look at it over time, it’s going to take some time to get everyone engaged. So this channel optimization, like I said before, is a continuous process. We’re continuing to improve the way that we analyze this data, and we’re continuing to refresh data and pull things in and see where it goes over time. So definitely an ongoing process here that’s been really iterative and great for us.

14:23 When we talk about the competitor launch– so as I mentioned before, this use case specifically was very intriguing for them because they had never had a competitor before. So how do you go about this when we don’t actually know how any competitor has ever impacted our business before? Something you should know about this competitor drug. So it was approved as a second line of treatment. So it’s actually okay that we do have some people adopting this new drug, right? You want some of these people to adopt this new drug. So the idea behind the second line of treatment is if the current drug fails, then you would move on to the second drug, right? So if I have 10 patients, the likelihood that one of them fails and will need the alternative drug is high, right? That makes a lot of sense. You’re going to have people that adopt for a good reason. What they really didn’t want– what we’re really trying to focus on here is eliminating the people who are adopting incorrectly. So they just don’t like our drug and therefore they want to try the other drug. They don’t like the side effects of our drug and so they want to try something new. And that’s what we’re trying to avoid here. So we built a predictive model that was the likelihood to adopt, so if you think about high likelihood or low likelihood to adopt. And we had this whole group of people that were high likelihood for such a– in this rare disease, it was we can’t attack all of these people, right? We can’t attack 2,000 doctors in the next month. It’s just not feasible. So who do we actually want to contact in the next month? And it was, okay, let’s overlay the people that have high patient potential but low script counts with the people that are likely to adopt to make sure that we’re hitting the right people at the right time.

15:57 This predictive model was a little bit tricky for us, because like I said before, we didn’t have any historical competitors. We had never seen what that would do to our market. And so what we ended up doing was working a lot with the market research team. So market research did a lot of surveys and talking with doctors to come up with this idea of who they thought were going to be the people that were going to adopt this new drug. If you look at the callouts in blue here, this was from market research. So are they involved in clinical trials of this new drug? Were they an early adopter of our current drug? What’s their engagement like with our materials? What’s their engagement like with our competitor’s materials? Are they writing papers about this space? Very academic thinking that these are the people that are going to adopt this new drug is people that are invested in the space and are writing about it and are engaged in the clinical trials. Our hypothesis was that you’d actually have a better idea of determining who’s going to adopt this new drug based on prescription patterns. So think about a doctor who– within the oncology space, they are prescribing drugs for a different line of treatment that does have two indications. So they are actually prescribing on both indications for another area. And how can we relate that back to us? So we took Medicare data. So Medicare data is actually free online. And so we were able to take down Medicare data, see the spread of drugs that HCPs are prescribing – so are they prescribing just one drug? Are they prescribing just the generics? Are they prescribing across the board? – and use the information from market research to determine, based on their assumptions, who they thought were going to adopt this new drug, and then we were able to project that to more doctors based on their prescribing patterns. We had a new prediction just based on those prescribing patterns.

17:51 We were able to say, “Okay, our label was generated from that market research, but then on the–” when we’re actually predicting, we’re saying, “No, it’s based on prescribing patterns, not based on these assumptions.” And so what we actually found was– we had the predictions from market research as well as our predictions, and based on those first couple of months– so this drug only went live in October. So only been a few months here, but we’ve identified three times a number of people that we thought were going to adopt the drug than the market research team did. So it’s this idea that machine learning can compute a lot more variables and incorporate a lot more information than the market research or maybe what you’re thinking originally. And so it’s that pushback to make sure that any assumptions that you’re making are worth it, right, and that you’re– it’s that combination of market research and machine learning that gave us this ultimate solution here. So ultimately, the lesson learned was it’s a combination of things, because there were things that we needed to understand from market research because we didn’t have that historical view of things. So it was a combination of the machine learning and market research that made this so successful. Again, this was a one-time analysis that we did. If we were to do this again, we would– we have a lot more data now, because we can actually see who is prescribing. This was all pre-launch, so it was very difficult to validate, and now that we have some people that are actually prescribing this competitor drug, the way we would go about building this model now is very different, right? Just the availability of data is different. So when we think about what is production, it’s this– we were able to use this analysis and take action on it very easily, but it’s not something that we’re going to continue to run weekly or something like that.

19:38 And then finally, we also did some analysis on a new product launch. So my client right now, even though they’ve only had one product for about the last 10 years, they have three in the pipeline to go live this year. So they haven’t really had to do anything on the launch side in a long time. Data has changed a lot since the last time that they did this. So what does that mean for them? How do they want to attack these new products? It’s just understanding the HCPs in an unfamiliar space, because they’re so engrained in the disease state that they have now and they understand it so well, and then they have these new products in a totally different disease area and a totally different space that their reps are just confused. They’re like, “Do they like to talk about the same stuff as the other doctors? What are we into? How do we attack these people?” And so initially we just did a lot of targeting. So targeting for HCPs and then organizations as well within that space. So what are the top organizations we should be going after? Who are the top HCPs? And then we ran them through a similar channel optimization that we did on their main drug so that we could say, “Okay, these are people that are more inclined to do in-person versus these are people that are more inclined to do non-personal, so the emails that they’re sending out versus the face-to-face interactions.” And then finally we did some speaker identification. So I want to talk through that a little bit. What we actually did was we predicted whether the attendance for each of the different– for everyone, basically, for the 10,000 doctors in this universe, we were able to say, “Okay, we’re predicting these are all the people that are going to attend if this person is a speaker,” and if that’s the case, we were able to narrow down the 10,000 doctors in the space to about a little over 100 doctors that seemed to be the best speakers– speakers that were going to get the best attendance. Right?

21:21 And so what we did there, we actually took a bunch of attributes and put them into three different buckets, because even though– and we used PCA and some other things here to do some dimensionality reduction, but we were trying to understand– even though we have 100 doctors, we really only needed like 20 to be speakers, right? This is a very rare disease, again. We don’t need a lot of speakers around the country, so how can we narrow it down from that predicted 100 to a better even 20? So we had a couple levers that we wanted to be able to pull here. You might want speakers for different reasons. For example, in one area you might want the speaker to definitely have a high patient count. You want somebody who knows the patients. You want someone who understands that patient space. And so that would be this clinical bucket here. You’d want somebody who’s really high within the clinical space, versus maybe I’m a sales rep and I’m trying to choose a speaker for my next speaker program and I’ve had five speakers bail on me because they just don’t know who I am, then maybe I want someone who is high on this client relationship. So someone that I already know, someone who’s in the space that I’m a little bit more familiar with, who knows our product, who wants to speak with us. Maybe that’s all I really care. It’s like, who of the top 100 speakers do we already have a relationship with so that I know that they will say yes when I ask, right? And then on the influence side of things– a lot of this came from some of the stuff in the graph as well, but it’s, who’s influential in this space? Outside of just my client and outside of just the number of patients that they have, are they writing a lot? Are they involved in panels or on boards?

22:53 And so these are three big levers that they wanted to pull, and we actually created a RapidMiner web app that was very, very simple, but it was a good way for us to deploy this type of interaction with the client. So different groups wanted speakers for different reasons, and so they’re able to come in here and just put in weights for the influence, for clinical, or the client interaction, and then a score is generated based off of that, and then they can– that score is sorted and then they can just download it and say, “Okay, I’m focused on someone who has a high patient count and that’s all I care about. This is the top people for me.” And it just narrows down that interaction. We built this as a web app just because the people that were using this were changing their minds a lot and asking us back and forth. We were like, “We’ll just give you this and you guys can play around with it,” is basically how it came out. And the RapidMiner web app was a really easy way for us to deploy that to them and give them that interactive piece. So finally, when we think about impact here and some lessons learned– so like I said, they have three new drugs coming to market in 2020, so this is now a very repeatable process, something that we’re going to take that first launch, learn from, learn what worked, learn what didn’t work, so that the second time around we’ll continue to iterate better. And it’s just that ease of now we have a plan, right? Before they were like, “Sure, doctors. That’s who we want to contact.” But it was like there was no focus there, right? And so now they have a good way to focus, a good way to interact with those people, and it’s just the reproducibility.

24:27 Lessons learned. Because we were working in a new disease state in an area that we were very unfamiliar with, purchasing data was huge for us. It was very impactful to get a bunch of different sources to consolidate together and kind of validate some of the analysis that we had done with some of the analysis other people were working on, as well as just to get a better understanding of the space overall. We just had to purchase data, and there was no way around it. So don’t shy away from that. We found it super impactful and– yeah. Highly recommend if possible. So thanks, guys. I just wanted to talk through those three use cases with you guys. They’ve been super helpful over time, and we’re continuing to work through that channel optimization piece, continuing to make it better, continuing to improve our predictions there and incorporate feedback.

25:19 Three awesome use cases. Does anyone have any.