Maximizing Lead Conversion Success Using Predictive Marketing Analytics

Hello, everyone. This is Dylan Cotter with RapidMiner. I’d like to welcome you to today’s webcast on Maximizing Lead Conversion Success Using Predictive Marketing Analytics. We’re joined by our speakers today from Anbliks, a RapidMiner certified partner, Muddasir Hassan, as well as Nikita Naidu, a data scientist at Anbliks. I think we’ve got a good agenda in store, so I’m going to start off with an introduction to RapidMiner for those of you who are new to what we do. And then, hand it over to Muddasir who’s going to talk about challenges that customers face in marketing. We’ll then talk about processes for how to use machine learning to overcome some of these challenges, particularly focusing on customer acquisition, nurturing customers, and then, ultimately, converting customers into clients. We’ll then hand it over to Nikita, who’s going to walk you through some demos so you can see it live in action. And then, wrap away with some key points for consideration.

So RapidMiner, our core focus as a software company is making software that helps analytics teams be more productive, right? And we do this through an Opal platform. Really focusing on if you think about, sort of, that lifecycle from preparing your data – and that includes blending data sets, joining data sets, cleansing data sets – to actually applying the machine learning, to ultimately deploying models of production. Many of you on this phone are part of the user community, so you’ll find that we’re worldwide users who are contributing to this open community to help really drive knowledge, and then help each other get more value out of data science. And that innovation, it’s thanks a lot of you who are on the phone today who helped contribute to that. At the end of the day, it’s about driving business outcomes. And so, predictive analytics and machine learning can be applied for revenue challenges, so to help improve revenue, help save costs, and, more importantly, mitigate risk. And these are some of the use cases you’ll find RapidMiner’s used.

As a data science platform– again, thank you to the user community. Very strong. You’ll find universities using RapidMiner. So many of you may have used RapidMiner at university. As well as a good client base. What I would encourage for those of you on the phone, again, who are new to RapidMiner and want to know what do end-users say or what does the analytics community say, you’ll find that Gartner, who covers the space, has recognized RapidMiner as a leader for the last five years in their Magic Quadrant. Forrester has also recognized RapidMiner as a leader for the last two years in predictive analytics and machine learning. So reach out to the analysts and converse with them to find out more. You’ll also find a great resource is KDnuggets, where RapidMiner’s recognized, but also a great resource for just data science topics in general for those of you who are learning. And then, another source to really find out more is, “Hey, what are other end users like yourself saying?” And so G2 Crowd, you’ll find a RapidMiner, really strong endorsement. That should give you some confidence that others have tried out the tool and it can be used for your data science challenges.

So things you’ll see today as the team goes through and to help sort of differentiate RapidMiner is, again, speed in stuff that you do over-and-over from a data science standpoint. That’s really what we’re focused on, and so things like improving your performance, making it easier to collaborate among analyst teams, those are some core capabilities. You’ll also find the platform is very open, so, from a technology ecosystem standpoint, easy to integrate with other technology that you may be invested in. And it’s extensible, so if you want to plugin your own machine learning algorithms, if you want to integrate, again, with other technologies, RapidMiner provides the hooks to make that happen. And again, it’s one platform, right? So not having to jump from tool to tool, but, really, from a data science perspective, being able to cover your prototyping needs, model validation needs, and ultimately putting those in production. So hopefully some of that will come across in today’s webcast. And with that, I’m going to turn it over to Muddasir.

Thank you, Dylan. Hello, everyone. Thank you for joining with us today. My name is Muddasir. I work at Anbliks as a data scientist, and in this webinar I’ll be talking about the current challenges in the marketing industry, how machine learning and RapidMiner are revolutionizing this area, and I will take you through the different concepts of predictive marketing analysis. I am joined by my colleague Nikita here, who will further take you to a case study to help you understand how this is done using RapidMiner. So coming back, there was a recent study by done by Dun & Bradstreet for their global marketing teams where they highlighted the top-three challenges in the current industry. Of the teams who participated, 57% of them say that they are not able to understand their audience, 46% are not able to track the ROI, and after all the marketing efforts, they say just 17% of their leads convert. Right now, there are so many popular marketing automation tools in the market, but still, they are not able to fully understand their customer or track ROI events. As a result, a lot of data doesn’t always make the big data suits. These challenges can be addressed by developing a predictive marketing analysis solution using RapidMiner as a data science platform which helps you analyze the data to understand your audience and tackle all other challenges as well. You’ll see how this is possible in the next slide.

So this is how a customer journey looks like. I’ve simplified them into three different stages to show how data science can be used to improve the output in each of the stages. In the acquisition phase, data science can help you easily understand your customer and data mine channels that are working well. In the nurturing phase, it can prioritize your leads and data mine products that are more relatable to your customer. And in the last, conversion phase, it can help you reach out to your leads with a discount offer they’re highly likely to accept. So we’ll deep dive into this concepts and see what they have to offer. The concept of segmentation, a part of the acquisition phase, slices and dices your audience to simplify your understanding of the customer base. For instance, if we take the image on your screen as an example for a mobile phone company, we can understand that there are three basic personas in their audience. The first one is a business user, and the second persona is the casual user who maybe uses the phone for the sake of it, and the third one is the power user who uses the entire features of the phone. So this way, the mobile phone company will know who their audience is and they can reach out to them in a very personalized way, and also chart it for marketing strategies. Channel attribution, also a part of acquisition phase, gives you an all-around perspective of your customer’s journey. It essentially tells you what channels are working well and gives you an idea of where a prioritization is being made in the customer journey.

In the nurturing phase, lead scoring is a methodology that helps you find top-quality leads by assigning a school to each profile through its learning on their past behavior. This way, you’ll be investing your time efficiently with leads that are highly likely to convert. Moving on, product affinity, also a part of nurturing phase, gives you an insight on what products are closely relatable to your customer. It gives you an outside perspective of what products can be bundled together for sale, hence helping you chart new and intelligent product strategies. This discount recommendation, the concept comes in the last phase of the customer journey. It is that last chip you need to push a customer to an end line. The fundamental idea here is to furnish a discount which is highly profitable to customer and the vendor.

So now, Nikita will take us through a use case from one of our clients who was looking to improve their marketing efforts and, also, include their customer journey. She will take us through all of these concepts and also show how the insights from each of them helped our client in every phase the customer journey. So over to you, Nikita.

Thank you for the introduction, Muddasir. Hi, all. This is Nikita, and I work as a lead data scientist with Anbliks. And I’ll be taking you through one of the use case, which is based on one of our clients. It was an educational consultant based out of UK, and they provide comprehensive education services to international students to prepare them for graduate and undergraduate programs in the UK. The main business objective was to maximize lead conversion rate and minimize sales and marketing costs. And one of the biggest challenges for this client was in prioritizing the lead pipeline. They had a very huge number of leads coming into the system, sometimes as high as thousands of leads in the peak season, and it had become nearly impossible for recruiting agents to reach out to all the leads in the system. They experienced very low lead conversion rate, but then they also had to understand what kind of products or discount incentives should be offered to these leads. So we followed a consultative approach. We first understood their business model and business processes. We broke the problem into multiple stages corresponding to the stages of the sales model. We identified the problem areas which required enhancements or improvements, and came up with a use case, which would have direct impact in the lead conversion and marketing and sales costs. So let’s deep dive into these use cases.

This is the solution architecture, and the client basically got leads through two distinctive channels, online more and offline mode. Online mode is through the digital channels and offline mode is through direct recruiting agents. They use Marketo as their marketing automation tool, and all the marketing and nurturing campaigns were created and ran through Marketo. Lead activities were captured in Marketo. Activities like email trends, email open, resources downloaded, website visits, etc., were recorded in Marketo. This was very useful information, which we used for understanding the lead behavior. Use the Salesforce CRM tool to run their sales and engage sales and lead engagement through data business processes. We used RapidMiner’s Studio and a RapidMiner Server to create and deploy machine learning models onto the client’s system. RapidMiner ran data from Marketo and Salesforce data warehouse and advanced machine learning models. Some of the models, like calling, runs on a daily basis. We then used Tableau to summarize business insights and create informative dashboards and reports, which were used by recruiters, stakeholders, and management.

Let’s take a look at the data flow. So the client gets records from two different sources. Leads come through digital channel and recruiting agencies, so data related to demographic, academics, lead activities, etc., is captured in the CRM tool. A dump of that is taken into data warehouse. These data is not clean because the forms are hosted on multiple platforms, and some of the forms are not even in English language, so this data was not fit for modeling. We had to do data preparation to make the data fit the modeling and reporting. To give you stats on the size of the data, we used two years of historic data for modeling, and it had around 500 thousand lead profiles, 150 million line items of the lead activity. We had to aggregate these lead activity data, and create a summary for each of the leads based on activities they have performed throughout their journey. And this data was spread across 60 odd tables with thousands of attributes, and it was not consistent. So we had to clean that up and create a consistent model, which was saved again in data warehouse with a customized schema. This whole data preparation step has taken 189 days. We had three data scientists working full-time on this project. It has taken 60 days to do the data preparation step.

So let’s take a look at the use cases one-by-one. We started with segmentation first because it was important for us to understand the lead person as what kind of lead provides a presence in system, and if there’s any pattern where most of the leads are– if there’s any profile where most of the leads are converting, or any other profile where the leads are not converting. So we used K-means, a gaming distance-based clustering algorithm, for creating the clusters. And we used Rhythm’s business metric to find out optimal size of the cluster, and we found that five is the optimal number of clusters. And we have used the lead activity data and lead demographics data for segmentation. So let’s take a look at the RapidMiner process. So this is the data used for segmentation and each row represents a record for each lead activity, and you can see that we have summarized activities performed by the leads across different activity types, so their journey. Some of the attributes are coming from the lead activity data and some of the attributes are design by us. That is some attributes like age in the funnel, recency weeks, frequency leads, breadth of touchpoints, etc. Breadth of touchpoints here represents how many different channels does the lead use in order to interact with the client.

So let’s take a look at the process. This is the RapidMiner process for lead segmentation, and I’m just showing this process to show you the results. So we are using Cluster Model Visualizer to summarize it. For reading the data and selecting attributes using it, which explain 80% of the– only those attributes which explain 80% of the variance in the data, normalizing it. We’re supplying it to the clustering in and using Cluster Model Visualizer. So let’s take a look at the results. Here are those five clusters, with Cluster 0 being of the smaller size, and cluster 3 being of the largest size. We can actually take a lot at Centroid Table to understand the characteristics of each of the clusters. And this basically creates the center of each of the clusters based on those attributes that you have supplied. And we can further use the Model Visualizer to actually summarize each of the clusters. So let’s take a look at the presentation as to how we have summarized the clusters and what were the different ones found.

So these are those five clusters, with the first cluster being named Highly Interactive and Inbound based on the observations. This cluster was very responsive and was proactive in asking questions or responding to the emails, and a high conversion ratio was observed with a conversion ratio of 3.9%. The last two clusters are named as Pretends to be Involved and Minimal Interaction because there response rate was low and the conversion ratio is also low at 0.5. So we used the cluster model to cluster a new lead coming to the system and tag them according to their behavior, classifying to one of these categories. And this was very useful for the agent to understand the behavior of the lead and understand how they should be engaged in the future, or what could be expected of them, so.

Let’s move to the next use case, which is channel attribution. So once, when there were leads coming from multiple channels and it was important for the client to understand which of the channels are performing better, or where are the best quality leads coming from, and what is a better one channel over the other. So we have used Markov Chains, which is a concept of simulation. Basically, we removed one channel at a time from the system and studied the effect of that channel over lead conversion or the lead conversion of other channels. And let’s take a look at the data first. Okay. Okay. So this is the data. And we had to arrange the data to create a lead journey or lead path for the different channels that the lead has touched during its journey. And you can see this is the lead ID, that is the first lead. And this is arranged in ascending order of the lead activity, which this is the different channel that the lead has touched. And let’s take a look at the design of the process. Okay. I just joined this thing. So basically, the output of this is three different results. The first results that explain the removal effect. That is when a channel is removed from the system, what is the effect on the lead conversion? And the first column here is the channel. We have 11 different channels. This summarizes the effect, sorting it based on the removal effect. You can see channel number one has the highest removal effect. That means if channel number one is removed the system, it would affect 91% of the leads. Followed by channel two, which would affect 66% of the leads, and so on. The other one is to understand when does a channel appear in the lead journey with the channel? Whether the channel appeared as the first touchpoint, or whether it appears in the middle of the journey, or whether it appears at the last touchpoint. Last touchpoint is just before the conversion.

So every channel is given a value based on where it appears in the lead journey. So channel number 1, it mostly appears everywhere– it looks like that appears. Yeah. Each and every channel has a value across these different variables, it’s first touch, last touch, and linear touch. Let’s take a look at the results. I just go back to the presentation and we can see that. So you can see, on X-axis, we have different channels. It is channel number one to channel number nine. And on the Y-axis, we have the count of leads which have appeared where this channels has appeared as first touchpoint, as a last touchpoint, and as a linear touchpoint. And each of the color here represents a type. You can see that channel number 2 is very popular and it kind of appears everywhere in the lead journey, and channel number one, here, actually appears most of the time as a last touch conversion. So channel number one looks like it’s very important because it’s the channel which could have influenced the decision of the lead. So when we start with the removal effect, channel number one has scored the highest. So here, it means that if channel number one is removed, it would affect 94% of the leads. Followed by channel number two, which would affect 64% of the leads. And to do this, we had used only those leads which have converted in the past. And this was very useful in understanding the behavior of the lead coming from different channels, or importance of the channel.

And further, we have also studied a transition matrix. Created a transition matrix to study the effect of one channel over the other. This is particularly a correlation type matrix. For example, if you look at channel number seven, the correlation between channel 7 and channel 10 is 0.85. That means 85% of the leads who come to channel 7 will transition to channel number. For this kind of study, it was very important to understand the effect of one channel over the other, and if it’s removed from the system, how it would affect the other channel. Okay. So let’s move forward to our next use case, which is lead scoring. And this was one of the most important use cases because, like I mentioned before, there were many leads coming into the system and it had been important for the client to quantify the leads based on the probability of the lead converting, categorize the leads into A, B , C priority, with A being of the highest priority and C being of the lowest priority. And we have scored each and every lead on a scale 0 to 100, and categorized them as a high or like A, B, C. And we have used logistic regression for classification. We obviously had a class imbalance issue because lead conversion rate was very off, and we have used SMOTE to boost up the minority class. We had a lot of features in the system, so we have used the Gini Index with feature selection and selected top-end number of features, which explains 80% of the variance here. So let’s take a look at the process.

Okay. So this data was used for leads coding, and we have used lead demographics data, lead academic data, lead activity data for its coding. And we have 72 regular attributes, so there was a need to do feature selection. And let’s take a look of the design in RapidMine across– I’ll run this one. It will take time. So we have filtered out those leads which are of a SPAM or fraud nature and taken only important leads for the modeling. Not just the leads which have converted, but the leads which have not converted into fact and which have a good amount of data. And I’ll just go inside here to show you how we have the feature selection. And here, you have done feature selection to select only top-14 features based on the importance. And we have used a GLM for training the model. So let’s take a look at the results. It’s going to take some time. I already have the results of it here. Great. So the output of this a score on a scale of 0 to 100 for each lead, and this explains the probability of the leads converting. This score is correlated to the applications who have converted in the past so that those kind of scores which have converted in the past would get a higher score. Plus, the leads which are active in the system would get a higher score. So someone which is having a good profile, but are not moving in the system or still and not moving further in the process will be given lower score.

Let’s go back to the presentation. Okay. Again, each and every lead, we provide a score of 0 to 100, and this is coming from Tableau. You can see two scores here. You can see lead score and application score. Application score is a score based on the affinity of the lead, product affinity of the lead. And I’ll explain that in the next section. And you can see that this first lead has a lead score of 65, and the second lead has a lead score of 72. And this score is correlated to the live status of the lead, whether it’s open, engaged, or outreach. And also, the stage of the lead as to in which stage it is, whether it’s in the known stage, or whether it’s marketing qualified lead, or whether it’s a sales qualified lead. This was very important for the agents to prioritize the lead, and to understand whether the lead would be converting or not, and how much time should be given to the lead. Let’s go forward. And once they can understand how to prioritize their lead, they also wanted to understand, “With what kind of products should we offer to these leads to ensure conversion?” And in order to do that, we have used descriptive statistics to understand what kind of profiles have an affinity to what kind of products. And we have created this correlation table in Tableau, and you can see that program number 1 that is Computer Science here on the X-axis– you have different programs or different products, and, here, you see the count of leads who have opted for this program. And the three colors here represents the category of the lead, with green being the highest quality lead, blue being mid-quality, and orange being the least quality. And program number 1, that is Computer Science, is highly turning because most of the leads coming into the system are opting for computer science program as a lead in the past. Followed by Mechanical Engineering. And we have supplemented this result with a score on the acquisition road. That is from where these leads are coming. And you can see that social media is the top one channel from where most of the leads are coming. But the quality of the leads coming from the channel is of mixed nature, with 50% falling in the B category, and 50% falling in the C category.

Further, we give a score on a scale of 0 to 100 against each and every application. And there are two leads here, for example, like in Donna, you can see that these two applications have the same lead score because it’s essentially for the same person, but they have different application score, which is basically a product affinity score that is of 28 and 19. And they’re owned by two different agents, that is Alex and Alisha. And by looking at the application score, we can see that Alex has a higher chance of closing the lead Donna compared to Alisha.

Okay. So the next use case is discount recommendation. And there was a need for the agents to understand how much discount should be offered to the lead in order to close the lead. And not only it required discount, certain profiles of the lead were price-sensitive who were looking for offers and incentive. But certain profiles were definitely not looking for price, but they were looking for other things like the rank of the university, etc. And in order to determine what is the optimum amount of discount that should be offered to the lead, we have used descriptive analysis by setting what has happened in the past and what amount of discount has resulted into what percentage of conversion. So let me take you to the results. This is coming from Tableau, and we have here in comparing two regions, that is China and South Asia. And on the X-axis here, you see different programs, that is program number one to program number five, which is compared across China and South Asia. And these three different segments represents the quality of the lead or the priority of the lead, high, medium, and low. And the color range here represents the range of discount offered in the past. Dark blue represents no discount, light blue represents less than 5k, dark orange represents between 5k to 10k, and light orange represents more than 10k. And clearly, it is visible here that China is the less price-sensitive over South Asia, and most of the leads have converted without discount. But program number three and program number four in South Asia region have enjoyed certain amount of discount. And so this kind of tells the agents as to which program should be offered, what amount of discount with is correlated with the region, and this has helped them optimize the sales cost.

This study was very important for the client to understand which kind of programs requires what kind of offers and incentives in order to promote their sales. And these were the five use cases, but there are many other cases which are there under predictive marketing analysis. And Muddasir will take you through the key takeaways and other use cases which are feasible. And over to you, Muddasir. Thank you very much. So that was great. Thank you, Nikita, for taking us through the use cases and explaining the data from each of them. We’ve seen how predictive marketing analysis can be an essential piece of marketing journey. The insights through PMA can expedite your efforts in winning a customer, but that is just one half of the battle. Keeping them engaged and, ultimately, retaining them is the real long-term success. Our team at Anbliks have created this long-term success story for organizations across the globe. Our PMA solution has practically simplified their marketing efforts, and their goal of delighting their customers with a personalized service is now simpler to achieve than ever. Based out of Texas, we are in business since 2004 and served over 300 customers. Our services are not just limited to predictive marketing. We are, in fact, a brand under which more than 400 technology professionals, data analysts, and scientists throughout the globe collaborate to help our clients navigate the digital transformation. All this by leveraging our advanced analytics, big data, and automation to ease their everyday process. The credit, of course, also goes to our partner, RapidMiner, and its product’s potential to build clean and interpretable code that helps data science teams design a solution that has a decreased time to market, which I believe is the most appealing feature for any client. So thank you again for joining with us today. Please feel free to get in touch with us to know more about Anbliks and our services and products we offer. Thank you.