About/Background

There seem to be new business intelligence software released all the time. There are great apps that track your marketing spends and trends, apps that predict customer churn and lifetime value, inventory apps that help your managers better understand the product pipeline; but very few handle all of the above, and those come at a dear cost. What if you want to measure them all at the same time while saving tens or even hundreds of thousands of dollars in annual labor costs normally associated with bookkeeping, forecasting, and reporting? With CB Abi, Cloud Brigade’s AI for Business Intelligence, this can be your reality.

Challenges

• Irresolvable Complexity – With all of the different software out there, how can you decide what data to track in your business, and what does any of it even mean? Do your many tools integrate well?
• Improving Systems and Processes – CB Abi allows you take action instead of spending all of your time learning what has already happened by providing dashboards to your business’s health and deep-dives into the predicted outcomes of various action options
• Antiquated Technology – With the arrival of IoT devices and machine maintenance sensors, your business should be using predictive maintenance to save money and reduce downtime
• Antiquated Technology – With the ability to use Unsupervised Learning ML Models to cluster your customer base, you no longer have to trust the gut of the highest paid person in the marketing team
• Skills & Staffing Gaps – If you’re already collecting more data than you can use, hiring a team of three or four software engineers to sift through everything and create a new system to maintain will not pay for itself for many years. Spending a fraction of that to hire a consulting team to do it will start earning you money without the downstream cost liabilities associated with added staff

Where it Began

In the winter of 2020-2021, Chris Miller, founder and CEO of Cloud Brigade in Santa Cruz decided that it was time to get a little healthier. As a data geek and inventor, Chris was excited to see how tracking his progress using his gadgets might help him stay on track or be more successful in his workout and diet plan. He asked me to build out a Data Science pipeline that took all of the various sensor data and health reports, do some predictive modeling with them, and build dashboards with the results.

Before I went back to school for Computer Science I was a Sales Director for an online wine retailer. It was a startup that sold for $100M based mostly on their ability to dial in the important Key Performance Indicators, or KPI’s. They modeled them out with accurate forecasting, anchoring on the Lifetime Value of each customer (LTV) and the Return on Investment of each employee (ROI). I built a dashboard for that in excel, which is what compelled me to learn how to do it in python, javascript, and to use Machine Learning Models and Statistical Analysis for even better prediction accuracy than multiplication factors alone. 

My project back then at the wine startup saved me 30 hours a week (that’s right–an entire person’s workweek) which reduced my job to a mere 45 weekly hours. It was clear that in creating this new workflow now, I had come to the marriage of preparation and opportunity, and thus CB Abi was born.

Solution and Strategy

Cloud Brigade is no stranger to finding the best technologies for your business. As an AWS Partner, Cloud Brigade has experience pairing the right technology with the right problem, and CB Abi allows us to use this same process to manage your company’s data. Working with your IT or Data professionals, we can custom-build ETL scripts or connect to existing API’s. Then we are able to restructure quantitative data and interpret qualitative data to recommend best practices for your data warehousing. Finally we use Amazon Web Services (AWS) to handle our workload in the cloud.

The end result is a branded interface that can live on a big screen in your office or boiler room like an ESPN reel or a CNBC Ticker, keeping you and your teams aware and excited about your daily business performance. You can also log in to this dashboard and drill-down into each segment of your business for a closer and more in-depth look at what is happening.

This Is No Ordinary Project

Building Machine Learning solutions that deploy on the edge is not as easy as it sounds. Sure, using advanced algorithms that in many cases were developed at Google or Amazon and applying them to your business using python libraries and a little statistical knowhow seems pretty straightforward. However, it takes a lot of time-consuming data engineering before you even get to the Random Forests and Neural Networks. 

If you don’t already have a handle on your company’s data and how to access it, that will add time and complexity to the job. However, getting into the weeds creating scalable systems for your business is what we do best.

Smarty-Pants Technology

Before we can tackle this issue, it is important to build solid data pipelines using AWS Glue (as is described in another article here), or Amazon Data Pipeline or EMR. Glue allows us to take data from our data lake in AWS S3 where our devices have been programmed to send the data on a daily schedule, run an ETL job (Extract, Transform and Load) on the data, and insert it into our Amazon RDS (Relational Database System) or Amazon Redshift Data Warehouse. We can do this in PySpark, utilizing the RDD and making copious amounts of data far more manageable. But at the end of the day, the microservice that delivers a scalable solution that is the right fit for your business is the one that we will choose. Or if need be, we’ll invent one. 

Then we have to explore all of the data, normalize it for comparison, and then re-write our pipeline. Along with restructuring the database to hold it in its proper form without duplicating, truncating, or clobbering other data. The objective is to have a self-contained software that can handle the forecasting and data reporting for your business that is easy to use so that nobody has to become an expert in using the software–you’re already an expert at what you do. CB Abi lets you spend your time doing what you do best.

From there, we can use a data dashboard and visualization software that renders comparative views of each of the different metrics. We may choose to build a visualization solution in-house using a python library such as streamlit, but for most use cases, Amazon Quicksight is a robust solution with a very low cost when it stays inside an organization. 

Finally, we deploy our in-house Data Science and Machine Learning team to run exploratory data analysis, predictive modeling, and more data visualizations in Amazon Sagemaker. For this step we’ll have questions for your marketing, sales, manufacturing, and IT teams in order to make sure that the models we are building properly speak to the specifics of your business.

So what does this mean? This means that we can build out CB Abi to monitor your sales and supply chain, track your customer segmentation trends and marketing effectiveness, forecast your budget and hiring needs, and even run predictive maintenance on your machinery or equipment. The actionable insights that CB Abi’s dashboards leave you with will free up time to do the important things, like coaching your staff, innovating to get ahead of your competition, and even taking a vacation to be with your family where you don’t have to constantly check your cell phone for peace of mind. 

Building the Model

Since every business will have its unique obstacles to overcome, CB Abi is not a simple one-day solution. We’ll ask to spend a little time with your decision makers and stakeholders to gain some knowledge into what currently informs your forecasts. We’ll want to get a sense of what questions you already ask yourselves every month. We’ll ask what KPI’s you are currently working with, and what rates of change you are expecting to see on a month to month or quarter to quarter basis. With a little domain knowledge and historical data, we can turn around predictive models that will live inside CB Abi with insights ready to help you plan changes for the future.

Technical Hurdles to Overcome

Since every system is different, there may be some data migration and normalization hurdles that will arise. Provided our teams are on the same page from the start, there’s no hurdle we cannot get over as we stay on track toward your custom solution.

A Successful Integration and a Technology Handoff

One thing we have learned through our work in Machine Learning is just how time consuming and intricate each step can be. A lot of times, the work of building up the infrastructure and lining up the data so all the moving parts can communicate with each other is actually more challenging than creating and operating Artificial Intelligence. This product is no different. The live application must be programmed to dynamically handle new data fields, data types, and data sources whenever the user wants to add them. The data pipeline must be programmed with the proper error handling and notification systems, and the data reports must be connected to the alert systems with bullet-proof security. However once the data science has been fully integrated, very little regular maintenance is required to keep the system working.

Opportunities

With a solution like this for your business, we can also build you a virtual assistant that can interact with your managers to alert them to oddities or answer quick questions. This assistant can utilize Amazon Lex, the software behind Alexa, to give a conversational overview of the top-line numbers every day. From threshold notifications to quickly processing how a new idea will change your profitability in a coming quarter, we’re here to let CB Abi keep working for you even after we’ve handed the code repos over to your teams.

CB Abi also integrates with Cloud Brigade’s SmartDispatch which labels, prioritizes, and delegates your incoming customer emails so that you can save time and money. The data from SmartDispatch can inform your customer segmentation models, your employee productivity metrics, and even your budget forecasts. With CB Abi you’re not just getting a report, you’re gaining a team member that never needs a coffee break.

WHAT’S NEXT

If you like what you read here, the Cloud Brigade team offers expert Machine Learning as well as Big Data services to help your organization with its insights. We look forward to hearing from you.

Please reach out to us using our Contact Form with any questions. 

If you would like to follow our work, please sign up for our newsletter.

Sign Up

Cloud Brigade is setting out to answer the question: Can AI and Machine Learning make you more money by taking your customer segmentation strategy to the next level using the data that you already own?

About/Background

Have you ever launched a new product line or a new business and wondered why everyone doesn’t see the amazing value of your product upon first sight? I have. I was working at a Wine Industry startup a few years back and as the Head of Sales, and I was asked to temporarily run the marketing program until we hired a marketing manager. There were only four of us at the company back then, and we weren’t even licensed to sell wine yet, so I was eager to start getting our customers to line up outside the door in anticipation. 

Having worked at an ecommerce juggernaut in my previous role, I thought it would be easy. As you can imagine, there was a lot more to it than I’d expected, and nowhere was that more true than in finding where our ideal customers were and talking directly to them.

I was no stranger to customer segmentation. I knew that we were targeting people of a specific age who were looking to spend $40 – $150 per bottle of wine, people who had higher-than-average incomes, maybe people that drove foreign luxury cars, and people who valued authenticity and the feeling of being part of an exclusive club. The problem was that these characteristics of our ideal customer were all from a branding perspective. We weren’t utilizing the data that we had available to find out who was actually using our website pre-product launch. If I had known how to build a Customer Segmentation model back then, our business might have progressed on a very different track.

Challenges

• Irresolvable Complexity – How do you know which data to segment your customers on? What if gender and age and zip code were only three of several dozen data points that you had available?
• Improving Systems and Processes – Customer Segmentation Modeling allows your CMO or Marketing Manager to run more A/B testing of your content on the best cohorts of consumers
• Skills & Staffing Gaps – Relying on gut feelings is important, but using data to drive your decision-making takes the guesswork and arguments out of your board-room

Solution and Strategy

In the spring of 2020, as a pandemic was spreading around the world, Chris Miller, Founder and CEO of Cloud Brigade started wondering if Machine Learning could lend insights into how this virus was acting on different populations. I immediately thought of customer segmentation and wondered if this problem would be anything like the marketing attempt from my wine start-up. In this pandemic problem, we ended up finding at least 147 different factors comparing county-level data in the United States. Some of these factors included the prevalence of public transit in a region, the level of education, percentage of households receiving medicare, average household income, race, age, etcetera.

Using clustering models and transfer learning, we were able to find some glaring factors that contributed to the disparity in spread and mortality of this illness, and you can read about it here if you are interested in that study. The study is directly analogous to the customer segmentation problem in that it helped to group together areas that were similar from vast amounts of data, and that we wouldn’t have otherwise thought to look into.

The great thing about customer segmentation models using Machine Learning is that they take a lot of the guesswork out of your marketing and financial forecasting. For example, like my high-end wine company, you may want to market to a specific customer because you think they fit your branding consultant’s idea of the “ideal customer”. Your gut tells you that this is the right type of customer to buy your product, but there’s an X-factor that makes your business unique, and that same X-factor means that the people that you have experience with in the past might not be the most valuable customers for this new product. 

In my case with the wine, the company founders came from the corporate side of the luxury wine market, and I came from a more scrappy online retailer. We were trying to marry these ideas by getting high net worth individuals and people with higher incomes to join a subscription wine club that they would pay for monthly. We found out after months of trying that our “ideal customer” was not the person most likely to interact with our website. We had missed the mark because of our own preconceived notions of those customer cohorts and how we chose to use our target marketing dollars.

With Machine Learning, we could have taken the tried and true cohort segments such as age, location, gender, and income, and added to them data points such as how long a specific customer spends on the website, or on a particular product page; the date of their last purchase, how many total purchases they have made, what web channel usually brings them to the site, or which physical retail store they shop at more often, or do they buy apparel and gear from us. 

All of these factors contribute to grouping customers so that you can find out what traits actually describe your most valuable customers. You can figure out how much it costs you to acquire that most valuable customer, as opposed to just knowing your new customer acquisition cost. You can run comparative testing, known as “A/B Tests”, to see which marketing strategies or sales funnels work best for each customer cohort, and optimize your spend for each. Through that you can learn the lifetime value of each customer cohort, and from there you can figure out which cohorts make you the highest ROI when you market to them.

This Is No Ordinary Project

Building Machine Learning solutions that deploy on the edge is not typically easy. In this case, the ease of creation comes down to what systems your business is currently using to collect data. You likely have some type of Customer Relationship Management Software, or CRM, that houses lots of contact data, and usually purchase information, about your customers, but are you sending that data to a Data Warehouse? Do you use a Point-Of-Sale system such as Aloha in your retail channels, and are you collecting your customer’s email address at the time of sale? Do you use a separate fulfillment system for deliveries, or an accounting software like QuickBooks instead of an ecommerce system? Tell us about your ERP solution. And what about your website monitoring–is it all facebook and google analytics or has your web developer already created a pipeline for your data and is it connected to a customer id and a database?

If you don’t already have a handle on your company’s data and how to access it, that will add time and complexity to the job, but be assured that getting into the weeds creating scalable systems for your business is what we do best. As a Data Scientist, being able to assemble this data and put it into a usable format is my literal pleasure.

Smarty-Pants Technology

There are several kinds of clustering models that we use depending on the type of data that you own, the spread and diversity of the data, and your main objective and goal (do you want to optimize for ROI or for total profit dollars, for example). We will most likely use a transfer learning algorithm that blends concepts in regression and classification with the clustering models.

That’s a geeky way of saying there’s more than one way to build this, and your company is unique. Whatever the desired goal or the perceived complexity, our product is not ready until it clearly shows you your different customer cohorts and allows you to put a dollar figure on each cohort so that you can decide where your marketing and advertising dollars should be spent. This can mean we’ve built a fully-managed and robust front-end with a no-code app that your marketing team can use, or it might be as simple as adding a label to your existing software through an API. Our team will find the best way to make you the most money and deliver the results that you’re asking for.

Technical Hurdles to Overcome

Since every system is different, there may be some data migration and normalization hurdles that will arise. We might decide to use microservices like Amazon SageMaker and Amazon QuickSight to expedite the solution, or we might even code it out with open-source tools from scratch. Provided our teams are on the same page from the start, there’s no hurdle we cannot get over as we stay on track toward your custom solution.

A Successful Integration and a Technology Handoff

One thing we have learned through our work in Machine Learning is just how time consuming and intricate each step can be. A lot of times, the work of building up the infrastructure and lining up the data so that the different parts can all communicate with each other are actually more challenging than creating and using Artificial Intelligence. This product is no different. The live application must be programmed to dynamically handle new data fields, data types, and data sources whenever the user wants to add them, the data pipeline must be programmed with the proper error handling and notification systems, and the data reports must be connected to the alert systems with bullet-proof security. However once the data science has been fully integrated with the front-end application, very little regular maintenance is required to keep the system working.

Opportunities

With a solution like this for your business in place, it’s really a small step to implement CB Abi, Cloud Brigade’s AI Business Intelligence solution. CB Abi also integrates with Cloud Brigade’s SmartDispatch which labels, prioritizes, and delegates your incoming customer emails so that you can save time and money. The data from SmartDispatch can inform your customer segmentation models, your employee ROI metrics and productivity metrics, and your budget and headcount forecasts. With CB Abi you’re not just getting a report, you’re gaining a team member that never needs a coffee break.

WHAT’S NEXT

If you like what you read here, the Cloud Brigade team offers expert Machine Learning as well as Big Data services to help your organization with its insights. We look forward to hearing from you.

Please reach out to us using our Contact Form with any questions. 

If you would like to follow our work, please sign up for our newsletter.

Sign Up