https://www.linkedin.com/in/abhikbanerjeeuc

This is a great interview with Abhik Banerjee. Abhik is a Staff Data Scientist at Kohl’s Technology (at the Kohl’s departmental stores ) where he leads a team on machine learning and data mining projects. He also provides strategic direction around these areas to senior executives at Kohl’s. Abhik has an interesting role because he has to marry online and physical store analytics to provide insights to Kohl’s. That’s not easy.

Abhik received his MS in computer science from the University of Cincinnati in 2012.

Here are some other things we talk about:

-How do you present your machine learning findings to senior management?
-What are your priorities?
-How do you implement a program like optimizing merchandising at your stores?

Transcript

David Kruse: Hey everyone. Welcome to another episode of Flyover Labs. Today we are lucky enough to have Abhik Banerjee with us. And Abhik is a Staff Data Scientist at Kohl’s Department Stores where he works on machine learning and data mining projects. And what he does is, he provides a direction around these areas to Senior Executives at Kohl’s. So I wanted to bring Abhik on the show because it sounds like he’s bringing another level of intelligence to Kohl’s and retail. So Abhik, thanks for coming on the show today.

Abhik Banerjee: Thanks Dave, thank you very much for inviting me. Thank you.

David Kruse: Definitely. And so, can you before we jump in what you are working on now – can you tell us a little bit about your background.

Abhik Banerjee: Sure. So my background has mostly been a mix of data science and software engineering. So I think I’m fortunate enough to have worked on both the areas. So I finished my Bachelors and my Masters in Computer Science. My Master’s thesis focused mostly on unsupervised learning, predominately biclustering and applying those on to gene data sets, bi-empirical data sets and sort of putting prediction based algorithms on friend based networks. And I have been working in the research areas of machine learning and big data for close around five years now post my masters. So I used to work for the AOL mail team prior to Kohl’s where we used to do a lot of text mining and basically lot of unsupervised, and supervised mining algorithms for identifying mails, categorizing them and well a bunch of things and have been at Kohl’s for the last two years now, where we apply a lot of the research concepts and try to build great products for our customers to make their experiences better.

David Kruse: And what originally interested you to machine learning?

Abhik Banerjee: So I actually. I have been like coding since I think from eighth grade. So I wrote my first C++ code yeah, when I was in the eighth grade. But I actually came – I would say close to my machine learning is probably my first course in my grad school. So there was a course called intelligent systems where I actually wrote my first neural network from scratch using the back propagational algorithm and another algorithms and I think there has been some courses which actually really intrigued me in my graduate school, which was most spanning across machine learning, artificial intelligent, backend recognition. So we like did a lot of courses and tried to see how these technical aspects or the research aspects were the problem laid on to the practical aspects across the industry. So it was like I think around 2009 where it really got too much into data and seeing how things can be transformed using neuroscience and machine learning.

David Kruse: You were a little ahead. 2009? I mean now machine learning is quite hot, but it wasn’t as hot.

Abhik Banerjee: Oh! Yeah. I think it’s like people have been working on it since like for the last 30, 40 years. Neural Networks has been from 60’s, but I think it started catching up much recently through a lot of the hardware advancements and other things.

David Kruse: Can you think of a project that you worked on kind of initially around machine learning, where like ha! This is really interesting. Like this is something that I can do and that’s really off just writing software. Can you think of a new project like that that kind of where you went Wow!

Abhik Banerjee: Yeah sure. So basically, then I think a few bunch of those basically – again, I’m like going back to my grad school. I was really interested, so at that time I think it was really interesting to understand some kind of link prediction that was at that time too much into the like how Neural Networks transform. So there were some projects that we were working on, which was around by – so that was a really cool and fun idea, which I think there was some start up where we saw who was trying to use that as their concepts and you get basically recipes across the internet, but then what do you do is you actually fine tune and fine drain from the ingredients and try to train your neural network to create new receipts from it. So this might not be existing, but then actually you take like real ingredients from those in order to create a new – something which might be going in proper proportions and how much should they be mixing in order to create a new recipe all together. So that was really like a fun kind of an experiment which laid really the foundations of these things can be applied to not only fun but also for revenue generation models.

David Kruse: Interesting. Did you come up with any fun recipes?

Abhik Banerjee: I think I had some – yeah, I had like a brownie come lasagna which was like a mix of lasagna items and brownie items, so it was like funny.

David Kruse: Nice, nice okay. And we can talk. I’m curious how you did that, but we can talk about that a little bit, how you implemented that?

Abhik Banerjee: Sure.

David Kruse: So yeah let’s talk about Kohl’s a little bit. Can you tell us your role at Kohl’s and kind of your focus areas?

Abhik Banerjee: Sure, I can do that. So basically I was – I joined Kohl’s at 2014, so I was one of the first data scientists in the team and I think I’m very happy that like in the last two years are team has grown close to around 20, 25 members right now. So at Kohl’s like recommendations had been a major focus, but I think I like to suggest some background around how things are actually currently at Kohl’s and how the whole organization is moving towards the direction where every area of the retail business, or every vertical in the company actually wants to use data more and more in order to make more informed decisions. So the various verticals probably want to use data to make directive, but also something called perspective Intel, like analysis, so which does not predict the outcome, but also helps us in understanding the reason behind the particular prediction and also how we can prevent that. So let me go a bit deeper into some of these areas. So like two of the main assets of Kohl’s are our customers and our stores. So our main goal as a whole team is to basically understand our customers better, personalize their whole experience, the whole journey starting from when they land in the Kohl’s pages by giving them personalized deals, personalized products, personalized recommendations, providing them offers and it also includes the whole machine learning algorithms so that you can understand and can predict when the customer is going to abandon the car, they can react to particular marketing campaigns, how can we actually get the whole personalized affinity for every customer, so to understand them better, and serve then better. So that’s definitely one of the areas I think which as an overall kind of higher level the company is moving towards. And apart from that its also the make the stores profitable. So Kohl’s has close around 1,100 plus stores across the United States and recently there was an article being published by our CTO Mr. Ratnakar Lavu on the Times Magazine, which was talking about something called Smart Store. So what Smart Store actually envisions is you have a store so powerful and so well connected, to the network that basically understands all the customers around you who come to the store, live around the store and then you can personalize the whole experience and the store for those customers. So you have the right inventory in the right location, you have the right merchandising allocations, you have various behaviors to the customs across the store, in the store as well as you have a lot of other external factors that is affecting, like the weather data, the macro economic data, the social trends around the location. These are all various things that actually helps us I feel in making our stores much more smarter and more connected and overall give a much better personalized experience to the customer. So the goal of our like goals is basically to become the best technology company in the domain of departmental store retail and I think we have a few initiatives also going on in the deep learning area, where we want to pass the experience by using image based recommendations and a lot of ex-mining and applying deep learning algorithms, sort of understanding our sales, our customers better. So there is a lot of interesting things going on I would say.

David Kruse: Yeah, your keeping busy. Yeah, this is a lot of – so your team helps tackle all those, is like going almost three to four years.

Abhik Banerjee: Yes, that is right, that is right. Yes.

David Kruse: So how – I mean from a higher level, how in the world do you figure out kind of your priorities, because like each one of those projects could take probably team of 20.

Abhik Banerjee: Absolutely.

David Kruse: So how you figure out what to focus on and do you work with outside parties on some of those projects?

Abhik Banerjee: Sure yes. So basically we – apart from our Kohl, like members who are full time, we also work with some partners in the company. So basically we have been collaborating with some of the recent stockiest in the valley and that’s one of the major reasons of Kohl’s actually having an officer at the sort of Bay Area, is basically to partner with these startups which actually can help us in getting much quickly the ROI which we are looking for in a specific area. So, I mean let me talk about some of the things that we look at, so something like – so let’s say we generally try to attack a project mostly in two sessions. So what do you want to do with this deal, we want to start with the minimal viable product kind of a thing, MVP and then we’ll keep enhancing those features. So as a result of which, what is also helps us is with the lead. It doesn’t let us loose the immediate ROI, because if you start perfecting our models on the very first get go, we might end up losing the immediate ROI which, sort of which is a problem we are like trying to solve. So we try to do it in an incremental enhancement kind of a fashion, so that actually helps us to get our data scientist and data engineers much more focused in pushing out the feature in really quick turnaround times.

David Kruse: So with the store personalization project, how would you start to kind of get that minimal viable product and then expand it beyond that?

Abhik Banerjee: Sure. So basically for the store one, again there has been data used earlier, but I don’t think it’s a vast amount of data that is freezing there is any better way apart from using like machine learning and data science algorithms to make our systems better. And then again, the whole idea of machine learning is assist human beings in making their decisions much more smarter. So one of the key – like two of the key areas in making our stores smarter which we are looking for is, to allocate the right merchandize in the right store. So it used to be as the biggest – what happens mostly in retail is, if the allocation on the merchandising is not right, basically you might end up with a lot of inventory on the floor and then anything which is on the floor and not selling is a key loss for the company. So to optimize that particular factor, keeping lot of other constants in mind, like the particular geographical location, the weather around the next, like weather in the next 14, 15 days the upcoming events in the area, and basically how people’s behavior have been changing over the past, over the time, which particular timeframe of the year it is. So keeping all these factors in mind actually and also our customers behavior, so like Kohl’s customers some behave much more differently than I would say definitely from other retail customers. So understating the customers and putting that back into the equation in order to optimize the whole inventory of the floor is something which is really challenging and I feel which we are trying to solve right now. So that is one of the core like part of the feature of the products you want to enhance and then part from that then basically we want to focus on marketing campaigns and other merchandising allocations, promotions and offers being applied to those particular inventories there on the floor. So those steps will actually follow through as more and more features are allowed.

David Kruse: Interesting. So how would you, you know if you went to another company or if you are talking to the students, how would you tell them to even start with their merchandising. Because I mean there is just so much data that you can look at, how do you know where do you even start with something like that?

Abhik Banerjee: So definitely, like so most of the retail companies have been. I think there are two streams to it. So one is definitely the internal data, which we call it, which basally is walking about, all the information about our stores about our customers, how our customs engage with Kohl’s properties, that is really valuable, but again one of the other driving factors that I think has been lacking in most of the analysis towards retail is basically the external factor. So the external factors also actually play a pretty huge role in terms of governing the actions, by customers and sales by customers and new action. So I think people have started using them recently, but again I don’t think we have been tapping into those resources that much. So definitely I feel if I – basically I will start from a problem from scratch. Definitely the best part is to start with the data. So like we all have heard data is the new oil. So definitely data is something which I think the need to understand and own in order to before even jump into the machine learning piece of it. So having the right data in order to generate the minimal viable product is definitely of the much essential need.

David Kruse: And how do you know that you have the right data? So like you mentioned weather data which makes a lot of sense or like the events coming up. Would you test out like a pilot on a store to see if the merchandise has improved based on like predicting weather events?

Abhik Banerjee: Yes. So basically it’s a very – so may be I can give some background on sort of tied like to most of the machine learning problems. So it totally depends on what problem we are trying to solve. So let’s say we are trying to solve machine learning based problem where we want to see the model that we are building in order to allocate the merchandize to particular stores, how well they behave. So generally, and this is the process that we follow throughout I think the whole, all of our data science projects. So maybe I’ll just over a bit more detail into it and talk through it. So it starts with basically an ideation phase or hypothesis phase, where we generally collaborate with the product management team and the business teams in order to come up with a particular requirements and the solution that we are looking for and the then also get their – so like lots of the times it has been mostly the art is the business and the product management team have been working though. So again those inputs are actually really valuable in order to decide on the features that need to be going into the model. So once we, so that we again – as a machine learning model we just split our data sets into training and test. So we build our models within training data sets and then test is something which we deserve for doing that offline evaluation metrics. So there are like pretty common offline evolution matrices for most of the supervised learning, machine learning problems, including position recall, left measure and cumulative distributive gain and other things. So we generally use all these metrics on the offline data set in order to see, if we would have done this on the past, let’s say three months of data, how well our model would have predicted it in the last months. That gives us some benchmarking against the offline evaluation metrics, like doing it offline before we decide to push in and try it actually on the retail customers. Apart from that we also have something in house tools where we also want to see to give this as an experience to the customers, how well the look and feel would be. So there are cellulite components where which we actually tie back, call the innovation app; it’s like internet to the teams. So they basically look at it, they give the reviews, opinions and that we go with arbitrations to fix, like modify our chance in all those algorithms. And again one more thing is like we generally try to work on two or three different versions of the algorithms. So basically there might be some tweaks, or high biometric tweaks for some different algorithm, opinions which the teams have. So basically we generally try to work on two or three different versions. So it might happen that both the second and the third, like both the two or three versions have pretty closely related results on the offline evaluation matrix. So then basically those are two primarily candidates to go into the AB testing. So then we have our own AB testing frameworks, which we mostly use for all our – like as well as being put to use. Like a lot of our marketing campaigns, lot of our line promotions, offers, personalized recommendations across the website, once they are put to the AB test then this will be run by the AB test for a few weeks and then we collect the matrices around monists. So our primary source is monists. We get our metrics from monist and then there are a lot of – from the retail online side there are a lot of metrics which is really prior to the revenue auto conversion rates, add to cart rates, click through rates and the same pretty much goes for the merchandising also. How much you actually – people that are interacting with this, are the selling. How far this views are selling, so those kinds of metrics actually helps us to benchmarked and see which ones are the best rate from the AB testing standpoint.

David Kruse: Interesting okay. And one more question on that. So I’m just using the weather example because its easy to everyone understand is that, what types of tools – how do you bring in the weather data and then how do you kind of actually test it. You know you mentioned you have like a testing period which makes sense or you back test it. How do you – yeah, what tools do you use to pull that together?

Abhik Banerjee: So basically – okay, good question. So maybe let me give some background on how actually the data has brought us. So I report the Director of Data Science who like reports to the VP. So generally the team is structured between two different groups. So I currently am – they can think of the principle data scientist in the team from the data science side. So there’s also a platform team which actually is mostly responsible for all the data being pulled in from various resources into our big inner cluster. So most of all, like almost all of our algorithms from our big data and other branches, so the data platforms being actually is responsible for a lot of scoop and other functionalities from a hurdle standpoint to be in the data, clean the data, do the minimal key processing the data and then expose them as visualization tools and tallow, are the dashboard and also in terms of high internal users. So basically we are completely a hardware shop. So all these data sources are pre processed, cleaned, aggregated in a form, which can later be consumed by the machine and the algorithms. So this is the part which happens with the data platform team. So like for the weather data there are some amount of effort being spent or had been spent by the platform team in order to find the right resources, get the right information, get it validated with the data science team in order to make sure that is the right and valid data which we want to use, get the proper licensing in place. There are like – they are not free resource, and freely available data sets and then we have the whole pipeline built into it, so that it can be brought in as a regular feed data. And then there are like post jobs in pay and other scoop jobs which actually goes and transforms, cleans the data set and loads it into proper format of the downstream jobs to be using them.

David Kruse: Got you, okay. All rights, that’s helpful. And so I don’t we have a ton of time left, but I had a couple of other questions. One was about you report to senior level management and I’m curious how do you know – do they ask to see or ask you certain question to report on or do you say, ‘hey, this is really interesting or you should know about this.’ or how does that dynamic work, because I’m guessing you probably know a lot more of the details than the senior level management knows.

Abhik Banerjee: No, that’s really a good question. Because I think I have been fortunate enough to actually present to key level executives around a couple of times, like for the last four or five times and I think really with the senior management it goes really very, very like encouraging and helpful that they really help us to let go and actually identify what we can do for the business. They have given us the full responsibility as well as full freedom in order to work towards the betterment of the company. So may be let me go wash all the details around it. So the way we want to show the senior management is not only the big picture of how the proposed solutions are going to not only improve the experience of our customers, but also how we can use data to make our more informed decisions in whatever we are currently doing as far as with the verticals. So let’s take an example. It’s like one of the key areas, like the higher level goal what we want to show the senior management of what the whole data science team is working towards is, rather than being more reactive we want to be more prescriptive. So let’s say like doing our analysis which we found or upcoming few weeks, the sales might be down. So our job is not only to predict the same that the sales should be downward also to get the whole dynamics around it, to prevent even before it happens. So generally for the senior management one we divide our like demos into two different areas. So one of the areas actually comprises of what we have already been pushed into products and they are performing, the metrics around it, what was the – what actually benefit does it give form the customer stand point or from a store standpoint. How is it going to improve the whole experience of the customer personalization, as well the whole experience that’s brought across the stores and the Omni channels. The other area s to the product, to the projects which we are currently doing, which may have some what of a long term vision and also have some of the research components into it. So which we are like still experimenting with and some work you wanted to see what exactly we can use it to materialize into something which we want to pursue further. So generally we try to segregate our like demos into two different areas; one, which is are already into brush and like is definitely needed for the business to make and improves the processers and how we are improving them, how it is being engaged with the customers, with the stores and how the revenue numbers are for those and those kinds of things. And the other areas is something like more in the – like the people of each site, where we are actually focusing, okay this is a very concept. You want to start implementing into our products and use cases, but again its might take some time in order to realize to the real value of it. So I fell like sometime it might be in a couple of months, but not immediately. So that’s what how you try to like segregate our – as far as you want to show the task map. There is the other key area which we want to showcase is basically, apart from the software development part of it and the algorithm part of it, we also want to showcase that basically we also have a good research obvious that we are taking ideas from journals, publications, to exactly the products and how they are actually improving the products. So no research is complete unless they are actually in the hands of the customers to be used as terms of products. So that’s a very valuable and key lesson I think which we want to focus on that anything that we do from a research standpoint actually needs to go in, should go in as a feature into a particular product which improves the overall goals for the company and the customers.

David Kruse: Interesting, okay. And another question I wanted to ask was what are some – I don’t know, I was going to say major tech hurdles, but maybe it’s not technology, but is there any major technology that you wish was available or is it more kind of working through the data and the processes and rolling out new algorithms. What’s the major hurdles you are facing?

Abhik Banerjee: Sure. I think I would say definitely, I think like one of the things I feel every company actually faces as they grow is, as the multiple teams are working on the different deliverables and various solutions, they are actually disparate places where the data resides. So the various solutions, the different teams build they might be using different data sources, this kind of data being like placed into different parts of the organization. So I think there is a – and I think there is a very good direction by our CTO again, but the whole organization is to have a single source of fruit. So we are actually getting all the data stores, one of the data stores, data mast, data architecture, in a form of a data, how about data weight. And big data, so that becomes our primary source of all the data being aggregated and stored in one single place.

And then the other part in between all the solutions that we are building, machine learning and data science standpoint, that can be readily available to all the teams to use and they are like, they should be not even in the form of dashboards or experience or reporting, but also in the form of API. So basically the way this whole ecosystem should be built, or if you are trying to build basically have the course source of data at the hub of it and then basically build solutions of top it which can be directly accessible by partners, customers across the organization, also external if needed, who can actually just by call up APIs user machine learning trained models or train their own models of can actually place them as they need.

David Kruse: Interesting, okay. Well, I think that’s a pretty good place to end it. So unfortunately we are out of time, but Abhik that was a – I learned a lot, so I appreciate you telling us more about your experience and what you are doing at Kohl’s and it sounds like you guys are doing some really interesting things. I’m excited to see what the Kohl’s Stores look like in the future.

Abhik Banerjee: Sure. Thanks Dave, thanks for having me.

David Kruse: Definitely. And thanks everyone for listening to another episode of Flyover Labs. As always I appreciate it and we’ll see you next time. Thanks Abhik, thanks everyone.

Abhik Banerjee: Thank you very much Dave. Thank you. Bye.