Everything You Need to Know about Data Science Consulting (Gleb Drobkov) - KNN Ep.23

we as consultants we take pride in our work but we don't necessarily always need the credit what we care about is that the people that we work for look really really good [Music] today i have a blast from my past we're interviewing glebe dropkoff glebe is a senior data scientist at bcg gamma the advanced analytics practice of the boston consulting group his career spans consulting in the financial services industry and these days he typically focuses on using data and machine learning to help brands and retailers connect with consumers in more personalized ways glebe and i actually went to high school together in washington dc so it's great to see our paths in the data science world converging this year in our conversation we talk about the unique opportunities and challenges of working in a data science management consulting firm the breakdown of time spent on a typical analytics engagement and much much more i hope you enjoyed the interview so thank you so much for for coming on glee as i mentioned in the intro i actually went to high school with you so things are really coming full circle we're i believe of our class the only two people that went into the data science path which i guess isn't that surprising because we had a pretty small high school class but a lot of social sciences at st albans but i'm very glad that we found the the the way of the jedi early on in our career exactly exactly and again i i mentioned uh earlier in the intro how you're working in consulting as a data scientist at boston consulting group and that's definitely something fundamentally different than than most of my other guests obviously i've had a little bit of experience in the consulting branch but my company is far smaller far less organized so i think the the viewers everyone listening is going to get a lot of benefit of understanding what the differences are between traditional data science and maybe the consulting group so first i'd love to get your background you know how did you break into the data science also break into the consulting um i think that that is going to set a really really interesting stage for anyone listening here yeah absolutely canon feel free to kind of clarify any other questions as we go along but the uh my path to data science actually started out a little bit non-traditionally i didn't study computer science uh as an undergrad i actually studied economics and i had a kind of a minor in statistics coming out of cornell that piqued my interest in data analysis and and quantitative analysis but at that time we were working primarily with excel minitab occasionally and stata just for uh there wasn't all these uh open source technologies like r python that are now so commonplace in the commercial space and uh my career actually started out in consulting as a manager management and technology consultant at capgemini the french consulting firm i started there because they recruited at cornell and i went through the typical case interview process uh and when i went on the job i realized that i was one of the team members that was more eager to actually dig my hands uh into the data and get my hands dirty with a lot of different modeling and prediction and demand forecasting analysis and i found that i really liked it um so my kind of reason and impetus to start coding which is kind of the primary i think unifying characteristic for all data scientists is that we write programming code whether that's sql or python or r or spark um and i started doing that only after i started my career so i kind of morphed into a data scientist over time awesome and how can you talk more about you know from the economics i believe you also did your your mba correct i did actually yeah i went straight through and did my mba after an econ degree at cornell awesome and how did you go about actually learning the technical aspects of programming did you take courses did you just find projects to work on what did what did that uh learning curve look like that's a really good question there's a little bit of both because at first i i just spent my nights and weekends reading uh and taking these uh massively open classes that have the ability to actually uh write a bit of code in a terminal and then test if you solve the problem correctly so i would spend not all my time so i was working pretty long hours but maybe two to four hours every weekend just uh taking these classes and building up my competencies in the main languages um but then what really i think drove my adoption is when i tried to take on some uh projects of interest for myself so i started looking into stock market uh pricing data based on the yahoo finance api and i was fascinated that you can just write some code and then uh publish it and it will continue rerunning and updating kind of over time um and uh i started to play around with things like our pubs which allowed me to create a free account and to create different types of uh kind of financial i wasn't even investing at this point i didn't have any money to invest but i was personally interested in running these back tested models and seeing if i had this simple heuristic how much would that return over x number of years and uh i did that kind of on the side until i started looking for jobs in finance and then i actually my next job after capgemini was at jp morgan where i worked in a more traditional finance role awesome and you know my my question again this is like in retrospect this is a while ago but how did you know that you were ready to take on one of those projects after kind of getting that foundation in the basics i think that's something a lot of people struggle with is that they're they're learning programming they're like oh you know i i know how to write you know i understand how variables work i can write some loops i understand basic object oriented program maybe not in python kind of fuzzy but um how do you know that you can take it to something how do you know that you can start to apply it is there a point where you're ready or you just kind of have to jump in yeah i think it's it's extremely like there's there's so much anxiety about applying for these roles because the level of competition but i think it it it doesn't have to be i think that in entry-level roles if you are an aspiring data scientist look for a hiring manager that's looking to help you grow and actually build build your competencies and you don't need to be kind of an expert in deploying models in production when you first start out a lot of the time that's not what these firms are looking for they're looking for someone who has an appetite to learn and has uh some kind of analytical rigor that they can show and talk about so um i i i think i i'll bring up the the side projects or the personal projects again if you're if you're passionate about something let's say it's sports statistics something that i know that you know a lot about and you and you do a project that builds your story of i i downloaded all this data from this publicly available source i crunched it i made predictions and i kind of bet against myself to see if i could uh beat the odds that are uh set up on gambling websites for sports like i know people a lot of people that do this kind of work and i know the people those people talk about that in interviews at top tech firms and that's the kind of job that the kind of project that gets them their first job of they were passionate about something they use data to objectively analyze a life situation and they're able to talk about it uh with an interview so i would say don't don't be afraid if you're considering making the leap uh our door is always open for people who've been in the space for a while to give you a sense of kind of how we first started out and no reason to be afraid awesome and so since we're on the topic of of interviewing i want to get start getting the conversation flowing about the differences between perhaps traditional management consulting and um you know management i mean uh typical data science consulting and a typical data science role yeah so um one thing that would be interesting to me is can you talk about how that interview process at a consulting firm for data science would work um and perhaps again how that is different from from an industry data scientist yeah absolutely so i think if you're applying for a data science role at a large corporation i know companies like unilever or the banks or any type of manufacturing company they now have data science arms those types of jobs typically require some sort of subject matter expertise where you uh previously studied engineering and you understand how factories work and they might need a data scientist who can apply optimization methods to a manufacturing process so the interview prep process might be very much like uh case based where you will read the uh like articles in the harvard business review about big decisions that were made using data in the manufacturing world for that type of application process when you're applying at a consulting firm you have no idea what type of case interview you're going to get it's an equal likelihood that it's going to be in consumer or in the financial services or in the manufacturing or in anything you might be in mining space and what they're looking for is a generalist attitude of being able to think through what are the data inputs that a system has at its disposal and how might you use those data inputs to either drive additional revenue or reduce costs or just make the company more competitive and gain a competitive advantage whatever it's doing um so i guess a long-winded way of saying that the the interview process is is more it's more focused on uh broad skill sets rather than subject matter expertise interesting and so one thing when when i was doing consulting especially at an entry level that in a lot of my courses and a lot of the conversations i had was addressing being client ready um i think that that's probably less important in an industry role i mean you obviously have to work with uh like internal stakeholders but from you know the perspective of actually working with external clients where you know you probably could get uh fired or taken off a project or whatever that might be yeah how important is that and how do you assess that in a candidate uh so that kind of uh client readiness is actually something that um i think is it's very difficult to fully understand without with a candidate that doesn't have a lot of work experience because what you want to know is how this this person will behave if you brought them onto the team and day one they might be sitting across from the svp of marketing or with the cmo or potentially in a meeting with the ceo and how do you know that this person is going to say something which will be a creative to the conversation that won't kind of slow things down or knock them off course i think in the consulting world we try to uh control for uh individual biases that go into any interview process by having a lot of touch points um i know that at bcg we have at least four in-person uh meetings or in i guess in the virtual world these are there's we're still recruiting it's video conferences but a chance where different uh reviewers can actually provide their perspectives on would they want this person on their team and uh what kind of contribution could they make from the technical side if they're recruiting for a data science role um i would say that the best thing i would go back to kind of the previous idea of if if you have something you're passionate about talking about then that'll come across in the interview if you uh if you tell an interesting story about how you maybe um helped a uh i don't know on a team project and or in a previous job role you helped drive a group of people to a decision that was hard to have challenges with it and you can talk about those challenges and how you overcame them then that's kind of the way that you can prepare yourself best for any behavioral questions of just having your uh your your story polished and um at your disposal a few examples of uh for the typical question areas that you might encounter absolutely you know i think something that a lot of people can learn data scientists software engineers whoever it is from the consulting approach or that interview process or even that you know lifestyle whatever it is is that consultants are always thinking about the results and the end goal and i think that a lot of the time technical people i mean i guess we're technical people now we get lost in the process and it's really important to let interviewers know or let business stakeholders know what they can expect out of uh something that you're working on or what or what you've done like the end thing that you've done rather than just talking about all the different models that you used when describing uh you know any of the past work that you that you've completed or the projects that you've done so i think that that's an interesting maybe like a quick quick tip from the consulting branch that can hopefully help a lot of people exactly i think like to build on that even the imagine the interviewer's perspective too the they might be kind of lined up to speak with four candidates in a row for one hour at a time for most of the friday let's say and the more that they can kind of enjoy the conversation and make it a two-way dialogue the more you can also kind of uh you you if you if you look for cues of like areas that they want to go deeper on and talk about then that's typically how i try to encounter people the first time i try to listen and see what they want to talk about um and obviously answer the questions and and drive the discussion if they're looking to sit back a little bit but um also to kind of see what they're trying to attain out of a conversation or an interview awesome all right so let's move on from the interview process and talk a bit more about how the consulting data science path versus the industry path differs in the tangible work you'd already touched on one is going to be a lot more subject area focused but in terms of how the teams are organized and team in terms of what you do on a day-to-day basis how do those things diverge yep so i'll speak from my experience of two years working in uh financial services and then uh about four years in consulting but um on the as a data scientist in a big bank we were working with a very very large existing data set and a mature technology stack so we had systems and processes for accessing the data that were centralized and we would just go and submit a request with a team to get different resources that we need to compute or store our data and on the consulting side uh we go client by client the projects are shorter in length um the typical project when i worked in fs was maybe six to 12 months and in the in the consulting world it's three to six months before the project might go on but you as a data scientist might wrap up your module and then get transplanted onto another project um and within those three to six months you have to do everything that the financial services firm takes for granted and you have to deliver the analytical models so you're responsible for getting on the ground and writing the first sql queries on the clients database you're responsible for writing or communicating with an engineering team what kind of pipeline and daily refresh process you might need to get data off an on-prem database into a cloud environment that a group of data scientists can share and work on and collaborate on models so um i find myself i think the latter work the consulting work is very rewarding in terms of the project structure because there's fewer uh kind of points of failure you're you're there and you have the mandate of the management to give you access to the data and to be able to perform your analysis because at the end of the day the contract is for a set length of time and at that end of time the consultants have to finish their work or ask for an extension and nobody really wants that to happen so uh you in the versus in the industry world if a project doesn't finish on time like the bank is still making money while we sleep so you can roll over a project another three to six months until additional approvals or different uh sign-offs are obtained and therefore it's a little more risk-averse it's a little slower moving versus in consulting you kind of get the chance to make an immediate impact because the system you're delivering will get deployed at the end of your project awesome also i want to talk more about you know the system being deployed at the end of the project i think historically in traditional management consulting you get on your six-month engagement you pass off your insights and then you like never think about it again maybe that's like a big stereotype whatever that might be but historically i think that that's more the norm than the exception to it with data science with software engineering with even data engineering you're creating things and you're passing off now basically products that these other companies are using how does you know bcg how do consulting firms make that transition to rather than just like saying here this is done and then like let's negotiate another contract saying oh we kind of have to do some maintenance to this or you know we have to make sure that the code base the the github repos whatever that is is usable by these other people how do you not necessarily ensure that but how do you like set it up so that there can be more success for the client going forward yeah i will tell you i've i've seen projects that were very smoothly delivered and handed over to a knowledgeable client team that maintained them and i've seen other projects that had more difficulty getting adopted and and deployed or they had challenges after the completion of development uh the best thing you can do i think is to treat the clients as the owners from the beginning uh we as consultants we take pride in our work but we don't necessarily always need the credit what we care about is that the people that we work for look really really good and that they they can build and deploy things and be owners of them and we also help them hire people too so we will go out and we will uh help them write job descriptions for the different engineers machine learning specialists or just front-end data scientists that they need to bring into the org and we'll help them design the org of the future to support the the production environment at the end as well but if that doesn't happen then i i would revert back to we try to write code in a um agile way so that we uh get something that works and uh is deployable early on in the project just so that we have kind of a certainty that we can complete on the scope and then we prioritize additional features and add-ons as we go along at the end of the project we sometimes have to backlog or de-prioritize different different editions so we know that when we hand over a software product it's something that has the guarantee that it will run um otherwise we're we're still available if we if if we if we say with our word that something's going to run it will run we're responsible for it very cool and so maybe i probably should have asked this first but what types of deliverables do you guys have you know what what are the main things are the dashboards are they api endpoints what what do those look like um or is there a huge breadth of that uh it definitely differs by project uh the maturity levels of the clients that we work for vary across the board we might be working with some clients that have to publish a response to an ad exchange within 50 milliseconds and then other clients that a batch process with daily refresh is okay and but um to your question of what we actually deliver it's usually a system a data system and a process surrounding it that we pilot in conjunction with a client team so in the case of for example a demand forecasting algorithm that might pull together different data sources like weather uh or traffic or other historical information um and and merge in real time to make predictions on a forward-looking basis we would uh hand over a we'd actually build the um the code base and the system inside a client's cloud environment so whether that's aws azure gcp we're agnostic to whatever systems we build it in our engineers will have user names in the system and we'll set it up and then we will work hand in hand with the client i.t team to maintain it to document potential uh trouble points that we've experienced in the past and to kind of create mitigation or service level agreements to support it awesome and so you've talked a little bit about the the client teams like on the other side what do your guys's teams look like are you like put on a team with a bunch of consultants and like an engineer or are you guys kind of a solid a system within bcg where each team will reach out to kind of a pool of data scientists for insights what is that um organizational structure when you go on a client project look like yeah um it's pretty uh pretty wean in terms of every everyone who's on the gamma side on that's the data science arm of bcg that i work in uh we are typically responsible for the uh interfacing with the existing data systems and uh the the marketers and the technologists but there's also typically a strategy and finance side to our projects so we might be working on a project after an assessment has been conducted by some very smart mbas and uh other experts in the in the particular subject if it's consumer they might have decades of retail experience and that they bring to bear to assess uh and scope what is the uh the market size or the opportunity of this type of uh automation um and uh this if this if this algorithmic insight is injected into the business process there would be uh net this much upside so once we work with the uh the core consulting team as we call them bcg to understand what is the value opportunity then our teams will work together to craft out uh what is the organizational change management plan that we have to implement like basically getting pilot participants and getting them to uh look at the insights and then change their behaviors based on those insights and then track it and create reporting for executives about how the program is going that type of work is typically done by the uh the core consulting side and we are part of every single meeting like working to develop the algorithm and make it better and better and more efficient and implement the features that the executive suite is interested in having there cool well so you said that the team structure is lean how many people are usually on a project yeah if it's a project with about two modules we might have uh uh two data scientists on each module and then one lead data scientist uh working on uh kind of the interconnection between those two and of the data scientists you might split one who's working on the data engineering and the pipeline and another one who's working on the uh the modeling and the ml um so that it's pretty pretty important then you imagine like maybe uh a parallel of that structure on the strategy and implementation side too very cool and so for the uninformed can you just explain what a module is sure uh we consider a module a self-contained project that uh i guess for from the bcg i'm a resource at bcg we think of it as like a single evaluatable performance project like i have two they're either two months or three months or four months and i have a particular scope that i sign up for with my manager and we agree upon it and uh then we track how we do over the course of the project at that scope and uh a module is just a way of of breaking it into a deliverable so i would be responsible for uh deploying a system into production and supporting a pilot process and those would be my main two goals of my module and i would work on that until it runs out and then i might start on another project or be considered for other projects modules at that client or in modules at a new client it's kind of like a like uber for uh internal processes in the consulting firm and so are you involved in the planning process of the modules or those scoped by um like i guess like bigger wigs if you will uh the partner team is very uh very open to discussing the scope with us and actually helping us craft it so when i wrap up a project if i before if i have time before i start another project i will help to work on a couple existing accounts to scope out and to prepare for another potential piece of work so but it's half and half like if the client specifically wants something that i don't think will work but they're focused on making sure it's tested then uh who's to say that like i i will still do the work and understand and test out if i'm wrong or if they're correct um but the uh it compared to other organizations i've worked in um bcg prides itself on being flat and having a line of sight to the as you say bigwigs the partner and the managing director and partner team and i think it's pretty equitable if they need our input on technical matters they're always bringing us in early so you know one thing that i was thinking of while you were while you were kind of discussing that yeah the the the is the the scoping prog process and a lot of people have questions about okay how do i build a personal project and i think one of the things that people don't often do well is plan the project before they start is scope it out and like really organize the the steps that you have to go through is there a process you guys use are there specific considerations that are really important when scoping a project that could you know help people in their job but also help people who are trying to start a personal project yeah i mean don't don't try to rush things too much if you allocate a full week to exploratory data analysis on like a kaggle data set that you're working on don't don't kind of uh preempt yourself and build a model and see how accurate it is in the first day uh before you fully understand understood what the features are and you've kind of disentangled them and and de-averaged some things to understand the trends better i think uh spacing things out as much as possible on a project and holding yourself to certain milestones like i will create uh some descriptive reporting and a short story about what i think the main drivers are are based on looking at a correlation matrix and just some basic descriptive statistics i think creating and documenting hypotheses is almost as important as jumping right to the gun and trying to make the most possible accuracy on something you're trying to predict some classifier you're trying to predict uh the more i worked in data science the more i've realized that a little bit of time up front of thinking through the problem and writing out a plan with a very rough timeline uh can help you pace yourself so uh so you don't burn yourself out working on something when you're trying a bunch of different things and it doesn't make a difference having those documented hypotheses in the beginning will it's a treasure trove of other things you can explore absolutely i think you spin your wheels a lot if you don't know what direction you want to go in and you also go and go in circles and it's it's really important to you know even even just a quick glance at the data understanding what columns are there you can start to form a mental model in your mind of what you think might be important and then you can go and test those things and you know something that on a project recently you know i struggled with was that we had a lot of data and we kind of knew very like a very fuzzy idea of the direction we wanted to go but there's really two approaches you can take right you can either start doing like exploratory analysis with some unsupervised learning so we're looking at clustering data we're doing pca we're doing factor analysis trying to like group things together or you can do hypothesis testing where you're comparing you know individual features with the with the dependent variable and looking to see if there's some relationship there and so i think it's like okay well which approach is going to make the most sense should we do both you know like even understanding that and framing your exploratory data analysis is so so important because in theory right eda depending on how much data you have could take an infinite amount of time seriously yeah you could just go all day just exploring the different features and distribution of uh of the dependent across them you and you could drive yourself crazy and lose your hair thinking about that all the time exactly exactly so i want to kind of shift gears start talking a little bit more about some of the types of problems your clients are facing the ones the stuff that you can talk about obviously there's some ndas there's some sensitivity there which i think is true at almost any company but i'd love to understand the types of problems you're facing and then maybe some of the initiatives that you have going on or bcg has going on uh right now yeah absolutely i'll talk kind of more broadly about one particular subject that i think is becoming a very common use case for data scientists to work on um and then i'll talk about one particular project that i think is very very interesting to me personally in which i'm i've wrapped it up now but i'm going to stay involved on all the future iterations of uh the subject matter is uh personalization and creating uh targeted uh marketing content for users based on their historical preferences i think that there's so much two-way feedback and interaction that can exist between brands and customers um especially as we transition more and more of all of our spending into digital channels i think the brands that build the best loyalty and have the most uh kind of driving that dopamine hit or the happiness of a of a user when they uh see an ad or purchase something i think that uh there's a lot of data science and behavioral analysis that can drive those decisions and the kinds of uh analysis that like facebook and google are doing um to to personalize your spending habits and your purchasing behaviors i think uh will transform our economy as we know it in a lot of ways and the traditional retail space is definitely going to transform as demographics change but data scientist projects there are typically very you're not that satisfying because you are making very very minute changes to maybe the the section that an ad is run or the the type of uh product that's shown in an ad or a different coupon that how how much it works or it doesn't and it's kind of a game of attrition where you have to test a lot of different things and measure measure measure things very very precisely so requires a lot of uh architecture and planning and kind of data analysis to stratify customers send them offers and to measure their performance on those offers i think there's a lot of work to be done there so i think that a very large share of all future data science projects will be in some way to personalize this the uh outreach or targeting of an ad so real quick there's something i i think it's really important for pro listeners to to notice is that a lot of data science isn't just model building it's like uh problem architecting and so a huge component of data science is a b testing but it's really hard to do a project where you a b test in the real world that's more of an industry like like on the project type of thing but it's something you should be familiar with i mean that's why people harp on t-test and p-value so much even though it's like okay this is pretty simple there's some obvious kind of issues with p-value in my opinion but um at the same time like those are some of the most important tools for evaluating if uh like a test you're running is working a live test or you can get immediate feedback you can there's so many systems especially with advertising uh that are just solely based on which ad we should run uh from collecting live data and and you know that is a huge component of it understanding that can really pay dividends on the job or in the interview process absolutely you have to speak the language and uh there there's a lot of resources out there just to to learn uh this process because at the end of the day it's testing and learning uh that's that's all the machine learning and data science kind of boils down to in a lot of cases um i want to answer your second question on another another project that i've been working on recently and i think uh this is an area which doesn't get enough focus from the data science world um and that's in projects that might not be necessarily for profit and i think that data science has a large use case for social good if it can drive uh systems in the developing world to make better decisions for example around healthcare the data already exists in many community healthcare organizations to track which children are most likely to not adhere to a vaccine schedule or most likely to uh kind of not have a uh the situation where a mother has a choice of whether to take her child in for medical care because of certain types of patterns in different parts of the world and we can empower those caregivers and mothers and children around the world by code that we write sitting in our homes in north america uh and all it takes is is is volunteering your time for ngos that might have data assets um that are currently being used and if you can anonymize it and and and make the data uh private and only used for kind of aggregate analytic purposes but then you can also create insights for an organization about how it can run more efficiently use its limited resources in the best ways uh i think that that's a really something that's underlooked and i really hope that uh the social impact we work on this a lot with the social impact practice at bcg we invest our own money and time uh of our resources to work on these things and i really appreciate the firm for that and i i would hope to see that maybe there's grants for other people to do projects for uh for for ngos around the world in the future yeah i mean i think that that's that's incredible and you're 100 right like that's that's an area where we can help out um and you know even for people that are just learning like you can still create value in those spaces there there's simply not enough people doing analytics related to ngos nonprofits whatever that might be and you know it's great to help all but but selfishly for for those people doing a project that creates real value that can actually help other people and there's tangible impact that looks great for you too in the job market whatever you're doing so like it's great to be altruistic but if you're in one of those situations like just volunteer your time like there is value for you i mean there's plenty of people that i've talked to they went and did some of these things purely for altruistic reasons or because they were interested in it and they either got job offers offers to be interviewed they got um you know blogging opportunities they got a lot of good things that came from just simply putting your work out there and and helping other people so i can't stress how significant those things can be even though we don't really keep them front of mind so so the last thing i wanted to ask i i leave the floor open to any of my guests at the end of each interview to talk about things you have going on in your life uh any projects you're working on any final words of advice that would be helpful to um to to anyone listening i mean heck you could talk about your dog if you wanted i don't know uh but but yeah the floor is open to you you're welcome to to kind of spitball uh thank you i'll just say uh i appreciate you and i i'm grateful that you're inspiring many people to demystify this path and to see that it's actually accessible to to anyone who puts their mind to it and has a passion for it take it from me i i studied economics and statistics and i picked up all the computer science along the way over the past seven years of my career but uh there's always a place for you in the data science world the number of problems is only growing and the demand for people like us is is has never been higher so uh keep at it stay focused uh feel free to reach out and i'm happy to be a resource and as helpful uh as i can be to any of your listeners perfect i'll have your linkedin linked in the video version as well if you have a twitter or something i can throw that up as well and i'll post i'll post a photo with uh my ken's nearest neighbor uh t-shirt on i i have it at home i love it we'll all be repping you know so perfect thank you so much for for coming on glee i really enjoyed it uh always a pleasure man always a pleasure ken have a nice day thank you

Channel: Ken's Nearest Neighbors Podcast

Views: 6,090

Rating: 4.9622641 out of 5

Keywords: ken jee, ken jee podcast, kjp, podcast, data science, data science podcast, knn, ken's nearest neighbors, knn podcast, ken's nearest neighbors podcast, knn episode 23, artificial intelligence, kens nearest neighbors, kens nearest neighbors podcast, Gleb Drobkov, BCG Gamma, BCG, BCG Data science, Data science consulting, Consulting and data science, data science consulting vs industry, data science consultant interview, what is data science consulting?

Id: UHZz7h5G56g

Length: 39min 18sec (2358 seconds)

Published: Wed Nov 25 2020