Interview With My Brother Who Sold His Startup For $60 Million | Machine Learning Engineer

and earlier this year we just sold to appFolio and which is where I am building out the product scaling it and adapting the product to their client base got it mind if I ask how much did you guys sell it for well it's public for people who want to know mind if I do my research and put in my video I guess I guess I can just tell you because you're gonna put it on anyways we sold 460 million before we continue this video I just want to say thank you brilliant for sponsoring this video every day brilliant publishes daily challenges on many stem topics like math science and computer science this site is extremely sleek and they have over 60 interactive courses which makes learning these concepts way easier because of the hands-on approach they also have a artificial neural networks course which is really really well made and I think if you're very interested in ml you should definitely check it out this is a great complement to University because you can do practice questions under well curated sequences of problems which allows you to master the topic you want to master you know I wish I had brilliant when I was in college because I'm more of a hands-on cam guy I learned by practicing and if I had brilliant I think I would have understood these concepts a lot faster so if you're interested you can go on brilliant org slash Joma and then the first 200 people will get 20% off all right that's it all right welcome back thank you thank you thanks for having me back yeah thanks for coming David MA my brother it's cool I think I'm your first interviewer that came back for a second round right am i right I think you are actually yeah interesting because you were high in demand so I had to bring it back cool so just a little context in the the previous video I made an interview with you and it's mostly about how you were a quant at 2 Sigma and then that video you told me that you quit your job to do to co-found a startup is that correct yeah so a lot of people were wondering why did you make the switch the switch there's a lot of small reasons I wanted to try something new wanting to see what was out there for deep learning I wanted to also look at cryptocurrencies I want to look at biotech but in the end Elliott the co-founder of dynasty reached out to me like a week after I said I was gonna quit and he was like hey have you ever thought about quitting and I was like did I quit like one week ago like how do you how did you how did you do this hmm and then didn't he said like yeah just come out to LA and then see what we're doing and then there's nothing no harm done right yeah I got one thing led to the next and you know Here I am now nice nice so between back then when I interviewed you to now any updates on the startup is dynasty right yeah there was a lot of a lot a lot happens since since the last time we spoke I joined dynasty at that time we had just pivoted into the the AI for for real estate business and during that time we built a product found product market fit and earlier this year we just sold to appFolio and which is where I am building out the product scaling it and adapting the product to their client base got it - I asked how much did you guys sell it for well it's public for people who want to know mine if I do my research and put on my video I guess I guess I can just tell you because you're gonna put it out anyways we sell for 60 million mm-hm yeah nice how many co-founders really yes so dynasty was initially a different business which did not succeed and that's very that's very common for a lot of startups so yeah at the end of the last business everything was we were like 10 people and I joined that as like the sixth person and at that time things we're not going well there was no product market fit and we started pivoting into the real estate and about half the people best oh so in terms of co-founder originally there were two co-founders and then this new will for the new pivot five of us were left just a tldr what was the previous product and what is the product now just to make it more tear back then we wanted to create I want to securitize real estate assets basically create a in exchange and then allowing people with a smaller amount of capital to take and take positions in real estate assets if you think about it right now you buy a house you have to have you know especially in Silicon Valley you have to have like 200k 300k just to put it put down a deposit it's not very democratic right and also your all your money is into it just one single assets that's very susceptible to to local changes so we want to just change that it didn't really work out we at least we didn't find how to make it work I'm not gonna say it's a bad idea because we still think that you know there are benefits to this world we're dreaming of but in the process Eliot's and other people who joined before me found out that a lot of real estate participants had a lot of trouble managing their assets so it's unlike stocks real estate as an asset that you have to you know it's a real thing you have to keep yeah there's up cave you want to like get people into for rentals and stuff like that I think half of the the income comes from rectals right if you take it the other half has appreciation so one of the big challenges was the operations of leasing a building or leasing your assets so we decided well everybody says it's a problem so let's do something about it yeah and that that's where Lisa lisa is the second iteration of dynasty so that's like the pivot that's your new business that's right yeah and of course Lisa is an AI for leasing mmm-hmm I was wondering now that you've you know dig your hands deep into ml are there any misconceptions about ml engineering that you want to debunk I think the the general excitement about ml is great it made a lot of people go into ml a that and that's awesome but like a lot of focus has been on how do I build models and how do I fit a fitter model to the data but like very little focus has been on how do i generate data how do I design a business process that will create data for the algorithm that I want to build how do I handle the outputs how do I build all the process around DML components yeah there's too much focus on building the models not in the focus on how to integrate ml into existing products and to be fair it's kind of a new field right not many people how to know how to do this because it's it's it's so new like an analogy is when you know computers were were first invented her like the internet was first invented people were finding out how to integrate that into existing business processes and you know it took a lot of trial and error but that's the same thing like not everything is just building models right yeah not everything becomes more useful if there's machine learning in it yeah right maybe not everything should use blockchain technology exactly right that's what that's the example I want to use like I always wondered a lot of people want to do machine learning now my viewers especially because I think it's because I'm in the intersection of data science and software engineering but I don't I don't really understand the appeal of machine learning at the job because in my mind what you do is like you said you make sure you have good data make sure you solve a problem with your ml so most of the time in my head build data pipelines to funnel it into your model you pick a model you play with the you tune the parameters and then try to optimize for that AUC and then that's it like is or is there more to it I don't know that is fun I I think like you're you're kind of right I mean especially if you're building products you know you don't have the time to do the fun stuff in ml research the way I see it software engineering is the the core skill and then there's like ml engineering that that gives you a bit more domain knowledge into how to build an old products or ml models but in the end once you once you've done that like a v1 of it that's it you have to you have to build all of the systems around it and that's not what the school is on what you really learn at school mm-hm so imagine the fun stuff that machine learning researchers do what is that what is the fun stuff what would be fun for me I think in terms of research would be you know investigating the latest algorithms and like understanding why they work why they don't trying different data sets on these new algorithms a lot of the things that you've seen out there like Gans like generative adversarial networks you make like funky images style transfers like these are all investigations and why do convolutional networks work as they do and that's research right and that's not it's not primarily those things were not built primarily for business good and of course lisa is an AI for leasing the pun one of the problems that people had was like when you put your your your apartment for on Zillow or something like that you get a ton of inbound you have a you have emails you have text messages you have phone calls like it's the it's very fragmented the things that you get and you have to take all of that in Bound coordinate showings take care of applications and I do all that move-in process so Lisa what we decided to do was to automate the the responses to text messages emails phone calls and on the other side we just produced showing so people just had to show up and obvious the leasing agents just had to help to show up and sell sell the property they didn't have to like coordinate and do all that stuff and that was the main the main focus of it I think a lot of people liked how you know their phone stopped ringing after they used they started using anybody said you know okay cool awesome so what about you what did you do in dynasty yeah you worked on Visa I guess yeah of course so originally I was hired for to do some research because you know I was a researcher blah blah blah but you know nothing turns out as we plan it and I decided to take that opportunity to dive into ml I mean I had some ml experiences back in college and a little bit back at my previous job but never deeply so I decided to build out the the ml components got into deep learning learn about NLP which is natural language processing and once that core thing was built you know we were still startup we just had to do other things so I got myself into a software engineering you know before that had never done like real software engineering but under our CTO I was able to learn a lot about you know how to build how to build products and like yes I did software engineering and product design yes basically you have to do everything at a startup mmm-hmm so why why couldn't other people just do the same thing just apply AI to real estate stuff and then would they be a competitor like how why what made dynasty successful mm-hmm we also thought that there was going to be competitors and I think there there are but nobody took the symbol of the exact approach that we did which was Lisa would talk to two prospects prospects being the people who would rent they were protecting them without telling them that they it's a bot or anything because in reality were not fully about either about 40% of our messages are handled by humans who we call operators the fact that we try to give such a natural experience to prospect is something that the clients really liked because in general people don't like to interact with the bottom so our conversations look very natural because there's a lot of humans that kind of that we fall back to when things go wrong actually I have a funny story about people not liking to interact with buzz so one of the one thing that we measured was how often would people reply when we send messages back to them right it's like reply ratio whatever initially we would reply instantaneously because people would message in and were like hey we have a showing at 2:30 you want to come like within seconds and then the I mean people were probably freaked out and replied less than if we were to wait two minutes before replying so although we can reply very quickly sometimes we wait a minute or two before doing so that's interesting so another part that made us successful I think was the fact well I think the biggest the biggest success was the the market validation that was done prior to the pivot but that aside another thing was our willingness to just build what was was needed and like not focus on technology too much and just get something out out there we get it in front of people iterate and build it as simply as possible so that we don't waste too much energy trying to optimize a system that was not going to exist like a week later yeah and also these days AI products are like very I really hot a lot of people spend a lot of time optimizing that small small component which it in retrospect like I could have spent more time on that but if I did that I wouldn't have liked use my time to build out the product in terms of the software engineering part right deciding to say okay that's good enough and work on like the most important part for the business was at where something was I was spirit that like everybody had at Dynasty and I think that really mmm-hmm you know pushed us beyond the edge okay give me a concrete example of that thing you just described like what could have you optimized on like how did your ml system work as a whole what was that part of that ml system that you could have optimized on but what did you work on instead to make your product better in the backend the the first ml component that we we created was an intent classifier so we would take in messages and understand was the intent classify them one of a few intents like do they want showing did they accept something like whatever Oh actually before we continue can I just kind of have a high-level overview of what Lisa does because right now I think it's kind of just like a chatbot for leasing yeah and then is there anything that does like how does it interact with the operators and also the clients right so Lisa is a pretty complicated product in terms of how you would explain it like with components that you understand because there's a lot of interactions one interaction is with the prospect so with through there we it's just a bunch of text or emails the first text comes in we ask them do they want a showing then the conversation can go anywhere they can ask questions about the property they can't like you know decide not to do anything we have to confirm them for the showing like we asked them how how it's going that's one interaction with the agents the leasing agents we schedule things on their calendar we ask them questions that that we don't know the answer of we answer questions that they they have for us and there's like an interaction with bosses well which bosses the bosses a real estate agent boss correct okay sometimes the same person but like often it's not they want reporting so that's a that's the main interaction and then there's our human operators which uses Lisa as like an interface to the outside world they have what we call the command center and it's mostly like a messenger interface but like augmented with a lot of information and also they have a concept called the quick action which they can quickly find a a commonly used action to reply to prospects awesome okay so that means back then real estate agent will have to communicate with the prospects do the showing scheduled your own showing manage your own calendar but now they actually don't even need to talk to the prospects and you just have Lisa that is kind of like a layer in between that's right down oh that's really cool until they get to the showing which I don't you know they want to use their human specialty to do the cell you know well I mean one day you could build robots and just replace that - okay one day way cool okay so now that we have a good overview let's go back to what are the ml parts that are very specific I'm pretty sure there's ml in the quick reply there's probably ml in trying to classify what messages are so what weren't the other stuff that you could that you worked on that was better for the business and then just optimizing that okay once we got the the intent classifiers we could have you know tried better models like bird or Elmo like the stuff that came out in 2008 it was really hot we could have tried using that to you know gain a few percentage points of accuracy or those models like NLP Manos that's right okay yeah we used something like pretty simple like a Tec CNN for our core models and it worked it worked fine but like you know sure could we have done better we're still trying someone we have time but like there are more important parts to work on in Canada mm-hmm like the what like the question Tiger was born out of necessity and like they were they were answering the questions over and over again so like finding other places to to the package by the ML solution is more impactful than trying to optimize the existing components got it yeah mm-hmm so what do you think most startups hmm make as mistakes using ml like what are the common mistakes startups make using ml because I'm guessing you're comparing your dynasty to other startups how'd they do what would they do wrong my guess would be focusing too much on the latest technology that's out there especially the academic literature and trying to apply that to their business it takes time to pour academic literature to mature for business and a lot of a lot of literature is sometimes not reproducible mmm and it's a common problem so like investing too much there will waste a lot of time and as a start-up you don't have a lot of time got it yeah I heard a lot of people saying that in theory the paper sounds great and it works with their data set but applying it is a whole different story yeah applying you have to so finding a way to apply ml to a business environment is difficult because you have to specifically know which problem like this isn't in itself is like many many problems and you have to carve out one specific problem for ML it has to be worth it or at the time of research you have to be able to find a process that will generate data for that problem and once your algorithm is working you have false positives you have false negatives you have to have a process to handle that and then not even including monitoring like making sure that your models are are always performing as well as you think they are in our business with Lisa it's generally pretty it's not too bad because like English generally doesn't change but you know maybe if you're working on ads or like trading your environment will change and your algorithm performance might decay and you have you have to like know about that know so there's a lot of stuff that happens around that there's no small ml component so from what I understand it seems like it's better to build out your system understanding what you have to do how to do everything and then identify what are your spots that need ml and then solve for that rather than some other companies with identify maybe even an mo problem and then build out a solution like everything else which ultimately may be that solution wasn't even that useful for the market anyways right okay awesome so oh yeah I think I think another pitfall would be trying to solve too much with ml all right like there was a you know there's a few companies out there that wanted to build personal assistants and it turns out the job of personal assistant is like extremely difficult and there's like infinite variations right even in our business of leasing we found that it's a very subtle hmm taking the the very simple example of what's the rent you know you think that's a simple answer it's a simple question at first but then you have to say you think okay well actually what are they talking about the one-bedroom a two-bedroom are they talking about moving now are moving later are they talking about one one one-year lease or two-year lease oh wait are you talking about the like about all our properties or like only the ones that are available yeah there's a lot of tricky C's intricacies in in a simple question just as like what is the rent yeah and I also know it's quite difficult because I'm trying to hire a real personal assistant and that is still very hard so I'm guessing you know because if you can't even solve my problems with a real human I don't even know how to start solving with them like machine learning model because usually I think if something is easy to solve manually like a replica lots of repetition then you can kind of replace it with AI but yeah yes that's actually one of the guidelines that we have if we want to carve out something for the 4ml first solve it with humans see how I could work see how it works if it's really repetitive and like people make don't make a lot of mistakes right some heuristics not even ml and run run with it for a while handle your handle your false-positives handle your false negatives and if the system is humming then you know try to increase the accuracy with with ml mm but without these intermediate steps like don't even think about it mm-hmm all right so Dynasty are you guys are you guys hiring you know yes yeah we're definitely hiring right now you know we need a lot of good engineers the the way we we want to hire is hiring people who have a decent software engineering experience because we don't currently have a lot of time to to bring people up to speed because we're still a start-up even though we required and we want general software engineering skills we think that that will translate well into the ml parts or if you focus on you know more systems reliability or like more product but like the core software engineering skill is what we're looking for okay what kind of technologies would they be working with if they want to be an applied machine learning engineer we we use like typical stuff like tensor flow you know the Python packages and stuff like that our stack on the backend is Java again our pragmatism we don't we don't try to use too many fancy latest technologies we just use what is tried and proven so really that's kind of our core philosophies mm-hmm right so what would make a good machine learning engineer a good hire for you what kind of skills are not just skills but attributes personal attributes yeah willing to dig into the details understanding the business not like focusing too much on on the ml part being good at software engineering in general I dynasty we're not like ml engineers are not are not like oh I'm the ml guy and you can you guys can do like the back end like you have to do you have to do this on for engineering you don't get like an assistant for that mmm yeah so what's next for you and dynasty ended next five years mm-hmm yeah five years is a long time I mean just a year and a half ago it was a different business yeah now - yeah two years ago thanks it was a different thing definitely for me I want to see Lisa built out to its full potential hopefully you know the viewers will well someday rent an apartment I'd be talking to Lisa and after that who knows I think I'll be at appFolio for the foreseeable future yeah well maybe I'll be building other ml products hopefully finding other ways to apply ml into the real world mm-hmm awesome cool yeah thank you so much and I just want to say best of luck to Lisa dynasty and appFolio if they do want to apply to dynasty do they have to do it through appFolio website or is there a separate dynasty website we are fully under at full wear now okay so you should apply to a folio exactly so and then if you want to prepare for appFolio don't forget to check out Tech and be pearls all right I got taking intro if you're interested in getting ready for interviews oh yeah I also want to plug my Twitter em a David J M a David J a ghost's cool all right thank you so much for being here really appreciate it Thank You private all right

Channel: Joma Tech

Views: 1,323,346

Rating: 4.9325142 out of 5

Keywords: joma, vlog, machine learning, deep learning, data science, artificial intelligence, machine learning engineer, machine learning models, startup, appfolio, software engineer

Id: fB7nyxXaczY

Length: 29min 19sec (1759 seconds)

Published: Wed Oct 16 2019