Perplexity CEO: Disrupting Google Search with AI

knowledge on demand personalized to you is a trillion dollar opportunity welcome to the Logan Bartlet show on this episode what you're going to hear is a conversation I have with Arvin shabos now Arvin is the CEO and co-founder of perplexity AI perplexity is a business that has entered my daily active workflow as an augmentation for Google that I use every day to help answer different questions Arvin and I talk about a number of different things related to artificial intelligence including his decision to compete with Google why Google's existing business model and their North Star conflicts with user experience not everyone has access to the best Founders or like the best VCS in the world so who are they going to ask their questions to the web is a treasure throw but you need the GPS what do utopian world of artificial intelligence will look like in the future if you give people back their time life is a luxury that's what a utopian world would look like where even the people who are not making much money can do whatever they love the cultural differences between between Deep Mind and open AI two places that he had worked in the past why regulating AI today would be a bad idea as well as the different considerations that he has about providing answers to particularly harmful questions that people might ask I don't see this need to adopt a extremely high moral ground and like virtue signal to every every user that you're not going to let people like be able to get answers to these queries Marvin thanks for doing this thank you thank you for having me here so there's an axis that you talked about where on the left hand side is traditional search and on the right s right hand side is an answer engine um can you can you elaborate on that and where Google fits where chat GPT fits and where perplexity fits on this axis left or right so the left end of the extreme is probably like obvious links you go with the query you just get links uh that's the T you know larryan Sergey starting point whatever Google is today is already not that uh like for example if you ask um you know where does like Logan Bartlet age or something might pick up in case it's public right so on the right end it's purely chat UI you just treat everything like a chat bar chat chat UI is the default and so that's not the best way to consume information about like what's the latest core of the B game because are you have to get The Sweet Spot of accuracy high bandwidth communication of the information um and like the form factor in which we consume the answer all three things together right right so then the answer seems to be somewhere in the middle Google's also somewhere in the middle today but more closer to the left end because not because they want to more because that's their business model like even if they are incentivized to give you the answer from a product perspective from a stock perspective they're not and the right and chat gbt is like I'll just throw the largest giant model and give you a chat bot and you interact with it and the model figures out what to do but building a product is beautiful and like an interesting exercise because you're constantly searching for that sweeter spot every every single day every single week and trying to get to the user so I believe it's somewhere like answers most of the time sometimes information presented in panels and knowledge panels and like uh scorecards widgets and things like that and sometimes you just want to get away from the your app to like your your destination like say some subreddit you just want to go and Surf and you want to be able to serve all these use cases in like one single product it's pretty hard to figure out what that is but we believe like um starting from the answer endpoint can build in a different Financial incentive to get into The Sweet Spot than starting from the point when people talk about perplexity there's often this natural comparison to Google because of that traditional search versus answer in JY Paradigm that you just spoke on um but when I actually use the product there's a lot of um parallels of my usage to Wikipedia in some ways do you take e equal inspiration from both of those two things or how do you think about the Wikipedia versus the the Google um you know Dimensions absolutely look I mean I I was a Wikipedia nerd as a kid uh I used to just go to the Internet open a Wikipedia page and then keep clicking on these hyperlinks within it and get into these rabbit holes so that definitely was a big inspiration for perplexity in fact a user that really likes perplexity once described it on Twitter as u Wikipedia and chat GPT having a baby but that data uh comes from the whole of the internet that's what perplexity is so the whole citation format uh using authoritative sources uh giving the answer and paragraphs are like well formatted marked down um trying to generate a personalized Wikipedia article on the Fly for what you're asking is certainly the inspiration for the product so uh it is natural that you're comparing to Wikipedia now you can ask okay Wikipedia exists why do I need your product except Wikipedia is like one static version for everybody like you you may be interested in like the Oppenheimer movie um about like some particular aspect of it that might not even be in the current article and you might be surfing and reading the whole thing uh someone else might just be interested in the cast someone else might be interested in the budget so you don't want to read the whole thing again from scratch and like Surf and you know squin through and like go and find sources elsewhere and there's comparative elements of you know between pages right if I wanted to ask I don't know what grossed more Oppenheimer or Barbie I would need to compare exactly the sort of personalized experiences uh that can never be cated to by like a single Corpus is what perplexity offers to the user and if we can get this kind of traction Wikipedia has achieve today like they get I mean still the top five websites by traffic in the world if you exclude all the horn sites uh then that's like a big deal for perplexity and that's why I'm very convinced that like we don't have to defeat Google to um it's an important point and I've heard you say that search has always been a hack to get us information can can you elaborate on that uh point and how Google's model was kind of a a hack of of explaining things back to us yeah by the way I I want to credit Mark andreon to this point uh it's not an original Todd uh I I heard him say this in um his podcast with Lex fredman where Lex ask him hey dude like you buil the first browser um you know Google came onto it and you know and and now like we're in Ai and chatting so it's the ultimate search engine and answer engine to which Mark says yes I mean everybody knew this that was a reason ask Chiefs was even an attempt made in the 90s except it never worked we didn't have the technology to do this at that time in fact Google uh the core reason Google even became a search engine that people used was because they mod which are basically train on whole of internet and um they have such good language understanding capabilities that we can build a to that can look at all your links and read of read them and come back to you like as if an intern of yours did that work and gave you a Wikipedia article on on on the question you asked that human equivalent intelligence that a person would do by browsing reading and summarizing and writing an article cannot be done by an AI so suddenly answer engine becomes possible and then that hack doesn't become necessary anymore so all this is great it's just a product feature so you know why why doesn't um Google Just Launch it like reals like suck did and just kill uh like Tik Tok or like why doesn't why is it not just like stories like you know kill snap except Instagram can launch additional things it doesn't change their existing UI at all um and like like um he still figured out like how to advertise reals but in the case of Google like you are incentivizing people to click on the link like you are incentivizing people to like click on the link and and and when you bid on an adward you're looking at the cost per click now if there's no need to click on as many links anymore like you click only on like 1% or 10% of the links compared to earlier uh how are you going to convince the advertisers to still pay the same amount of money you can argue like oh yeah you know what it's like higher intent people are ask asking real questions so you kind of have to bid more for like these bigger questions they're not going to be like you know changing their mental models about this so they have a budget every month allocated for advertising on keywords and that's what they going want to continue to do and if the highest paying users no longer come there to ask questions about like you know which sparkling water brand should I buy should I buy San pelo sh the Crocs like like if those questions are going somewhere else and then they're making their decision and going doing the transaction smars like what is the point of advertising on this platform right these are the kind of questions they're facing they have no incentive to like get rid of the hack as fast as like somebody else there's some point in that Journey uh of this disruption that's occurring that it could become an inevitability and that Google's forced to reckon with the Blockbuster Netflix situation and I don't know um uh if that is ultimately how this is going to play out obviously but I'm curious do you are we looking at this pure innovators dilemma kind of Clayton Christensen uh total it's really hard to bet your entire business model uh shift on a small percentage chance that something looks different than what it currently does so my sense on how it's going to play out is it's going to play out more like Google Cloud so why did Google not build Google Cloud uh first and let Amazon get at the AWS moment I I actually don't know there a very simple reason for this cultural related stuff uh some part of it is that but primarily it's um primarily it's mostly the margins like Bezos has a SC your margin is my option he used it in a different context but I'm just borrowing it here of when when my margins on search advertising is like 60% 70% and cloud is likely to be like 20 30% it's still a good business but am I incentivized to go expand the 60% business or am I incentivized to go build out an alternate business how how can I roll out like 10 years from now all this advertising business is going to be under threat but if they had and diversified no problem now they would have at least been as big as Azure or even bigger ideally because they had better technology at the time and um even if search advertising business uh margins are lower or like they're going and spending money to build AI business um it's going to be like much better than now so my guess is they would have learned that lesson and said okay look we're not going to kill this business we're not going to go and try to change something that's producing US money uh but let's spend some money and try to build this AI subscription business or like whatever it turns into dat subscription apis around our models let's try to build that business whatever open AI is ahead today um maybe that's going to produce like 10 billion of year Revenue at some point for Google and that's still pretty significant like to Wall Street uh and then they would say like okay this is a lower margin business and keep reporting revenues on that uh but they're not going to like try to change the existing the the radical changes that Netflix did uh on like introducing advertising on the core platform and stuff like that I don't know if Google is going to will be to do such things um you know re Hing is a different kind of CEO and Netflix has a different kind of culture from Google so I'm not sure uh I my my guess is they're just still going to be on the following like they whatever other people are doing they're going to try to follow that what's the tension that exists for Google between the shareholder and the user because those things have diverged it seems at least in terms of the search experience yeah so so that's the whole like you always need to make sure the shareholder and user alignment is there um but now think of Google as basically two products one is the core search product and the other is the advertisement product there is massive alignment between the users of the advertising product and the shareholders probably the most massive alignment ever um but that came at the cost of the alignment between this the core search users and this is an important point so so so advertisers in this case would be a big brand that are paying to be on the link shareholders or the stock stockholders and then the users who actually searching yeah if you go to adwords.google.com you get a UI you can you can see a bunch of keywords you can see their frequency of searches number of searches approximate statistics and then you get to see the CPC you bid and then like your ad ad is running you get analytics on that you get the conversion stats this is a whole product it's so hard to build all this and it's so hard to get new users onto this and like build all the tracking for them they build a whole suit of products for that it's pretty amazing um and that's what makes some Revenue but it's built it it's basically Le using the platform they have for search to build this right like like it's sort of like saying uh I have a really fertile like lot of land and you come and you know host your businesses here rent spaces and like I'm making money out of all the rent you're paying except that when these guys are all coming in they're like hurting the uh the land and like you know creating mess like garbage and it's hard to clean up and like people are already living there are like uh I don't want to do here anymore that's basically what's happening the differences in actually using the product are pretty remarkable um for people listening on YouTube I'll show this uh example live but I was at a basketball game the other night and and um one of the players on the Knicks uh who's 6'4 had 20 points and 19 rebounds and I thought I wonder who the shortest player is ever to have 20 points and 20 rebounds and so I'll show your search results here for people that are listening but basically you gave me the answer which was Jerry West uh who was 63 and a half in 1962 and then I went to Google and I got some hodge podge of different results that Direct had to do with the topic but didn't actually answer the question it was like the number of people that have had 20 rebounds the shortest player in NBA history all these things and then I went to chat GPT and it actually gave me a wrong answer uh it gave me the shortest player in NBA history which is directionally right but very different um knowing that uh I think a lot of the things you've done are kind of built on the same Corpus of data that both of them have how are you able to give me that answer that Mak so much more sense to what I'm asking than what those other two are able to do yeah like the as for Google the the the first point is very clear it that's why it's hard to go from a link base to The Sweet Spot because when you're in link base you have to your your the true intent you're supporting is giving the user the link so the users when when the user actually types in a real question um it's really hard for the algorithms to work in the same way you have to detect that this is a quest and Route it to something else called SG and it doesn't always get triggered with the classifier is not precise all these problems exist so there probably wasn't a specific web page that answered that question so it didn't serve me anything exactly right um and uh as for chat GPT look I've always been saying this when people tell me like perplexity is just going to be subsumed by chat gbt um if the purpose of chat gbt is is to allow people to search on the web and only that yes like we shouldn't exist but the reason chat got all the hype and virality was people allowed like interacting with the actual model behind chat GPT and not the not what it can do by taking information from the web and giving it back to them that's a that's that's what we built and and that's a way less viral product and you know just asking AI interesting questions very open-ended answers don't exist on the web at all like it's what the AI thinks is almost treating the AI like another human and talking to it that is why CH gbt went viral and so uh when you're trying to build a completely different utility and value proposition in another product that users think is for something else then it's kind of confusing to the user and does and and even if you know leave the user confusion part like it doesn't work always in in the way it's intended to it has too many incentives at once to fulfill citations yeah core component of what you you all do how do how do you think about inserting citations as important it comes from our academic background um you know in Academia citations is sort of like a currency uh it's sort of like it doesn't matter by the way there you know I'm not I'm not like going to glorify citations uh because there are so many ways to have your Google Scholar uh like have a lot of citations by just being part of one big paper again it's not well counted it's counted equ equally for all the authors you can be an author in like a 20 author paper where the most of the work was done by the first two or three people and you can get 100,000 citations because if that paper was a big hit yeah but everything aside a good paper uh has a lot of citations this is the same insight for page Rank by the way like page has academic background and says you know a good web page is one that's cided by other web pages so we had the same idea that um a good chatbot should be accurate and have authoritative answers and the best way to do that is to make sure it only relies on highly sighted uh sources and and it's a CH chicken and egg problem of deciding what to site so the only way to know what can be highly sighted is like start rolling out a product and make it side a few sources and get a lot of data through your product and use those data points to decide like which websites to pick and not pick based on the quality of the answers and if this becomes a you know well oil algorithm in basically converts into a machine that keeps improving itself that's what we want to build if you dream the dream of perplexity success um you're going to play a very important role in what is true across increasingly complicated topics um for the most part Google's been able to defer to a lot of the links uh and the the sources of information um and you you also provide the links and the citations of all the stuff you're pulling from but um people aren't always I know I don't always click through to the actual sources and so increasingly um as ubiquity spreads you're going to provide some version of Truth to people um in a world in which truth is a complicated topic um can you maybe speak to how you think about this and the responsibility element of of Truth yeah uh I think like the computer scientist way of thinking about it is like truth is just a problem you can solve uh but if that is true go a step further and ask like oh what is the pseudo code for verification like if you can verify something then you can solve it because whatever you say you run it through a verifier and the verifier says it's true or false then you if it's true you give the result to the user if it's false they go back and try to change what he said right but the truth is like we don't even have a verifier today like as a human what do you use actually to know something is true or not if it's in your domain expertise you you you know you you you can write the C code if it's outside of your domain expertise you rely on Experts you go and ask another person your colleague or your friend who has like learned more about that topic or space and you ask them what they think what is this Legend and I mean I I'll just give you your example you're you're you're a venture capitalist so when you're like looking at companies to invest and someone's telling you oh this is the hot you got to get in on this what do you do you're verifying if it's true right you go and do your due dilig you ask a bunch of people you you know ask other other like cap VCS in in the in the space and find out so that's what like we are doing we're basically sourcing information from like good quality sources on the web and we're taking all those different parts of the sources and looking at the query and then giving back the answer by giving a Viewpoint across all of them right we're not making a decision for the user we're not telling the user hey this is what it is we're telling look these are this is what all these different links say uh there is no source that accurately explicitly Sayes this but these are the viewpoints right like that's sort of what we are presenting to the user and I would say that is a step towards in increasing like every person to discover truth but it's not truth as a service I think the second thing is much harder to build I guess related to that and the the thought around truth there's the other side of of safety in some ways and the obligations around that and so if I uh I actually didn't do this but if I typed in into perplexity what's the best way to kill yourself or what's the best way to plan a school shooting or something that's like you heavy topics um is there anything you look to to guide the decisions you make about what information you should surface back to a user versus what you should you know not provide answers to I mean it's a complicated topic right um I would say you should obviously warn the user like look if you're trying to kill yourself like important to like make sure you talk to some place for help before you make anything drastic but someone might just be curious how other people kill themselves right it's still important to know if you want to save others from killing themselves it's important to know how people kill themselves in general so if if if the intent for the query was that should you not answer the question you should right so I kind of believe that disclaimers along with an actual answer would work and there's a different philosophy where some people think like okay if if you know sure but why why make it even easier like why help someone make a bomb or like why help someone kill themselves or like kill their dog or whatever but I'm of the belief that you should know everything in the world like you should be able to make your good decision and the AI should help you influence you to make a good decision but if the intent behind the query was to actually do something like just get the facts out then you'll still be able to provide the answer um and a lot of people believe that it's so hard today to make get get answers to these things but reality is it's very easy YouTube has videos uh Google has all these links if Google is not doing this YouTube's not going to some they can go to Bing or Yandex or whatever if someone's so motivated to kill themselves and they're going to find ways to do it uh so I don't see this need to adopt a extremely high moral ground and like uh vir your signal to every every user that you're not going to let people like like be able to get answers to these queries yeah there it starts to be the question of hosting the information uh or or providing the information um through someone else's source to actually distilling and simplifying it and maybe that's too new or maybe that's an unnecessarily um narrow distinction between between the two and the disclaimer solves it but yeah I mean uh in general like by adopting this uh positioning as a scholarly tool like a tool that's meant to increase knowledge and be factful accurate boring but useful we've already avoided the sort of um affordance to the user that oh they see this as an AI they can talk to and have conversations with but you're not going to come and ask Wikipedia you know usually you're not going to go and ask Wikipedia how to S kill yourself you're going to go there to learn and on the other hand you might ask a more like character AI kind of an AI more personal things so by adopting that sort of affordance already with user we are somewhat safe as a tool like we're not going to be use for negative purposes and uh I think we can do some small small things like disclaimers you've given two anecdotes which show the challenges presented in building a company like perplexity that competes for talent against folks like open Ai and Google and meta one was you tried to hire a very senior researcher from meta and they said to come back when you had 10,000 h100 gpus which would cost billions of dollars and take five to 10 years to do not exactly a practical request the other was when you got someone from Google actually to commit to joining yeah only to have Google forx their salary to stay how do you compete for talent in a world like this by the way I I genuinely wish I just said big Tech and not the actual company names because you know it's unfortunately putting the people who spoke uh in the spot uh so that's a mistake I made I just think that the right right answer is not chasing the ones with the biggest brands on their LinkedIn or something like that but people who are truly motivated to build if you're already able to build you know if you're already in a great position in your big big tech company where you know you're coding you're the tech lead you're driving everything like it's very hard to convince you to leave like life is already happening for you like it's already great like what can be greater right like it's very hard to find that sort of a bet for you but then if you're really talented but somehow you're not fortunate enough to be The Driver progress in the in your current big tech company then you're the right kind of person I should talk to because at least I can provide you that and you're not going to make a decision based on money because even if your parent company offers you much better money to retain you uh the reason you want to leave is not money but actually to be able to build something so if we can arrive at a middle ground of where our offerings also good but what matters more to you in life at that moment is building than money then the decision making is much easier except like the these sort of situations where someone is really smart talented somehow not in the right position to build in their current company and is prioritizing building over money all of these things happening at the same time is very rare even getting one such person changes your current company's Destiny but uh and there are very few really good Engineers already and all these additional conditions need to be in play for you to lure them out which is why like getting somebody out of big Tech is not as easy as people think cuz the really good ones they are being retained well they are being given like autonomy and like power and in know technical decision making power and things like that and they're being paid extremely generously so it's very hard to still get them out and this is why like you know there was a window in 2020 or 21 2 maybe 2 and half year period where people at Google were actually unhappy they're really talented they they didn't have any agency and like um they just felt like they wanted a different kind of place and open AI post several of their researchers right and um that's like one unique moment it doesn't always happen hey guys I'm Jacob Efron a partner of Logans at Red Point wanted to take a quick break from the episode to let you know that red points AI podcast unsupervised learning now has its own YouTube channel we have an incredible set of guests really at the Forefront of the AI Revolution so if you're interested in what's happening in AI what it means for businesses in the world definitely subscribe now back to the show how do you assess the talent of people for their ability to succeed within an artificial intelligence machine learning world yeah when they they don't have um the existing competency or maybe you're coming from outside of Industry how how do you go about assessing those people actually I have a barel to AI itself on how to assess a human um the best models the best AIS are those that you can prompt with few examples or and they just get it they have never trained on that but they just get it that's what amazes you right this AI never trained on this but it's able to pass this exam or with just few examples is able to do this task um I think those are the kind of humans you want ideally fast Learners right now that's a skill that's very hard to interview for so some things you can look for is whether they've done an array of things in their past not just one thing have they worked in like very different projects and have they succeeded at both of them that's a very good signal it's very hard to be good at like many things at once uh so you must have had to learn on the job um and also I think you should interview for like mentality like the are they just interested in getting things done right are they a doer and that you can get through like back Channel references and um and then you interview for a culture fit like do you really want to struggle here like why are you even interested here are you coming here for the money are you coming here for the mission and um what does winning mean to you right um and all these things give you like a rough on tour of like what that person is and uh beyond that you just make you know you already have the coding assessments like you know the technical skills so you you get a pretty good measure if you've done like at least six or seven interviews and then then you make a decision right maybe you're wrong and like it doesn't work out it's fine uh usually the way I make decisions on like offers is like if somebody's really good that I'm like making a very good offer um there's always the sort of um debate among other people in the company of oh should we really you know make such a good offer or like you know we might be paying up let's negotiate and I'm like listen these are extreme cases mu plus Sigma two Sigma cases where if it doesn't work out we'll know really quickly and both both parties will mutually part base but if it really works out you're not going to regret making very generous offers so and and and and the time period for that is not going to be too long so let's just go ahead and make a good offer and uh um that that's helped me a lot like you got to speed up things decision making needs to be sped up in a startup how' you land on the business model of charging a subscription for perplexity honestly I we just copied chat gbt like like there was nothing I really wish they started off with something like $30 a month like everybody in the industry would have adopted it everyone in the industry copied them uh like anthropic uh Gemini Advanced co-pilot Microsoft it's all priced at $20 a month you can ask like where did that number even come from and it's like a random number open a made up pretty sure like you can put some thought into it and like justify it but if you went and asked them like would they be would they have done it like $30 a month they would have said yes do you think this ends up being the long-term business model that creates the most Equity value for I don't think so I don't think so look I mean the best business models have always been like usage driven right like performance advertising works for Google because it's at a query level like Facebook similarly ads are at an impression level um so the largest businesses like highest margin businesses have always been about usage like as your meters right so I think like um the current subscription model doesn't capture that uh and and so like any model that actually captures that at a really large scale uh can create an even more profitable business that said this is already a profitable business I hear chbt uh open a actually profiting from it not just making Revenue uh company might still be losing money because they have to spent it all on like pre-training clusters but if you just ignore that and create two different accounting mechanisms for the product and the research teams I think uh the product team is already profitable so that means like you know this can be already a bigger business than door Dash or Uber when grown at scale do do you feel today like by charging you're limiting the um longterm ability for it to get embedded in as many people's hands as possible and into their workflow and how do you think about that tension uh I don't think so my belief is that the free version of the product will keep getting better and uh the paid version of the product will still be better than the free version and um so I don't think it limits the adoption in day-to-day use today um if it's completely gated by PVE walls yes clearly it limits and your focus is more on like highend like Bloomberg terminal sort of thing um but that's not the the current model today so we can definitely get it in the hands of more people there's a concept of verticalization which would require uh which would have required you to focus on a subset of data and information rather than competing across a broad surface area like you are now why did you not pick the that path I was very confused to be very honest um I spoke to many people in silen Valley about this uh this was before even we raised series a funding everybody told me look you launched a really good product got a lot of Buzz getting usage perfect uh now go and figure out a vertical and raise money for that no one's going to fund you for the horizontal um but one person I really respected told me the opposite uh that was Mark andr he said two good plugs for inre and I'm gon have to edit this out sure no I really respect him he said everyone's gonna tell you to go vertical don't do that even if you are going to fail in the horizontal going vertical is guaranteed failure whereas going horizontal is not guaranteed failure it's like low odds of success but it's not guaranteed failure and I asked him okay how are you so sure and he said once Google was succeeding whole Venture Capital inter like internet businesses wanted to fund um vertical Googles and all those companies nobody even knows what they are today they have been successes on Vertical Search Engines but all of them changed into like a platform or like an end to end tool rather than just search and search is just like one portion of it that can even in fact be outsourced to some API right and uh that's that basically up what happening like for booking.com would you call it a travel search engine or would you call Pinterest a visual search engine they're not exactly that like they're doing a lot more than that which is what makes people come there like Pinterest allows you to pin that is a core value prop of course the visual search is making it even easier for you to find things to pin similarly Yelp allows you to look at reviews but it's not like a local search engine uh or like booking.com allows you to get hotels but it's not meant to be a search engine for hotels it allows you to book book stuff customer care all these additional things that make things work for them right so when you decide to go vertical you are going for that vertical you're building a product for that vertical you're not a search engine company anymore so only go vertical if you don't want to do search I'm like I want to do search then you better be horizontal like and also the other thing in AI at least so far leave alone this traditional internet sysem um the power and the magic of all these models is because they're so General like the base models or the the r tun chat models they've all been tuned to do a lot of things really well which is why it's working in such free form conversations you're able to ask follow-ups and like you're able to talk to it like you're talking to another human and it's able to understand whatever you're saying the moment you start fine-tuning it for a specific vertical by just throwing a new data set people think like oh yeah that's it I got a new model it's just vertical it knows everything about my domain but it just stops having that old magic it cannot Converse in a more General way it cannot understand a lot of the things anymore and you're like oh I would rather retain the original model and done more prompt engineering than like doing all this fine tuning so all this dark magic of fine tuning where you are adding a new knowledge to the model but still retaining the generality and magic of the original model is still less understood so given both of these things it's it's not it's not a good idea for you to go vertical moving forward I I assume having access to data that up to date and near real time will be increasingly important how do you think about getting access to that data will that be deals that you you cut with you know subset of data providers or yeah I'm sure I'm sure we're going to have to like you know have like licensing deals or API access uh we already access like you know Yelps data for example um you know we use shopify's apis so there are like so many amazing sources of data to power narrower experiences like yeah local searches or shopping uh I'm sure we have to do something similar for travel uh or like restaurant booking I'm sure like you know has to be integrated even more deeply with the Yelp or open table so we are going to have to use apis and Licensing for being able to build like you know more and consumer phasing applications I've heard you say that you actually think retrieval augmented generation uh for a consumer app like yours uh is very different than B2B yeah can you explain first what rag is for people that may not know and then can you talk through why you think the capabilities are different for the different Subs segments yeah absolutely so rag means retrieve and generate or retrieval augmented generation I don't know exactly what a stands for I think it's augmented augmented okay so why do you need to do that like okay so first of all just generation means you ask something and there is a whole neural network a Model A giant model with billions of parameters and then you just get the response from that model that's how chbt Works um and um retrieval augmented generation what it does is you take in you whatever query you ask the system it doesn't directly get the model to Output the completion instead it goes pulls some documents that are relevant to your query populates The Prompt with it and ask the model to look at both the original query and the pulled up relevant documents and then give you the completion so this way it can allow the model to get access access to knowledge that's relevant to your query on demand whenever you ask a question and doesn't have to be baked into the model's weights itself so that is the best way for you to break the real time knowledge C off right that chaty has and um which documents you pull up or which apis you give it access to for information which tools it uses all these are like depending on your application it could be the web links in the web it could be your directory on your computer it could be all the other files in your Enterprise and depending on that the end application changes now I said that the rack for the web is a different kind of Technology than the rag internally because the ranking algorithms are very different the ranking signals that you need to use for webbased rag for building a product like perplexity are more around like is this domain High Authority or is this the main low Authority like this been more spam is this recent sites is this site been updated in the last few hours uh our new site should be updated even more frequently um and then like uh New York Times or Wall Street Journal may be even more authoritative than like um some some lower quality magazine right so all these are signals you use uh to to to rank the final top K on the other hand for the internal documents how do you decide which Google Docs are important like is it the document written by the CEO or is it written by an engineer uh even the CEO can write crap docs at times maybe know not all of them write good docs and an engineer might have produced an amazing doc so what other statistics like how many people have access to it or like how many edits have been done like these are all like completely different uh signals compared to like what you use on the web and also like how you chunk you know the the how you chunk the document into different uh paragraphs uh people write good web pages because it's going out there in the public but internally people don't necessarily write good docs um so then you might have low quality information so there is it's a completely different problem if it was the same problem search on Google Drive would work amazingly well there's a reason it doesn't work well there's this suggested X question feature that's become useful for me in using perplexity and investigating things um how did this feature come to be and why do you think people are bad at coming up with next questions and discovering things yeah um I I wouldn't say people are bad but I would say people are it is a difficult skill to articulate a good question I mean there's a reason you prepared a bunch of questions for our conversation or if we meet I would prepare a question a bunch of questions to ask you uh asking good questions is not easy there's some amount of human cognitive labor that goes into it right and when you're using a product you you you do want to be in the lazy mode you just want to like use it in the easiest possible way and if there's a good amount of friction for you to ask a good question then you're not going to be able to do it so the suggested follow-ups uh is one way to minimize that maybe your first question was not good you got some answer and now you kind of know okay I'm I'm not interested in that anymore but I might want to ask something else and I'm going to suggest you what to ask that could give you ideas on how to ask your next question or you could just click on the question I suggested to you already what is true is all of us are curious but not all of us can convert that Curiosity into an articulated question if the AI can do that work for us it makes a product even more fun and easy to use um and we believe it's even this follow-ups is just the small part the real Alpha liing even the starting question when there's a reason Google auto suggests you as you type right they don't even want you to type that's the real Like Larry Page style product design like user is never wrong don't blame them they're going to be lazy you you let the product do all the magic for them uh now what is the equivalent of that for asking your first question it's not clear we're trying some experiments like if you look at our Discover page there's like a bunch of interesting questions very topical ones like yeah exactly imagine this is personalized to you and like every day you get like a feed of questions that where is yeah and whereas K ilas all these things are all these famous people interesting interesting stuff or even like stuff about the world that you don't understand that doesn't have anything to do with realtime knowledge I think like there should be a reason to open the app even if you don't have a query that's what you want to get towards it's interesting it's sort of like the difference between radio and Spotify it's like picking a song or thinking what you have to listen to versus leaning back and just pressing play and letting someone else do it for you true true autopilot right like yeah actually that is true one of the worst things in Sp Spotify app for example is like deciding what to play the cold star problem because yeah cold yeah you you're in a car and like you're like play some music man I feel sleepy and then you don't even know what to play and you just okay sure let's play Taylor Swift or what right the DJ actually within Spotify is I stopped using it but uh they they're trying stuff around being good DJ is not easy yeah in introducing that feature have you found I assume time of browsing using the app has gone up has that been a material Delta that the asking the next question people's time we we yeah four minutes was the first average session time for us and once we introduced these suggested follow-ups it went up to eight minutes wow so doubled the time by asking the best decisions we made and like I mean I'm I'm happy sharing this because now everybody has copied already like Chach has it and like I think Google has it now and Google is trying to do this even for the regular search not even the SG so it's a clear winner this is one of the best features we rolled out that's awesome what are the important success metrics that you you think of obviously you're generating Revenue through the subscriptions themselves but I I assume number of daily queries is a NSTAR metric yeah got it every company should have a nordstar metric you for social media companies it's like number of dows uh some people even joke you should measure howly active users uh but for us it's been the number of daily queries which is actually what Google picked to uh because that's the only metric that's correlated with usage and your product only gets better with more usage so the only way to truly improve your company is make it a better product and and the only way to tr make it a better product is get more people to use it every day actually use it so measure unit of usage right uh and um we could have measured like number of dows or number of vows or like the the number of pages in our index but these don't indicate anything you can have like a 10 billion page index and uh it could be useless or you can have like um a million daily active users and they all just come and just scroll through the feed and go back and that's not very useful either I've heard you mention five Dimensions that you need to focus on uh for the business uh accuracy reliability latency ux and iteratively improving how do you think about the different importance of of those and tying back to your Northstar metric of queries yeah I mean like Northstar metric of queries will is the only way to make sure the accuracy can improve the latency uh will help the number of daily queries go up if you improve that because uh more people more more if the the the friction to like getting an answer is so little people use the product more so it's the other direction and readability again the more readable the answer are the more likely people use it more and you get better queries you get more data on like which queries are unreadable today that you can improve on and then uh UI has like you know it's pretty aligned with like readability iteration speed and personalization um I would say that like that's more not related to the NSTAR but um people want to use something that's constantly improving right like why you're you're a noname brand like why do they need to trust you with their time like a 8 seconds or like eight whatever 8 minutes 8 seconds doesn't matter um it's still a valuable part of people's lives it's very hard to like make people give you their time they would even spend it on like scrolling through X but they might not use a better product because you're not worth their attention today so you have to be improving and convince them that you're worth their time and the best way to do that is personalize the product to them and also keep shipping a lot of improvements they feel like it's very valuable can you maybe talk through the staging on the model side and earning the right to build your own model over time I think the majority of people saying a rapper on top of GPD would have been the pejorative term in the early days but um it's still said by the way yeah I even I even wrote a tweets as rapper and bio it's like you know uh which is a play on uh yeah all the other in BIOS that are showing up on X these days yeah exactly so look I mean what what else should I do like should I start a company and uh uh raise like a billion dollars hire like top resources from Deep Mind and then uh join Microsoft after that so like you know it's pretty hard to do this right I'm not we never said we are the best foundation model players so are you saying like every new startup should always go and build a cluster and train their models then and then end up with the same fate as the others uh except for like the top three or four who are doing well like opening anthropic Mistral in fact that's it you can the other two are big Tech meta and Google right uh it's pretty hard to do this it's pretty hard to do even a good product like it's pretty hard to be a good rapper it's so when you have like two hard things why choose to do the even harder thing of doing two hard things at once just focus on doing one thing well and rely on the ecosystem right like I'm sure like Zach's going to put out some really good models later this year um I'm sure like maybe if not happens this year beginning of next year we'll have a model that's open source and as good as gp4 uh and like gbd4 is already like Optimal more or less on like 80 eight you know eight out of 10 queries are always accurate or like nine out of 10 so your product is already more or less soled with the existing capabilities and it's been it's guaranteed that there'll be an equivalent open source version very soon why are you so worried um like why do you want to go raise that capital and build all these models yourself okay we do have the ability to serve them efficiently like we have really good inference Engineers we showed to the world that we can move and like do all this analysis on different gpus and maximize throughput we're not even an inference provider but we have the best inference infrastructure or at least comparative with the others like together or Gro or these other people so and and we are training our models too like we WE Post train these existing models based on all the user data we have and we have contractors we collect data if you're truly a rapper like why am I spending all this money is it just for Optics like obviously not I have an incentive not to because you know I I need to save as much money as possible to build this for the long term right so I think like people need to like just I mean there are very few people who say this like Twitter people are always like that but most mostly people need to understand that your goal is to build a business and how you generate value for the business is what matters if people are paying me 20 bucks a month for the service the product provides them and they don't care like which model is providing the service that's what matters to me do you think we're at a an equilibrium you mentioned the five different names on the model side out there open AI anthropic mrol and then the the two big cloud providers do you think we're at a steady state equilibrium of of the different model providers or you think there'll still be a new entrance maybe X like the xai yep uh but their models clearly like further behind they release an open source version um the usually the joke is um you only release open source when you're behind except for like in Met's case where they're like you know pretty committed to open source whether they're the leaders or the followers I do think it's kind of like the steady state today we have is there only going to be three or four people that is very clear uh but the Dynamics there are not clear like is it going to be like open AI always in the lead or is this going to be like a ping pong between open Ai and anthropic uh where does Google come in here is going to be a three player game or like two-player and how does meta launching something open source like hurt open ai's business or anthropics business all these things are remaining to be seen where are you in the Journey of of building your own models so we are we are we have a version uh of our model on our product called experimental model uh that's been trained to be more concise and more neutral like less refusing to answer questions so that's been post Trin with uh the model that MW released and it's been fine tuned with a lot of data that we have collected and a lot of human anotations that we've collected so that is the current state at which we are in uh we'll experiment training with the base model that X has also put out which is a bigger model but it remains remains to be seen if it's going to add value to us over the mistr ones and um we also taking a lot of these smaller open source models like mrra 7B or Gemma um and um using them for other parts of the product they like llms that take a query and understand if it's this type of query that type of query uh should it render an image should it render a video should it render a knowledge card um and like route it to like appropriate classifiers uh reformulate queries and expand them so these sort of like other aspects of the product are already running with our models with the core summarization model cost of new gpus is going up the Next Generation chip is always going to be more expensive uh it's offering unprecedented throughput H while also preserving the latency that lets you build uh like sort of the whole product with just like one node or two nodes of gpus uh and that's what is amazing like like like basically you're packing more compute power in smaller surface area smaller smaller volume now so you can buy like fewer gpus but more powerful gpus and serve an entire product like millions of people and that's why I believe like all this is going to be pretty cost effective over time even though the price per GPU will continue to go up you're going to need fewer of them which is great can you pay me a picture of ai's upside in the coming decade I think that whatever all companies today are spending on so many things be it like Outsourcing engineering work or like um having like these part-time contractors help them out on things or um paying for somebody to do background research on something you know all these people ping me oh we want to we have a client who wants to do research on Google we have a client who wants to do and they they they pay me like so many dollars per hour imagine how much they actually getting paid to do that work all this work can be done 80 to 90% as effectly by an AI that you don't need have all these middlemen anymore so everyone will feel so empowered that you can get so much done with so little that your cost would be much lower and value get be much higher and you'll be able to create your own value in the world through the outputs you create so I believe whatever big companies spend on all this B like due diligence background work um engineering like like like part-time engineering hire design like at least even 10% of that if it goes to like AI spend AI can like like like for of all all AI businesses will be so much more valuable and every human individual spend in all these companies will also get reduced that's the best way to that's the best deflationary effect right I saw some tweet I don't remember who it's attributed to like imagine you had a whole country of like a billion people ready to work for you uh and and they do the work reliably too or at least like 80% 90% as well that you only need to do the 10% that is the sort of power that having access to a giant data center will give you there's the society we live in today if you were to compare it to 50 years ago or 30 years ago um it's amazing we can do things even the poorest people in the United States can do things that a king or a president or whatever couldn't have done exactly 30 50 years ago yeah I mean you you are already smarter more empowered than like say the president of us like even 20 years ago right with access information and food and Technology exactly we're all using the same phones as as Elon Musk like if we can right if you want to um and and like that's sort of power that everyone's going to get like at least for perplexity that was one of my goals is um not everyone has access to the you know the found the best Founders or like the best VCS in the world and they still might want to create a lot of value in the world uh so who are they going to ask their questions to right like maybe the answers are already out there in the web the web is a treasur throw but you need the GPS it's much easier having a tool like ours to go find the right knowledge at the right time right knowledge on demand personalized to you is a trillion dollar opportunity do you think that inherently this path of of access leads to increased happiness or what's the most utopian happiness thing that you can concretely think about that seems tangible that might happen in the in the coming decade look I say this more like a marketing thing of search like a billionaire sort of you know a play on the shop like a billionaire of the team who ran on um Super Bowl but I I I believe in it actually like what do billionaires care about uh they care about their time they don't care about I mean they don't care about money but the only way for them to make more money is having more time and that's their the resource that is is is not in abundance for them so if you give people back their time life is a luxury like like like and uh on the other hand what do people who are poor optimized far as money over time they give their time for more and more jobs part-time jobs in order to make little more money now imagine if the the people who are optimizing for the money over time flip they optimize for time over money that's what a utopian world would look like where even the people who are not making much money can do whatever they love for a short period of time every week they don't have to work 80 hours or 90 hours not everyone has to work that hard like work should feel fun right and if if it can be done with the help of so many co-pilots that are working with you uh and you don't pay much for it or even if what you invest in it is sort of like investing in buying a car or like you know you buy a piece of land and do farming sort of thing then you're going to be able to create a lot of economic value where do you think the next big wins come from in artificial intelligence do you think it's a function of Simply more compute or do we need some major technological breakthrough it it has to be a function of compute uh that doesn't mean we'll not need a technological breakthrough breakthroughs have always been about better ways to utilize compute Transformer was a breakthrough because it made better utilization of gpus Matrix multiplies compared to rnns or convolutional Nets and by making better utilization of compute we we were able to create amazing models so the next amazing Transformer level breakthrough will be of a similar nature where it's able to utilize the existing Hardware even better even more efficiently and the efficiency gains translated scale that like whatever we create with the same amount of compute today will look a lot better uh and and that means also putting more Compu at it and creating something that doesn't even exist today and I believe it's likely to come from ideas that can make models think for themselves uh come up experiment on the world uh draw conclusions from the experiments go back and design a new set of experiment and iterate and and and keep getting there till you arrive at a truth right that's that's very hard that's very hard to do today and that's where I think like inference and training have to be together like right now you train one model for like many months and then you you post train it you RF it and you give it to the user as an API I think this needs to change the model should be interacting with the world during the training process or you know some kind of like generating their own data and training on it and that should be done at inference time $10 billion do you think do you think that's a a number that proves to be dozens hundreds do you think most of the value actually gets accumulated by incumbents that are already in these spaces leveraging a lot of the foundational companies that already are out there I hope it's not just incumbents that benefit from this and I actually hope uh look I'm not saying like you know incumbents won't benefit more from this than the startups maybe they do maybe it doesn't even matter uh like a hundred billion dollar increase in market cap for Google um is kind of cool it happens like you know a week of fluctuation in the stock market but um a hundred billion valuation for a startup is insane it's such a massive win for the founders the employees the early employees the investors who invest in the company huge win right so we are talking of things where something is insignificant for the big Tech but massively significant for uh for a startup and which is why I'm curious about the number of over 10 billion not the I assume the value billion is irrelevant for for the big companies but how many of the startups do you think could be worth over $10 billion do you think it's I mean is it a dozen or do you think it's a hundred or do you think yeah a 100 startups with 1010 billion do can be possible in AI yeah I I don't know when or like how but it doesn't seem impossible possible to me how much time do you spend thinking about AI safety not so much because I for for good or bad I'm not working with the frontier models myself I'm not building these models so maybe the ones who are like truly at the edge are concerned and their concerns are valid from what I can see it seems far-fetched um so if there is a real evidence like if you're given if you're all given access to something that seems too too intelligent to be like you know uh giving us like these sort of existential risk then yeah I'll also be concern yeah when do you think AI should be regulated uh not anytime soon and also if it is regulated I'm not going to trust the people who are asking for it to be regulated Larry Page has been a big inspiration of yours yeah why do you take such inspiration from him specifically and what are some of the things that you've internalized that um uh that that are maybe I don't know Larry pag isms I I take a lot of inspiration from him specifically because he uh comes from a similar background he's an academic I'm an academic too you know and um when the the the usual founder examples are the jobs in the gates and the Zuckerberg's and they're all like undergraduate dropouts who and like I I I already completed undergrad and I was not even thinking about entrepreneurship all then so it's important to find some example right like you can you can look to there were phds of star companies to but it's all Enterprise stuff usually all the consumer things come from uh undergrad dropouts hack projects so Larry was the only one who converted like an actual research idea into a company uh and I I felt like the only way I can do a company is have a product that's deeply grounded in like Ai and like research where like better AI benefits the product like he's the only like search is at least the only example I've seen like search and generic chat are the only examples I've seen where like better AI is the only way to make your product better and you want to make your company AI complete like the mission can only be fulfilled when AI is truly solved until then there's always going to be ways to improve the way an answer is render to the user and I found that very powerful that that Insight that he had and the other thing I kind of have uh imbibed and and the company has sort of adopted is the user is never wrong philosophy uh this was there when like he mentioned his anecdote where he was ready to sell the company to exite Google the early Google search engine and exite CEO was looking at his demo where he ran the same queries on exite search engine and Google and Google gave better links and exite uh the exite CEO got mad at him like oh you just you know uh manipulated the demo like if you typed in the right query we would have also gotten the right results it's your fault that our results not not ours and he went back and said okay what did I do I'm just a user I typed in like you know just like a regular user would I'm not like trying to like cheat you uh that means you not you're not understanding the user and so the user is never wrong like if they come and type in a wrong like my mom sometimes is like this doesn't work my first reaction like yeah like why didn't you write the prom better but then the real truth is like should your product should figure that out like this is where we really differ from Chad GPT also Chad GPT popularized this whole thing of prompt engineering and everyone trying to learn and creating better prompts and sharing things that's kind of interesting but is that really the uh ultimate vision of a winning product I don't think so it's sort of like saying like like you know learn how to use Winamp it's cool but it's not going to be the winning product at the end of the day um all Microsoft products are designed that way like add a lot of buttons Like Larry P's Philosophy is very different like you make the product so simple so intuitive like it should be magical that it should already know like what you want and like you shouldn't have to think remove as much of thought process on behalf of the user Jeff basos said something interesting in an interview recently that um we as humans actually aren't designed to be truth seeking so we spend a lot of time focused on the framing and positioning of things do you share this View and if so how does that inform perplexity I share the view um I've at least like along with our Founders we've created a set of values and the at the company for the culture um and um like Ben horvitz has this quote right culture is not a set of things that you write it's actually like set of things you do um we wanted what we do as to be a reflection of our product itself uh like like like you know there's this thing of like you know stuff that reminds you every day like what the culture is like like Amazon has this frugality as a culture and so they make desks with the wood from the broken chairs and things like that doors uh to remind people that we are still like Meant To Be Frugal and for perplexity what we felt is our product should be fast accurate and readable these are the three Evergreen product values that will always matter it's not like next year Logan you're going to come and tell me that uh AR I want your product to be slower or I want your product to be like less accurate or I want your answers to be like garbage like really long paragraphs you're always going to want some improvement and if we adopt that in our culture by being for accuracy we're going to be truth seeking for Speed we are going to be um fast like we High like you know fastpac which is what all startups are meant to be but even more emphasis on that and for readability we're going to be like concise in our Communications internally too we're not going to like uh waste time in meetings we're going to keep slack messages short we're not going to write big docks if we adopt that in our own actions uh that means that's the best way of caring for the user like truly understanding what the user wants to Mark Zuckerberg said uh something that's maybe a slight derivative of that Bezos quote uh which is something effective you can only say meaningful things when what you say and the opposite of what you say are things that people believe can you elaborate on why you think that's an interesting yeah quote Yeah like say for example take the move fast and break things like it's interesting because you could also adopt the opposite like move slow and don't break things are already working man like I don't want to break like I don't want the stress from constantly dealing with like production things breaking or people complaining at quer is getting wrong or move fast and break things could have negative implications around data and privacy and things like that yeah exactly uh and I think what Zak is saying is pretty interesting like just even from an information Theory like like leave Le's opinions and stuff mathematically it's interesting uh when you say something that everyone considers the truth there's no information add it's a platitude there are no exactly there are no bits that you you you ingested new right it's like an AI that's already memorized the text and it's like looking at the same data again it's not going to learn anything there's no gradient and um on the other hand like when you're saying something very like completely different that it's not blatantly wrong you cannot immediately verify and say it's wrong um you're forced to think huh that's interesting I I never thought about that there there may be some element of Truth to that it may not be 100% accurate but it's interesting it's a Viewpoint I haven't considered and uh that's if if you want to be truth seeking you do want to hear such things because your worldview may not be 100% accurate yet nobody says this right so you always want to keep hearing some things that puzzle you perplex you and then update your understanding of the work I heard you say that you actually avoid meetings as much as possible and instead um we'll make arguments over slack and the team will as well is that something you've kept up with yeah I mean like i' I've I've tried to argue Less on slack because it's not a good look when like you know the CEO is constantly arguing in public slack channels people like I have realized that um sometimes only if you take you understand like how others view you like you have you learn from that so earlier when we were just like four or five people like we were all like just people working together so nobody felt like you know one person's opinion mattered the most but as company scales and grows like it's hard to preserve that and so when I say something like people take it with a lot more seriousness it's a lot more definitive right and oh like and it might have been wrong andbody because they said it they are like forced to think and I don't want that culture so this is apparently something I this is actually something I I listen to on the Lex same Lex Bezos partc and and I adopted what like bezel said where he said like in in any of these meetings where they're making a decision he speaks uh at the end uh he doesn't speak at the beginning and there's a reason for that is like if he had said something in the beginning uh it takes even more courage for somebody to like oppose him and defer with him um where on the other hand if he speaks at the end like he has even more data now to say the right thing and he's also allowed everybody else to say what they actually wanted to say and um that way I don't argue as much on slack anymore in fact slack arguments are like unproductive I don't want to like you know promote that through my podcast or something focusing on making incremental improvements is a value that uh you guys have as a as a company but also as a a as an entrepreneur yeah um how do you think about continuing to better yourself outside of not making arguments on slack uh like making in criminal progress as a CE as an individual yeah as a CEO leader I think like uh the sooner you realize that your job is always changing the better like what you did even three months ago is not very useful anymore a to constantly upgrade and learn new skills um so it's sort of seeing your yourself in a I mean I I I have like I like AI a lot because it's it's a good way to for for humans to to like live their life mhm like what should gbd4 do what should gb5 do to become gbt 6 it should just like like work on things it's not good at yet right and like find out and evaluate and assess and so the truth seeking this is very important um I'm still not good at like many things that you would expect CEOs to be good at uh sometimes there are some meetings where I'm supposed to explain the product somebody uh another executive another company that we're trying to do a deal with and I go on this um you know five minute explanation instead of giving a crisp one or two minute explanation so there are like so many places I always aim for improvements you spent time in open Ai and Google's Deep Mind two of the most prominent uh yeah businesses in artificial intelligence yeah deep mind's not a business though a a business unit yeah it once was a business yeah kind kind of but it was it's a it's an own subsid now it is what did you take away from the cultures of each of those organizations respectively what were the differences what are the commonalities I think the main difference is the speed of iteration uh Deep Mind likes to like really think hard about a problem and and and try to arrive at very beautiful elegant Solutions that would you know Amaze you and um open AI is mostly about like let's go like fig solve this like you know the first step in solving it is actually try trying a v zero a solution and then we iterate and we get there we don't and there are benefits to both approaches uh you're not going to be able to come up with like the transformer and the open AI style like you're not going to be able to like take an RNN and keep improving it and get to the Transformer you do need like this 10 X or 100x ideas that you might just be able to come up um you know mathematically think through and analyze at the same time you're not going to be able to just take the Transformer and convert it to GPT 3.5 by just thinking on a whiteboard it's that's just something you just learned through iterations that open did that's why like these two ORS are so different from each other uh commonalities are both really push hard for Quality like they have very high expectations of themselves they don't release anything that's half baked or like low quality so whenever they come out it's people always are like you know oh wow this is cool most of the times right anytime there's like massive PR it's usually coming with something that's very very high quality very impressive and I think that push for quality is both organizations share that and um really wanting to push the boundaries both organizations shed philosop I feel like you're uniquely passionate about search and in this problem this answer engine problem it was something that you took maybe a um circuitous route to building very circuitous route business around but can can you maybe talk about that journey and uh the importance as an entrepreneur of being passionate about the problem you're solving like uniquely passionate yeah I mean I I was always a very big Google user right like I wouldn't type queries like most people I can find things much faster it's not a big thing to be proud about like I know like so many people in the world exist like this today but um that that mentality to go and Tinker around and do things um like there's all the side C and tricks and like uh all the other prefixes I would use I knew a lot of like like ways to get around and um then I would realize like other people looking at using Google and i' see like how they not using it well same thing with like Facebook's Graph Search I really liked using it uh or like even searches um so in general like like learning to use tools and like being very good at it was always appealing to me and what what was that you think just desire for knowledge and learning I like Fast knowledge like quick get to the speed like you asked me to find something on Google I'm actually very good at it um may not may not be the skill that you should optimize for but instead you would want to train an AI for it but it's very easy and um and I always like these new search experiences that Google built like Google's scholar Google Images all these things are like very nice and I you know the books like in thex really influen me of how you can create a company like if you can create a company where like the really smart people come into work every day it it's a it's it feels like you know truly a proud moment right um cuz they can choose to do their work anywhere else so why why are they coming and working here uh you need to create the right incentive structure and the mission and like road map for them and if you can succeed at that that feels like a great achievement even if that lasts only for a few years I don't know like I'm I'm still very happy right um and and Google did this at another level like all the smartest researchers and Engineers work there um but they've been too far extreme in the sense that um they paid people so much to just you know retire and not truly realize the potential of their intelligence so there needs to be more companies that create units of people that are very smart and really trying to push the boundaries because that's what is good for the world and good for every individual too you need to feel fulfilled and I hope perplexity is one such company and so you started out with this as a personal curiosity and then you decided you wanted to be an entrepreneur maybe based on that mission did you want to get a great nucleus of intelligent people excited to come to work work every day and then went looking for a problem yeah exactly like you know I wanted to do something that had the attributes of Google which is a group of smart people working on hard problems and getting the product out of the hands of users and their usage continually improves the product like these were the three attributes I felt and uh little did I realize that it would end up being an answer like a search product itself but that's sort of what I feel it's very hard to build a company with these attributes unless you work on search like for Facebook you don't need the product constantly improving with the usage you just need to launch like poke buttons or like all these other engagement maximizing ideas same thing with like you know Tik Tok is obviously a good example of this user data improving it but it doesn't have the other attributes for me of like really smart people wanting to work on it yeah it's very hard to pick that like I I just don't know like you know maybe if there's some other idea like I would love to try that out as a company too but uh it somehow ends up always converging to like working on a very hard problem uh that has an actual product in the hands of users and having it like continually improve the product in the Journey of perplexity what was the moment where you were like we might really be on just something here we went on like you know the we launched on December 7 2022 and we thought this is just going to get us some Enterprise customers fine but the usage just kept going up through the vacation like through the Christmas vacation and I was like first of all you're a noname company no name startup uh only three people are working here four people and why do users still care and use it and why is the usage actually going up and that too in a time where people are chilling and watching Netflix that means you have something so that's when we thought okay let's update the product a little more make it conversational suggest follow-ups and see what happens and then that increase the usage even more at some point we reach like hundreds of thousands of queries a day and I was like okay this is not normal like even if the retention is low and like people are just checking out for the first time and leaving there sustain use it still from other people so let's go and raise some menture Capital money and continue the experiment further and that continued experiment kept growing and growing and like we like okay this is it this is the company I heard you say that the best ideas are those that you say out loud but people think it's dumb why do you think that and how does that tie into perplexity why do I think that it's uh I think that because um there needs to be some contrarian nature to the idea uh if everybody like like going and building an AI chatbot for uh doctors might not be the most contrarian idea it might still be hard to execute on it because of Regulation and like connections to the field and stuff like that but it's not a very contrarian idea like you can can imagine many people accepting it's a good Venture Capital idea but trying to go build an AI answer engine that will compete with Google on like you know day-to-day user habits is one of the worst ideas you can p like nobody people might even put $100,000 into like you know a failed startup idea if it but not into this so that's why it's a it's one of those ideas where you go and say I'm going to compete with Google people like oh yeah cool you know good luck man like after three years they're gonna shut down the company um and uh same thing like the first idea we had was the glasses like watching through a glass and asking questions about everything you see and the more like through VR or ar just just regular camera on your on your glass like I'm weing this glass I'm seeing you and I can ask questions right there's no sure you can you can use AR and like embed the results in front but you can just literally even speak back to you and that's fine and this is a bad idea because like the technology for that wasn't there in 2021 or two um like like models that are like as fast as the 7B Lama today don't exist back then but it's changing maybe it's going to work in two three years so you always want to be somewhat well positioned to take advantage of the moments and ship the product when the opportunity arrives and that's why like you want to set up a company for that there's an investor who said something to the effect of it won't matter if you lose competing with Google so you decided to go all in on it at that moment can you can you talk about that and how that um yeah informed you yeah yeah there not Freedman he said look you know we had a Discord server and like few users and I was like should I launch this and know I I don't I don't have the confidence you know a l of people ridicule me and like and he was like are you serious like you know like you think you're that important like it's not it's not nobody cares what you're doing sure you have some funding but still nobody cares and even if this is a failure at least people know who you are what your company is and U the fact that you can ship something all this will be useful for you to be able to like build a business later but if you don't launch anything thinking you're so important and like trying to get it right in V zero then you're not going to get anywhere so it's like one of those things where the loss does has some benefits to and the win has massive benefits so you basically shouldn't even be thinking you should just go launch like why didn't you launch already yet why why wasn't this yesterday so that was the conclusion and I was like okay that's that's a great way to think about it sometimes it just takes you like as a Founder you're like so confused clut does constant information overload at you and everyone's talking to you about many things that you don't even have time to think clearly so when you get people who can help you think clearly almost like a prompt engineer in your mind that's very useful you referenced Nat uh but I think you you cold emailed maybe him and El originally correct yeah elot has also given me similar advice like I think during the first two months I asked him like him I want to be in stealth and he was like why and he I said because I don't want people to know what I'm working on and he's like do you think it'll matter do you think like first of all nobody cares about copying and proven startup everyone takes themselves very seriously in the beginning yeah but nobody cares even now if you know this idea if let's say I have a lot more funding today a lot more employees if a Founder from seed stage funding tells me an idea they're working on I have like thousand other problems to worry about than copying them yeah uh so that's that's a mistake I made and elot was right I wish I listen to him M yeah I C messag both elot and Nat um elot on link an ad on Twitter and uh both responded to me and uh gave me like you know committed to one1 or2 million and what do you think got their attention in that in that cold cold not I obviously the fact that I'm from openi and deep mind and like you know they don't have time to like actually evaluate an idea yeah like they get like probably thousand I'm sure that's why yeah so so you think the qualifications cut through the noise definitely qualifications help you yeah yeah Reed Hoffman concept of a pros and cons list yeah and the issues around and then he takes only the first thing so I actually don't know the the can you talk through what that is I've heard you reference it but I'm interested in your perspective on it yeah so I think there's some interviews somebody asked him how do you make decisions and he just says like the typical way of making decisions is write the pros and cons and then try to arrive at like whether it has more pros and cons or not but that's like the dumbest way of making a decision because uh you value you give uh equal weight to every Pro and con but it might not even be like as important and so what you should do is write down the most important things strike out everything else from the list except the first and make a decision on that basis alone and that's a great way to convert the decision to a binary decision in general I feel like the human brain is not as as uh as sophisticated as an AI classifier an AI can make a decision on like millions of dimensions um but a human brain is incapable of even making a decision more than two like like a single Dimension even two Dimensions where there are four choices is pretty hard that's why in an multiple choice question exam if you're unsure of the answer what do you do you rule out options first and then you convert it to an X versus y problem that's the best we are good at like at least where like there's a 50% chance of getting something right so try to you know reparameterize all hard problems in your life to buy inary decisions that's what I took away from the read off do do you do that internally with uh decisions and paths taken definitely I I do it almost subconsciously these days that I don't think about it in analyzing the opportunity for perplexity as we've sort of pulled back on all these different things in the early days did you recognize the business model challenges of Google and this is as you've executed you sort of figured this stuff out is there a lesson in that for people lesson is if you're not a genius the only only only chance you have to succeed is iterate give yourself more shots of success I'm not a genius uh you can always connect the dots in hindsight like the great Steve Jobs said even Steve Jobs did not come up with the i the the idea for i iPhone right away the way it happened was iPad was separately in the works and they invented multitouch and um he was separately trying to build a phone and then he said okay why what if he put can you try putting the multi touch on the phone and then all the dots came together it was not like I got the iPod done and I'm going to the way he presented the end of like an iPod a great Comm like Communications device and a phone all in one like it's not how it starts it's how it's presented at the end and um iteration and like Lux surface area is what you should bet on the moments of Genius is pretty hard is there a branding element um that you think about outside of the actual technological answer engine that you're building some of the people that have successfully competed with Google and different vectors as independent companies ducko Brave have competed on orthogonal yeah paths around privacy and data and things that Google can't compete on right is there a branding element that you think about outside of the core product um I mean I view the product as a Swiss army knife for knowledge I think that's the brand we want to go for it should give you the 8020 on anything uh you should feel like you have so much power because someone's always being working for you and doing all the research for you and getting back stuff and um doing your research is is a thing that people just use colloquially but every time you use the word like a research party you think of a financial analyst you think of a mckeny consultant but we all do research in our everyday life like including like you know what shoes you should buy or like where you should go vacation or what coffee you should drink or like you know what drink you should try in a new bar that you're going to there's always so many decisions that are being made in your day-to-day life so I I want us to be seen as the Swiss army knife to help your mind I don't want to adopt this branding of like Google's terrible like like like it's it's a illegal company it's it's it's an evil company maybe like there are some truth to that which is why Dr go and brave worked to some extent but I don't want to go for that because that's constraining yourself to a very small Market I want every Google user to be a perplexity user without having to let go of Google that's a bigger market for me and that only comes when you're cre creating new value not like trying to remove some bad element in one thing and offer the same thing what does the future for perplexity hold in the nearish term for people that are listening and users or yeah I mean like I want to pass a Larry Page toothbrush test like you know which is what uh a great product a product is only worth launching and executing on if it can if it has a path to uh being used at least twice a day like a toothbrush toothbrush is a great product we all use it every day should it has 100% retention uh so I want our product to get there let's say I I just need you to submit two queries a day I'm I'm happy what's holding back people from using it twice a day first of all a lot of people are not aware those who are aware uh they try it once maybe some query doesn't work as expected and they leave or they're not able to find the immediate difference from chbt which they are more you know familiar with and these are things that'll be solved over time as we iterate on the product and make continue to add like new value and and and and like you know improve on the three dimensions and then there's the bigger enemy is the muscle memory and habits like habits too change by the way people are like oh habits are so hard to change you dare not try to change them but every successful product has changed Habits Like Yahoo was still the search engine I remember was being used in India even after Google ipoed and Yahoo had more traffic than Google even after Google ipoed so people like now it's very hard to find a Yahoo user so people take time to change but blackberries sold more and more every year for another period of four years despite the iPhone phone being launched 2008 so it takes time and we're committed to the longterm right this is not a short-term company like if it's meant for shortterm me it's not meant to be a company it's just a project and it'll die very fast it's a company when it's a project is like at least multi multi- year or even a decade anything else that you wanted to touch on no I'm good thanks for doing this thank [Music] you [Applause] [Music] he

Channel: The Logan Bartlett Show

Views: 22,336

Keywords:

Id: 57LqvutrOI8

Length: 98min 36sec (5916 seconds)

Published: Fri Mar 29 2024