Anthropic Co-founder on Claude 3, Constitutional AI, and AGI | Ask More of AI with Clara Shih

I think that we'll have ai systems that can really help us to do scientific research to address poverty inequality um but at the same time I think that we'll face a lot of uh a lot of challenges welcome back to ask more of AI the podcast at the intersection of AI and business I'm Clara Shai CEO of Salesforce Ai and I'm so excited to be here this week at Trailblazer DX with our amazing developers and admins I'm about to have a conversation with Jared Kaplan co-founder and chief scientist of anthropic fun fact Jared and I went to high school together in Illinois and also College together at Stanford tell us more about the release of Claude 3 that is all over the news what drove this launch how long had you been working on these new models and what what sets these new models apart yeah so we've been working on them pretty intensively uh really since Claude 2 um it's it's sort of the natural outgrowth we think that uh these systems are going to be able to be made sort of more honest more reliable we're going to be able to decrease hallucinations make them more useful um so it's really been an ongoing process and uh I guess what's most exciting is that uh we've been able to bring kind of multimodal performance and Tool use to Claude so that there are so many different ways that you can uh make Claude productive for for for business let's talk more about trust um anthropic is an AI safety and research company and again you mentioned constitutional AI for for those who haven't heard of that before what exactly is constitutional AI how does it work and how why should people trust it yeah so it's a good question and I mean all of these all of these research ideas are Works in progress um but constitutional AI accomplishes a few things it allows us to transparently articulate a set of principles guiding uh claude's behavior and then use AI itself to train our systems Claude to abide by those principles so the idea is that we instead of necessarily having uh uh lots and lots of human labelers labeling data in order to train our AI systems we can have Claude uh effectively supervise itself to abide by uh this list of principles and so it allow for transparency it allows us to iterate really really fast because instead of having to go out and find or or or ask ask people to label data we can use our own systems overnight to uh to try out new constitutions to update Claude and and and really sort of do research at the pace that AI is developing isn't there kind of an Inception problem though because you have to trust that initial supervisor of itself absolutely no that this is this is a this is a really great question and this is why we also need people to interact with and evaluate Claude in order to ensure that it's safe so part of the training process of course is constitutional AI but we test these systems very thoroughly with red teamers and all kinds of different categories to uh uh ensure that we're actually hitting the benchmarks that we want can you share examples of of anthropic success with Enterprise customers yes so I I I mean I can just give a few um we work very closely with uh gitlab on uh on coding um we we we've worked with uh fizer using Claude models for uh for all sorts of uh medical drug discovery type type type use cases um we work with Lexus Nexus on uh on legal AI so it's really exciting how many different domains how General in other words Claude is and so it's it's it's really fun sort of seeing how all of the different companies uh are able to deploy Claude and are these companies deploying Claude as is or are they fine-tuning a version so um there are a couple of different layers that you can use to customize Claude um I think the first layer and the first line of defense is really prompting and providing Claude in context with the the data that's relevant to your use case and I think that's that allows a a great level of flexibility it's really easy to get started um and I think it's it's surprisingly powerful now a lot of customers come to us and they they have really great data they want to fine tune clot on their data and uh and and that is possible um but I always sort of encourage people to think about what is the goal they're trying to accomplish with AI before they turn to fine tuning CU in many cases you can accomplish really really valuable goals without sort of the back and forth the complexity of fine tuning but I think that um often you can really go farther than you might expect um without taking that step I couldn't agree more we're seeing the same thing and I mean hence all this discussion about Rag and our data cloud and you know fine for the last 10% final mile you can fine tune if you have the resources such an interesting um perspective so so you're serving consumers and you're serving Enterprises are there differences you see in in how to make each successful yeah so I mean I think that at our core I think we're mostly focused on Enterprise however we think that it's important for anyone at any company to be able to engage with Claude and see what's possible so for us it's important to provide Claude um to Consumers and really to anyone at claw. um just so that they can get started so that they can experiment but we're we're a bit more focused on uh on Enterprise use and I think that the Enterprise case I think sort of the safety message reliability really really resonates now a mutual partner of both of ours um that I'm going to be interviewing this afternoon is Amazon so can you talk about the options for deploying anthropic models through Amazon bedrock and how that differs from other ways of of consuming anthropic yeah I mean we've really tried to keep Bedrock as much as possible at parody with our own first party API we want customers to be able to use Claude um uh on AWS in exactly the same way they they could if they were working with us um so really all of the Claud models I mean Claude 3 Sonet is available now on Bedrock um and very very soon Claud 3 Opus and Haiku will also be available there um and so you should be able to sort of uh deploy Claude in whatever way is is best for for your business yeah we love giving customers Choice um Which models and how um anthropic is also a big customer of both Salesforce and slack can you talk about that yeah so I mean uh I can talk more about slack probably for Salesforce our sales team was ecstatic once they got access to Salesforce but of course I'm a research leader so I'm not I'm not using Salesforce as much uh for slack though uh honestly um you I mean this is this is really very genuine like coming from academ I think using slack has been very transformative for sort of medium to large research teams um our our our research teams are very demanding that say uh they're allowed to retain all of their content on slack because they have so many research discussions in slack threads and slack channels and it really allows people to have an idea suggest it to their colleagues at any time of the day uh share data share plots share discussion and that's how a lot of our uh our research productivity happens what do you expect models to get much better at in the coming year versus two years versus five years it's hard to answer some of these questions without sounding crazy um good so I mean I think I think about the uh capabilities of AI systems on kind of two axes so back when I I think we were in high school maybe like deep blue came out and it was better than human beings at but it's a very very restrictive task playing chess is not not not all that valuable it's not AGI it's not AGI so uh so 1X is kind of like what is the generality of the environment that an AI system can operate in and so we went from deep blue and alphago which were very restrictive but powerful to large language models which you can interact with in any kind of textual situation obviously very quickly added coding and now sort of multimodal capabilities where we have systems that can understand images and and with Cloud 3 I mean you can give Cloud 3 many frames of a video and it can understand sort of the Dynamics that are going on so I think one direction that we're going to see explode very quickly I think is use cases that take advantage of all the different modalities to bring AI more flexibly and fluidly to anywhere that that we do business so I expect that uh just as Claude can follow instructions and do tasks purely in text we'll soon have systems that can look at your computer screen as an image and tell mouse keyboard Etc what to do in order to do in order to utilize all the interfaces that are already available to to people so I think that's One Direction and I think probably following on that will be uh moving from research to deployment cases where these systems can operate Robotics and presumably start to enter sort of the the real world the other direction that I tend to think about is sort of the time Horizon on which Claude can operate So currently uh you can ask Claude to write a short memo you can ask Claude to read a document maybe even a book and and summarize or answer questions these are tasks that might take a person a few minutes maybe an hour but uh I think current systems as they're deployed tend to make mistakes they're not necessarily good at making plans and so it's not necessarily possible to use them to do tasks that might take you many hours or a day or a week Etc but I kind of expect that there will be further Pro progress in sort of the capability the robustness the reliability of these models so that you can use them for tasks that are agentic that that involve taking many actions getting it right backtracking if they make mistakes Etc and so I think those are the two directions that I see um I think progress on this uh is probably going to be very rapid um I think with respect to say robotics I expect the next 2 or three years to look like the last year or two in terms of uh seeing seeing starting to see real world deployments and I'm not sure sort of when we will be able to deploy sort of these agentic AI systems safely um but I think it's probably coming sooner than we might expect what's standing in the way what are the key block building blocks that need to be created or perfected yeah so I think that uh one of the things that's been a lot of fun moving from theoretical physics to to AI is just the pace of progress in AI is really rapid I think there's we're really breaking new ground it's not that we got smarter uh but we found sort of a way of making research progress uh easy uh with with with with scaling and so I think that uh the main blocker is really safety and robustness um and reliability um Claude Claude 3 Opus can uh get maybe half of the questions right that a PhD expert in a particular domain uh might be tasked with in a PhD program but the half that it gets wrong it doesn't necessarily know how to fix its mistakes and I think that kind of reliability is what we're working on and then what I expect is coming and and and currently is a blocker to to these kinds of uses what does that mean for PhD students um I mean hopefully it means that uh we can be a lot more productive and engaged I I have friends who are still in theoretical physics who use models like Claude to brainstorm and get feedback on ideas um to generate code for experiments and simulations and so hopefully uh hopefully like we can have a lot more fun and and and and make progress in science uh more efficiently can you describe the day-to-day of what it's like at anthropic I mean are there you know a set number of experiments that are being run how's the how are the research teams organized and how often do you check in how do you know if you're you're making progress yeah so I think um we really try to be very systematic and try to do kind of the simplest possible thing that might work just because when when things are moving so fast you don't necessarily have to try a very esoteric idea to uh A's razor yeah exactly exactly so um uh and it's different from in physics where I might spend 3 months 6 months just banging my head against the wall trying to find a good idea um so I mean people are very flexibly organically organized I mean they're people working on uh safety and evaluations for current and future models there's our interpretability team that uh I think I think is is very well known and which is really made tremendous progress in kind of reverse engineering how AI systems work which is really important right because I think I don't know if a lot of people know this but we still don't exactly know how these large language models work even the world leading AI researchers don't know so you have a team focused on figuring this out exactly exactly so they're focused on trying to understand internally what are the what are the neurons inside these neural networks actually mean how can we decode that and then how can we turn that into full circuits that you might think of as analogous to little computer programs uh that that govern how systems work so that's a huge Focus we hope that we'll be able to bring what we learn from interpretability to make AI systems easier to monitor safer and better to understand but absolutely what you said is 100% correct this field is moving so quickly that even though these systems are very powerful we don't have a deep understanding of how they work it's it's nothing like physics or biology where people have been working on the field for decades hundreds of years um and that's why that's sort of the Common Sense reason why anthropic worries a bit about safety why why we think caution is important because it's all so new and it's all moving so quickly is it kind of like quantum mechanics um I think it's all in a certain sense uh simpler and newer than quantum mechanics I mean quantum mechanics was built on this edifice of classical mechanics electromagnetism um all of these really deep insights uh that preceded it um AI of course is built on information Theory computer science optimization all kinds of things like that but I think it feels at least uh it feels maybe as revolutionary as quantum mechanics but but it's all I mean quantum mechanics from say plank in 1905 to uh Quantum field Theory Fineman in in the 60s and 70s it was occurring over decades and this is all it feels like it's happening just over the last few years and a few years from now I think it'll be very different I I think the stochastic element too I mean there's a little bit of a parallel there absolutely yeah yeah tell me more yeah like I mean you just you don't know for sure right there it's it's it's probabilistic absolutely yeah yeah absolutely yeah yeah people ask me about creativity in AI like can Claude be creative and I mean it's a It's Complicated question but exactly the this the stochastic nature means that you you can you can generate sort of new new ideas and new insights so you and I benefited from this amazing public school education imsa is a public Math and Science magnet boarding school I asked you about phds earlier what do you think about K to2 education I mean we both have young kids how are you thinking about educating your children and how should we think about educating the Next Generation to to succeed in this AI era one thing that I think we're working on is to make AI systems that are sufficiently safe that we could use them as tutors I mean that's something that like internally anthropic we I often enjoy like you can ask to teach you a new language or a new idea so I think that AI systems can probably help a lot with uh with with with education but I do think there's sort of a question of just as with like a calculator it makes it less important to learn arithmetic what exactly are the skills that AI is is uh is sort of making redundant and and and how can it help um I think that it really depends on how much progress we make and what's what's possible I think if uh uh if the progress we've seen is indicative of what's what's to come then could be very transformative or we could or there could be some some sort of plateau and then uh uh and then there will still be a tremendous amount of work for uh for people to basically uh use AI systems but check them ensure that they're robust Ure that they're getting the right answers which outcome do you think is going to happen I mean obviously I'm someone who had a a career I enjoyed in physics but I moved to AI so I'm very optimistic about AI continuing to make more progress um but uh but I try to maintain some sense of skepticism just because uh expecting that the tech the thing that you're working on is going to be the biggest thing ever is uh feels like almost everyone who's thought something like that in the past has been wrong and so uh try try to keep us sort of healthy skepticism I don't know I mean it's a good founder trait so let's let's put aside that skepticism for just a moment since we want this to be interesting and entertaining right no skepticism what does that Vision look like let's play it out 10 years it's 2034 what does the world look like I mean what does the world look like is really hard to answer because it involves Society economics governments Etc um let's start with the AI the state of AI do you think we will have achieved AGI by then again no skepticism a lot this particular conversation well I mean if there's no skepticism of course I mean I think that uh I think that we'll have ai systems that can really help us to do scientific research to address poverty inequality um but at the same time I think that we'll face a lot of uh a lot of challenges because um I think that Humanity has been uh distinguished by our ability to cooperate um and obviously our intelligence so if there are AI systems that can kind of do all of the cognitive work that we can do now um I think that uh that things will be very very very different so let's talk about that so how how would it do all the so would we would we still be sitting here what would be what would happen I mean I think that uh his historically the way that these things have happened has been that uh these systems enhance productivity um but we still are are important for all sorts of different uh different Niche activities so I mean cars displaced horses but uh but but people are still relevant I think that I guess what I imagine is that uh individuals in in a in a in in the positive direction if this goes well I think that individuals will just have a lot more ability to learn um spend their time in the way that uh that that is most enriching and enjoyable for ourselves and also build the things that like make our vision make our dreams sort of come true I think that's that's sort of the the the positive vision and I think that's a vision share and and I think it's up to us right and that's why I'm I'm really proud of the work that I think both of our companies have done to engage regulators and lawmakers and government leaders to not just Envision the future but to take the right steps to make it a reality it's it's I think it's so important um so you know if if we think about um the next just more of the immediate term right the next 12 months what advice would you have for all of our Trailblazers who are at this event and online in terms of how they can skill up and really take advantage of of these new capabilities yeah I think maybe the the most important advice is sort of uh dream big um in the sense that AI systems I think can do a lot now but a lot of the things that don't quite work right now will work soon so a lot of the thing one of the major things I think about anthropic is uh if AI is going to move very quickly how can we sort of deliver the most value the most benefits uh and keep Pace with with with with research and so I think um something I'm always telling people internally is that doesn't work now but get it ready because with the next system it might work and we should be prepared for that and so I think uh that's sort of the advice I would give for anyone kind of developing uh using these AI systems is um try really ambitious things and even if they're not ready now even if you can't deploy them um keep them in your back pocket and try them again it's Claude 4 um Etc so I think that that's that's the main thing and then I think um a lot of these systems can be can perform even better if you kind of orchestrate them so uh uh there are all sorts of simple use cases like Claw is a chat bot and you can chat with it and ask it to do all kinds of work but if you're willing to sort of uh orchestrate Its Behavior have it generate many possibilities and then refine them um you can get even better performance and that's at least as a researcher that's that's what I always love to see from customers is sort of doing things that are really ambitious and kind of push the envelope I think both of us when we were in high school we wanted to be physicists and we both interned at fery National accelerator laboratory in bavia Illinois and it wasn't surprising our our high school the Illinois Math and Science Academy was co-founded by Dr Leon Letterman the late Dr Letterman who was a Nobel Prize winner in physics and was very inspiring to a lot of us um but you carried this much much further and longer than I did you spent the first 15 years of your career as a theoretical physicist you got your PhD in physics from Harvard after we went to Stanford together can you talk about the intersection of physics and machine learning and how you made the shift in the world to the world of AI yeah um probably we were both excited about physics because of the potential impact it has on technology and I mean I was really into science fiction as a kid I was sort of envisioning like what could my career uh what kind of would my career have on on on the human future um and uh and so I pursued theoretical physics because I wanted to build warp drive and faster than light travel like kind of like silly things like that um but I think I was always very Mercurial and so a couple of things happened um when I was in grad school Dario amadi who's the CEO of anthropic and I like became friends and we really hit it off because we're both excited about how technology could have kind of of an impact to make the world a better place and we stayed friends and so I became a postto at Stanford and then then a professor but uh I I spent a lot of time living in San Francisco Dario and I actually liveed together and Dario was transitioning from cosmology to biology and bioinformatics and then to Ai and so we had a lot of uh conversations about how exciting AI was becoming because sort of in the ancient days when uh when I was in school uh AI wasn't making as much progress is as it is today and so then I basically took an opportunity to go on sabatical and study AI with a lot of other physicists and because it's such a sort of rapidly moving and kind of New Field it wasn't that difficult to learn about it and and start to get involved and so back when open AI was a nonprofit I actually volunteered there because Dario worked as uh the the head of uh head of research at open Ai and I just started spending more and more of my time on AI research um and kind of transitioning and so then in 2021 when when we founded anthropic I went on an extended leave from my job as a professor and it's just sort of been AI for the last few years and probably into the future that's such a random and such a great story I love that um so you've played a critical role in defining Core Concepts like scaling laws and reinforcement learning from Human feedback right everybody loves rhf um what's it been like to be a part of these formative Milestones from industry and again are there parallels to the academic world and physics that you draw from it's just been a tremendous opportunity I think I've been really lucky to be kind of in in the right place at the right time and maybe asking the right questions I think that uh the perspective that physicists theoretical physicists take is one of kind of focusing on Simplicity and generality we want to know like what is the big picture and so I think that helped a lot with kind of the development of this idea of scaling laws because obviously like long before scaling laws people were talking about Big Data um obviously sort of AI progress is was was going going strong in part for that reason and so sort of the questions that we asked were kind of like how could you uh quantify that like how much data how large could models get and and what would that mean how much computation could be used to train them and so that's that's kind of I think the the way that that physics helped us really just in kind of what questions do you ask yeah it's such a such a great Point um okay let's shift to what's happening at anthropic today um what are the goals shaping your work who are you building for I think anthropic is famous for having a healthy level of anxiety about AGI how does that manifest in the day-to-day and how you lead yeah I think it really gives a lot of coherence to the organization so in part because of ideas like like scaling laws uh since uh since since long before sort of Claude and chat GPT we believed that predictably AI was going to get better and better more and more capable um basically through large investments in compute data uh larger models we were going to be able to make AI systems that could do things that were were impossible a few years ago and I think that kind of core Vision that AI is going to get better very very quickly and therefore we have a serious responsibility to try to use it for for for for the good and to make sure that these systems are safe really kind of guides the organization and that means that kind of even though we're doing a lot of research it's not like everyone is working on a lot of different projects that are incoherent everyone's kind of focused on that goal of how do we make AI as capable as safe and as useful as possible well Jared thank you for all the groundbreaking ways that you're leading us um in furthering humankind with your AI work and doing it in a really responsible way it's so important um please join me in thanking Jared what an amazing conversation three takeaways for me one even the world's leading AI researchers don't exactly know how large language models work they're trying to figure it out in a field called interpretability two the pace of AI development is exponential and so even something that doesn't work today try it again in a month or to and maybe it'll work it's just astonishing last but not least anthropic is leading the charge working with a number of healthcare and pharmac companies to really accelerate drug development and amazing breakthroughs in science that's it for this week's episode of ask more of AI follow us wherever you get your podcast and follow me on LinkedIn and [Music] Twitter

Channel: Salesforce

Views: 820,912

Keywords: salesforce, ask more of ai, clara shih, jared kaplan, anthropic, claude 3, claude 3 news, anthropic interview, claude 3 launch, AI, trusted ai, ethical ai, artificial intelligence, business ai, AMOAI, AMOAI Podcast, saas marketing, saas customer service, saas crm, crm for software companies, partner lifecycle, prm solutions, prm software, product led growth

Id: YeeeiSjXzDw

Length: 28min 53sec (1733 seconds)

Published: Wed Mar 13 2024