AI achievements unlocked: Emergent abilities in large language models (GPT-3, GPT-4, PaLM, Gemini)

April 2023 and these models are getting bigger and bigger and bigger GPT 4 estimated to be a trillion parameters that's six times larger than gpt3 what's happening is these models get bigger I'm talking today about achievements unlocked you might have heard a buzzword about emergent abilities which is just a technical term it's been around since the 1970s to describe all sorts of behavior within different models and not just models in artificial intelligence but models in physics or biology or economics or computer science consider ant hills and Termite Mounds here in Australia that can grow several meters high from these tiny little creatures that are a millimeter or two across that's an example of emergence of this scaling up as things get bigger we're getting new behavior that was unexpected did that is just being documented for the first time another example is the Sun and we can feel the effects from this massive ball of Fire I've created a new visualization and I've documented it based a lot upon the work of Dr Jason way at Google he's done some fascinating literature review of the emergent behavior in these models particularly based on the big bench which is the beyond the imitation game Benchmark by Google and the mmlu both of those pretty big in terms of making sure that we can actually document Beyond a particular ceiling level what's happening in these AI models let's step through I've documented four different sizes here small medium large and extra large models let's start with the small which might be something like GPT 3 13B this one was unreleased it's a whole lot bigger than Curie but it's not as big as DaVinci and also Palm 8B by Google we're seeing as achievements unlocked at this small stage things like emergent arithmetic when it was not taught how to do maths it was able to be stepped through how to do very basic operations at this small level and that gets better and better as these models get larger code debugging the ability to go through and find what's wrong in a particular piece of code and very basic comprehension maybe at a grade one level these models can actually read and understand moving right along to the media models this is going to include gpt3 DaVinci and to a certain extent chat GPT I've put Lambda in here at 137 billion parameters Palms kind of medium model at 64 billion parameters and chinchilla 7B given its scaling is the equivalent of a lot larger model so we start here with Linguistics puzzles from the big bench the ability to convert movie titles into emojis which is kind of fun GRE comprehension so we're moving up there with its ability to play around with comprehension at different grade levels metaphor understanding physical intuition and understanding of the world and then logical deduction starts showing up as an achievement unlocked at this medium level gpt3 size let's move up to the large size now this includes Google palm 540b and the massive deepmind chinchilla 70b which is the equivalent of a much much larger model and they were uncovering some really cool things that came out of these large size models first one here geometric shapes understanding of shapes and geometry Proverbs in different languages the phonetic alphabet we start testing here the pronunciation of different parts of our English language elementary math you'll see here we've progressed from modified arithmetic and we're actually able to do math at a much more advanced level once again these models were not taught how to do math and somehow they've gone and learned what the operators mean what happens on both sides of the equal sign and they've learned a lot about numbers especially at this large level causal judgment might be stepping through causation and maybe correlation and code line description was found in Google Palm here's where we get to the super fun stuff and we're still going to be uncovering what's possible with gpt4 for a number of years we don't even have access to it yet the API is just the chat interface it's very limited as to what you can do but we do know about some of the capabilities from the different papers that have analyzed it from the hidden end Microsoft paper in particular is pretty revealing about its capabilities so both gpt4 here at the extra large level and I've also just estimated what might be possible with Gemini which is the project that combines Google and deepmind brings together their phds brings together their compute budget and allows them to train a one trillion parameter model very similar to gpt4 Let's jump in we start with college level exams you've seen what gpt4 can do here scoring in the top 10 percent in the SAT and several other exams including medicine and law you'll note here that this was just not possible with gpt3 it was getting decent scores on subsets of the SAT like the analogy subset but it wasn't able to do exams in any subject and get very high scores paper on reflection with a slightly different spelling with the gpt4 model is able to self critique its own answers and actually up its accuracy by analyzing and then modifying its own results building lots of documentation on this how gpt4 is able to generate both code and then entire files and directories for the creation of apps spatial reasoning for gpt4 similar to deepmind Flamingo it's able to see the world you're able to send it photos so it's got more of an understanding of how objects fit within space Advanced creativity we're still uncovering what it's able to do here they've tokenized both chat gbt and gpt4 allowing it to actually rhyme which is an interesting example of creativity and I've put in here at the end embodiment options we've seen Microsoft do this with chat GPT we've seen Google do this with palm but I think gpt4 just gives us a lot more options for embodiment particularly with seeing the world and being able to make decisions that's it that's the models we know so far I have put one little extra line down the bottom here called Next and all of these are essentially guesses or predictions or hypotheses here's the first one grounding the ability to know what it's fact and what is not fact I think the truthfulness rating is going to go Sky High with GPT 5 or whatever is next and the ability for it to critique and reflect and make sure that it is telling the truth long Horizon planning this is something that open Ai and Microsoft have spelled out as something to look for not just in the realm of power seeking but also in the ability to plan long term persuasion we saw this a little bit with gpt4 testing they couldn't get it to go too far but it was an interesting experiment and I'll see more of this in gpt5 Advanced embodiment we'll definitely see this with the new One X investment by open Ai and the Neo robot awareness this one's controversial but I'm going to leave it in because we've got people like Professor David Chalmers talking about this Marvin Minsky and Alan Turing were talking about this back in the 50s and 60s and last week even Professor Jeffrey Hinton said how can we be so sure that these things are not sentient so for all of these experts to go on record about this I thought let's put it in there such an incredible and interesting concept emergent abilities things that we wouldn't expect to come from these large language models as they get bigger and bigger and bigger we discover these unexpected achievements that have been unlocked thanks so much to Dr Jason way at Google Who provided literature reviews he found 137 emergent abilities I've just Consolidated these down to something that's more accessible if you've stayed this long through the video I've got a gift just for you it's never been released before you get to watch it for the first time recall that many years ago I was an expert in the research field of very high intelligence gifted education high performance specializing in prodigies I worked as chairman for one of the nerd organizations where I had oversight of 54 different countries and the families within them and how they would actually raise a child that could do very outrageous things that had achievements unlocked or emerging abilities at a very young age this is a book from 2017 it's a picture book that I wrote very quickly in a matter of minutes and then took quite a while for my illustrator to illustrate it beautifully you're going to get to see it now with an AI voice behind it the AR voices by narakeet.com the child's name the AI child's name is Tara with an Australian accent the book's called people like me and I want you to consider how our large language models right now are progressing in some ways like a child prodigy we're seeing these emergent behaviors that shouldn't exist but as we scale up and it might be scaling up the brain or the capabilities in a prodigy we're seeing some really cool stuff being unlocked people like me thanks for joining me see you soon to me by Alan d Thompson illustrated by Judith willings this book is based on true stories of real people this book is for you [Music] Adam is one year old he is curious about the world he has taught himself how to read the newspaper he loves reading so much that he does it as often as he can even when the words are new to him Isabel is two years old she thinks that dinosaurs and other creatures are incredible she is learning a lot about each dinosaur including their Latin names Grace is three she likes to memorize numbers she already knows the first 31 digits in the mathematical number called pi she can say them out loud [Music] roshni is four years old she loves language and talking with adults she has taught herself to type on a keyboard she writes her own letters to her grandparents in Singapore it is a little bit slow and sometimes her fingers hurt from typing she enjoys using new words and improving her sentences Carl is five he wants to know more about many things and loves asking questions he is especially interested in geography and astronomy he knows the names of every country on Earth he also knows the names of many galaxies in the universe he reads every day learning more and more Alma is six years old she loves playing the violin and piano she is in the middle of composing her first opera it is about a beautiful fairy tale Alma hears the music playing in her head sometimes she hears the music in her dreams sometimes she hears it in her mind while she is spinning her skipping rope around her head then she has to work hard to write the music down and make it fit together is seven engineering and physics are his favorite subjects he loves building things with his hands when he designs a long bridge sometimes his model breaks or falls down James picks up the pieces and tries again sometimes taller sometimes stronger sometimes in a new way [Music] Tristan is eight he loves lighting code to create software he also loves gnats his first program looks simple but it was hard to write it can calculate and display prime numbers when Tristan's software displays an error he has to find it and fix it his software programs are getting better and better Nicholas is nine he sees new inventions in things around him he likes nature and machines for fun Nicholas designed a new type of motor using bugs attached to a windmill he wants to continue to discover and invent new things for future Generations Clara just turned 10. she loves science especially chemistry she also spends a lot of time playing and having fun while she was playing with ball and stick models in science class she discovered a new molecule it is called tetroneutra toxicarbon unique but I'm not alone there are other people who think like me there are other people who love learning like me there are other people with Advanced brains just like me I listen and make them my friends I play with them in the park I share with them in groups I can find some of them in my school some of them are older some of them look different some of them live very far away I seek them out I find them because I know they are there creating things making mistakes finding Solutions living life sometimes I have to look extra hard but they are there people like me [Music] thank you

Channel: Dr Alan D. Thompson

Views: 47,576

Keywords: gpt3, palm, wordtune, grammarly, ai, pwc

Id: qAUJwKvm_lQ

Length: 15min 47sec (947 seconds)

Published: Wed Apr 05 2023

Reddit Comments

[u/AutoModerator]

Hey /u/Pretend_Regret8237, please respond to this comment with the prompt you used to generate the output in this post. Thanks!

^{Ignore this comment if your post doesn't have a prompt.}

We have a public discord server. There's a free Chatgpt bot, Open Assistant bot (Open-source model), AI image generator bot, Perplexity AI bot, 🤖 GPT-4 bot (Now with Visual capabilities (cloud vision)!) and channel for latest prompts.So why not join us?

PSA: For any Chatgpt-related issues email [e-mail]

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.