Separating AI Hype from AI Reality

welcome to this episode of Dev questions with Tim Corey join us as we tackle the questions you are asking about a career in software development understanding the industry and new technology if you are just starting out or you want to grow stronger as a developer this is the place to get your questions answered now here's your host expert developer and online educator Tim Corey how do I tell when AI products are real and when they're fake there's still a lot of questions around AI seemingly every week we get a new announcement about new capabilities of one AI or another and some of these can seem like life-altering changes like when the Devon team show off their AI that entirely replaced a software developer now of course it turns out that they faked a lot of the demo so how we determine what's real and what's fake in the AI space that's what we're going to talk about in today's episode of Dev questions so let's start off with an understanding of what current AI is because all AIS right now kind of come to the same route so let's talk about how they work and what they can can't do now what we call AI currently is a system that's basically providing the next generation of a search engine you may say Tim that's not right no they do more than that no no no it's pretty much a search engine so let's talk about what I mean so the way AI is built it takes a lot of inputs and by a lot I mean billions trillions and so on of inputs and it breaks them down into small parts it observes how items connect to each other based upon how many times it sees those items connecting each other in its training data for instance it might see a million pictures of a maple tree and learn that maple trees have their leaves that typically connect at the end of branches which then attach attaches to the trunk so it may see all this and say okay I've seen enough to know that you know if I see a maple leaf it probably connects to a branch and that Branch probably goes back to a trunk and a trunk to the ground and so on it observes what it sees and sees those connections and then it sees that over and over and over again says okay I see how this could be so then based upon that information and these associations when we ask an AI to give us information it pulls pieces from various connections that it has seen before and makes a new mixture of information based upon the perceived relevant data so that's the Trap okay so it takes these different correlations and hopefully that they're they have a true causation connection but creates a new mixture based upon the perceived relevant data and it gives it to us and again that's the Trap we see this as a new mixture and Associate intelligence with the action at least in our minds we see that the act in terms of a human act so we start to assume the AI is gaining in intelligence as it adds capacities so when we see it AI come back with this you know this paragraph of information that seemingly we couldn't find on the internet because it's not from One Source but it's from many as put together well it's learning right like that's how we learn so we read a book or we read a web page or we watch a video and we absorb that information and then kind of put that together with other information we know and we put out a new conclusion so that's what the AI is doing right not really so what's that's not what's happening the AI is only mashing together things that has seen before into new combinations based upon the similarities to previously seen combinations you may say well Tim it's still creating something new no maybe okay yes you may see some new things that maybe didn't exist anywhere but that doesn't mean they're first of all right because a lot of times they aren't and also it's based upon what it's seen before and just kind of regurgitating that but in taking a smaller chunks and put them together so it's kind of like instead of you know building a whole house it's taking Lego pieces and putting them together and hoping that the result is a similar type house to what it's seen before that's why to this day with the latest chat GPT 40 which just came out a couple days ago if you ask it to generate a nine sided shape it gives you shapes with six sides or a different count I try it just a minute ago and I kept telling it no that has six sides try again no that has six sides try again and it's not figuring it out and if you are a toddler one of the things you learn is how to count 1 2 3 4 that's pretty early on human learning but yet AI doesn't know how to count it doesn't count what it does is it takes previous information and repackages it so it's not reasoning it isn't counting the sides and going wait you're right six sides in fact as a human when you look at that you go wait 1 2 3 4 5 six I see six I don't see nine so you immediately know something's wrong the AI has given me this confidently saying this is your answer and you say no that's six sides it will confidently give you a new one that also has six sides now in your example it might be eight sides or it might be 12 sides but for my examples over and over I got six sides when I asked for nine now when I ask for three sides did it every time there's a triangle there you go it even calls it a triangle if I give it a fours side shape yep no problem there's there's four sides but nine side shapes are unusual it's not something that the AI does well because it's unusual it's not intelligent it's just putting together what it's already seen and it hasn't seen a lot of nine-sided shapes so with that understanding of AI we now know the basics of what it can and cannot do it can create really specific searches for us so I love AI don't get me wrong when I am developing something I say hey how do I do a I've been working recently to create different the HTTP verb commands in C for a project I'm doing online well just asking it hey build this I go ah it's not quite right can you tweak this and it does that's super helpful it makes me faster because instead of going to Google and typing that question in searching through a whole bunch of web pages may stack Overflow getting the wrong answer a few times or getting a piece from this web page and a piece in that web page and kind of piece them together it's doing a piecing together for me instead of giving me whole web pages it's giving me specific parts to my solution and piecing them together for a result that's tailored for me now the downside is that that piecing together might not be correct and if there isn't enough training examples to allow it to generate a similar answer to an existing solution it's more likely to be incorrect it's going to assume things and as we know as software developers assumptions are a really bad thing now that's what AI can do it can be a really specific search engine and piece things together in a much better way than just a Google search can but what AI cannot do is generate new answers on its own it's not inventing it's not creating something new you may say well Tim it's piecing together new answers again yes and no what it's doing is taking existing answers small chunks of them and putting them together it's not building new answers it's just taking existing answers in smaller parts and put them together and give them to you so it can't invent or create the only seemly new things come from when it has seen something similar beforehand unless you count the remix which is its best guess at a solution so at the very most what is doing is taking small chunks of things from other people that people have created and then put them together that's the best that you can call creation but it's not really so now let's talk about what how to differentiate hype from reality when it comes to artificial intelligence we can call anything type that involves an AI creating something completely new we can also identify something as hype if it involves autonomous actions by an AI autonomy requires logic AIS don't possess logic nor can they they imitate existing patterns that's why it can't create a nine-sided shape it doesn't have logic to Think Through I need to count the sides it's just to a best guess so when cognition introduced Devon the in their term words the world's first AI developer your hype radar should have been going off like crazy being a software developer isn't primarily about writing code we talked about that I think last week it's about building logic understand a specific situation and building something that fits for that situation that's why senior software developers so often use the phrase it depends when you ask them for advice just throwing code at a problem isn't the solution so what would Devon have to do as a software developer well it have to design a solution specific to the situation for example the right language work in existing codebase have current data and use that current current data because it might that current data might change how you design something figure out how to work inside a specific industry maybe that's how you do certain security things or something else number two you have to understand the needs of the customer including knowing when to ask follow-up questions that's really important knowing how to ask a follow-up question because maybe the user thinks they want something but they don't realize what they're asking for Devon can't think that through logically and go you know what you're probably asking for this but what you really want is that okay knowing that how to ask that follow-up question that involves logic processing and something that AI cannot do and also number three it know has to know how to evaluate if the code does what it was asked to do okay so when you get requirements you have to be able to say did I meet the requirements not just do the unit tests pass that doesn't really matter in the grand scheme of things if you asked me to create a method that added two numbers together and I create a method that multiplied three numbers together and I create a unit test that makes sure that yes it does multiply three numbers together the unit test is going to pass but I didn't do what you asked me to do so I failed the assignment even even though I have working code so knowing how to evaluate your requirements and then evaluate Your solution to make sure it matches the requirements is something that AI just can't do at this point all these tasks involve logic and intelligence remember that's not what AI does And yet when Devon was launched they showed it solving problems in code they showed it learning from documentation they even show up performing tasks on upwork as if it were a real software developer and that's when you should know should have known that yeah this is hype and this is not real okay now if you look Clos to the demos you're going to see a few things the upwork task that Devon says it completed it didn't even result in a solution that met the initial requirements again coming back to can it evaluate its existing code against what it was asked to do no now of all the things that AI can't do yet I think that's the easiest one to maybe come closer to but even that isn't possible right now with AI uh number two the problem that Devon identified so there's a video where he the the AI identifies a problem it debugs it puts some print line statements in and then it fixes it well the problem is those problems were actually caused by Devon it caused the errors because it couldn't f figure out how to clone the repository and build the code even though there was clear instructions that were included in the repository instead it created extra files that it then had to debug in order to try to get those to a working State when the files weren't need in the first place so it caused the problems it then is proud of fixing I mean that's what we do as software developers too sometimes but that doesn't really show any type of of value as a tool because it didn't think anything through logically it started throwing code at solution when that wasn't the way to fix it and then number three in their llama 2 Benchmark demo they said that they were turning over to control to Devon and that Devon is going to be the driver seat however you can repeatedly see an operator correcting Devon and redirecting it throughout the process basically the operator is providing the reason and logic and Devon is just attempting to write code which means that the operator was doing the software development role and Devon was doing kind of a co-pilot role so it turns out that the Devon demo was faked and it didn't actually perform much differently than GitHub co-pilot or chat GPT so that's something that you have to think through is what can the AI do and what can't it do and is some of this going to fake or hype as opposed to reality everyone is trying to get in a gold rush of AI there's going to be a lot of misinformation out there so you have to be able to evaluate what could it possibly do there is a lot that AI can do but being able to evaluate the hype versus the reality is really important now now that you have an understanding of what AI is how works and what it can't do hopefully you'll be better equipped to separate the hype from the reality of AI and there are a lot of great things coming out with AI so for example near realtime language translation that's awesome better code writing assistance it's amazing I use it a lot incredible web search capabilities can really make a huge difference in our lives AI can make our lives better it just won't replace the true purpose of a software developer now it will however make software developers more efficient and effective so that's the other part of the fear of AI as well it's going to replace software developers no absolutely not it can't do logic okay so well but software developers going to use AI to be more efficient therefore be less software developers kind of yes because software developers can do more things because they have an AI assistant that's always been true of our tooling though when basic not Visual Basic when basic first came out people thought that the the career of software developer which was barely a thing they thought it was dead because anybody in their basement could write code and so they thought that by making basic that it would essentially eliminate the the field of software development going back even further when they made assembly they thought the same thing it's going to eliminate software development because it's so easy now to write applications these things didn't happen in fact software development has been on a hockey stick curve of growth it's been booming so yes there will be some more efficiencies added to developers that AI will help with because AI is valuable but that doesn't mean it's going to destroy the market for developers or even new developers because it actually makes new developers you know it allows them to get in the market more easily so what it will do is it will open up new jobs and new possibilities and new allow you know smaller companies to bring on software developers and really grow how many different areas can employ software developers so it really is actually a good thing for the industry so instead of being centralized in a few companies which I know we're not but everyone thinks that you know if you don't work at Google or Facebook or actually meta um or Microsoft or one of those that you know you're not actually a software developer I don't know but people look at those companies and say Well they're laying off therefore the the industry is dying that's not how that works there's so many different companies that hire software developers with AI they'll be even more which means it diversifies the industry it allows for more possibilities of working in more places so AI is a good thing AI is going to be helpful but we have to make sure we don't get on the hype train and believe that AI is somehow more than it is so that's how we separate the hype from the reality when it comes to artificial intelligence thanks for the question and as always I am Tim Cory [Music]

Channel: IAmTimCorey

Views: 5,611

Keywords: .net, C#, Visual Studio, code, programming, tutorial, training, how to, tim corey, C# course, C# training, C# tutorial, .net core, vs2022, .net 6

Id: lD3KJg7h2Bs

Length: 19min 49sec (1189 seconds)

Published: Thu May 30 2024