Prompt Engineer: The New Job Created By AI

hey guys I'm Smitha and welcome back to my channel where I talk about all things Ai and machine learning related cat gpp has truly taken over the internet and just over the past couple of weeks we've seen many large tech companies come out with their own chat Bots which are built on large language models exactly like chat GPT however now that millions of people have started using things like chai GPT and other chat Bots similar to it we've begun to see quite a lot of issues with these type of large language model chat Bots and that is that they require very specific instructions in order to get the right answer and specifically they require very engineered instructions as well so a lot of people have begun to write on what works for them and what doesn't and now this has actually become a proper role within Ai and a lot of people are speculating that this could be the next big job that AI actually creates and that is called prompt engineering so why exactly do we need prompt engineering why exactly do we need prompt Engineers to begin with and is prompt engineering an actual job there's quite a lot of speculation on this and also we'll talk about what exactly can you guys do to learn how to prompt engineer better so let's get into it so first off let's actually talk about why exactly do we need something like prompt engineering for child GPT which is based on large language model so the issue with large language models is that they are not easy to understand meaning that they often behave like black box models a black box model is a term used within Ai and machine learning to refer to something which is useful which can provide useful information without revealing any information about how it actually works so essentially when you have something which works like a black box model there's quite a lot of disadvantages because you can't really understand exactly how it works and that is the issue with having really large language models like Chad GPT or GB3 is that you don't exactly know how it works simply because of the huge amounts of parameters within it and it's really hard to figure out why exactly it's giving you a certain type of input based on its prompt and that means it's also difficult to understand how exactly to best make use of child GPT in order to get the output that you are looking for so now you know that large language models are not easily interpretable because they have a lot of parameters and that is exactly why it's hard to figure out exactly how they work so now let's actually look at some common prompt engineering techniques which people have figured out actually works for things like chai GPT I found this really amazing documentation created by coheer dot Ai and I'm going to share a link of it in the description box below but essentially they have version of four main principles which which they suggest will work really well for prompt engineering so let's take a look at all of them so number one a prompt guides the model to generate useful input so if you need a summary of an article for example a large language model trained on enough data can generate a summary if you guide it as such and number two try multiple formulations of your prompt to get the best Generations when using generate it is useful to try a range of different problems for the problem you're trying to solve so different formulations on the same prompt might sound similar to humans but can lead to very different Generations that are quite different from each other when you give it to something like charge ubp for example if you are doing a summarization example and you want chat GPT to summarize an article for you you can use words such as in summary or to summarize in plain language or the main point to take away from this article is so these are different language that you can use in your prompt the third thing that they suggest is to describe the task and the general setting so your prompt can be structured like this where you give a prom task description you give your current output and then you give your output indicator so indicating whether it is correct or not so here's an example so you give the task description where this is an article followed by a summary written in an informal language and your input would be the actual article itself and then your output would be the summary so when you feed that into a language model you're able to get a much better response and number four is showing the model what you would like to see so here's an example of showing the model what you would like to see where you give it the task description you give it an example input as well as the example output and you do that as many times as you can and essentially you can think of this type of prompt engineering as you training the model to identify exactly what you want in your output so it's able to give you a much better answer so here is an example of something like that right so this is a problem that you can give it so this is a movie review sentiment classifier and you give it the first review which says I love this movie and you say this review is positive and you give it a second review which says I I don't know it was okay I guess and you say this review is neutral and the third review you will give it saying this is this review is negative now finally you pass it an actual review which you wanted to classify which says I really enjoyed this movie this review is and the model is able to understand exactly what you're looking for and is able to tell you that this review is positive now these four ways are just some really simple ways that you can start off with prompt engineering there are no fixed guidelines in fact a lot of machine learning engineers and AI practitioners are trying to exactly figure out how large language models work and what are some of the best ways to get useful outputs out of them so this is something which is still being figured out and this is something that even you guys can explore with by playing around the chat gbt although prompt engineering is extremely important especially right now I do think that this is a very short term role if it ever does become a role because there's a lot of research which is being done to actually automate prompt engineering itself so people are building AI models out there simply for prompt engineering so this one is one of the latest type of research that we have out there and this is called large language models are human level prompt engineers and what they have essentially done is they have created language model which they call ape ape and it stands for automatic prompt engineer so it is automatically able to generate several prompts the prompts that ape has actually created turns out is actually much more accurate than even human design prompts so that is definitely something to think about and in fact in the future as we progress further on we can definitely see a lot more research which is going to be done simply on prompt engineering simply because large language models are going to be widespread read and prompt engineering is definitely required there's going to be you know research being done on prompt engineering itself and in fact I'm pretty sure that a lot of the large language models which are coming out they might actually be optimized to create better prompts as well so that is something to definitely think about thinking long term so I highly doubt that prompt engineering could become a long-term uh actual established role within Ai and machine learning but who's to say there's been a lot of surprising things which have been happening within machine learning and AI so that is definitely not off the table let me know what you guys thought about this video and what do you guys think about prompt engineering is it something that you think is going to be a huge role with an AI or not or is it something that you think is definitely more useful for the short term thank you guys for watching and see you in my next video

Channel: Smitha Kolan - Machine Learning Engineer

Views: 7,481

Keywords: machine learning, learn machine learning, self taught machine learning, How to earn money with machine learning, how to become freelance machine learning, software developer to machine learning, how to become a machine learning engineer, Prompt engineer, ChatGPT Prompt engineering

Id: TPIXDkaLsZM

Length: 8min 35sec (515 seconds)

Published: Tue Feb 21 2023