Using ChatGPT for Language Learning

hello this is Tom golly of the University of Tokyo it's been a week since open AI announced their new large language model interactive version called chat GPT and in this video I would like to show briefly some experiments I've done using GPT for language learning in other words now that we have this interactive multilingual AI that seems to be able to grasp meaning and to interact with the user to some extent I want to see how well it could be used by language Learners and perhaps by language teachers as well too for the learning maybe even the teaching of second languages so first I will look at how will chat GPT can explain the meanings of words in context for example if a student is reading a text in a in a foreign language and there's a word the student doesn't know can you ask chat GPT what the word means in this particular context many words of course have multiple meanings next I will see how good it is at correcting and explaining grammar for example if a student makes a grammatical error and is not aware of it can chat GPT correct that mistake and then can it explain the reasons for that mistake um next I will look at how good it is at sort of a traditional tasks that could only be done by humans which is um defining words giving especially multi-sense words words that have more than one meaning can it give definitions explanations as a dictionary can and also what about example sentences for learning words in another language is often helpful to see the words not only in isolation but also in a context can it produce example sentences that would be useful to a learner uh next is a traditional form of of language learning dating back probably hundreds of years is to have the text in the in the second language with uh detailed vocabulary notes and maybe a translation a side-by-side translation or even sentence by sentence translation can chat GPT prepare that automatically and then looking largely from a teacher's point of view one standard sort of learning and evaluation method um in language classrooms is quizzes vocabulary quizzes also Learners will sometimes use them to test their own ability can we just ask chat GPT to prepare a vocabulary quiz or not well let's see okay so first of all for explaining the meanings of uh words in context so let's take a case where the user is me in this case is asking a question in English about an English word in an English text so in this case I I said hi I don't understand what murky means in the following newspaper article I think this came from the New York Times can you explain it to me and there's a paragraph there and you see at the very last line it ends with uh the research is reported because the definition of a whole grain for food is murky well murky is the sort of word that a learner of English might not have learned yet might want to have to look up in a dictionary and so let's see what jet GPT says well it's in this context murky means unclear or not easily understood so it gives a I think a correct definition of this meaning in the context and it says the article is saying that so it summarizes the article and the meaning of the word in the context of this article of this paragraph and I my reading of that is that it's basically correct so it did a good job um I got a little curious about etymology I asked what is the origin of the word murky and chat GPT tells me that the word murky comes from the old English word Merce Merced remorse which means dark or gloomy uh according to the Oxford English Dictionary this seems to be correct so it's really more for Merck rather than murky but that's okay though you notice it then says it originally referred to a dark and murky appearance so it's using the word murky to Define murky which is a No-No in dictionary definitions but it's not a serious serious problem then I asked for some example sentences can you give me some other examples of murky used in a sentence and chat GPT replied um yes here are some other examples of how you can use it so let's look at those number one the water in the lake was murky making it difficult to see what was beneath the surface number two the company's Financial records were murky raising suspicions about their business practices three the future of the project is still murky as we were waiting for approval from the board and there's also four and five what I want to say about these is that these are excellent example sentences these are the kinds of example sentences that writers of Learners dictionaries and vocabulary study books strive to make the reason why is that in each case the use of the word murky is idiomatic the collocation of saying that water was murky the financial records were murky the future we're working with murky I think those are all reasonable combinations Plus in this particular case the second half of each sentence explains or augments the meaning of murky going back to number one the water in the lake was murky making it difficult to see what was beneath the surface so it helps to explain the word that's being um exemplified this is what um people writing example sentences strive to do I've written hundreds thousands of those sentences for dictionaries and textbooks and the like over the years and so this is a really remarkable that it came up with five sentences just off the off the bat like that um then I I said well what about if you know the learner will might be more comfortable asking these vocabulary questions in the learner's first language so if the learner is first language is Japanese well but he would ask the question in Japanese so here's a another article also from the New York Times in which the um there's this phrase on line three there's a former football coach for the University of Michigan and he had a football helmet that had go Blue written on it okay so this is something that uh uh a Japanese learner of English might not understand what does go blue mean and so like that's with the question in Japanese asks what what does this mean and I asked for an explanation in Japanese and so we get an explanation in Japanese that this was the words used um for the supporters of the American college football team in Michigan Michigan Wolverines um and the following explanation is basically correct um and so it's definitely that was remarkable I think um and then there's a phrase at the beginning of that text but it refers to this slab of a man what does slab mean in this context well it explains the original meaning of a slab uh the Japanese might be a little bit redundant but it means a large piece of something um and it but says in it so the original it explains the original meaning and but then it says in this case it just means this is a big person a big man um maybe slab has a little more Nuance in English than just big I think but it's pretty close okay next I asked about um the winningness program which is the winningest program mean um and the one with the most victories was the answer in in uh in Japanese um there's a little bit of a mistake in the Japanese it says there's a it's a getting the biggest number of wins within the program is what it says on the third line which is a misreading or misunderstanding of the word program in this context but I think even though this explanation is not not perfect it still would be sufficient for a learner to to grasp the meaning then I got a little meta I said I asked in Japanese um if a learner like me um what should I do to be able to learn vocabulary like go blue or slab or winning this and then I got this very nice set of advice about from chat GPT about how to build my vocabulary you know to make sure I understand all the meanings first and then to actually use them in writing and in conversation with friends and and to make sure I use them more so that they they become well established within me as words that I use use them frequently I don't know I don't know how often you want to use go blue or winningest but as general advice about building and retaining vocabulary it's it's very good advice and then I asked for more examples of the of slab used in the same meaning um and so a slab of bacon Cut the Cake into large slabs so The Rock was so heavy that it took the two men to lift the slab so I think these are all fine fine example sentences of it and then the the last four lines in Japanese are explaining translating those meanings into into Japanese so this first example of words used in um uh context of looking at the meanings of words used in context worked quite well let's go on to the second okay in this case I wanted to focus on a word that has more than one meaning the word oversight is interesting in English because it actually has kind of has opposite meanings in when it's used as an uncountable noun it means sort of supervision or control and when it's used as a countable noun and oversight oversight it means mistakes errors omissions and so one is a positive thing the other is opposite depending on the the form of the grammar and so it depends on the context therefore to what is the meaning so first I ask um I understand the English word oversight has more than one meaning can you tell me what those meanings are and it defines them first as an error or mistake and second is the act of overseeing or supervising something defining oversight with the verb overseeing is probably not ideal for a definition um also the fourth line it can also refer to a lack of supervisional oversight I think that's wrong I don't think that's correct though the second part of the fourth line a failure to notice something that is correct um okay so then I ask about the case in a particular um newspaper article uh so here's a sentence from The New York Times a couple years ago um that on the line five it has their strengthening civilian oversight what does the word mean in this context and it gives us an explanation in this context word oversight is being used refused to refer to the active overseeing or supervising something especially in the context of law enforcement and then it gives an explanation which is mostly a paraphrase of the rest of the paragraph but it does have some sort of new information added that I think is a reasonable extension of that a reasonable interpretation so it's a it's a it's a it's a quite good um explanation of what that word means in this case well here's another example also from a newspaper article which is oversight means here and you see down there towards the bottom Mr Inslee apologized for the oversight during a news conference so we have the oversight and then a little later says obviously we regret that mistake so human being reading this understands that this oversight does not mean supervision it means a mistake and if you read the entire text it would be clear well what about chat gbt is yes in this context the word oversight is being used to refer to an error or mistake that resulted from a lack of care or attention and then it explains it partly with paraphrasing but you know it it reads well and smoothly and the content matches what what appeared in the original um and so this is a very very good summary in English of what that word means in that particular context here um chat g p t did not do so well um it was able to correct grammatical mistakes but it was not able to explain some of them it was able to explain one not another and the one it didn't explain well was really really bad so first of all I created this not completely natural and natural sentence but I I so I have this sentence three three a few sentences here I was born in 2004 and tochiki prefected Japan when I was a baby my family lived in an apartment when I was five years old my parents bought a house and we moved there um I did not underline it in fact you cannot underline in the in the input form for chat GPT but so the the article a house was missing okay and so I asked to correct the mistake and then chat GPT did correct it it inserted a but now then I ask why why did you change house to a house and the response is I changed house to a house because the sentence was originally written in the singular form but the subject of the sentence is plural My parents in English why won't go on anymore this is completely wrong okay it's completely this is not the reason that house was strange and that a house was correct so this was a worthless explanation um I it was a little bit better in this case with number agreements so I um asked about please correct any grammatical mistakes in the following paragraphs schools and places of work teaches us so there's a problem there um the two learn was no problem I don't think um assembly was I didn't I did not have a comma after the assembly in the original one um it added the comma after assembly it deleted the two for to learn and to grow up on the lines uh two and five and that was okay either one is correct the two could be there could be not be there so those were not necessary changes but it did correct it did correct the two number agreement problems so on line one is corrected at schools and places of work teach us not teach us and on line five discipline that helps us grow up to grow up and become happy so I asked about the first one on the first line why did you change teaches us to teach us and this explanation is okay I T I changed teachers as the teachers because teachers is a third person singular form but the subject of the sentence schools and places of work is plural okay so that's that's the correct explanation but it even having one correct explanation does not quite counteract the completely wrong explanation earlier so for um correcting grammar it's for this very very small sample it did well um but for explaining the changes was not so good okay this is where chat GPT did the best in my opinion um so this was similar to the earlier case with um oversight I chose another English word that has a variety of meanings critical has a lot of different meanings the adjective critical so I first asked what does it mean and it gave me several meanings uh [Music] um it's interesting it doesn't it doesn't include the first paragraph doesn't include the meaning to to say bad things about people to say negative things which is maybe the most common meaning in conversation um it gives a meaning um important and relating to analysis evaluation analysis and also medical meaning and so um but anyhow as I saw a first after getting those definitions which were all fine as far as they went I asked for some example sentences can you give me some examples sentences of the word critical used in the meaning important significant or essential so I focused on one particular sense one particular meaning give me some sentences these are somewhat longer than the sample sentences in the previous example but they're also all very good they're very um appropriate collocationally um sentences are not quite as explanatory as those previous sentences which is interesting but they all are fine for somebody wanting to remember the word critical by seeing it in a variety of contexts these are all good sentences so I did the same thing with the next meeting related to the evaluation or analysis of something once again I got five example sentences that are all appropriate for that meaning of the word critical then I asked about the medical meaning and I also got five examples of all of these in the medical meaning critical head injury Critical Care Unit critical illness policy um all the great okay then I asked does critical have any other meaning so one of the features of chat GPT is it it remembers what happened before so I don't have to it remembers the meanings it's told me quite a ways back and now it gives me in this paragraph it gives me some other meanings including the judgmental or censorious meaning of meaning in chemistry a meeting in mathematics um which I think are all correct and now we get some of these meaning of important critical point critical decision uh I'm sorry these are these are different senses these are different senses that are being um Illustrated here see the last one the critical value of statistical test is the um mathematical meaning I think these are all correct um and so then I asked for five more examples in this one particular meaning the judgmental or censorious meaning we've got five examples there and then I went on uh kind of a spree first of all I started a new session so that it forgot I think it would have forgot what it had told me before and I start again I say I understand that the adjective critical has several different meanings one of those meanings is judgmental or censorious can you give me 10 examples sentences of critical so ask for 10 examples it gives me 10 examples they're all quite short but they're all very natural very reasonable ones and I asked for 10 more gave me another 10. I think these are all good as well too as for 10 more and so it gave me a 10. so I got 30 example sentences of the word critical in one particular meaning and so you know how long would it take a human being to think up those sentences or defines a a similarly short and um Illustrated sentences um in a corpus it would take a while this just took a few seconds I by the way I just noticed um it is um alternating the gender of the pronouns it's digital I just noticed this samurai oh it doesn't well do we have I in one case here otherwise it's it's he he he he so this is the the ethical and ethical um feature of of TI gcpt as of now I guess trying to balance gender by the way this is you know there if you go and look at say dictionaries written 40 50 60 years ago English dictionaries when the example sentences were made up by the authors it was almost all male um subjects to the sentences and very oft depending on the dictionary there would be there were some old dictionaries where if if a female was the subject of the sentence it was in some negative context so this is they've made some efforts in that regard and I asked for 15 more can you give 50 more examples and it refused I'm sorry but I'm not able to generate 50 more sentences using the word critical um a lot of different requests uh to chat GPT now give excuses reasons why I cannot do a particular thing in this case it's um one thing it notices it points out is it cannot browse the internet and so it's not able to go out and search for information but I I yeah I am you can say I'm personifying it and I thanked it and it said you're welcome back to me in this case I um sort of took a sort of a more traditional approach to language learning um this the use of annotated texts for language Learners um dates back as I said a long time I've used them in my own language study both many years ago when I was in college studying Russian and later when I was studying Japanese I've also written and translated some several books of this sort of readers of Japanese for example where you have the original text in Japanese and then vocabulary notes on the facing page and so I will so I took a paragraph from um the first paragraph of a short story by Louise Idris from a recent issue of the New Yorker magazine um and uh it's if you look at it you see it's a little bit I mean it's not sort of the standard academic um language textbook English and so it's the kind of thing that um a typical language learner would face um you know a lot of words to look up if you were trying to read it um and so uh can we automate that process of looking up the words and so I asked please prepare vocabulary notes explaining words and phrases that might be difficult for intermediate Learners of English to understand and there it is antecedents colon ancestors or forefathers Bonanza Farm a large and profitable form plagues swaps dire weather this is really good I mean these are just the sorts of words that um an intermediate learner would have to look up in many cases lethal Misty Curtin scour and so I think these definitions all work so it's an immediate learner might prefer the vocabulary notes in the first language so I asked for them in Japanese and we got the same words and we got you know Japanese glasses which are all look very good to me so that was that was remarkable I asked for a translation into Japanese um so chat GPT can do translation just ask it to translate from one language to another um the tests I haven't done a lot of tests yet but it this translation and the few others I've done are on the level of the best machine translation that I've seen so deep L is currently the best translation engine between Japanese and English and there are a few mistakes in here but um you know for somebody you know wanting to get the the meaning of the words and not wanting to sell the translation as a correct translation but to use it as an age of learning this would be this would be good enough so then I asked to change the format for it this time display the sentence this text sentence by sentence shows the English sentence then the vocabulary notes in Japanese for the words in that sentence finally the Japanese translation of that sentence in other words just make it easier to use one sentence at a time we get a lot more um we get more vocabulary notes in this case and some for example the Translating the Name of the bar is not really meaningful in this case it's just a phonetic rendering in Japanese also beer is translated as alcoholic beverage which is not quite correct but history farming I think if we go back and read the context it's the Japanese translation is it's probably probably close enough um maybe not perfect but it's still pretty good and it continued that way so in each sentence then it had the translation in Japanese with the entire sentence then we have the next English sentence the notes for that the Japanese translation um so I did about five sentences and then it stopped responding this has happened several times when I've asked a question that required a long response chat GPT just Peter's out at some point um I uh open AI is offering it free at this point so I can't complain about that maybe after they start charging for it you'll be able to get longer responses okay finally um I asked it to do all right to save some time for language teachers so I asked to create a vocabulary quiz for an intermediate learner of English and the words it chose for this quiz so you can see it's happy small hungry hot fast early loud dark those are all reasonable words to ask about but this doesn't work as a quiz so for example the second one which word means the opposite of small a large B tiny C huge D enormous well I think either large huge enormous any of those would be would be a correct answer and looking through the rest of them um number five which word means the same as fast well we have both quick and Swift as answers um and so it looks like a quiz it seems reasonable the words at least fall into the same semantic categories but it doesn't isn't able to narrow down the answers to to just one correct answer um then I asked it or I did the same thing let me make a quiz for a beginning learner of English and some of the words are the same actually but these are all very basic words that it comes up with so yeah so it does seem to have some grasp of the level of vocabulary I asked for a quiz for an advanced learner of English we get Tranquil miserable vivacious tolerant well those are all more difficult words and then I asked for now make a vocabulary list for a highly educated native speaker of English and we get words sort of in America that used to be called s-a-t words Insidious incredulous irony prudent ostentatious Lucid arbitrary um and so it for making quizzes it doesn't work at this point maybe if the prompts were changed it might you might be able to Inc you know get it to to offer only one right answer but it is able to choose vocabulary items of various levels based on a very very simple prompt so there might be other uses I haven't thought of yet for using chat GPT for preparing um materials for language learners okay well this is still a very preliminary test that we've done so far and so it's too early to draw any conclusions about whether chat BT will be able to evolve soon into a fully fledged language learning support tool for individual language Learners or whether it's going to throw a lot of language teachers out of jobs but um it has that potential I think it has a it has a potential um to to Really revolutionize language learning and maybe eliminate the need for language teaching depending on how well developers are able to integrate it with um sort of human-like interfaces and and sticky sticky applications um I once again I was especially impressed by the way it was able to um disambiguate to use a default code word disambiguate um polysemous words in other words to to um identify the meaning of how a word is used in context and to create example sentences for particular meanings of of multi-meaning words um that was quite remarkable so that suggests that there's a lot more that it can do um as we figure out what kinds of prompts are needed figure out what kinds of tasks you can do I'm sure there are many other things to try and so I encourage language Learners language teachers to keep trying it if you have any bright ideas share them with me share them with other people through through social media and your other contacts there's still a lot to learn about what this uh remarkable and somewhat frightening tool can do so thank you very much for listening

Channel: Tom Gally

Views: 32,187

Keywords: ChatGPT, language learning

Id: l41hZLRsDos

Length: 30min 47sec (1847 seconds)

Published: Wed Dec 07 2022