ChatGPT and Education: Threat or Opportunity?

one good afternoon and welcome to another in the faculty of Science and Technology science for today series uh in November 2022 open AI Unleashed upon us a product called chat GPT which is really a front-end a large language model called gpt3 this program is capable of responding to questions that would normally be put to humans and it has demonstrated an amazing capacity to even create new text from simple prompts of course the ability of it to respond to these kinds of questions has created quite a stir within Academia because of all implications for student learning essentially how do we know that what the student turns in isn't something generated by chat GPT for example it's that good so this afternoon we'll be exploring the impact of this new tool on how we teach how we assess and perhaps what we ought to be aiming to teach in the future uh I am Daniel core professor of computer science in the department of computing at UE Mona and I am joined by a very capable panel this afternoon I have Mr Matthew Stone are you a PhD student specializing in Ai and he's going to tell us all about how those how large language models work we have Dr Wayne Wesley CEO and Registrar of CXC and we're going to hear from him how he sees and CXC sees what uh chat GPT represents uh we have Dr Sean Miller uh U Electro in the department of computing and we'll hear from him about what possibilities there are for chat GPT within the discipline and we have Dr Ruth and Baker Gardner a U.S lecturer in the department of library and information studies and we will hear from her on what the implications for academic Integrity are so a wonderful spread of um contributions and we certainly are looking forward to hearing um in in great depth and towards uh what should be a fulfilling and uh productive discussion and conversation afterwards uh to begin I'll invite Mr Stone Matthew would you please share your presentation while I read your bio so uh Mr matthewstone is the CEO of his own startup Stone Technologies limited he's currently pursuing a PhD in computer science in the department of computing he's re researching an area called reinforcement learning and trying to make it more efficient one of the relevant fact series that reinforcement learning is actually quite pertinent to how the large language models backing GPT is created so without any further Ado thank you uh Matthew for being present um please proceed thank you professor all right um good afternoon everyone so I'm guessing a lot of you have learned I have heard about chat gbt in the news and some of you have probably even used it but a lot of you probably are not exactly sure of how it works so this presentation is going to take you through the history behind chat GPT where it's coming from and hopefully give you a clue as to where and the technology is importing chat GPT is going all right so where are we coming from first of all let's talk about what is track GPT so we can understand what the chat in the chat GPT stands for it's a conversational agent but a GPT it stands to generative pre-trained Transformer now most of you probably hearing that have no clue what that means so hopefully by the end of this presentation you have some clue what that means but essentially a generative pre-trained Transformer is a large language model and what essentially it it's trained to predict a likely word from a sequence of words so here's an example so for example if I had I like cats more than if I were to ask you most of you to fill in the blank for that statement most of you would say dog essentially what the large language model does it gets a lot of data and it tries to essentially given a sequence of word um predict what the likely next word will be so how does it do that well all of this is coming from the problem of sequence modeling so essentially given a sequence of tokens or sequence of words can we learn to produce that said sequence so for example a sentence how do we essentially generate that sentence starting from some starter word so for example if we started out with the word what what's it like what's the likely word that can come from that so for example time then is and then it and then of course a question where now from the early days sequence modeling the first model that proved very popular for doing this with something called a recurrent neural network essentially a recurrent neural network works by taking in uh the input from the sequence as well as it keeps track of something called a hidden State think of a hidden State as sort of that thing that keeps track of the history of what it's learned so far or the the set of inputs it has received so far so it uses all of that to be able to learn a pattern of how to generate um further words or tokens in the sequence now the problem with these recurrent neural networks is that they were not so good at remembering large or long sequences so if the sentence was too long it would struggle at essentially knowing what was said uh further down in the sequence so some improvements were made on this model most notably something called an lstm essentially to put things simply terms it works by or getting some of the input and essentially just remembering the important parts of the input so that way it doesn't have to keep a track of the whole sequence but only keep a track of the important bits of the sequence that it is needed to predict the next likelihood now this led to some improvements but it still struggled with predicting long sequences so here comes attention to the rescue so essentially attention works by placing focus on the parts of the input that are essential to predicting that next sequence or next token so essentially instead of looking at all the the tokens in that sequence are all the words in that sequence you want to look at the words that are most important in figuring out the next word so we have what's called attention now in 2017 Google developed this model called the Transformer so in their heads they essentially did away entirely with the recurrence model and simply used attention models alone now this led to the development of the Transformer now here's a picture of the Transformer architecture I won't go into detail as to exactly how the architecture works but essentially it works on the same principle giving a sequence of words it learns the place importance in that sequence on the words that it's required to generate the next word in that sequence no GPT only.com about essentially GPT was fusing a Transformer with all the text and the data from say Wikipedia read it on the internet so it they essentially trained this large uh they essentially created a trend a large language model with um these Transformer models and it took all the data from the internet and from that it was able to essentially learn to predict all the texts that it got from that now essentially from here it led to a big race in terms of creating these large language models using Transformer architectures so essentially what they did was they kept growing the size of these transform models so went from gpt2 which had about 1.5 billion parameters so these are the actual configurations in the model essentially the idea is that the larger these models the more complicated sequences they can model and the better they should get at the results they produce now this led to gpt3 I think this came out around 2020 durable now this had 175 billion parameters now with the release of gpt3 this actually led to a lot of amazing things being able to be done like this large language models so now instead of just being able to generate texts now they can generate code if you ask them to so this is an example of it actually generating some code and some of these Technologies are actually being used in coding tools that programmers use so this is actually a GitHub co-pilot I can also write essays so this is an example of just asking a question so you give it up prompt essentially how this language models work you give it to a prompt it uses that prompt as a form of context and then it's is able to predict from that prompt um the set of words or a sequence of words that is most likely to respond to that problem now they've actually used GPT to even pass exams so they've used it on the bar exam they've used it on the GRE they've used it on the SAT they've used it on AP exams I'm guessing someone by now will be trying to use it on the C6 or keep exams um these languages large language models are also actually capable of creativity so uh some of the members from even our own faculty most notably Dr Man Singh myself and some other members from faculty of law we did a project a collaboration with a university in London City University where we were looking at how AI would affect copyright it's most notably in the Caribbean space and we actually this was back from the pandemic days we actually use GPT this was when it was GPT three so we use gpt3 to see if we could generate poems in the style of Louise Bennett so on the left was a poem that we generated uh in 2020 so this was using gpt3 as you can see it does a pretty decent job of kind of understanding how pathway is used but not really good but on the right is actually using chat GPT so as you can see the output is a lot better and it sounds a lot more realistic now yeah so now you guys might be wondering obviously these language models have existed for some time now why is it that no in 2023 or 2022 is it getting all this hype you know they've existed for some time you know what's that secret sauce that essentially made it this good right now most of you have probably interacted with chat GPT all right so chat GPT is the latest iteration of the GPT model so chart GPT the original GPT was based on GPT 3.5 so that's a slightly bigger GPT model but the problem they realize is the output that GPT outputted was at times it did not feel human-like a lot of times it felt made up um a big limitation with a lot of these models is that they hallucinate so they make up a lot of stuff now essentially what they did is uh they paired that training of the language model with something called reinforcement learning with human feedback essentially the output that the language model gave was scored by humans and essentially the language model used this feedback as sort of a reward and the ideas with reinforcement learning is you're kind of using this word as a signal to be able to now produce outputs that are going to lead to higher Awards but this essentially ended up doing is it gave outputs that were more human-like and a lot better so I think well I think this is the uh the main component that made the chat GPT the way it is right now so if you actually use chat GPT uh the interface looks like this you're asking you ask it a question or you give it a prompt and the main ingredient of chat GPT that really I think has given it the um the sort of hype it has right no is the fact that it outputs its information in such a way that you know almost anyone can understand it so a child and it all puts it in a very structured and user-friendly way now what are the limitations with these models now the thing with these models based on how they work now remember you just essentially give it a lot of text from the internet and it's able to learn patterns of how to generate this text so with some probability it decides what the next likely word would be so there's a chance that the model will make stuff up in fact I've used it before to um even do some research and it's literally giving me fake references um to papers that don't even exist so I would caution a lot of people using these language models to ensure that you check what it outputs because it's likely there is a chance that it has made it up and there are many examples on the internet where these models have literally um hallucinated facts or just gave the wrong output it can also be confidently wrong so even if it literally gave the wrong answer you can check the probability of how right you think thought that answer was and the researchers found that it was very confident to not answer so you know that's also a very big problem it also has a problem of possibly outputting hate speech racist remarks um harmful speech I remember it's actually getting data from the internet as we know data from the Internet is wild wild west we have hate speech on the internet we have racist remarks on the internet harmful speech on the internet so it's likely that the model will output data that it has seen on it so combating that they've used several tools so they have an API called the moderation API that actually is able to filter out the speech which is why when you try to actually prompt GPT to um you know output some racist remark or even um you know say something bad it will tell you that I'm an AI model and I cannot do such things um also I think this is actually one of the biggest uh problems with these large language models it's a fact that they can reflect the human bias of the creators so remember they're trained with human feedback now the human feedback can have some bias to it so if a human says this answer sounds better than this answer you know why why was that the case may be the case that the human had allowed some of its bias to be reflected in that choice also it's not real time so chat GPT was trained on data I think up to 2022 so more than likely if you ask it for information on current events it will it will not be able to give you the answer to that now where do I actually see these language models going so as you can see there are several problems with them so where do I see it actually going as we've seen well what I've seen in research is they're now moving away from just being able to produce text to also being able to understand other forms of data so images audio video the idea is the the overall goal of AI is to create something called an artificial general intelligence so essentially a future AI that should be able to do any tasks a human can do or even better and of course if it's going to be able to do that it should be able to understand not just text but should be able to understand um images it should be able to understand audio and video so Google recently introduced um Pawn e which was a large another large language model um based on the same Transformer architecture that was able to use not just um text but can also use images not only that it's able to actually generate commands that can be passed to say a robot activator to carry your tasks I think that is a very big achievement um I know Google is releasing something called Bard AI and there are also other language models Facebook is also releasing llama which is a smaller model than GPT but based on the researchers saying it's also it's the results is actually competitive with the large language model so I think going forward in the future with these large language models we're going to see more and more multimodal models then gpt4 dpt4 actually is also multimodal so you can actually prompt it with images as well they're gonna try to go bigger and bigger and bigger frankly I think this is a mistake I think there's at some point you're going to find diminishing returns um to how um much improvement you'll get with bigger Nets but it seems to be the general consensus in the field or the general approach especially among these big tech companies to try to get bigger and bigger Nets um but I think the technology needs to move to a more hierarchical form of learning so um a child isn't going to be taught about Einstein's theory of general relativity before learning Newton's three laws of motion right so and AI today does not learn that way AI is literally you just throw it um information it's not it's in no hierarchical fashion it's literally random order which is why I think it does hallucinate because if it did understand if it actually understood its output it wouldn't um fall prey to a lot of the errors that it does right now so I think more research is going to be put into that in terms of being able to learn in a more structured and hierarchical way than how it is right now um I guess then the presentation I'll just make some of my opinions known on the education factor I know a lot of people have been kind of worried about how these large language models will affect education you know will it lead to more cheating but I think there are more benefits than harm going forward so I think generally the education system is going to have to move away from the standard Industrial Age style of Education so instead of rewarding root memorization and regurgitation we're gonna have to start rewarding more the ability to create and think out of the box um I think assessment is probably going to have to move from the typical take-home projects and assignments to more real-time in-class discussions such debates I think it will lead to more tailored learning so the this is one of the limitations of the large language models right now um they're not tailored to the actual user using it I think more of them will start being tailored to the user using it so the chat GPT that you use may be different from the chat GPT that I use based on the experience it has had with me so maybe when a child is using it it will actually respond differently or um output different text than let's say a professor or a lecturer uses it I think it will democratize education so you know it doesn't matter if you're in a um yeah it doesn't matter if you're not you don't have the ability to go to a certain School everybody would be would have the ability to learn just the same um I've actually spoke with a lot of teachers that are actually using it to assist with lesson plans and um coming up with problems but I think overall GPT or generally um any generative AI will increase human productivity I've even used it in my own daily life to draft emails um you know help with research you know anything that involves me having to write um I've used GPT to help me with that um arms disinformation misinformation so it's a powerful tool with using the wrong hands it can lead to a lot of um ill effects so it's easier for Bad actors to spread misinformation I think it's going to get a lot harder to tell real versus fake going forward so you're not going to know essentially what what's real versus what's not real um for example looking at these images if I were to ask you you know which one of these are actual real people versus AI generated people yeah most of you would probably struggle at telling me that um so I think I will probably end it here um unless yeah I think that's it got any questions thank you thank you very much um Matthew Mr Stone all right um we will um we will have time for questions and discussion afterwards I want to allow each presenter to go first and then we'll have hopefully what will be a robust discussion afterwards um uh Dr Wesley uh while you are bringing up your slides I'll just um while I read your bio could you please um prepare your presentation for sharing uh Dr Wesley received his PhD in industrial engineering from Florida State University he's currently the registrar and CEO of CXC he has served in the past as the executive director of high trust NTA chairman of the Caribbean Association from National Training authorities senior lecturer and program director at UTech and a board member of ucj he's also currently serving on the CARICOM new school model committee and the Caribbean Community administrative Tribunal so plenty plenty of experience in the education sector welcome Dr Wesley we look forward to your presentation thank you very much Dr core it's certainly a pleasure for me to participate in today's event during that this year's CXC is celebrating 50 years of existence of service to the Caribbean people I think it is fitting that we are also a part of this new and emerging area with respect to assessment within the region and so my presentation will try and look at the threats and opportunities that exist the previous speaker really did go into some fundamentals with respect to chat GPT and I would necessarily react those uh he also went into some of what I would say I'll just elaborate a little bit more on them and hopefully we can get the time back in in question and answers so good evening all and those listening right across the Caribbean thanks for sharing this moment with us I will start first by talking about the intelligence artificial intelligence and really as indicated before it uses existing knowledge and I emphasize here whether factual or not to generate a creative response to an inquiry and whatever that inquiry might be I'm focusing here another first Speaker indicated it will create information as well and so it's going to be important that we pay attention to what it has to offer and how that information is generated for use according to them I just want to quickly look at the evolution of content creation because AI is simply another step in how we utilize information that would have been created and using that information to solve problems and so we have started from the early days the stone tablets that you have you know the Ten Commandments were written and tables of stone and so tablets have storms so we know that information was created in that form as we evolve and the Bible tells us about knowledge and that knowledge can increase the increasing that knowledge they are also increased in the way that knowledge is actually being uh preserved and so then we move to the Scrolls and manuscripts we move to books and books create libraries and libraries create the internet and internet created Google and when Google came on you know all of us got used to the idea just Google it knowledge has been transformed and created in such a way that pretty much anything you want to do now somebody will say to you go Google it something is wrong with your laptop you can Google a video something is wrong with your furniture at home you can go and Google a video something is wrong with your car you can go and Google a video and you're given information that you can now use the song your problems Google assignment is given persons will go and Google so what we found when Google came out it individuals would desire less to go to the library as we know it in fact the library started transformation into an online database in preparation for what we have now been come to to accept the chat GPT which is not what we are grappling with now chat GPT takes that information whether factual or not and prepare a response as indicated before much as a human being would do it mimics our cognitive capabilities in generating a response and I pointed out before it can do that if you're an artist or if you're a singer whatever form uh it can actually mimic that language form and give you a response accordingly but even as recent as chat GPT guess what has happened already now we have GPT 4 open AI which was just recently launched so now the creation and how knowledge will be utilized it's already changing we we are not yet even uh fully died we are not fully digested Chuck GPT three but chat gp34 GPT 4 is already on us so what is this saying to us this is saying to us that content and the availability of content and access to content will come in varying forms and that will present to the education system a particular challenge in this article from the BBC News actually this particular point was made and I think the previous speaker actually alluded to it however like it's predecessor this is chat GPT for now open AI has worn that chat gpt4 is still not fully reliable and may elucidate a phenomena where AI invents facts or mixed reasoning errors so we have to be very calm and ends even if we begin to incorporate such within the education space we have to provide an environment in which we allow for the errors that are inherent in the current artificial intelligence software are mitigated uh through the provision of support so having said all of that at the Caribbean examination we are still uh not averse to the utilization of AI in fact our particular position is that AI must be embraced technology must be embraced but as we Embrace that technology putting sufficient Safeguard to protect the Integrity of the teaching learning and assessment process suspect guard against plagiarism because you know that as pointed out before if students are using this to generate their term papers or whatever then we have to make sure that there is a mechanism in place to State God against this from happening at the Caribbean examination counseling within our own uh Regional space as we administer assessment we would have developed our own quality assurance mechanisms that built into it our own plagiarism policy in fact we do have a zero tolerance once it is detected and our double layer quality assurance system where part of it is done at the Caribbean examinations Council and parties done within the school system and a validation processing food to ensure that the work being done and produced is that of the students so all our subjects are actually would be subject to this policy position and provision that we that we mechanism to ensure that we guard against plagiarism so this is going to be something that is critical as we seek to protect the Integrity of the teaching learning and assessment process we also have to be very careful that we it doesn't facilitate what I call intellectual laziness in other words begin to become overly dependent on the AI in generating and providing them with a response and it will now cause for uh underdeveloped cognitive uh competencies and and cognition that would be required for one to interact with with the content and material that they would be taught and so we have to put insufficient Safeguard to ensure that the competencies that we want students to display are actually embedded in the kind of environment that we create for teaching learning and assessment and the other safeguard that we have to put in there as I pointed out before it uses factual information and sometimes and information that is not factual so we have to have a process in place that will assist in the verification of that information the ability to test the veracity of that information and that's where uh teachers and educators will have to be a bit more Vigilant in their uh in their supervision of what obtains in the classroom so these are the three quick areas I want to also contemplate as it relates to also embracing AI but implementing the represents safeguards to ensure plagiarism is not something that becomes a reality or intellectual laziness it uh would have been well the development of intellectual business and our ability to verify the information so what of the impact on teaching so the first one here we know sometimes within the teaching profession do difficult to find content and material to help supplement their subject matter that teachers are required to deliver and so in in in their preparation teachers can use the chat GPT has an additional source of content material of course the represent verification that the information is correct and this will help to enhance the teaching learning process so when we talk about additional about the subject for is the teacher is perhaps needing an example of an essay on example of of what is right from what is wrong they can use strategy to generate that content and then engage the student in a process of analysis inquiry comparative assessment of what is being presented whether or not what is being presented follows a particular sequence and addressing the challenges that they're trying to address in the problem-solving situation that they might be exposed to so there are a variety of ways that can this can help to supplement the content material for the subject being delivered where there are challenges in identifying suitable material for learning there's also the need here for teachers to keep our Educators in general to keep abreast with technology technological development because GPT has come now and it has demonstrated that it has the ability to do things where knowledge exists about it and also now showing us that if the knowledge does not exist I have the intelligence to just create it and make up these facts right trying to think like a human being so teachers themselves will now have to evaluate how how they utilize this information and be very much opening with what is happening in the technology space because students are moving ahead they find this enjoyable and interesting and intriguing as they interact with with the technology it should facilitate dynamism in delivery approaches teachers will have varying ways of using this you can actually give students assignment just to go and use GPT to see what it comes up with uh on a particular topic they are discussing in the past and then when they come to class they have varying material with which they can interact and determine whether or not uh what is being said is accurate and how we can determine if it is indeed um suitable for the content they are covering so there's several approaches that can be utilized so teaching will be impacted but it has to be in a controlled environment and later on you see what I'm talking about it requiring greater vigilance from teachers now the impact on learning here the need is going to be to focus more on competencies with appropriate content coverage because as you can see content and knowledge you can get that pretty much anywhere so there is no need again for students to be regurgitating knowledge because knowledge is available the it now is for us to place greater emphasis on the competencies we want students to acquire and Achieve as they progress through their learning experiences and encounter within the classroom environment and so whatever we're doing now we now have to give greater focus on competencies being Acquired and through the development of crucial competencies can be facilitated like I said before depending on the topic being discussed we can generate essays from that you can generate sample questions from it in fact we have done and we also did it with our chemistry multiple choice items and it got the the gpt3 got all law multiple choice questions correct we put that in and it got all current with the chemistry paper we put that in and it's like one incorrect and this then brings up another Point uh and I asked one of my psychometricians to have done this analysis for me as pointed out by the previous speaker as well it has a challenge in some even though you saw the face recognition capabilities it had some challenges some challenge recognizing some challenges recognizing the symbols which were that is why why a chemistry question incorrect in fact in generating Arbitrage question it actually gave plausible explanation as to why that answer was correct and so that in itself can become a tool teachers utilize in helping students to understand and learn certain Concepts uh quite well no of course when you give it some other things it gave uh answers which were not plausible which is what we talk about it giving answers that are not correct in generating its own responses and and prayers that we want to call them its own alternative facts to keep it modern technology but the the idea here is that it has some capabilities and capacities and we have to find a way to treat with those capabilities to enhance learning search the information I'm not readily available chat GPT can help in the learning space or that and certainly it can facilitate independent environment and firing by Learners and teachers as they Seeker expand their own knowledge base and information that exist on the internet so there are opportunities then for us to rethink and that is how we are recalibrating our assessment at the Caribbean examination Council to give greater emphasis on the on the competency students are required to display and less so on content we are not saying content is not important content is important and very valuable because it is based on with certain decisions is made but we will no longer focus on regurgitation of that knowledge called right because we are now focusing on how will students interact with the information give to them to solve problems so on [Music] the thinking excuse me to generate arguments in a logical sequence becomes important and finally that the impact on assessment I just related to you the fact that we have put into the test and it would have done a certain sorry it would have done certain assessment and this would work but what it means therefore that as we move forward in the design of assignments and assessments we have to give greater freedoms to formative evaluation rather than summative evaluations and the formative evaluation allows us to assess students competencies as they as they progress through the education system and acquiring these competencies along the way we can validate them as we go so it will require greater vigilance regarding the design of assignments you can't just give another time you know that can be generated uh by an artificial intelligence system and now we have to think carefully as to how we incorporate that in our operation it will require greater level of this of the abilities of our Learners so teachers will have to have greater level of interaction and and um dialogue with their students to ensure that they understand at a deeper level the students capabilities to perform and deliver certain work and from the summative point of view it's throughout the year if you are tracking the progress and watching how the student is performing you will have a greater chance of mitigating the impact of chat GPT and how students perform within the school system I must stay at the Caribbean examination Council none of our assessment really a person would have access to chat GPT to to assist them in generating our final paper so that is not necessarily a trick our final examinations are done under strict conditions even our e-testing examples where students will not have access to any other device while interacting with our assessment platform our family examinations are also done where students will not have access to to online devices or any such internet access to for us to have that impact of chat GPT impacting their performance our SBA are also not summative our sbas are actually a formative assessment that should be executed over the years rather than at the end if that is being done and if we fought formative of chat GPT would be minimal and certainly the utilization of a progressive assessment framework that will now see us checking in and validating students performance as they progress throughout their learning engagement within the school system so as they progress through the school year we are assessing as we go and that way you a student at the end of the year don't suddenly provide an essay that is beyond the students ability that you would have uh become accustomed to because you were interacting and finding out monitoring the students progress throughout the year and of course a formative assessment should yield greater benefit to the student as the student will benefit from continuous Improvement and opportunity to improve their performance as they move throughout the year ladies and gentlemen it's going to be quite an interesting time I think the technology is here for us to embrace to be utilized but we must put in the requisite safeguard that will allow our teaching that will protect the Integrity of our teaching learning and assessment processes and that means that there are some policy implications for our Educators across the region with respect to teacher education capacity building and utilization of Technology within the teaching learning space ladies and gentlemen thank you very much for your attention and I'm big more than willing to answer any question at the time so appointed thank you very much for listening thank you so much Dr Wesley wonderful I'm actually very very very pleased to hear that CXC has taken a wholesome view of chat GPT not one of Let's bind this thing because we can't control it which we've heard some school systems respond in that way and I think the approach that I'm hearing coming from CXC is certainly the way to go let's not fight what is inevitable let us figure out how we're going to make it work for us and and and enhance everything that we do um I had said that I would have all the presentations first but actually we've been getting some questions from the from the the uh live feed on YouTube and so I thought perhaps uh it might be more useful actually to field some of those questions directly so um I'm gonna pause between the presentations to actually do that uh the first one is for Matthew um came from Revel Chambers and this one is asking what I'm stage prompt has entered over time I was discarded to those I'm adding are they incorporated into the learning of the model so um the problems so they're actually saved so you can actually if you use GPT you can go back let me turn them again yes so if you use chat GPT on the left there's a option to actually go back to your old problems it's actually not used to fine-tune the model in real time so it's not like based on your prompt it's going to you know the model itself is going to get better um what it will do is just if for example you go back on an old conversation it will just use all that information and continue from there so it's not like itself it's gonna improve in real time interesting I said actually we'll save that state and resume it yes okay all right I heard about uh interaction with Bing that it became rather touchy and they decided that the history was too long so is that part of a challenge here um yeah being I think the being AI issue uh it was it was essentially gpt3 but um or gpt4 really but with little content moderation right um actually if uh open AI did not uh heavily invest into that moderation side of things uh GPT would be outputting a lot of very hungry all right let me move let me keep moving uh Paulette McFarland asks is it able to understand facial expressions and I presume should be talking about gpd4 now because gpg3 is text only right yeah GPT um the original GPT was based on GPT 3.5 which was only text input but GPT 4 they're moving to being able to actually input images as well so right so the question is about facial expressions can it understand those I've never tried it with facial expressions but I'm guessing it it probably should be able to uh okay um this one from Marvin singer much she had asked which of those images that you showed were actually AI generated I guess at this point they're going but all of them had a feeling um and and okay I think I'll save the plagiarism question there's a question of plagiarism um is it is it please resume if you input the data and synthesizes um I think I'll say that until Dr Baker Gardner speaks and that can perhaps be saved for the end um uh I'm seeing some endorsements of the ideas that uh Dr Wesley presented says perhaps this is Elizabeth Montoya steaman perhaps you have to provide students material and have activities where they can discuss the idea ideas and I that you were actually um seeing some of those um very points Dr Wesley um Dalton Lewis asked how an illegal framework around this be crafted from a local perspective I'm not sure who might feel that question um but I suspect it was perhaps aimed at Dr Wesley because it was while he was speaking well and that's why I ended the presentation earlier about the policy imperatives that will have to be addressed in all of this because the issue of figurative and the fact that chat GPT doesn't give credit necessarily I think perhaps it's something they will be working on it doesn't necessarily give credit as to where it generated the idea from uh will cause some concern with respect to intellectual property and the utilization of certain phrases so framework then we'll have to address to copyright issues and within the within the context of the proliferation of knowledge who owns the right to information that is why they're available on the internet and all right ensuring the appropriate recognition is given to wherever we have received the source of that information yes and that's going to be a um an issue so when I I want to allow Dr Baker guy to see what she has to say or with regards to that before we go into a full-some discussion and I probably should not post a load that question to impose first at this stage um but this question actually was perhaps more specific to you um Janet Johnson says I have an essay to write can I use chat GPT and wouldn't it be penalized as that's what one of the things that I've indicated is that teachers will have to be very familiar with students ability they have to develop a greater level of discernment interaction with students to know what they're capable of and what they're not capable of so and submission of a piece of work should suddenly drop somebody from the all the way up to if uh if it is that person showed certain deficiency but that being said of course if one utilized GPT to at least pray about something for the student to do well the student needs to go and do some verification is the information correct because once it is presented to the teacher all content will be verified that all of those things will be checked to to put forward is actually accurate and not a fallacy right or information created that doesn't really exist anywhere so from that standpoint we just have to be very careful and student and that is why assignment now should no longer be just the turning a term paper because if that's the case there's something somebody who can write it right exactly in here is competency that I pointed out before the aim from a teaching perspective what are the competencies students are displaying it's not just upgrade now and exactly fun competence that we have to now design things that pull those competencies to the floor right and and I think specific to this question perhaps that entirely comes down to the interaction between the teacher and the student if the teacher said please do not use chat GBC and you do it then you're a sort of in violation of the code of conduct that the teacher set out but I think your words were sort of to encourage schools and teachers and Educators to find ways to embrace it and incorporate it in their teaching and hopefully the learning of students and so you know I think shorthands to that question it depends on what the teacher said and if you teach us a no then perhaps maybe encourage him to find ways to incorporate exactly I simply Daniel understands if it is that you you can interrogate the students as a student about the piece that they have written and you will know whether or not because if somebody really was intimate with that document when you begin to question them they should be able to articulate exactly what they would have expressed in the paper without it exactly and that comes back to what we're trying to assess when you ask a student to write a paper what exactly were you hoping that they learned and when you assess what's your opinion and we're going to get to that in a discussion but um okay so one last question Progressive assessment will be based on the school's standard or cxcs I think that was speaking to your last slide [Music] that you want students to acquire as they progress throughout the school years there are several competencies now uh critical thinking problem solving analysis you know all these things once you begin to lay the standards for those then assessment of the students will be against those standards as they interact with the learning material that would have been provided to them so it won't be a school day standard because school day standard you cannot control them remember now as a regional examining body it is important that we maintain standard so that whatever is done in Trinidad or in Diana Jamaica uh Saint Lucia Dominica Barbados sorry Grenada we should be able to stay this standard is the same right across the region so it cannot be school specific or country specific it has to be driven by what the international standard for these competencies are an assessment done against those okay wonderful and I suspect we'll have more to say about that after all the presentations have been made at this point now I'm going to invite uh Dr Miller while I read his bio please go ahead Sean and begin to share Dr Miller received his PhD from UWI Department of computing he is currently a lecturing Department of computing as an AI Enthusiast with a healthy Obsession in the process of creation he's driven to discover New Paths that will have impact on the world and improve the lives of humankind straight from his bio not my words uh welcome Dr Miller we look forward to hearing your presentation afternoon everyone so so my presentation will be an adjusting to a large language models in education So currently I'm lecturing at the University of the West Indies and where I teach the first year programming courses and Cha GPT has really upended some of the assignments and want to talk about how can we actually get into some of the examples of how can we adjust to learning large language models as it relates to education nice so the fact is large language models are here to stay this technology is definitely rapidly um advancing as they say in the previous um presenters they're rapidly advantage and has the potential to revolutionize the way we actually learn and communicate and pass on knowledge not only are they here to stay but students will have access to them whether or not they are abandoned the university grown or whatever High School the students are at they will have access to it and they will use it so it can either be an amazing tool to supercharge the learning process by if Educators incorporate them into planning how they plan the lesson or can be simply a tool for cheating if we we don't take them into consideration when we're developing course material now the outcome depends on whether or not Educators such as myself addressed to this new technology and there is there is actually much benefit to adjust into this technology and finding ways to incorporate it into the teaching but what can we do really well we need to to see it as a tool to support learning as the previous presenter made very clear we need to see it as a tool that we helped the students to dig deep I help the students to understand um how different topics relate to each other we see it as a tool all right to support the learning are not simply as a mean for up to get easy access to assignment answers so they have the key they have capabilities that can be of great use in the classroom right is able to explain complex topics in simple term I find that very useful when doing certain lectures I want to bring it up bring across a certain topic that students might find difficult to say all right let GPT find a way to explain this in in the like as though it's relating it to a five-year-old and when it does that I who know the material would see if it true and see if it actually makes sense and then present that and say okay it's a generate ideas right there it can engage in conversation around that specific topic and does all of these in clear human language so there's much benefit here and we can leverage these capabilities to to enhance teaching if we take them into consideration when we design the course materials so most of the adjusting obviously will be on the end of the Educators and for large language models to to be a tool for teaching Educators must make large language models work for them so we have to make it work for us it's it's not as pretty dressful mentioned just as simple as Banning this thing and if it's going to work for us we have to figure out what it can and cannot do for our specific domain so whatever so our individual teacher individual lecturer would have to interrogate our put his or her material through GPT to see if it can just simply generate the correct response so are we checking for whether or not the student gets the correct answer or are we checking for the process of how they got to the answer can it explain you have to we have to figure out what it cannot and can and cannot do as it relates to our particular domain and also learning outcome that we desire and in other to do that we just have to load up GPT and see what happened for my for me I teach um computer programming for the first year courses and chat GPT is very good at solving some some of the simple programming problems I mean the other day we were looking at some of the programming problems that we usually give before chat GPT and just just testing I just cut on paste the all the questions without reading it and chat jpt actually generated code that passed the test case because the online platform that we use we simply we it simply grades the code that students enter by itself and when I started to say all right cool so we need to we need to change next semester we cannot be doing questions like this so yeah that's work the next thing we'll need to do is we design course material with these models in mind so after we find out exactly what it can and cannot do we need to design course material with this new tool with large language models in mind because definitely if we don't do that then the learning outcomes won't be achieved in students will simply use it and the truth will be revealed during the examination because they won't be able to use your GPT then no what it can do will be specific to to my course domain so I am doing programming courses so I'm looking all right cool what can this do so for me it can understand a simple problem and produce an answer so right it can write simple code here it answers a simple programming question to find the average of the absolute value of three numbers so this is very simple now this says to me as an instructor who wants students to understand Python and basic programming for themselves that this is not a good assignment question to give this is just asking all right just write code here to do this this probably not a very um good indicator or whether or not the students understand the language understand programming understand python itself but it can also explain the code that it wrote so it goes ahead and explains the code so this might be integrated the explanation of the quotes all right right code are generic code that does this and it's and explain it so this might be integrated into an assignment or tutorial as a means to spark conversation similar to what was um said by the previous um presenter a more of a tool that helps teachers and and students to have deeper conversations about things that they would normally get into um during the lecture of time because we will find as we as we we teach is during the tutorial sessions where students have the ability to really interact and ask questions that's really where a lot of learning takes place and chat GPT could actually be a tool the large Dragon run can be a tool that facilitates students get into that place quicker by having assignments like these all right cool what it cannot do so this is equally important we need to to know what it cannot do right it cannot be relied on to to get the answer right all of the time sometimes it just gives the the wrong answer and as we said it's it's very confident in the incorrect answer um I find also that you struggle to solve problems that require layers of knowledge instead of just what is uh instead of just a what is or write this program to solve this problem cool so this is an example of it's getting something terribly wrong now uh ask it to write uh to find the error in this particular function and it very confidently layered a description of what the error is and here is the python code to solve it but when we check the actual prompt I saw that it actually just it was it just regurgitated the original question and made software as it relates to why what the error was so this is clearly incorrect right here now to be fair I really didn't give it much of a prompt by addressing find the error in this function but that could actually be a question right here for for our students to to be able to drive so that's that's interesting we get to some more after so the second thing is that we need to design course material with these models in mind in mind right we need to so after we understand the capabilities so based on whatever course or subject that is being taught we understand all right chat GPT our large language models can do this you can do this but you cannot do this and it cannot do this so there there right there is the work for for the educator to find that middle ground where all right stop gpe can do this so how can we use it to get the students to a point where they will engage with the material and also provide assessments and assignments in the gray area where gbt our large language models cannot really get the work done I mean also it it can help teachers to break things down so a scene right here um this is actually one of the examples that I use when teaching um one of the core the courses that I do it's it says explain icmp floating to a 10 year old and what it does it really breaks it down in a very interesting way um very simple terms that can be easily related and sometimes this and this is a asset for me personally as when going through the lecture slides and trying to break things down um I might break something down but to me it's clear what the students not clear using GPD can help generate ideas on how to break down a certain content a certain topic or a certain term and you can do it for 10 year old or five-year-old and it really it really does what it says cool and one the next thing it cannot be relied on to to get things right all the time so this was a prefix example but I realized that hey I just told it to find the error in the in this code or giving it much context so just by knowing what it can and cannot do I did that but when actually updated it and give gave it some more information it actually got you right so there is probably a a sense in which if the student can actually come up with a half the at a decent prompt to allow GPD to get it right they probably understand enough but is that enough for for our learning outcome that will have to be decided based on the type of course and learn nocon of course so far example right here we gave it more um information so find the error in this recursive function that returns seven win x equal to and y equal to 3 it should be six and then we gave it the same function and it actually gave the the correct answer cool yes so setting a question around the the use of large language models like this may may look something like find the error in this code and explain the solution well GP can explain the solution as well but the student will have to be able to explain the solution and his term he's our her term and in his or her own voice and that might be the way to to assess if the student is learning what the the Stu the lecturer or the teacher wants cool and as I said educators will have to address based on what they teach so right here we we saw where we asked you what did Aslan allow why did Aslan allow Edmond to come back even after Edmund betrayed him to the white which this is from the land which on the Wardrobe so it's a literature book So based on what you teach you can determine all right what can gbt do and what it cannot and cannot do and this is me following up with different conversations so it's really under the educator on the lecturers and the teachers to to know what the learning outcome is be very clear on what the students should be able to do at the end of this course at the end of this class and then design materials in such a way that your gbt can be Aid and not a hindrance so that's what we have for you today thank you any questions uh thank you very much Dr Miller um I actually very much like that you're identified things that it can't do things it can do and that somewhere in between it's possible for a student to get it to do something that's it perhaps gets the right answer and the student had to know a little something and there's a open question is that enough is that enough to meet the intended learning outcome and that's I think something that we're going to have to struggle with as Educators how do we balance um the questions or prompts in such a way that even if they can use chat GPT and get the right answer that we end at the right place that what they've learned is what we hope they would learn um I haven't uh okay I I I understand that there are some questions there but they're kind of General more about the plagiarism issue and I was I want to save that until after Dr Baker Gardner has given her her talk so thank you very much Dr Miller I don't see any media questions for you but certainly um you will be involved in the discussion post presentation uh Dr Baker Gardner I will now introduce you please proceed to share if you can that was faster than I anticipated that's a bigger Gardener received her PhD in educational Administration from the northern Caribbean University she serves as a member of the executive committee of the library and information Association of Jamaica she was awarded both the library and information Association of Jamaica information professional of the year in 2017 and Community Service Awards by liaj in 2017. she's a member of the International Center of academic integrity and she recently released her first single author book titled academic Integrity in the Caribbean plagiarism policies perception prevalence and possible solutions alliteration right there so certainly we look forward to hearing her perspectives on how chat GPT will have any influence if and to what degree and perhaps maybe what might be able to do about it thank you Dr Baker Gainer for being here we look forward to hearing your presentation please proceed so let me first greet the organizers of this event fellow panelists and the audience I am more than delighted to be here and I actually attended this a similar presentation that Michael D on this two weeks ago even the very title was similar um the Caribbean's immediate response to church EPG is not so much though about its advantages but it is fueled by fear of how students will use chat GPT to facilitate cheating it is interesting at a very thick blanket of Silence covers the discussion of plagiarism in the Caribbean and I say plagiarism because the concept of academic Integrity is not as well known well as soon as you say the word plagiarism especially in the tertiary context people understand what you're talking about so in reality in terms of its academic Integrity system the Caribbean is about 15 years behind places like United States Canada and Australia we have plagiarism and academic integrative policies but for the most part we have no systems for academic integrity and these systems are represented by the academic Integrity Center websites featuring academic Integrity resources and so on the region is so far behind that I had to ask the International Center for academic Integrity to add Jamaica to their list of countries so that I could register for the conference which ended last week in Indianapolis so the Caribbean does not yet have a significant presence on the academic Integrity state so that was some background so we live in a society where corruption is evident as shown by Jamaica's core the Caribbean country score on the global corruption index so any discussion of academic Integrity in the region must take into consideration the social context we must also give consideration to the fact that the education system is really a microcosmographic society so corruption within the society is likely to be a predictor of academic misconduct and academic fraud with no institutions the challenge is that no one is really talking about it however the high level of interest in shot GPT which is just another tool which enables cheating says that we have a problem all right so I found this page online so our although cheating has been actively promoted there's no corresponding promotion of academic integrity the issue that the Caribbean is experiencing goes beyond GPT what this new technology has done is highlighted an existing problem that we have ignored for far too long so we have tried in our individual institutions to make do of the situation but that is obviously not working and for the people who work in the UWI system you recognize that as a tent so it's not about the technology it's really about academic Integrity from copying to textbooks from from copying from textbooks to copying from online from contract cheating to AI tools students are cheating and advancements internally just give us a new face to an old problem so software to detect AI generated text is plain catching up and I saw that question in the chat so use of these will not sufficiently deal with the problem in addition there is not equal use of this currently available text matching software by Educators across board so on April 4th I was invited to Eternity in release of its new technology to identify AI created products on April 4th I got an email however shot gpt4 is here so we have to ask our question ourselves a question when that technology was being created and if gpt4 was not around is it able to detect it efficiently so a lot will rest with the individual educator in each classroom all right so there's a lack of education on academic integrity at a secondary level and detection is problematic so this is a clip a newspaper article it's it's concerned the 2013 um cheating scandal at Jamaica college so and this is just one of the many articles so cheating occurs at the secondary level education the Educators sorry are sometimes involved and it occurs in both public and private schools all right so this case of 70 JC sixth formers would have gone unnoticed if the teacher involved did not send a letter to cxe headquarters after she was involved in a conflict with the school administration all right so in those schools which have librarian students might be provided with some lessons in information literacy which might provide them with some exposure to academic Integrity however not all schools have Librarians and there's no curriculum for information literacy school librarians are therefore left on their own to decide what to teach instructions in academic Integrity typically boil down to a few lessons in citation run about the time students are required to complete their school-based assessment so I I I listened to Dr West to talk about the role of the teacher and this is something we have to think seriously about how many teachers actually do their due process shouldn't we create a system where we overtly teach students about academic integrity and then provide them with the opportunities to practice these skills so the link is this cheating behaviors are learned and practiced at a secondary level and this is research based students then engage in the same behaviors at the tertiary level the behaviors are very difficult to reduce and eliminate once students enter tertiary institutions and I am sharing with you briefly from a significant research I did all right and this is a social sciences Forum so social sciences students represented 38 of that sample it is important to to talk about the fact here that the interest generated by this piece of research completely surprised me I handed out 1040 questionnaires and we know how students don't like to do these questionnaires but at the end of this I got 1039 completed questionnaires that could be used so only one was discarded and it was because it had no demographic data the students when I went to collect the data engaged me in discussions and even some of the lecturers got involved in the discussion so they were engaged in what was going on but I want to share just a Smith snippet of the data that I collected all right so in order to get an understanding of students favorable attitudes towards cheating we need to pay attention to the first five items and I want to also mention that Walcott from um one of the other UE campuses I'm not I'm not remembering if it's cable or Saint Augustine also did a research among computer study students and fund favorable attitudes too so the sum I strongly agreed agree and undecided for all of the first five items range from 37.8 percent to 51.3 percent so many of the students in our institutions do not share the same perception of cheating as the institution when 31 of a sample does not agree that plagiarism undermines independent thought then we know that the arrival of GPT has just raised the states once students have a positive attitude towards teaching no matter they use the sorry the method that they use to cheating does not matter but what's also significant is what I put in blue students who are saying talk to us about it teach us about it I also want to share a little bit of the qualitative data so from this 1040 students 267 of them gave qualitative later now this is high considering we know that students know really like to write the greatest number of concerns were with the penalty for and severity of plagiarism and the need for education and plagiarism again we have startling data 33 of those who are concerned with the penalty for and severity of plagiarism believe it was wrong but could be justified and 10 believe it was acceptable no in terms of research in plagiarism slash academic Integrity in the Caribbean we are just emerging just joining the discussion so we currently have six Publications and three of those six are Publications that I did so the Caribbean is a late arrival on the academic Integrity scene I contacted two of the authors to explore two of those other authors to explore the possibility of collaborative research however they indicated that academic integrity was not their research interest and the Articles were written in response to issues in their classes and they were both lecturers in Computer Sciences all right so what I've done is summarize the research findings from those Publications so like elsewhere Caribbean students are cheating the Caribbean has a punitive approach to academic Integrity with all he is having a policy and some of those policies were so um short that when I did an analysis of the policies I had to exclude some of them most of these policies need to be Revisited based on the literature the attention in the cabin is focused mainly on plagiarism so some students institutions offer ad hoc educational sessions mostly through the library and Foundation writing courses so at no point do we see our students getting any organized educational academic Integrity one of the factors identified as contributing to plagiarism education is lack of instructions at the secondary level and three of those six papers found that to be the case students have indicated a need for education and academic integrity and educational academic Integrity has been effective in changing knowledge and attitudes of computer students so chat GPT has generated considerable interest but the issue is not being correctly framed from my point of view so there's a panic response to chat GPT there are seminars and webinars to talk about the issue however the challenge is not being framed as an academic Integrity issue as the concept of academic Integrity is not Central to the educational discourse contract GPT in the region all right um I want to just glance a bit this is Australia's um Australian government cherishing education quality and standards agency it's their accreditation institution and their position academic Integrity right in the middle of their web page so their HEI sector is therefore better able to deal with the use of AI as a cheating tool as they already have an existing structure for academic Integrity to date in the Caribbean there's no Regional or national approach to academic integrity and so what is likely to happen if this does not change is that there will be an ad hoc response as institutions attempt to deal with the challenges however this kind of response will never be effective I wanted these These are resources that the academic the um Australian academic the Australian accreditation agency sorry has on its website so they are big on academic Integrity this kind of emphasis is very very absent from the Caribbean all right so what are universities overseas doing they are embedded their discussion about GPT as part of their academic Integrity programs and system so I have clipped the um pages from several universities to show you so this is the University of British Columbia outside the region the practice is to categorize the use of GPT as an academic Integrity issue and treat it as such University of British Columbia University of Waterloo all right this is the University of Waterloo but this is a page for stars this is University launch our London Metropolitan University this is University of Nicosia in Greece all right this is Montclair University and it offers a page for staff in terms of helping them to respond to chat GPT this is University of Calgary another page for staff to assist them with the issues of chat GPT again positioned as an academic Integrity issue University of Saskatchewan staff page for assistance so although the Gleaner Jamaica's most widely read newspaper has identified the potential followed in the in Academia due to chat GPT there has been no coordinated response from the academic Fraternity in the Caribbean and note I said coordinating in terms of how to manage the potential followed this is because we are lacking the academic Integrity systems which other universities are using to manage the challenge and So based on all of this and I actually started researching on academic Integrity before chat GPT became a thing um I this drove me to research this lack of research on academic integrity resulted in this first volume academic Integrity in the Caribbean and it provides an introduction to the subject for the average reader whether you're an academic a graduate student or administrator it contains significant research covering academic Integrity policies accreditation policies perceptions of plagiarism and prevalence of plagiarism about the secondary and tertiary level so I got data from ccxc both cape and csec and I also got data from four tertiary institutions and I analyzed that data and discussed it it presents recommendations for establishing an academic Integrity system at the institutional National and Regional level so looking ahead there are some major project projects I'm getting involved with I am currently I currently have a proposal before ethics that was approved to use the academic Integrity rating scale which is a standardized instrument used internationally to assess the academic Integrity practices of 94 Caribbean universities and this should create awareness at the institutional level of what ought to be included in academic Integrity systems and program and it will also provide Baseline data for each Institution in addition to that I have another proposal before ethics where I want to examine the impact of educational academic integrity and I want to do this using 300 undergrad students because I believe if we have data then we can engage in a more wholesome discussion in terms of finding solution so education is the most effective tool in decreasing cases of plagiarism is a win-win situation for students institutions and faculty policing plagiarism is very stressful and time consuming all right so I close off with Food For Thought does the region actually have a cheating problem if no why would we be so concerned about the impact of GC after GPT if we have a cheating problem that we choose to ignore what will be the likely outcome are the current academic Integrity policies comprehensive enough to deal with the use of artificial intelligence tools and what is the role of text matching software in the context of secondary and higher education what are institutions doing with the data gathered from the use of text matching software and are there institutional policies guiding the use of text matching software and who is monitoring these policies conclusions educational institutions have a pivotal role in developing students who practice in Integrity while protecting their institutions the advantages of checked GPT should be harnessed and I heard the previous speaker spoke about that as we prepare students to live in a technology-rich environment in terms of its ability to compromise Integrity educational institutions forecast approach would have targeted education component embracing the positives and decreasing the negatives and I'm recommending the main recommendation is the need for the region to implement a policy for the development of academic integrity and in my book I argue that CXC and the UWI which are original institutions would be the most ideal candidate for this and from this Regional policy so say sorry let me back up cxe would be in the position to develop policies for the secondary level and the UWI for the tertiary level so from this Regional policy each territory will then need to develop a national policy now that is the way that organizations are going they're going regional policies National policies and then institutional policies and heis need to develop institutional systems for academic Integrity which reflect the elements of the national policies but which take into consideration a uniqueness of each institution and there needs to be development and implementation of introductory programs on academic integrity at the secondary level thank you very much thank you so much Dr Baker Gardner I very much like the way that you have positioned the bigger problem of dealing with cheating uh it's not specific to plagiarism and attempts to be the focus in many of our institutions and a partner that might come from how many students have been educated through our high school system they sort of have to meet requirements of exams if you produce the right answer nobody challenges so you know that and you might be able to amass enough points to pass and so it gets into this um modality of Simply thinking that whatever is put down on the paper that's that's good enough I don't necessarily need to comprehend and the competence is sort of all by the wayside so yeah I'm not surprised that we have this over emphasis on plagiarism I think it's quite indicative of what we see in our high schools I agree with you that we need to broaden it a couple of things though I think that the context of chat GPT it sort of as you said has raised the conversation has raised the issue so that we're talking about it and uh the way that you pose it is that if we thought about it as um academic Integrity we probably have to be able to deal with it one data point that I'll add here and then why I agree is that if you narrow it down to just plagiarism and I think perhaps allow Matthew to comment on this if if in fact I miss anything here is that technically when you prompt GPT for content and it gives you back something very often that text especially if the prompt is complex has been synthesized and therefore to try to label it as cheating because it's plagiarism you might not even be able to find the source in order to prove Pleasures we have to show what that was submitted is a so-called report of and if it's a synthesized for you by your prompt you may not be able to establish plagiarism but it certainly is in violation of the spirit in which the assignment might have been given if the assignment did not want the student to use strategypt so I think your point is is well taken especially when we can frame it in that context of just trying if you're too narrow you won't be able to establish what the violation actually is um if you want to add anything to that in terms of what tragedy can do and in terms of whether it is fresh content would is plagiarism uh the right kind of tool to use to capture what when when students are violating uh you know if you give them instructions and say please don't use strategy pretend to do it's plagiarism gonna gonna catch him yeah there has been a lot of discussions around um AI tools that can detect content written by chat GPT and use of those tools but I'm honestly in my opinion I think trying to detect whether an AI wrote something or a human wrote something um it's it's yeah I don't think it's it's something that's it's worth your effort because um even open AI has admitted that their own tool that they've created to detect AI text works like only like 30 percent of the time it's very easy to just change a few words in the text and it's the tools say oh well it's not AI generated anymore and how do you deal with false positives so what if the AI tool says oh this is plagiarized but it didn't the student didn't actually plagiarize or use chat GPT right or the student not going to defend themselves oh I actually wrote this like how are you going to deal with situations like that so more or less you're going to have to deal with the fact that you probably cannot um um at least just looking at the text differentiate between what a human wrote and a Air route right you are also actually speaking to a different point uh Dr Baker guy to mentioned it and I see there are some references to it turnitin claiming that they have tools that can detect whether it was you know auto-generated by an AI which is you know is that plagiarism well it depends on that that is really thin edge of the wage if you start calling that plagiarism because it causing the question how much volume of text generated by your Tool uh needs to be done in order for it to be considered not yours um and and I think well the point I wanted to make is that when turnitin claims that they have these detectors what I hear you saying and in fact what I have heard myself is that perhaps those will not last very long because the it's not a better mouse trap problem right the smarter the minds keep getting smarter and obviously you need to keep building better more straps yeah that's the yeah that's it the issue that will help because as as these AI detectors improve so will the the AI tools that generate the text so um yeah even if you create a very good AI detector tomorrow the generator will just get better and fool the the air detector so yeah trying to trying to differentiate the human versus AI generated text is gonna be very very right uh Dr Baker Gardner if if I may on this point given what you've heard that it's actually would be very hard so um I if you want to claim plagiarism so you're already saying we should move away from plagiarism and you're hearing that it's very difficult to reliably uh identify plagiarism if indeed we want to call it that when somebody turns in work generated by a GPT how would you recommend that that generalization as you're suggesting looking at it from an academic Integrity perspective what are the ceiling points that you imagine such a successful policy being able to bring out that would make it different from this over emphasis on plagiarism at the moment and in particular how might that framing capture some of the things that we're talking about no and as you spoke I uh what came to my mind is if your entire class of computer students chose to use AI tomorrow to cheat on an assignment can you can you can they be penalized under the current University policy um well I say to my students yes I say to my because it depends right um when we give the students an instruction you are to turn in your own work does that count that comes but if in other words if they use charity does it cost us not their own work right so you would have set parameters for the assignment right but in an instant where you didn't say that somebody could pretty much Sue UWI because the policy does not cover that the current policy speaks to um plagiarism but how it speaks to plagiarism it does not take into consideration the use of AI tools so that's just an aside so what I'm saying is that this is really a critical issue that we have to contend with no I agree with you I'm asking for suggestions if you have any at the moment about hope that might broaden rather than it's also being so narrowly focused on plagiarism retweet what are if you have ideas I don't mean to put you on the spot no man yeah but how might not be broadened so that it's not just so focused on Pleasures because as you're hearing just even trying to be very strict or plagiarism certainly might completely miss the mark which students who are using chat GPT because you can't prove it right so so so so so my solution to the problem and this is a solution that has worked is that and um Dr Wesley spoke about zero tolerance what has worked in terms of zero tolerance is educational monitoring hand in hand so we educate them we ensure we give them opportunities to practice so something like chat GPT you take it into your classroom show it to them show them the advantages and the disadvantages talk to them about their role as writers and that would be a part of the educational process and that is why I'm pushing for Education um Bahamas there's a university of Bahamas lecturer who did um research in plagiarism with computer students and she had zero zero cases so it can work but we have to be willing to spend the time to develop a kind of systems that are focused on students and let me tell you I've been a teacher for 30 odd years when I am stressed out I am not going to be spending any time to figure out whether or not the the AI wrote this paper I think you are consistent with some of the ideas that Dr Wesley put on the table um I mean he was I mean he's right here but he was really saying that we ought to find ways to teach differently or to find ways to use it to get students to engage with the content in perhaps ways that don't normally in other words you could ask them to ask chat GP to generate an answer and then have them come back and reflect on what it all put and so you're actually getting them to engage in a way that perhaps they may not have when you've given on the first assignment I think those kinds of interactions no you are speaking specifically to the education of academic integrity and I completely agree with that but I think that those two go side by side hand in hand if you have exposed students to the use of track GPT as a tool not not as uh you know a cheating mechanism that you need to hide and talk about but a tool just like on your calculator at home um and there are times when it's appropriate to use the film not same way you know if the school system if our education systems in general embrace it in a positive way then the kinds of things that you're talking about that it is possible to use charge GPT and not be in violation of academic Integrity it is possible to not use it I'm still being violation of academic Integrity policy and I think I mean what I'm just trying to say is that I hear what you're saying sitting alongside what Dr Wesley is saying I don't I don't actually see too much conflict there I do fear you say teachers will have issues adjusting because they are stressed but that's a separate problem I agree I completely agree and that's why I am saying divorce academic Integrity teach and I'm teach academic integrity by itself then integrate it into the classroom because if we leave this to teach us to teach it while they teach their content it might not get taught right so ensure that there's some program that students must be involved in the learning about academic integrity and get the opportunity to practice teachers in the class who need to ensure that what is taught over there is practiced over here and we might also have to teach our teachers too we have to be realistic absolutely absolutely all right let me I see a question from the field um this one was directed to Dr Miller um well let me just ask the the serious question because there was a joke about are you gonna offer a new course on prompt engineering but the serious one this is from Christopher Darrell says do you foresee learning prompts as another layer over programming just like a python wraps over lower level machine code [Music] so is it that we're going to teach persons how to use chat to prompt chat gbt well I think that was a yeah implicitly that's a question in other words if you're teaching them to use GPT to help them write code and you're aware that they might use it the question here is might it extend to the level we're actually saying hey this is a tool that you must use and here's we're going to teach you how to use it properly uh no no they will that would currently that's not the EM of the the course or the what we teach we're teaching um how to understand Computing how to um solve problems with computer science and well no that that that's not on the table currently though I expect that they will learn how to use it over time I mean was there ever a time where we teach people how to use Google I'm not sure I remember that let me let me put the question back to you in another way today if you have to develop a graphical user interface uh you do not start from scratch writing text well if it's complicated enough very often you'll use one of these graphical user interface tools you drag and drop the components you can't see it you can see it being developed as you build it and very often the only code you have to write is when the actual action needs to be taken place the code for layout the keyword for the the you know the timing all those other things that are graphical user interface related are auto-generated for us is that so different well well if you put it that way we're not speaking about the specific course that I mentioned so I do see where in other higher level courses where um these details in how to program is not necessarily um the focus or the learning outcome that um using gbt and learning how to how to get better prompts might be useful in such a course so put it like that then yeah right okay um well okay we've covered the academic Integrity uh topic when chat GPT um I don't understand if this is a question it's actually a comment creeper Educators need to define a new knowledge worker practices llms will enable to Def was this I'm not sure what hour Define the new competences workers students will need redevelop courses to teach and assess appropriately uh Dr Wesley I suspect that some of that is actually very much in line with what you are seeing in your talk that it's about redeveloping courses to teach and assess appropriately so after change I would teach change how we assess we have to define a new competences that we actually want our Learners to graduate with um so it's another question but I'm going to put it to you because it was right up it's in your wheelhouse anything you want to add because you've already said it but perhaps people join later may not have heard well no I don't have much to what you have indicated we have to stay focused on content knowledge by itself will not create innovation what and the regression the regurgitation regurgitation of that knowledge what creates innovation is the development of critical thing in students and if we continue to focus on content as they end up you're gonna always have displayed as a challenge a student developing the ability to solve problem and displaying that in front of a teacher it's quite different from somebody leaving to the confines of their home writing a paper to turn it in and we have made the point about formative evaluation because you're almost assessment that will be utilized in formative evaluation and it's going to be totally impossible that's quite um for us to monitor everything was doing that that's the way they're not dead uh being academically correct in in produced but certainly it is the work of the educator to ensure that the competencies are acquired by those students and it will shift therefore should now be on those competencies and let's go on what can be regurgitated and reproduced because that's not the aim anymore when a student walks into a workplace you know because when a student walks into a workplace and the manager says go and solve that problem what the manager wants to see is the solution that addresses the concern the problem would have put to that organization whether that student goes to GPT and get some ideas or go to Google and get some ideas and go to their father mother anybody and get to the idea because the issue that we need and processing in the fundamental principle that we need to inculcate within our students it's not necessarily about more resourcing but it's being resourceful how do you utilize what is available to you and when we shift from content keep going keep going you're back with us I thought you were sorry I don't even know where you messed with you uh when you shift no we need that fundamental shoot okay right all right from content to competency absolutely and and in fact what they have to do when they hit the workplace when they're in society how will they function right absolutely and I think that is actually I mean I'm happy to hear that as a region we're trying to go in that direction that is where certainly in my own discipline that we're a professional bodies have said that's where schools are agreed that's where it needs to go it's competence oriented confidence-based I see a comment here Christopher Darrell really is supporting us and I think it's worthwhile seeing our load we have an opportunity to LeapFrog Jamaica's production and future development here not just in computer science but in general please let's not let's make sure we don't strangle it to death with policy so um but I think In fairness this panel has been fairly open to the idea of using it as a tool while being quite aware of the challenges challenges that it causes to the current framework of how we do assessment and how we do our teaching and our learning but I'm I'm very encouraged to hear the kinds of views that have come come forward today um I would like well perhaps I if any other panelists would like to ask questions of any of the other panelists perhaps tomorrow would be a good moment um and if not let me give you a moment um I I if not then it's coming up to just about 6 p.m at this point what I would like to do is just sort of um you know kind of throw something that I uh in a way we've kind of touched on this but perhaps to break it down just as a kind of closing remarks perspective I would say that you know when we look at what the threats and opportunities might have been in education from our perspective as Educators and people involved in education uh we kind of with you know there are different areas so you can look at learning outcomes we can look at assessment and we've discussed how some of those things have are likely to need to change um I wonder whether we are any closer after today um to being able to formulate what those might look like so what about those opportunities in terms of learning outcomes or how we might expect students to learn more for example in blue our objectives are often written at different levels according to Bloom's taxonomy might we expect to see that it becomes a matter of course that more outcomes graduate to the higher levels as you're asking about moving away from content to application so you might see less describe and list and identify and more apply evaluate synthesize is that something that we might expect to see um do we have a sense of which ones are threatened and I think as a challenge there's this fundamental question you want students Learners to develop competences but I have to start somewhere nobody can learn the highest levels without first stepping up through the low ones and if GPT is going to be incorporated as a legitimate tool how do we make sure that it doesn't circumvent the learning at the lower levels as you say the mental laziness how do we make sure that doesn't happen um while still being able to embrace it in order to reach those higher levels I don't know any thoughts from any any perhaps if you do I'll take it in order I think we might need to go back to how we train our teachers I think that is where we are going to have to begin that level of change we train people and the training sometimes allow them to get these degrees knowing only content and not not much application in some cases all right so we are in a very fluid technology environment when we send teachers out to the teachers colleges because there are going to be some people on the field we can't change their minds anymore they are fixed in their ways and there's absolutely nothing we can do about them but can we educate a a casual of young teachers who understand how to teach people to think to work through the issues and the problems so they would have already learned it and so they are able to pass those skills on to their students the reality is that the workplace does not need content anymore we don't need to test content if a student sits at work the student can get all the content they need from the computer that is that is like by the way content is becoming obsolete so fast that by the time some of those students leave College what they would have learned is no outdated so my thing is can we involve our training teachers are those going into the universe the teachers colleges in the kind of learning that makes them so flexible so ready to suffer knowledge so critically on the edge in terms of how they think that they are able to engage these students because those authentic products will not come from teachers who think the regular way okay interesting so teach the teachers how to learn how to learn and they will teach the Learners how to do that right thank you uh Dr Miller or Mr Stone any final thoughts yeah I think um yeah generally in terms of assessment I think some of the other presenters have touched on this I think we're going to have to move more to a real-time form of assessment um so again move away from the take-home assignments and more to evaluating in class so you know I've always imagined you know students having you know live debates or discussions on topics um more so assisting a student's ability to think out of the box um you know create new thoughts rather than just um regurgitating what you saw in a textbook um yeah I think generally uh as a tool GPT will allow you know not just students but people in general to be way more productive I I found myself to be way more productive um like things that would take me always you know takes me a few minutes right there was a timeout then I was on Google search for something simple that literally so you think this is going to be an extraordinary productivity enhancement tool yeah I think and as far as the impact on education is if the focus is on being able to use it to do that that's where we're going to realize or the biggest impact yeah I think it will actually Force Humanity on a whole to transcend um you know in terms of intellect um because you know especially with the whole discussion of you know taking away jobs and you know AI being able to do a lot of the jobs that we as humans do it's going to force us as humans to know Force our brains to actually think at a much higher level rather than stay at certain level we've been at for Generations okay it's actually going to force us as a species now to think more at a higher level rather than exactly completely better yeah I'm thinking it's going to force us to get better as a species really okay I like that Outlook Dr Villa you have any final words yeah so um I really it's so the the current course uh this question as um yeah we're reviewing some of the reviewing the the first year courses we actually found the the way how we do the lab the labs in Computing to be quite safe because the way I would do it is that persons come into the love and they explain the code and we interrogate them about their quotes so um that seems to be the way to go in a large language model era where persons can simply generate answers because we see multiple times when persons come into the lab if they before if they actually just copied the code when you interrogate them and ask them why did they do this why did they do that they they don't know and it's not like they will be able to interrogate charge gbt on an infinite number of possible questions that we would ask so that that's definitely so it's very interesting that we thought that to be safe that type of assessment to be safe and the others were actually looking into no so yeah I think those that's the way it's about so real-time assessment of the important competence that they put in forward definitely okay okay good um thank you for those thoughts on Dr Wesley I give you the last word well the last word from original perspective given that we heard the rich space and one of the the new re-imagined philosophy that we have worked on starting from our qualification management framework and safety philosophy and assessment we would have been given consideration to a wide range of modalities to assess competencies trial and assessment observations you know whatever it is that is needed to demonstrate that students are operating at a particular level we also recognize that given the fact that learn as we go along how do we re-engineer our qualifications to begin to take account for competencies acquiring As you move through the education system and the requisite flexibility for persons to be able to achieve a full qualification with this kind of thinking and then we're looking at assessment will now provide some kind of guidance to schools right across the region as a root pointed out earlier about CXC influencing what happens at the secondary level how do we provide this standards that can be used across the region because anybody getting a grade one a grade two anywhere across the region it must mean the same thing internationally right and that is not only dependent on content that is dependent on the competency students should have demonstrated in a variety of ways and and that is why we no longer just have a summative assessment of the student work even though OST second kid is at the end our school-based assessment is so designed that as the students are progressing throughout the year we are capturing the competencies they have acquired so I I fully concur the there will have to be some new policy shipped in this regard to account for I think somebody asked the question earlier about the intellectual uh intellectual property copyright that will have to be considered as people use information and to recognize that we must give credit where credit is due but it's not that you can't use the information and I think that's the important thing perhaps what root is advocating for that if students understand that you can use information but you must give credit for the information you have used then they stop putting tossing on other first people's work as their own I think he's going to be critical of the building uh within our own workshop and seminars that we have will be crucial as we seek to take advantage of Technology because technology is making information available to us quite easy right right thank you so much Dr Wesley Okay so we've had a very productive conversation very wonderful presentations thank you so much presenters very informative and basically we're coming away with I hope the audience shares my sense of optimism rather than pessimism with what chat GPT means for us in education sector certainly there's no denying that we're faced with an immediate challenge but what I'm hearing is that really untruly that is a representation of how we have done things before and so if we weren't prepared to move towards competence-based teaching and learning and assessment chat GPT is making us not our choice and we're gonna have to get there I'm hearing that but how do we make sure that students still gain those lower level skills there are ways around it we can do this sort of interrogation of their understanding of what they've produced there might be other ways that we can do it of course or closed written assessments still preserve that Integrity of giving a student credit for what we know that student knows not somebody else so submitting work on their behalf and we also hear heard many ways that we can anticipate GPT finding its way into the classroom from teachers enhancing what they do to uh in enhancing lesson plans we heard about prompts being perhaps vague because when you try it with chat GPT it gives you a wrong answer and you realize maybe it's actually the question that's badly formed I think hopefully if teachers Embrace that that means they'll ask better exam questions or frame better questions on their assignments and and projects and so on so there are all kinds of ways that it may affect teachers and certainly the Learners if we can encourage them to see it as a tool and then have all that sit inside of a framework of academic Integrity so that it's not about am I cheating and I'm using this on the cover but it is that there's a clear policy and you're educated about that policy and the framework in fact and so there's an understanding of how you know when you can use it how do you know when it's appropriate you know what kind of use is appropriate context and that becomes clearer and so it becomes another tool that's available for use in a context in which everybody can do it with a clear conscience and of course then we sort of can and have that happen at the lowest levels so that by the time they get to the higher level institutions it's not you know a major problem as we currently have framed it um thank you so much everyone for participating thank you audience for being with us I hope it was fruitful and productive for you as well I hope you all learned something this afternoon thanks again to our speakers good evening all good evening thank you to Daniel hey you're welcome okay thank you very much real fruitful thank you all right blessings everybody

Channel: Faculty of Science and Technology, The UWI Mona

Views: 2,015

Keywords:

Id: I-TNmCDzXIc

Length: 132min 50sec (7970 seconds)

Published: Fri Mar 24 2023