Crafting Valid and Reliable Assessments for Distance Learning

me oh uh uh oh what's up [Music] okay okay what [Music] um okay hello and good afternoon um viewers um everyone to our subscribers followers um for today's webinar on online distance learning technical teachers know you adding um a third installment adding a webinar series for distance learning okay of course um uh i think is uh presence support and with me today you know uh a very special guest for today um so um for being with us uh here today starting a webinar series on online distance learning as a distance learning part three okay so foreign to help you cope with the ship to distance learning okay and again word in the distance [Music] [Music] [Music] of course we have two speakers for today uh each one taking on specific um aspects crafting valid and reliable assessments for distance learning so we'll have mr galvin for today and also miss gem astronomy also um with us today okay and um evaluation links will be made available at our website by 5 30 pm okay so um website so will be made available by 5 30 pm so 5 your registration so it's based on your evaluation so please make sure for evaluation and lastly before time start now um questions about the presentation should be reserved during the question and answer portion is questions so please make sure audience know our participants nah it is [Music] um our first speaker for today we are very lucky because this person is available for today yeah she is a good friend of ours and she is also a colleague so without further ado i would like to i would like us to welcome miss gino c astronomer a teacher from the south greenhills so um i'm adding first um speaker for today okay so yeah and welcome [Music] [Music] [Music] um reminders for all our audience okay please make sure that um your um your evaluations okay will be sent um uh this afternoon so if a postponement published website uh at 5 30 pm okay so again the links are available um link or we're showing it right now and also in the description and um in our um um comment section so now yes [Music] [Music] so i would like to introduce first myself no um i miss jamel astronomy name code it's a poster that's that's my nickname um i'm 19 years old in the academy in the in the uh teaching or nothing i don't want to share my age talaga i'm 19 years in terms of academia i'm an etta coordinator and a part-time teacher in the adult night high school so i i work at la salle green hills as the ethic coordinator it's relatively new and um but young part-time teaching adult night high school most of my examples later on foundation and then um just so i can at least share something about myself even without being really physically together you know um i like watching movies uh alone um i like watching movies alone and i hope almost okay and um today's session um uh since i i like to be uh the session to be an interactive one i hope um if not everybody will be able to join so um valid and reliable um assessments no so if i may just ask is it all right if you um open any browser and then just go to the url www.gosobox.com and then tap or click join event or sign in so you don't have to sign in or sign up you just have to join the events last 10 seconds 10 so hopefully my audio is now a little better nine eight seven 80 with or without joining 10 9 8 7 6 5 4 3 2 1 okay foreign uh 81 to 120 pesos five percent he gets a 120 one person number of responses nothing against mac users check out some microsoft powerpoint 2010 version um if you are connecting to a url you can just um clearly view it here on the slides okay 77 19 4 1 so one type of assessment that we can actually ask and very simple for our kids no or for soria but very ferocious students belong kids let's say um so this one at least gets to give me an idea of the age of my audience no so because this is something i can relate to so yeah when we do our assessment manga simple formative assessment it has to be something that our students can relate to and then we have to give meaning to is it all right if i ask you to click on go back to event because i'm going to be opening another question so yeah go back to event hopefully go back to event yeah um and then just click the recently open question is and of course i speak in filipino and in english [Music] so these are very simple assessments on how we can check also the students know um that only their understanding but um how well they are no it doesn't necessarily mean if you're creating assessment it's always cognitive no more specially at this time we have to ask them how they are because the the physical presence that they need distance you get to have a connection first with your audience before uh before you know you're giving out what is you do to them i think responsibilities so we asked them first how they are and then we get to deliver what they what's due to them so thank you very much respondents assessment uh is um take it easy um because it takes time to practice and um as an adult it's something that i have shared to the to my partners in the alternative education so merun coming department part-time students and we are on mainstream medan coming hopefully what i will tackle in you know in 30 minutes would cover at least uh just this is these are only reminders objective is for you to recall you know whatever face to face etc okay so yeah so i would like to cover at least in general um overview and assessment in the online or distance distance learning so the validity and reliability um some challenges encounter at my encounter nothing that at some strategies again i i'm not here to offer any solutions but some strategies that uh i feel would be helpful also for us teachers also does not rely on the score that our students will uh will get no because the support that they are getting is highly dependent on how you made your assessment that's why the topic for today is how can how can we make um at least a valid and reliable assessment in the distance learning assessment online directly um to be there not to to answer their questions basically so we always have to open our communication with them is one of the most challenging aspects of the academic experiences to navigate in an online context traditional teacher i call myself a traditional teacher um well it's very ironic um um essential diagnostic assessment [Music] asynchronous [Music] okay so yeah [Music] [Music] then video demonstration so return demo alternative [Music] return demo and then you have to submit a return demo and there's no other way on how they will be able to submit a return demo other than using a video demonstration so it also has to support you so [Music] [Music] give me 20 seconds just to plug it up sorry i enjoyed it yes sir [Music] modality approaches not in some aware winners adding uh different modalities [Music] um synchronous and asynchronous um there will always be a framework that you will uh that you may follow so but in terms of assessment in terms of writing and assessment whether it's um modular based whether it's asynchronous or synchronous formats [Music] will always begin with either your competency and then later on discuss objectives objectives so uh that's at least a general idea of young assessment why it is uh why it's important not only for any modality no education in itself uh next is yes so it has to measure what intent what it intends to measure and it is dependent on the purpose behind the test later on actually statistically speaking you you have a formula on how you can get the validity you know so um not the number in itself but based on how you know that the test is valid or your assessment is valid so again um tests or quizzes paper intent these are only examples of um assessment if i did not think anything also a reading test meant to measure literacy that is given in a very small font size [Music] face validity is also another aspect of validity example [Music] m [Music] i'm sure you know this already so font size matters font style matters color matters like like right now no i i didn't expect nah hindi masha readable so i i'm so sorry but you know i i i will share them yeah um uh so yeah specifications [Music] [Music] departmental or standardized tests and then yeah so threats to validity validity matrix and so the way we give instructional procedures the way we administer you know and scoring procedures and student characteristics are very important to validity [Music] fourth quarter it has to be distributed um [Music] [Music] [Music] [Music] why assessment is very important to teachers because this gives us data this gives us data uh if uh we were able to we were able to use the right strategies not to deliver the to deliver the content and of course we have to make sure that the assessment is fair to students of all backgrounds alternative education mainstream also before or [Music] [Music] [Music] so that's validity how about reliability so for reliability um uh refers to the consistency [Music] [Music] [Music] [Music] so how do we improve reliability you know so you have to give clear and specific these are only some um strategies now so um clear and specific rubrics yeah performance subjects so i guess this one is something that would be very useful also a printed self-learning kit whether you're online rubric is one of the best form of uh is make it as simple as possible so um hindi 11. one sentence and then of course so they are also experiencing similar um situation as ours [Music] 30 minutes or uh 60 minutes to 90 minutes [Music] [Music] um [Music] [Music] [Music] [Music] as an educational technology coordinator i have always been asked this several times now how can i improve the system your online system nothing without the students really um copying answers or opening others [Music] is [Music] [Music] there's no other way for for the parents to perform it other than that also and um one thing that i can share know for those who will be teaching in the lower grades always expect that you know the parents will always be there watching so nothing is is baba or i know um my grading system that's mostly based on numerics not so britain works so this is also management so when you are writing your assessment uh it will really take some time [Music] um and then what are sample strategies now in crafting assessments for distance learning so um know your objectives and then the proper assessment so for an assessment i write a simple sentence assessment [Music] assessment is very important another one is to set expectations um [Music] [Music] it's a combination of um skills that they have learned and then they apply uh in one performance task so similarly with competency-based and problem-based indicating every week so you put together you know competencies so um these are summative mostly summative has a grading system performance um some strategies when you create performance based best that um this are something related to to to the to our students and also [Music] it's nothing [Music] [Music] reflection papers [Music] solution papers presentation papers [Music] um my quality quality assessment and also um use uh use multiple methods of assessment so like uh say some competency hindi necessarily [Music] [Music] one i use alternative methods but just a reminder um i know you are all very creative ferro um it it will take some time for our students to digest everything performance tasks so [Music] [Music] there's always going to be um vertical articulation and horizontal articulation and lastly of course we have to involve our students [Music] [Music] towards is really very important and we will also try to empathize with our students how we can make quality assessments for um different study and then um yeah so i saw it yeah so formative assessment so collect evidences over time so those are formative assessment uh ask them for feedback um like towards the end for example you can just simply ask them um the confused bahayo or mastering these are very important especially for distance learning so we don't always expect that for example for example um we are not sure if they they are still you know interested in what you are saying so um most importantly you have personal conversation but you have to be careful no so personal conversation sometimes face us no the words on the emotional side no so um you have to be careful as teachers so we have to be very careful though it is an important also aspect informative assessment that we give or or offer them in personal conversation [Music] [Music] students will students well-being and of course use the data in the formative assessments um and then i move on to the next note so because these are data that will help you okay so you know i'm on my last uh last slide sorry over time yeah so um uh why is it important to use validity and reliability no so we every educator every teacher should always have validity and reliability in the data so that um statistically um conclude by looking at merely results no results i'm saying results because other than the numeric technology research and collaboration with our other teacher partners and also this can help us have young validity and reliability and validity and reliability allows educators to make decisions that improve the lives of their students both academically and socially as these concepts teach educators how to qualify [Music] and yeah on my last slide um you know this is something that i want to share now so as teachers hindi it's a bit different and we have to upscale there's there's really not nowhere to go but to do this and uh so in order for us to continue moving forward with education because we we don't want our students to really stop learning so thai only smoking and then so let us be the students that we want to be so that ends my presentation uh sorry over time um um a question and answer so um yes [Music] whatever we do whether we are doing it online or face to face we didn't have to trust our students so i think for me that's number one we have to trust our pupils whether assessment or kite simple activities we have to trust them that they will really try their best to do it on their own and then from there if they if the teachers trust them yes learning while we're at it viewers maybe ask the you also share this um live stream to your um timelines live and we'll be back in five minutes see you later [Music] this uh [Music] thank you okay okay you [Music] [Music] uh thank you [Music] um okay um [Music] um okay so uh for our second part let's uh we'll be having uh mr galvin the assistant director of ateneo assault institute for those who will be asking uh what's uh salt institute of ateneo basically an undertaker facilitate my training uh faculty development now among teachers and faculty and staff okay so okay so let's uh welcome uh our second speaker so with us um mr galvin and hey afternoon sir hi yeah [Laughter] [Music] vocabulary of what we need to think about when we think about valid and reliable assessments so i requested franco to help me out here i'd like to begin with just a short experience gem a while ago we'll try to make this a little interactive to begin with uh maybe also add a breaker between the first and the second talk so what i'd like to do in the next few minutes is just to ask a question uh through another tool that we're going to use it's called mentimeter now so uh the instructions are and please don't do this yet i'll give you some instructions and then i'll tell you when to start uh if you have access to a browser uh if you can switch to a browser for a moment uh this zero zero four five now uh what are you going to do i'd like to start with just asking you about this particular question a while ago when we were talking about the things that we picked up from uh teacher gen i liked how the conversation led to really thinking about what is it like from the student's perspective so i'd like to begin that way you know that uh what are three words that you think come to mind when a student hears the word assessment or for example we're gonna announce my assessment i o uh what are three words that come to mind that you think they will immediately think about now if i noticed in chat a while ago that some of you might have internet connectivity problems so no pressure um if you can participate in the multimeter activity go ahead if not you can also put your words over chat because we'll give you a few minutes and then i'll also look at the words over chat just in case some people aren't able to submit here so again the tool that we're using here is mentimeter if you haven't done it yet just go to a browser just like a while ago go to mentee.com and then enter the code 6d0045 so i can see we already got 22 responses i'll give you a few minutes uh maybe a minute or two just to contribute to our word cloud and again no pleasure this mobile name is switch to exam test quiz so for those who are looking at the screen right now if it's your first time to see a mention activity what we're building here together is a word cloud and the way to interpret this is to watch out for the bigger words not so the bigger the word that means more people submitted exactly the same word so it won't become bigger if for example there's a variation in how you submitted the word so as we can see our word cloud evolving uh one of the things that we'll notice is that uh what seems to be the impression of students not so if we're thinking about the student perspective is that when you talk about assessments difficult difficult this is the quiz test exam hard performance evaluation pressure no pressure nervous okay study anxious fear hard score great scoring words pressure test exam um stress no stress came out a couple of times performance came out a couple of times a measure evaluation this may 142 contribute maybe i'll give you a few more seconds let's see if um there are new ideas that emerge study so study suddenly became bigger studying for an example studying for a quiz or or an assessment maybe that's one thing that immediately comes to mind for a student cramming students tiresome uh paper activity confusing okay happy uh it's interesting so if you haven't submitted your words again please feel free to contribute and to edit over chat but i wanted to start this way uh because i think that we think when we think about assessment uh more often than not the conversation begins with what we plan to do as a teacher how we plan to design the assessment and i think that's important after all if you don't design any assessment but then i think that hopefully in the next few minutes i can help you reflect on the student's perspective and in online learning as i will sort of emphasize in the next few minutes having students on board or changing students perception of what assessments are is necessary changing what students think of when they think of assessment is i think a necessary ingredient if you really want to make online or distance learning work at the end of today's session in the next few minutes as we talk more about reliable and valid assessments if you if you imagine doing this activity with students in the future what will what words do you want them to be giving you it will be in words so we'll keep this in the back of our heads uh as i share some examples and things that hopefully we can reflect on in the next few minutes so allow me just to set up my screen share so sir franco uh i'll set up my screen share now so that we can switch to my slides yeah salamat no salaam it's a contribution i can see that people are still submitting their words over chat you know so i am a pressure exercise learning learning also in the future so um first of all uh just a caveat um to be honest i really learned a lot again from teacher gem uh and i think that my role in the next few minutes is necessarily to reiterate some things that she said already uh but inevitably as teacher gem told us in the beginning of her presentation when we think of assessments although distance learning a lot of the things that we still believe in or hold true in terms of what valid and reliable assessments are still continue to hold true whether you're talking about face-to-face learning while they're talking about distance learning so i'd like to frame my sharing with three questions so these are the three questions that i hope that i can sort of address and help you think about in the next few minutes so the first question is what do we mean by valid and reliable okay so bangit nani teacher gentle extensive gem extensively but what i'd like to do is just to maybe add another layer to that conversation and then the second question and i'll spend a lot in the second question is in distance learning what is the role of assessment and this is very much connected to the word cloud a while ago that if we think that when we ask students about what assessment is to them you know words difficult challenge pressure stress assessment and finally we'll talk about the challenges and potential solutions um i agree with teacher gem that i don't think any speaker in his right mind will be able to give an absolute solution to everything nobody will put my speaker or may make foxygo you'll be conspicuous but rather hopefully there are some strategies that i can share that might be helpful for some things that you feel might be challenging might might be problematic when you actually implement assessments in your distance learning scenario so over chat i can see how active you have been a while ago and i've been monitoring the chat and i think that's great so i think the way i usually like to use the chat portion of a synchronous session like this is if if there's something that strikes you feel free to share it over chat no but that may s so that it can become a more active experience for you as well so i know that this is not ideal again much like teacher gem i'm a classroom teacher primarily and i do miss that energy that can be exchanged between a teacher and a student in a face-to-face classroom but just the same we'll work with what we have so let's begin um again this might feel like a summary more than anything but uh jem mentioned a while ago that if you summarize it uh popham in an article in ascd uh said that validity is really about accuracy you know so how accurate does your assessment assess or evaluate a particular learning outcome for example and reliability is really about consistency but if you think about an assessment in the role of the assessment to be able to say who actually learned something and who did it or who actually learned more than the other then that's really what we're talking about in terms of reliability but in this article they pop him what i like about it is what he says is that more often than not when we talk about validity and reliability of assessments we often situate those conversations in the realm of tests or exams and it doesn't mean that this is bad or this is wrong in fact a lot of the conversations about validity and reliability most especially if you look into the statistics of it are really about exams less items among other things but i like how this particular article introduces this layer of inference now that according to the article when we think about validity and reliability hopefully we don't just think of it as measures of validity and reliability of a particular exam structure so so to speak but the validity and reliability of the inference that you can get from particular exams one way of putting it is this know that when we bring in this additional layer to the definition of what we uh talk about when we talk about valid and reliable assessments what we're really talking about here is that um valid and reliable assessments are assessments that accurately and consistently provide evidences of learning uh and when talking about assessments i like this phrase evidences of learning in the course that we recently ran in ateneo uh we presented this metaphor uh it's the metaphor of a prisoner or for example or someone in front of a judge what one thing that you can say is something like this learners should be presumed in a sense of learning until they are proven guilty beyond reasonable doubt no so if we just contemplate about this particular phrase i think that this very much talks about what assessment really is or what assessment ought to be that the role of the big role of assessment is so that we can actually validate if learners are actually learning and i think it's worth thinking about this because i think this is a necessary shift from an attendance mindset mindset rather to an assessment mindset you know usually you can say when we talk about teaching a lot of us to be honest myself included uh we really come more often than not from an attendance mindset so my students went into my class listen to my lecture so my student learned but did the student really learn um we're never sure if the students learn if we're just basing it on attendance but rather the invitation is to shift from an attendance mindset to one assessment mindset to create and craft and design and facilitate more formal ways of really finding out if students are actually learning and when you talk about distance learning i think that i'm preaching to the crowd when i say that this is probably one of the things that we're worried about that when we don't see our students in the classroom they don't react to the things that we're saying uh just like a face-to-face lecture all of these senses that we usually rely on to check if students are learning they're now gone no or they're now upset so to speak more has this theory that is uh mentioned a lot in a lot of distance learning scenarios i won't get into the theory too much but i'd like to use the theory just to explain what's happening and and what we problematize when we think about distance learning so according to more when we are physically distant from the learner when the teachers are the teachers are physically distant from their learners or from their students it doesn't just create an actual physical distance but it also generates what he calls psychological distance and again without getting into the theory too much uh one simple way of explaining this is that transactional distance is really acknowledging that there could be a gap between these two things the teacher's intent what you in that intent for students to learn and the students actually experience them actually learning it if you think about it even in a face-to-face scenario there could be this gap right uh i could be explaining to you something right now and i might be assuming that you're getting it that the things are sinking in that you're able to relate to the things that i'm saying but i'm not really sure no so again even in a face-to-face scenario this gap is also present but then in a distance learning scenario more often than not this gap is wider mainly because of the limitations of distance learning so for example a while ago overchat a lot of you were asking about modules so if your modality in your particular school is a printed module and the student experience is that they're reading through the module and then the teacher is not presence at all present at all then there is really this risk of that gap not that you're assuming that what's written in the module is enough for your students to actually learn from that what's written in the module the instructions the content there is enough so that it will get your students towards the learning outcome that you actually set for them another way of putting up putting this gap or facing this gap is called the absence of the teacher and a lot of us talk about this now when we shift to online learning but i think it's worth challenging this is the teacher really absent or are there ways for us to actually insert our teaching presence and my pitch this the next few minutes is maybe we can use assessment to actually insert our teaching presence maybe we can think about what we do as teachers and then build it into the distance learning scenario through the form of assessments of course assessments are not the only things that you can include to make your teachers presence more failed but then surely i think that this is one clear way we can do it so in the next few minutes allow me just to share with you an image uh i like when i teach one of the things that i like as a strategy is i try to think of an image to explain a concept because i feel like if it's an image that is relatable to students they will be able to hinge on the concept easier so our image for this afternoon is a map that when we ask this question so what is the role of assessment in a distance learning scenario the simplest way that i think i would describe it is that it's like creating a map and there are three components in this map we set the destination so assessments are about setting the destination in this map called learning second assessments are about pinning milestones not pinning milestones towards the destination in the learning scenario or in the journey of learning that your students will be going through and third assessments allow you to to navigate charting the path from the starting point to the destination that you hope your students will get to so we'll continue to use this analogy as we move forward so that we can have an imagery in mind know that imagine that when you're assessing students it's really like creating a map and will try to break it down for you in the next few minutes so the first is to set the destination um what does it mean to set the destinations if we sort of use this as an analogy as a metaphor for what assessment is what do you think it means by setting the destinations so can you head over chat right now and then just give me your impression um if we try to juxtapose this image of assessment as mapping learning for our students what do we mean by setting the destinations i'll wait for a few seconds and and see if there are some thoughts that come out over chat what do you think do we mean by setting the destination when we talk about assessments the goal okay the goal objectives a heart heart no so okay so think about it no uh what does it mean to set the destination the goal objectives knowing the product knowing where we're heading setting the learning goal objectives of the lesson right so a lot of you have mentioned things that are actually here on my next slide as if in a predicament easy opinion but when you think about assessment as a way for setting the destination it's really about two things it's really about establishing clear learning outcomes some of us call them goals some of us call them learning objectives some of us call them for example expected learning outcomes we all have different names for this but whatever name we call them they're about really the objectives that you hope for your students to learn and it's also about aligning evidences of learning and i don't think i need to overemphasize this because i think teacher gem talked about this lengthy a while ago and really emphasized how important it is to start with that so this is rule number one assessments uh in distance learning is really about setting the destination i'll give you some examples of of how to do this according to fisher and frey fisher and frey rather they talk about three things that are important in an assessment or feedback system the first thing is called feed up uh what they say is that in a good feedback system and i'd like to think that uh when we think about assessments we're really designing a feedback system one of the first things the teachers need to do is to feed up meaning to clarify the goal among students and i think this is what i was saying a while ago when we need uh when i said that i think we need to help students shift how they see assessments because more often than not when students think of assessments they think of the teacher as the judge they think of us judging them but then if we begin here if we begin to feed up if we begin to make our learning objectives more transparent to students then maybe we can shift the relationship a bit so one one very explicit and simple way to do this is to make sure that whenever you start your module you begin with learning objectives but more than just writing your learning objectives to make sure that your learning objectives are understandable to your students and i think this latter bit is important because of course taiyo mala teachers uh we've been educated in the school of bloom alumni words from from remember to evaluate to create and you know for teachers who have been teaching teaching for quite a while this these vocabulary words are so easy that to to just pull in and and make part of our learning objectives but when you read it i think it's worth asking yourself if you're a student do you really understand what this learning objective means do you really understand what what the learning objective is actually saying so it's important to think about that um just to share a little bit about this in a school that i observed an elementary school there's a very interesting strategy that i saw one of the first things that things that they did in their class was for the teachers to actually present the learning objectives as they would in a lesson plan and then one of the first activities they ask students to do is to rephrase the learning objective now so for example construct effective online assessments that provide valid evidence of student progress um of course grade school you know example appropriate but if it just just to relate it to that uh what they would ask students to do is to rewrite the sentence in their own words so that might be an interesting strategy that in your modules maybe you can include something like that um to not just force students to read the learning objectives because to be honest um i i most probably a lot of students don't actually read the learning objectives part seriously um so maybe just to make sure that they actually process the learning objectives it might be worth maybe asking a question about it asking students maybe to rephrase it so that the learning objectives are not just presented but really processed and again the goal here in feed up is to is to is to allow your students to know that this is the goal of this particular module the mana when they're going through the module they become conscious that at the end of the module this is what the module is really all about these are the skills that i want to learn from from this particular module another strategy i've seen some in some schools is to rephrase uh the learning objectives to i can statements i don't know if you've tried that before now for example instead of saying plan formative assessments to offer helpful feedback the way they will present it to students is to say something like i can construct a formative assessments that provide helpful feedback so even just rephrasing the learning objective can really do wonders because the language will really get in the way sometimes when if we really want students to own the learning objectives um a second thing that i'd like to pitch today is don't just present your learning objectives in the beginning of your modules but also present your assessments uh and i guess for for some of us this might not be this might not be something that is that we're used to doing because usually when you think of assessments there's usually happening they usually happen at the end of a particular semester at the end of a particular quarter uh but then if we present what we want students to be able to do at the onset then i think the value of this is it frames their module taking so for example if you're teaching an english class and your learning learning objective is for students to be able to uh let's say let's just do something that's more elementary to be able to write a complete sentence with the proper grammatical uh uh following the proper rules of grammar let's say that that might be a learning objective um if if at the end of the module you actually want them to introduce themselves for example in the language for example introduce yourself in ten sentences in english it might be worth including that in the beginning of your modules to tell them that at the end of the module at the end of the module in english or in the particular language in your particular subject area so the goal here again is to try to shift the relationship know that that assessment is not just the teacher saying but rather changing the perspective and saying when we begin to shift these relationships become someone who is a guide rather than a judge so um of course when you talk about learning objectives it's a mix of your knowledge skills and attitudes or in the department of education your competencies those in attendance we call it the competence but i guess the counterpart based on what teacher jim showed a while ago is your competency and the competency is really an application of knowledge skills and attitudes so how does this happen and how do you actually create this how do you actually measure um a while ago there were some questions about uh are exams still valid or should we still give exams in online distance learning or in distance learning um i think there's still a space for for exams no but i know that when you talk about the exams for example the screenshot that you see here is a quiz in my in my online course uh there are a lot of things that really come to mind know so elephant in the room when we talk about assessments uh we ask the question is the student really the one taking the exams so issues of integrity are there opportunities for cheating students and there is a grade attached to the particular assessment uh you want to make sure so uh i'd like to pitch two things though so first is if you really want to give objective exams here are some exam parameters that might help you and again i guess the caveat here is there is some technology that might be involved that will allow for you to do this um so it might not fit all kinds of modalities so for example if you're doing some sort of online learning even if let's say it's just your assessment that is not online you might want to look into some of these parameter parameters so for example number one is to schedule your exam usually scheduling an exam or a quiz or graded quiz will help avoid cheating scenarios a timed exam is also a way of doing so so for example the exam will happen from 9 to 11 a.m it's a 20-minute exam so if you're using a learning management system for example i've seen that post about moodle or being or rolling out the learning management system timing exams is one of the features there that you might want to look into another feature is randomizing questions also again if you think about it this is not new no so palace and randomizing answers as well so again these are not surefire ways of preventing cheating but these are some provisions that might help you address the potential of cheating at least another strategy is to display one question at a time so for example if you're familiar with google forms and you know that you can create quizzes on google forms uh one thing that you can do is actually to use sections so that you ask one question at a time rather than putting all of your questions on one page and the value of that is is really just a harder it will just make it harder for students to pass on exam questions from one person to another but again all of these are sort of work arounds and i still strongly believe that my students i like how teacher radha mentioned a while ago that at the end of the day it's really about the relationship that you build with students and the trust but just the same if you feel like these parameters are things that you could set up they might help another strategy is to vary your question types and so for example there are two question types that i found helpful um so that the exams are not easily cheatable so to speak the first is a call is called multiple right answers so for example uh we're often used to multiple choice question exams but then there are exam types wherein there is more than one right answer so in those cases it's a little harder it gets your students to think a little more more especially for example if you don't say how many are the right answers there so they really need to evaluate all of the choices so that they can choose which one they think is correct another strategy i've heard of from some educators is to mix multiple choice questions with open-ended questions so for example um if we're so worried about cheating and integrity it's really the multiple choice choice questions that is really easier to sort of cheat in but then if you have follow-up question that that asks you questions about the multiple choice question that might be a strategy that you can use to to sort of um make sure that it's not purely objective for example uh you have a question uh for example in an assessment uh what do we mean by reliability and then you have uh option a b c and d and your and then that that's followed by an open-ended question that might sound like uh based on your answer can you give an example of a situation now the situation is not something that that that students can easily cheat about because it will be pretty obvious if you if you present the actual situation if a student if student one and student two will actually submit the actual situation so these are some strategies that you might want to to include if you feel like uh in your summative exams you're worried about integrity you're worried about cheating but another i think broader strategy is i think just to build on what teacher jim mentioned a while ago to ask yourself what you're testing that are you testing what students know or do you want to test what students can do with what they know and the latter is really a test of competence or competencies so if you're teaching testing competencies maybe exams like that might not be the best medium for you that when you're teaching competencies maybe you might want to look into more authentic forms of assessment in again in the course that we're teaching right now in latineo uh this is how we present authentic assessments authentic assessments are assessments where students are given an opportunity to apply their knowledge and skills um they're usually a complex performance that requires justified decisions and the reason why i'm showing this this definition is that sometimes when we think about authentic assessments um and maybe performance tasks are things that immediately comes to mind sometimes we think that for example in an online learning scenario but authentic assessments i think are not necessarily always authentic mainly because of the medium of the assessment video is a medium a performance is a medium but authentic assessments the authenticity of an assessment is really based on the task that you actually give students and why is this why is it important to to remind ourselves of this i think it's important to remind ourselves of this because this means that even if you're teaching in what you might say is a low-tech scenario in a paper and pen language for example you're doing printed modules and then your plan is they answer some questions they submit so the modules the worksheets gets picked up and then teachers will send the check worksheets later on even if you're in a paper-based scenario you could still generate authentic assessments for your students depending on the question that you ask them for example i'll i'll give this example at the risk of uh being mistaken because i know that we our moderators for today are adeline panlipunan teachers know so for example if your if your question is if you're if you're doing a unit on the branches of the government and your question is simply what are the branches of the government of course that's not an authentic assessment right because that just measures knowledge and skills but if let's say you create a scenario and that becomes the question that students write about for example in the time of kovid right now if you were to write to someone from one branch of the government to send them a letter suggesting ways to improve how we're dealing with the the current uh the current virus for the current uh pandemic who will you write to write a letter to imagine that you're writing a letter and write an actual letter to someone from that branch of the government and give your suggestions so if you think about it it's those two are writing exercises no uh you can easily imagine students doing that in pen and paper but the latter example just because of the scenario that you put just because of the nature of the task that you had students think about actually made it an authentic assessment so i think this is something that's that's interesting to to think about that authentic assessments are not authentic because of the medium they're authentic because of the task that you give students again which means that even if you're if you're just on paper and pen it could be an authentic assessment no so the keywords here are complex performance and that requires justified decisions that there may not be one single correct answer so here's an example from my class i teach a class called visual thinking right now sorry medical college example but i'm sure that those of you who might be teaching in secondary education can also relate you know uh i ask them she says they are business students i ask them to create a business plan so if you think about this a business plan is not necessarily something that's so techie something that will require a lot of multimedia in fact a lot of the work here is offline um and i think that even if let's say you're working on an online learning model for distance learning remind yourself that projects and assessments need not be online all the time need not be the students working on the computer uh all the time a while ago in the chat people were asking about things like pe no this champ recipe um if your learning objective is to be able to for students to be able to demonstrate a particular skill in physical education i agree with what teacher jim mentioned maybe the best medium best evidence might be a video of them actually doing it you know so and the actual doing it is really not the video making but the actual doing it is really practicing the demonstration and then just using video as a documentation so also think of assessments as things that students can do outside the module outside the printed module or outside the learning management system if you're an online learning scenario so i think that's my first point that i'd like to share for today that one rule for assessments is that assessments really help you set the destination that if you're able to clarify what your students what you want your students to be able to do at the end then hopefully that becomes a clear impetus or the clear catalyst for them to want to learn because they know that they need to prepare themselves to be able to get to the destination so it's really about making your uh end assessment or somebody that says it's more transparent you know and in a way we can say that this allows us to teach to the test because you've already told the students what the test is so to speak uh such that when they're going through the modules uh they will be very cognizant ideally of how the modules are preparing them to actually meet the needs of the assessments that you set for them the second rule is to pin the milestones i know some online shopping uh sites and one of the things that we actually learn in general is how to pin our locations so if you imagine the imagine distance learning uh i think this is one of the most important things that we'll need to do as a teacher to be able to set up milestones most especially if your your mode or your modality in distance learning is highly asynchronous for example if you've decided that in your school or if your school decided that it's mostly printed modules so if if you imagine that for most of the quarter uh the experience of the student is that they're going through the modules um the absence of the teacher there is to to help them say that i know that okay you're on the right track how do us how does a student know that he or she is on the right track and i think this is where our formative assessments really come into play uh and i'm glad that in in education parlance in basic education and even in our higher education we've been talking more about formative assessment over the past few years and fisher and free comes in here again to say that feedback is the second element of that feedback system or that some of that assessment system that ideally in a business learning scenario or in any teaching scenario there are opportunities for students to actually test their knowledge and force for teachers to actually respond to student work and i think the latter part is very important that responding to student work or student submissions is really actually one of the main things that is that makes formative assessment important it's not just the giving of the formative assessment it's what they get in return because formative assessment in in terms of the nature of it or the intention of it is really to help clarify where students are developing that's why i like the analogy of pins in a map but imagine that the imagery here is that formative assessments are really pins are really milestones that you set so that you can be you can tell students i think you're on the right track i'm not so sure if you're getting it right and things like that so how can technology help if you're doing quizzes in a platform one of the things that is usually readily available is preset feedback so if you haven't done this yet it might be worth looking into it for example the screenshot that you see here is using google forms so if you use google forms as assessment tool then for for the correct answers and for the incorrect answer answers you can actually write feedback and for me the feedback is one of the most important things that you will put in your in your automated quiz because it's in the feedback that you actually insert your teaching presence it's in the feedback that the teacher can actually be present to say yes you got the answer right or no you might have misconstrued the answer to be this but the actual answer is this because blah blah blah blah blah again because the goal here is to help students learn a little more and to confirm their learning and to confirm misconceptions another assessment strategy so ah sorry so when you talk about quizzes if you're not online uh maybe an alternative here is an answer key and uh i think i feel a little funny being able to say this because i think it's pretty obvious but then if you're not doing things online making sure that you have an answer key and your answer key ideally is not just the right answer um i think that's a good strategy to think about that if you have an answer key hopefully the answer key is more than just saying this is right or wrong hopefully the answer key also tells the students or informs the students why this is right and why this is wrong another strategy is to use portfolios again if you're in an online learning scenario this example is from my class i use google slides and i use i ask students to maintain a google slide deck throughout the semester so that i can ask them questions throughout the semester and then they can continue to build on this portfolio i know that portfolio assessment is something that is also common in modular teaching for example in alice i know that uh in in the al scenario the mere diamond compilation of the output of all of the modules so i think this might not be new to you um but i think a portfolio assessment is something that is really worth looking into as a formative assessment but again as a formative assessment the important uh thing to design and facilitate and to plan out for is uh when are the points where it will actually check the portfolio and provide feedback because that's important and i know that to be honest this is one of the more challenging things most especially if there is zero online as part of your distance learning uh because if you can if you can imagine that um what is the scenario where your students will be submitting their let's say their portfolio the portfolios will be shipped to the school for example the teachers will pick up the portfolio from the school the teachers will check it and then the teachers will return it to the students so um i know that this is one of the challenging things uh but if you are to think about something and to plan something out i think this is worth thinking about how can you how can you make that loop smaller what i've seen is is the insertion of some online learning as part of distance learning that is not online for example i know of a center that what they do is they have a messenger chat no and what happens is even if the modules are offline what they do is that in the messenger chat they do check-ins with students and this is where they can actually exchange ideas and this is where teachers can actually provide feedback so if you were to invest in something i'd say invest in that invest in a modality a way so that you can actually reach your students and provide some more timely feedback so feedback is important to validate the quality of answers and uh one of the ways by which you can do it is through adding rubrics so if you're in any learning management uh system uh one of the generic tools that you have there is an online discussion just like the screenshot that you're you're seeing here uh and i like online discussions because it may makes physical discussions because students can actually contribute to questions and you can actually see their answers but the challenge with online discussions i think is that if you have a large number of students you will also have a large number of things to read so imagine that if you have 10 online discussion forums and you have 30 students imagine the amount of time that you will spend just reading all of their entries so again it's important to be able to read all of your entries but imagine the latter scenario wherein you want to give feedback to them and i think this is where rubrics actually help a lot uh the rubric that i'm showing you right now is my actual rubric and of course i'm running at the moment i'm sorry about that working from home and it's a single point rubric and this is what i why i want to show you this because uh lately i've been a fan of single point rubrics now single point rubrics uh are rubrics that look like this you spend more time breaking down all of the criteria and then more often than not the points are binary you know so did the students submit it on time yes or no was it complete yes or no the quality of contribution uh from zero to two uh what does it um how do you rate the quality so having single point rubrics or a couple of these things um are really valuable because of two things first of all when we train students to use rubrics when they're working on their assignments it actually promotes self-evaluation that for example if this is a rubric that i'm looking at as a student and i'm contributing to an online discussion forum then i'm actually already self-evaluating myself that when i'm answering the online discussion forum i'm making sure that my answer abides by the criteria that is set by the rubric at the same time it also allows teachers to provide more timely feedback and quicker because all you need to do here is to score the room written so that might be something that will help again rubrics are not necessarily online exclusive tools uh it could be an offline rubric but to make a rubric available and to invest time in creating a rubric i think will go a long way in terms of feedback if you're using a learning management system i just want to show this just to to push the envelope no and just for you to explore what you can do with it that in most learning management systems for example you are allowed for students to actually respond in different ways such as posting images among other things so that's really about pinning milestones that when we talk about formative assessments the one rule is for these assessments to to allow students to confirm and as students and teachers to confirm or to clarify misconceptions along the way um and again i guess one major challenge here is really the loop how fast can you get that loop uh to be uh and again that that might depend on our context but it's good to think about how you can close the loop faster as much as you can so the last thing i'd like to share with you is that assessments 30 30. it's really also about charting the path and maybe this latter part is not about assessments per se but it's about what you do about assessments or what you do with what happens in the assessment and i think teacher gem touched on this a while ago in my classes one of the things that i like doing as a strategy is at the end of my module i always have what i call an exit ticket so an exit ticket is simply a question that you ask students so for example this is an example exit ticket from uh from from one of the modules that i'm teaching right now uh so for example based on module one how has your thinking or opinion changed or has has been influenced uh so usually i just have one or two questions and students will answer in two or three sentences but usually exit tickets are good ways for me to to gauge where they are what they're actually getting so far but i think assessments in this third role it's really an invitation for us to do what fisher and freight talks about when they say uh feed forward you know and this is what teacher jen mentioned a while ago when she said in the latter part of her presentation that when we talk about assessments one of the important things to think about is how you will use the data and ideally assess assessment data is used to modify instruction um in a face-to-face scenario this is easy to imagine for example if you give a quiz and then they they don't seem like they're getting some parts of the items uh then you'll probably reteach no uh in an online learning scenario maybe it's something similar you know so for example in the in the example here uh this is from an actual class that i just recently thought i had to adjust the course content uh because i noticed that in the in the online learning management system uh when i read through the interactions with my students i was giving them too much work so i had to adjust a little bit of course if you're not in an online learning modality adjustments might be harder to make so maybe the way to feed forward is to do interventions uh but again the concept here is to think about what the data means to modify your instruction or to modify the learning experience of students moving forward now so here's another example in in our class in ateneo we don't really endorse a lot of synchronous sessions again mainly because we know that not all students can connect to the internet so we want to minimize synchronous sessions so most of our courses are actually asynchronous and what i try to do is before the synchronous session i ask them to give me some questions that they want me to answer um presupposing that they've been doing the asynchronous sessions beforehand so this might be something that you might want to think about also that when you actually so right now i know that a lot of us are designing the modules but as we approach the school year when the school year begins the students will now shift to actually working on the modules so your role as a teacher will now change so what will be your role and i think one important role is to be able to get some feedback so that you can feed forward that maybe every week i don't know if there's some way to communicate to students to check on what they've gotten the mojo so far what is not clear that might be a good thing to invest in so that you can actually feed forward and do something that will help them moving forward because again when we think about assessments in distance learning hopefully it's not just something that happens at the end but assessment uh for me or assessments rather actually helps you map the learning experience no by inserting these things i think the intention is to really make the teacher more present that in the absence of the teacher in most online learning scenarios or distance learning scenarios most specially because most of it is asynchronous assessments actually serve an important purpose because it allows you to help minimize if not eliminate the transactional distance and the misconceptions that might happen along the way the gaps between the teachers instructional intention and the student's actual experience or what the students are actually learning or not learning so i'd like to end with just some key ideas i think the big idea i'd like to share this afternoon is that to reframe how we see assessment that assessment most especially in distance learning is really about making transparency and communication happen between the teacher and the student it's not just about asking students to do things but it's really about using assessments to guide them in their journey in online learning and i'd like to end with this quote from william in topham what william said is assessment is the key process in instruction it is only through assessment that we can find out whether what has been taught has been learned so again the first part of this uh this quote uh it's about the shift i mentioned earlier you know the shift from an attendance mindset to an assessment mindset so not just not just to think about attendance are they viewing my module or not not just to think about that question but to think about are they learning or not because the learning or not is a more important question than are they viewing the module or not assessment is therefore the bridge between teaching and learning but in as much as we spend a lot of time designing for teaching most especially in a distance learning scenario it's still the learning that's important and as we know good teaching is not just giving students content and leaving them to their own means good teaching is making sure that learning actually happens and so i think that when you think about validity and reliability broadly speaking it's really about how assessment continues or will continue to support learning uh and most especially in distance learning i think that's important to think about so thank you thank you very much for the invitation to share with you this afternoon and i hope that there are some tips that has been helpful salaam assalamu [Music] uh if you have questions or um things that you'd like to ask to our speakers uh sir galvin and miss gem uh please uh feel free not to uh to ask them um yeah so valid the reliable puba and online assessments um in a very brief suguro uh recall sir um sir galvin um how do you see this young question of someone doing it or a question of integrity or assessments [Music] cigaro i can start and then to teach you them go ahead go ahead guys um i think that uh one of the things that i also learned today from teacher gem is a lot of us discuss this both gem and i discussed this lengthy today you know that it's really in the design of your assessment that actually makes it valid or reliable not necessarily the form of the assessment um whether you're thinking about assessments online or offline it's really how you design the assessment that will lead to validity and reliability um not necessarily the medium of the assessment and thank you thanks for getting jen any thoughts on that um [Music] um now we as teachers we should act not as their judge but as their guide so whether assessment or other activities is valid or or not first we have to make sure first that we guide our students that our pupils were properly guided by us and then we can also get the feedback from our pupils i think for me that can be also we can also consider that that's one way to know if of what we're doing is valid or not um some specific question i think this is from your presentation in the comments [Music] the obvious answer here is uh the assessments were in you have answer keys in should not be your summative assessments or should not be your graded assessments um i think um maybe to clarify uh this will probably be your formative assessments i think the value of the answer in a formative assessment is the feedback um again if we're working with a model we're very limited communication uh that's really to a certain extent no well students are going through the modules that's really one of the only ways by which you can clarify their understanding so um maybe just to clarify it i was really talking about a more formative assessments associations so definitely uh [Music] um as a form of data collection about our students learning or independent learning set as a way to improve instruction to improve other activities etc so again formative delegation [Music] um [Music] um you have to make the leader feel or the students feel that you are with them even if they are just nearly as bleeding even if um foreign is it all right to give points for formative assessments um anyone caught from from the panel uh recorded you mean may i just clarify you you want to record the formative assessment or i mean record included in the grading system or because you can always give points come on to your assessment but may i just clarify if this is um like including this as part of the greeting system component is [Music] ten percent of the hundred percent formative assessments we also want to give them you know proper you go one [Music] a big percentage also or chunk in the gradebook the primary objective objective why you have used formative assessment is for you to have a check can be necessary for them to um is really more for example the timeliness because i wanted to be able to submit on time quality of the contribution so if you remember that room um if you if you decide to grade formative assessments um you just have to be careful that you're not grading right away their their understanding of the all the time of the content all the time your purpose informative assessment [Music] then i think it's okay just make it clear to of course number one to yourself [Music] so what is the best way to give summative tests to our learners formative assessments summative assessments [Music] especially for first quarter or second quarter and that's why we have been including our education on um authentic assessment performance solution would always require to have a periodical type of assessment then there should be nothing you can do but of course there are other ways like um rewriting the best questions there foreign a while ago it really depends on on on your school because we have different uh setup first try but at least we can do better next time so we just have to always try and try and use a guru we can get what is best for our students hi nice one thank you so much for that mr adan sir galvin [Music] i agree with what jim mentioned a while ago that measure competencies rather than knowledge skills and attitudes in the first place want to process the pandemic the situation to unlock the materials like in terms of like the zone of optimal development is young materials they're readily available to them so you lagging in easily [Music] how else can i expand or deepen their learnings in relation to uh to the competencies and skills i'd like to develop with them so yeah you are developing or crafting your assessment you have to make sure that you know how to do it you know um electronic portfolio e-portfolio it's very important that when you ask them to do something if you can give an example you can even demonstrate how to do it more especially for the online learning modalities is up and running so mama upon 5 30 p.m um visit the website um you can find it here sababa video nothing um comment section description one do the evaluation and two make sure that you have um complete um answers to the assessment part so my assessment part of the thing uh topic a crafting valid and uh reliable assessment performing assessments evaluation so make sure [Laughter] so that we as teachers uh there's so much to learn we just have to be open-minded anyway uh what we are doing is all for our students forces collaboration critical thinking creativity and communication in connection to ensembl tools would be online tools or uh in modular format naped young mitten to develop these particular skills in the 21st in this distance learning uh to continue developing this uh 21st century uh learning uh skills okay so abandoned next nascent now webinars um teachers and i will be saying goodbye now they will be preparing your evaluations for this webinar so my name is again next time you

Channel: Franco Nicolo Addun

Views: 247,195

Rating: 4.8206277 out of 5

Keywords:

Id: cpaPQiNozqw

Length: 177min 8sec (10628 seconds)

Published: Sat Aug 08 2020