Title I Directors meeting 11-18-2020

seeing that yes we can see it oh thank you so much so we have quite a bit to do today we have some wonderful um colleagues from usbe joining us this morning to help with presentations so here's a look at our agenda you can see that it's quite packed so i didn't plan a break but if you get to the point where you need a break you can feel free to take a break on your own becky peters is going to record the meeting so you should be getting a message asking you to click that it's okay to record the meeting so if you couple of minutes or have to leave early or we know you're busy you'll be able to access the recording later um some of us were in a meeting yesterday and the one of the presenters shared this quote and i i love quotes i love reading quotes i love finding them i love you know finding inspirational poetry and things like that but this one i really really love i wanted to share with you today and that is individually we are just one drop but together we are in an ocean and we're powerful and we have you know amazing things that we can accomplish so i appreciate you being here today joining us i i'm thankful for everything that you're doing on a daily basis to support education in your lease and in your schools here are some reminders a number of you have sent in your 50 carryover waivers if you have not done that yet the deadline was november 1st we're still accepting those but you need to get those in quickly if you don't submit those and get those approved then you run the risk of having some of your funds reduced and we didn't have an lea last year that lost funds uh because they didn't submit a waiver and they have excess carryover and so that's part of the federal elementary secondary education act statute for title one so please get those forms in if you aren't sure what to do please reach out to tammy long and um she can ensure that you get the form see i've got to cancel my alarm here that's going to go off okay um the desktop monitoring instrument is due december 1st your title one applications are due december 18th um as of yesterday afternoon we've had 25 lea submit their title 1 applications a number of those have gone back for revision and things that we're seeing that are requiring us to send things back for revision are schools with low poverty rates but indicating that they're a school-wide program okay that's not legal we've had to send some that we've had schools leas that are not indicating which of their schools are in turnaround csi or tsi status you need to mark those those are some of the common things sometimes people are leaving part of the application blank or not checking insurance boxes appropriately we sent out detailed directions on several occasions so if you follow those directions you should be able to get your applications in and approve without having to do revisions uh here are some upcoming dates for you to keep in mind and i'm going to go to the next slide usbe has arranged with christine and manassevit law firm they're a law firm located in washington d.c their focus is federal education law so they will be doing um a workshop it's free of charge you it is for federal program directors and business administrators usbe staff that works with federal programs and federal fiscal issues so you can register at this link but becky peters this morning sent you the powerpoint and there was a flyer about this workshop also so we encourage you to register for that those attorneys are really great presenters and it will be an opportunity for you to get some good in-depth fiscal training on federal grants the uniform grant guidance and the code of federal regulation which applies to all of the title education programs special education internal nutrition so we encourage you to participate in that the national esea conference which will be virtual this year registration is still open for that we're hoping that some of you will also be able to attend that this year since there won't be any travel costs it will significantly reduce the cost of participating in that conference and it's always excellent so um i'm going to turn some time over for these next few slides to tammy long who is our fiscal compliance manager for title 1 and also title three good morning um for the first slide here um i just want to let you know that the financial folks we all got together because everybody we had a lot of lease complaining that everybody wanted different items and so we kind of standardized the items that are going to be required on reimbursement requests and these are for you know federal funds including the cares act funds because these are all federal funds so um we need detailed ledgers when you're submitting for these funds and a journal entry is not considered to be detailed enough for us so we need more information like we need to know who is being paid with these federal funds and the specific purchases that are being purchased with these federal funds and you can either do a detailed ledger for those items that are being purchased or you can submit your receipts whichever works best for the lea it's it's up to you guys what you provide but we just need to know that if you make an amazon purchase of three thousand dollars that it was for um appropriate and allowable uses of the funds and make sure that as an lea that you keep all the receipts because all of these funds will be reviewed at some times whether they're from the state or the feds so we need to be able to support um all of the reimbursements that we've done and make sure it's been a little confusing on the cares funds um people aren't aware that they need to keep time and effort on these funds but they are federal funds and time and effort documents will be required so make sure that you get all those gathered up and ready for to be scrutinized because they will be in to look at how we have spent the funds next slide um the federal funds they've had some extensions on the federal fund dates and just so you're aware we kind of did a timeline here so your 2019 will be due september of 2021 september 30th and then also the federal funds the 2000 funds will be due in september of 21. and all of your esser funds they will be due in 2020 of september and it makes it a lot easier if you request these funds like every quarter so that you're up to date and don't have to do them all at once and it it's it makes your risk assessment go down too if you do them more often than just one time a year next slide so when you're doing reimbursement requests financial ops downstairs has set some dates that if you don't have your reimbursement request then in by those dates which are listed over here that you won't the funds will not be on your allotments that go out the end of the month and also if you wait till the last date and there's a problem we may not get it all approved before um the end of the month so it's better to do it a little early than to wait till the last date to um get them approved because it's it's kind of hard when everybody waits till the last and if there's a problem then we have to send them back so please submit those in a timely manner and just be aware of those dates so that you know when you have to have your reimbursement in and that's all i have thank you tami so if you'll be really careful about those dates on that previous slide making sure you're paying attention to the dates when those funds expire is really critical so we do not want to send money back to the federal government also you need to be aware that our state auditor's office is going to be doing a single audit at usbe between january and march title 1 will be part of that but they've also indicated they will be auditing the cares act money so that's why we need to make sure that we have documentation and you're spending your funds on things that are allowable costs i'm going to turn the time over now to our assistant superintendent of student learning darren nelson and he's going to provide you with some information about some plans and possibilities that will be happening this school year for assessment and accountability so darren thank you rebecca it's nice to be with you guys this morning and i see i have 20 minutes on the agenda and i'm grateful for for each of those minutes um it'll be a challenge to to talk about maybe all of the things that some of you might want to talk about but i'm going to begin by giving an overview of where we are with assessment going into the 2021 school year and what our what the board's uh interest and focus is in regards to that and then we'll translate we'll transition into a little overview about accountability remembering that assessment in many ways is different than accountability although they get confounded because of the connection that they are happen through a school accountability system so i want to start out by talking about the threats to assessment this isn't news to you but these are ones that we've identified and are addressing to the best of our ability as we move through this certainly unusual year the first threat is equitable access you know the question of do all students have a little legitimate opportunity to access learning and then also to participate in any state assessment that we administer utility is the next um idea that we talk about in terms of a risk or a threat to assessment you know under under the circumstances i think i saw on the news yesterday a number of around 40 schools right now are in soft closure statewide i think that number is pretty close to what they published i mean that number as you guys know fluctuates we have schools that come in and out of 14 day full school quarantines and then we have individual students obviously who go through quarantine and teachers who go through these quarantine processes and looking forward we have to consider how that affects the results of the assessment the next risk or threat is the limited resources these these are legitimate questions of if we have access to our students in a face-to-face setting that's limited to what we normally have and we we know that instruction is the primary purpose of school where does assessment fit into that and is it justifiable to allocate resources a time when we have students in a face-to-face setting used to administer assessments and then logistics which this is actually the primary risk and that is is it safe and feasible practical to administer these assessments to students this isn't a comprehensive list again but it's some of the things that we've begun talking about last summer and have involved our technical assistance committee and our advisory committee through the state board of education as well so the answer to to those questions or the question that i get sometimes is should we continue to administer statewide assessments we believe that the answer to that is yes despite the risks that exist perhaps more than any year that i've been involved in education there's a need for us to be able to get increased understanding about the impacts of the events that have occurred on our students learning through on there just a information i received from the hunt institute a survey that they did some of you might be surprised that that number is as high as it is i actually think in my experience in education that every parent wants to know how well their students are doing but in their survey 89 of parents want to know the academic impacts of coba 19 which again points to our interest in being able to go forward with administering state assessments to the greatest extent possible remembering also that these decisions are based on our current circumstances and we recognize there are still a lot of unknowns about the future so our assessment accountability policy advisory or actually there's someone talk before we get to that mitigation tactics so how do we mitigate the risks that are present for administering statewide assessments first one and this is this would be what i would call low-hanging fruit is to extend assessment windows wherever possible and that's happened with the kindergarten entry test it's also happened with the acadians beginning of year assessment we work with the assessment directors and to determine whether we need to extend additional testing windows and we'll be monitoring that as closely as possible the second way to try to mitigate some of these risks is to support remote test testing conditions whenever feasible and i'll talk a little bit more about that in a coming slide and then the last the last one i think is just an increased tolerance for reduced student participation if we go back to the threat um one of the threats to administering statewide assessments and that one of those being the health and safety and the well-being of our students we know that there are students across the state who've chosen under the direction of their parents to take virtual learning or remote learning as their option for this year at least to this point because of concerns about their health and safety of either the students themselves or other family members that reside in their home it'd be unreasonable to ask somebody who feels concerned about their safety and well-being to then come into a school setting for a test when they're also unwilling to come into a school setting for for instruction because of the concerns for their safety so just an understanding that we probably will have reduced participation i do want to pause for a moment here though we have been able to complete across the state the kindergarten entry assessment and just to give you an idea of participation rates we're just beginning the analysis of that data but we tested 43 725 kindergarten students this year and that represents 93.2 percent of the kindergarten students who are enrolled this fall that participated in that keep test in a typical year that number has ranged around 96 percent so last year for instance it was 96.9 percent of the kindergarten students completed the keep in addition to that kenyan school district recently one of their board meetings published um the the some analysis on their um acadians beginning of year assessment and they they indicated a 91 percent participation for kindergarten again this is on the acadians beginning of year and 92 participation on first grade and 92 for second and third grade as well so while we need to have a understanding of an increased tolerance for reduced participation it's also very feasible that the vast majority of our students will be able to participate in the various statewide assessments so the next two slides are just an overview of our testing timeline to give those of you who might not be as engaged with assessment an idea of how this window works so we've already completed the peep the keep and the acadians beginning of your assessments this friday is when we're scheduled at the state board level to get the last of our data sets for acadians many of your leas have already completed and have your full data set for that and have begun analysis or maybe completed your analysis on the impacts of kova on literacy early literacy screener the apple which is our dual language test is also underway right now that window is open the asterisk on there i just want to point out that asterisk represents assessments that we've been able to again if we go back think back to the previous slide where we said support remote proctoring where we're feasible these are assessments that we have been able to support remote proctoring so what's how do we define feasible well there are some risks uh with administering assessments that's remotely particularly with our students with disabilities and being able to provide them access and accommodations that they might need to to have those assessments delivered to them in an equitable way but we also have certain certain tests that have protected item banks so let's use the act for instance obviously those item banks the test questions associated with those tests are highly secure and protected and if one of those test items gets exposed in any state in the country in any form or fashion that item has to be removed from the bank and is not accessible to any act test from that point forward so we have some tests like that and remote proctoring has not been able to address and resolve concerns around test security is one of those areas so you'll see that you test fire plus our rise the ect are not currently available for remote proctoring options but our acadians peep and keep we've been able to make some advancements in that way so let's talk about if people will say to me then if we're able to um deliver assessments to our students and to the vast majority of our students why why wouldn't we use that data for accountability um well when we talk about educational statistics and and data associated with that especially in the context of making determinations or value statements about the effectiveness of individual schools this becomes very complicated when we consider all the variables that have been changed with within our students learning environments and also the test administration procedures that will take place so we're proposing we believe as well as many across the country the assessment data should be decoupled from accountability um which which again there's various rationale for that but that's the position of the state board so in their meeting earlier this month they made two motions one had to do with the way we calculate student growth or student growth percentiles if you those of you who are involved in assessment have probably recognized that in our accountability manual the way student growth percentiles is currently calculated and has been since 2014 in the state of utah is to use a annual calculation for growth in fact in order to get a student growth percentile a student has to have two consecutive years of achievement data in a content area test so third grade a student takes english language arts and math content area tests they get an achievement score but they don't get a growth score because they only have one year of data in that that content area the following year students who continue to be enrolled in the state of utah and participate in our state assessments second year that they receive an achievement score for language arts or math in this example then they also calculate a student growth percentile and they get a student growth target associated with that and that's the information that's used to determine the amount of points in our school accountability system related to growth so given the fact that this assessment was suspended in 2020 we have a one year where no students in the state of utah have an achievement score so it would be impossible for us to use single year growth calculation method for any students in the spring of 2021 even if we were able to administer assessments to every student in a fashion that we felt comfortable completely comfortable using the data for accountability we still have a problem with the way growth is calculated there is an alternative method for doing that and that's the skip here method some states use that as their growth calculation method um we have done analysis through the center of assessment to determine uh what kind of variance that introduces into our system and so we used the 2017 to 2019 rise data sage to rise data and calculated skip your growth on that and compared it to the singular growth calculations which we already have in our data set from 2019 to determine if individual students were impacted either advantaged or disadvantaged if schools certain school types were advantaged or disadvantaged by that change and then also student groups there was uh moderate change there so much to the degree that we anticipated that it would change the outcomes of overall school accountability in 2019 if we were to do that given the fact that that was under normal conditions and we introduced a pandemic into that environment and then if we were to attempt to do that as well um the risk associated with that and the concerns that we would have about the validity of that are high enough that again our state board is recognizing that we then take steps to relieve schools from federal and state accountability determinations based on the 2021 assessment data then people in fact the board when they looked at that motion said what does it mean to relieve schools that sounds fairly vague and there was some thought behind the exact language that the assessment accountability policy advisory committee used there because there's still so many unknowns at this point we have more clarity because the u.s department of education has released additional information for us about procedures and processes we can use and then at the state level obviously there are a couple of areas that we know directly impact school level accountability using the uh the annual assessment data so this first statute 53 e5 to 11 is the one that requires us the state board to award an overall school rating or overall school grade that would be a statute that we would need to have the state legislature extend that waiver or take some other action that would allow us to not be required to issue overall school grades or ratings the second one is related to the school turnaround program and again we would need relief from the legislature from our requirement to identify the lowest performing schools in the state of utah and award the money that's associated with with that to them as well then we have federal accountability and we are engaged in the process of uh determining what our next steps are here there are three ways that we can seek relief from federal accountability one is an addendum the us department of education has issued what i'll call an expedited addendum process i don't know that they call it that but it's laid out in a way that would allow state education agencies to fill out a form-like process for addendums and there might be areas where we need to seek an addendum the next option is to take a follow the amendment process to our consolidated plan and that's if we make changes that go more than a single year i've alluded to you that our growth calculation is impacted for multiple years when we miss a year of assessment or we have two years where we're asked not to use data for as our board said to relieve schools from response or school accountability related to the data in 2021 so it's likely that there'll also be an amendment at some future point the addendum is due by february 1 or the u.s department of education has requested states submit any addendums related to this year by february um first and then finally there is the the amendment and the waiver those are standard processes that are available in any year for states with their consolidated plans and the final option then is a waiver the state department of education has made it clear that unlike in the 2020s the spring of 2020 where they they awarded waivers to to accountability and assessment across the board to any state that applied for it they were all granted those waivers they do not expect to award waivers for delivery of state assessments in the 2021 school year um however they were not as in as strong a position related to accountability um so it's possible maybe even likely that in the coming weeks we will as through the state board of education need to seek a waiver for certain provisions in school accountability as well so those are those are the the steps that lay before us the path that we're on again to summarize we're moving forward and hopeful that we can collect as much state assessment data as possible and then we also are taking actions um i'll be discussing with the education interim committee this afternoon some of these slides that we've talked about this morning with the intent of of seeking state flexibility from school accountability and so information about that while it won't be final until the legislative session is completed we'll provide updates throughout that process the final thing that i wanted just to give you an overview on many of you may not be aware of this i think everyone had a chance to respond to our survey back in september related to our development of an assessment strategic plan this slide just depicts the steps and phases we went through that plan is scheduled to go before the board in december at their board meeting in its design that plan is designed to do a couple of things one is to inform assessment practices in the state of utah for the next five to ten years it took a look at uh next generation assessments and what should the state of state board of education consider um it has has a strong emphasis on creating a balanced assessment system it took into consideration portrait of a graduate and personalized learning initiatives that are gaining momentum in the state of utah and then finally the that plan will also inform the writing of our next grade three through eight rfp which is scheduled to be released in january of this year so that's an overview and i'm open obviously for any questions that you might have do any of you have questions or you can put those in the chat box and tracy vandiventer is monitoring the chat box for us no questions yet okay and i'll uh i'll stay on the meeting for a short time and be able to respond to the chat box as well if you have questions that you'd like to put in there but thank you very much rebecca for giving me time on your agenda and thank you to our title 1 directors for the the great work that you're doing um across the state of utah thank you so much darren it's always a pleasure to hear from you so we appreciate your time we know you're busy so next we're going to move to um sarah young uh sarah's going to share some oh i can't get the slide to go just a second there we go whoops sarah's going to provide you additional information and updates related to the cares act reporting that she talked about in the september meeting awesome thank you becky i appreciate the opportunity to be here today um can i share my screen would that be possible um while we are working on that let me just kind of give you a a why i'm here um moment and so really when it comes to um our estering gear reporting i had been with you in september and kind of provided you an overview of what our um what our plan was and our timeline was related to the reporting requirements that we had seen from the us department of education noting that they were in draft format um so as a part of that um they closed public comment and they actually responded to public comment which was great because we had asked for some clarification in spaces the unfortunate moment was that they then decided to do another round of public comment which was released to the field on november 1st of this year so as a part of that we have a new updated tool that they are receiving feedback on um but that put the state board of education in an interesting spot where we were um you know kind of debating the pros and cons of moving forward with asking for reporting um as well as you know then recognizing they could change some of those requirements you know later in december and that we we could have to come back um and kind of trying to figure out how to best you know thread the needle in terms of asking for information once um with confidence and yet not leaving you you know a window of only like a week with which to report um because the final report is due to the federal government on february 1st and they have not changed that date so it looks like i still can't share um i am wondering oh i hear i think there's talking that i can't hear i think if they make you a co-host you can share is that possible becky or should i just kind of give the verbal update and then provide the resource oh i'm a co-host meet your co-host so thank you so much perfect awesome all right friends so here we go um so this is version two of where we are so i'm just gonna provide an update of the changes from the first tool to now the newly released tool um and we're really pretty hopeful that this is gonna be um what the final version is the updates are pretty minor um but still worth going over um so again just a reminder they have now released a second round of public comment i will share this slide deck so that if you have any questions or want to see the resources i'm clicking on these links you're welcome to do that we will note that public comment closes on december 2nd which informs then when the reporting from leas is going to occur here momentarily so as an update i'm highlighting things in yellow that are different than we told you the first time in september so we plan to release the reporting tool for leas on december for 2020. so here's the plan basically public comment is going to close those comments are going to be posted that gives me and the rest of usbe staff the opportunity to look at those comments and see if there's any substantive that they could change or add or modify and then we'll use that to inform the release of the tool to the leas so that we are as close to the final version as we expect to be in releasing it on december 4th we are asking that leas respond to that tool no later than january 8th of 2021 so that will give you kind of the the um the reporting window that we're looking for from leas as far as the individual elements to be reported super pleased to announce they have taken away the methodologies for calculating equitable share um so it's just you know from the the changes in the department of ed rule to that being declared you know um not allowable and so they've removed that element of reporting the other two elements that we had previously presented still stand so that includes reporting on expenditures for equitable services in terms of the total amount of funds the leas have spent of their award within the reporting window on public schools versus those that have gone to equitable services um for private schools both for esser as well as for gear new to the reporting requirements um is luckily a yes or no i love when they add yeses or no's because hopefully that makes it just a lot more straightforward and less detail-oriented so um these are the financial use cases that they are asking um each lea to report on for their sea base awards so if you received a base award in regards to not having met that twenty one thousand dollar threshold your esther esser 90 award and then your gear award independently for each of those they are asking our least to reflect if they have purchased educational technology with that award yes or no um activities focused specifically on addressing unique needs of low-income um students so for everyone who received gear that's a yes um in regards to your esser and your base awards you would have to um declare that uh locally providing mental health services and supports yes or no sanitization and minimization of spread of infectious diseases including cleaning supplies ppe training for staff yes or no summer learning and supplemental after-school programs yes or no and then they have an other category so that's a yes or no with a descriptor if you choose yes in terms of reporting on that element um full-time equivalent positions has been revised which is great because we had a lot of questions about what that was going to look like in more detail and so the department has clarified that they are seeking the total count of uh full-time equivalent positions at each lea for the following dates so there are four dates that they are looking for leas to report on the first being september 30th 2018 the second being september 30th 2019. they're asking for you to draw a line on that march 13th 2020 date of the federal national emergency and then september 30th 2020 which is the end of the federal reporting period this will actually go into our reports for both esser as well as gear but we're just asking les to report it once because we can use the data set to report in both spaces um i've included a table just so you can kind of get a visual of what the department is asking for in terms of just looking for those numbers but it follows the narrative above as far as the remote learning and cares funding all of this section stayed the same except for they added one additional criteria that they're asking for leas to reflect yes or no on and that is if the lea implemented a method around participation in individual coaching or check-ins so that is a yes or no question on just one additional item that they're looking for clarification on again here's the draft slide you'll see that they have added on that particular item in regards to both elementary as well as secondary and asking lias to yes no reflect in that space as far as student internet access and cares funding all of this was maintained so they are still asking for yes or no's related to leas using cares funding for student internet access and then asking for yes or no specific to the strategies for internet access that the lea provided to the field and again i've included the the slide or the draft table on the next slide just again so you can have a visual to follow with the narrative student devices for lease this did not change either so they're asking for a yes or no if the lea provided dedicated learning devices for students in elementary and secondary and then the number of students with a dedicated device from the lea in elementary and secondary so we will go ahead and fill in your information related to the december 30th student enrollment count we'll use your october first counts related to that um but we are asking you to report how many students have a dedicated device in elementary and secondary related to your lea for the gear funding report um we actually didn't see any changes there they were all captured above and the items that will be reported for esther as well so i've updated this slide to kind of show you the breakdown with the additional elements such as the financial use cases the number of ftes for the following dates um and then making sure that it's it's co-reported in both ester and gear um as far as the state of utah reporting this is all the exact same as what we showed you in september um so this will just look at um asking about ppe procurement in k-12 yeses or no's related to did you procure these items using cares funding training for teachers did the lea use funding for training for teachers in relation to covid19 and if yes asking the lea to report the total amount of cares funding that was used to tune teachers again within the reporting period so through september 30th 2020 infrastructure remote and blended learning again this is the same as what we presented in september just asking for yes or no's related to the procurement of specific solutions for remote and blended finally the same as before optional narrative for this date this is where you have an opportunity to provide additional information with the reporting window being what it is part of that is also to be able to provide this information to our legislators in terms of informing their next steps that they're going to be taking in january february and um the first week of march in terms of um next steps for the education budget so this is really your opportunity to let them know what the continuing needs are and if you have any feedback within that space it is optional you don't have to include it um as far as logistics goes just a one last update um so the data will be collected through a qualtrics tool um so we will send an email asking for the reporting it will go to title 1 directors bas and the award points of contact on esther gear and esther 90 so i'm highlighting this because if you are one of um the um individuals um clearly as title 1 directors within those spaces you also may want to coordinate with your ba as well as your reward points of contact to figure out who's going to be responsible for this um and just make sure that it's not duplicated because we certainly don't want leas to be having more than one person trying to do this but we also want to make sure that we're really clear um that everybody has an opportunity to work together in this space um the full tool will be available on the first page of the survey we often get that piece of feedback where it's like i need to see everything up front um so we'll make sure that that's available quick and easy download it'll be included in the email as well so that you can prepare your answers and then just enter them one time so just a friendly reminder our plan is to release the tool to lease on december 4th 2020 um and we're asking lease to have that information back to us by january 8 2021 um and we'll be doing individual follow-up with leas as necessary from that space so i really appreciate the opportunity to give an update becky thank you so much and i certainly recognize that this is um as they've you know kind of created this cares opportunity and these awards um in a very quick time fashion that the reporting requirements have come after that um and that it's definitely you know a bit of a moving target so we appreciate you um you know and your leadership and working with us hopefully these are all very minor tweaks and changes um and if you have any questions you are welcome to reach out to me so with that i will stop screen sharing and i will turn it back over to becky thank you so much sarah we had a couple questions in the chat box first of all we can send your slides out right yep and i'm going to drop a copy um into the or a link into the chat box right now great and then the other question um one of our directors wants to know if they're only reporting on this uh point in time or what they plan to how they plan to expend funds throughout the year it's a great question so um we will be very clear that what we are asking for you to report on is going to be the time from the uh march 30th kind of earliest date or not march 30th march 13th earliest date of award through september 30th of 2020. so that is the reporting period so we're not asking about what your intents are moving forward there will be three reports associated with the cares funding so anything beyond september 30th of this year you will be reporting in your report this time next year there was actually another question if we could just confirm this applies to cares funding that's federal not county that is correct so um in all of these cases we will be very clear what we were asking for the numbers to align to so for example the use cases the financial use cases will actually ask you to break that down by what are the financial use cases for esser what are 90 what are the financial use cases for the esser 10 percent base um if you pr if you received part of that um and then what are the financial use cases related to gear so those are the three awards that we are speaking about um in regards to the use of this cares money thank you so much and just to make sure everyone sees in the chat that there is a link now that sarah has put for us to be able to access her slides and and have those available for use thank you cyril awesome thank you tracy for monitoring the chat box so i'm going to pull our powerpoint back up now so next we're going to hear from kathleen britton kathleen is the director of child nutrition programs at usbe and michelle martin is a coordinator in the child nutrition programs area so i'm going to turn it over to them i know you have questions about you know what's going to happen next year with um your free reduced rates and that kind of thing so kathleen and michelle well thank you becky um just want to thank everybody for including us and welcoming us to join your training um there's been many questions on october free and reduced numbers due to the facts with covet 19 usda allowed all students to eat free with that parents did not have to fill out a frame reduced at lunch application so we just received further guidance that these reports that we usually submit to usda have been put on hold because we cannot receive the data correctly our schools have been trying to do the october surveys but as you can expect most of the data was skewed due to the fact that parents did not have to fill out the applications what usda is telling us the department of ed has to make a decision on what year they want to use the data so we're waiting to hear what for you guys of what the department of ed wants to use if they want to use next year they'll use the 2019 but then going forward what do you use that's still all in the process right now of what the department ed can decide on and what usda is collecting i know our schools have been trying really diligently to collect this information but if a parent doesn't have to fill out an application to receive a free meal there that's why we're struggling to get this information schools do have to have submitted us the information we're still missing quite a few districts that have not and we'll comb through that but we've already seen kind of a discrepancy in that data so it's not really good data to pull for your title one funds um we'll be able to publish something in january if usda requires it of us does anybody have any questions on that part because i know that was the biggest concern for everybody the next i have is the smart i guess there was some questions becky had suggest had said to me that you guys were about the start smart utah breakfast that was supposed to be starting this school year through the legislative change that is we are now currently looking to submit a statewide waiver to the board to see if we can push that off for another year for the burden that it's caused i can have michelle step in a minute here and kind of talk a little bit about the start smart michelle do you want to talk about that a little bit sure so um we had heard that there were a handful of you that may not know what start smart utah is so i'm going to give you a little bit of background last year um house bill 222 passed and it was enacted this year and start smart utah is an expansion of the school breakfast program statewide so there are there's a a three-year implementation period and had coveted not been something we experienced this year schools who participate in the national school lunch program that do not currently participate in the school breakfast program would have to start implementing that program as of may 2021 and then if your if individual schools had a percentage of 70 free and reduced students or greater based on their most recent october survey um they would have had to and start implementing an alternative breakfast style and alternative breakfast style is defined as breakfast after the bell per the start smart utah program um and then in um and that would have had to been implemented as of may 2021 um and then following that in 2021 2022 50 um if your site had a free and reduced percentage of 50 or greater you would have had to start the breakfast after the bell and then the following school year if your um school site had a free and reduced percentage of 30 or greater you would have had to start um an alternative breakfast model there are um waiver opportunities uh if your school site is experiencing a logistical or hardship implementation issue with starting start smart utah but like kathleen said because of the current circumstances and the feedback we've received from the food service directors who are operating your school nutrition programs we are doing our best to make sure that we have a statewide waiver available to postpone that implementation if you would like to move forward and it is feasible and reasonable for your sites to do that you can start those requirements now but like kathleen said those that flexibility will be available this year and then if we still are issues due to covid um primarily one of the biggest concerns is actually staffing because school sites are quarantined and then you know shut down the food service staff are also part of that population so they may not be able to prepare meals so they're experiencing staffing issues is really the biggest thing we're hearing about as far as um being able to successfully implement this right now from their perspective um but there are i will make sure to pop in the chat um we have a spark page that has breakfast information um about this too that you can read in addition and then i'll also share with you um we have a child nutrition canvas page where you can access some additional resources that give you information about um start smart utah so i'll make sure to pop those in the chat but do you have any other questions i can answer right now no other questions are in the chat um if you do have questions please feel free to reach out to either of us thank you thank you kathleen and michelle for taking time this morning to join us if we receive any questions that come in to us any of us will make sure that we get those to you but if you think of questions title 1 directors if you pop those in the chat we'll make sure that we we get kathleen and michelle so thank you so much we're going to move on now to dr christelle estrada she created a new award just recently that was presented at our last state board of education meeting the first week of november i'm going to turn the time over to christelle to tell you about these schools and the criteria and how other schools may strive to achieve these awards as well so dr estrada so that was applause uh thanks uh becky and tracy for checking on the chat and thanks to each of you for attending um so we're gonna look at this and one of the reasons we actually developed this approach uh was based on an underlying assumption the underlying assumption is we actually don't have to go out of the state of utah to find really wonderful promising practices and excellent teaching when we did title three visits you know there were huge commendations for a range of teachers throughout the state and the underlying assumption with all of that is when we're learning together then whatever promising practice actually aligns resources to evaluate program effectiveness and make sure that the focus is really on increased learning outcomes then how do we recognize that how do we learn from each other um so i'm going to talk to you about the criteria as well as letting you know that on december 2nd we're actually going to have our first uh open conversation um on zoom with one of the uh school districts and all their principals so what was this like a success criteria we'll go to the next slide the success criteria was actually determined with tracy and becky and one of the reasons we picked an enrollment criteria of 400 plus is that oftentimes uh schools with a higher population i have to think really strategically in relationship to braiding funding which includes not only title 1 and title 3 but state funding as well the second criteria was 80 to 100 economically disadvantaged so these were all title 1 schools one of the reasons we picked the percentage of 40 percent students learning english and 40 to 60 percent of students who actually reached their annual growth goal was to focus on growth and one of the reasons we did this was about four years ago we took all the eccles business school four-year scholarship students and we did from we looked at their course taking patterns and we actually discovered that when students become english proficient or 5.0 on wida before they leave elementary school they have equitable access to more rigorous coursework right so make sure that you're really clear about the way in which the criteria was developed guess the next slide so these are the five leas and the 11 schools of excellence now we actually did two categories the first category was the criteria that we first showed you and if you look at this this is pretty impressive in relationship to the range of schools who actually are very clear about individual growth goals and what it is they need to do to ensure that teachers this goes back to what darren says that they actually have access to their annual growth goal and that annual growth goal is determined by when the students so it's very personalized and differentiated it's when they entered our school system and at what level of english proficiency so the point is if you're a high school student you're a refugee student and you come in at the ninth grade and you're a level one then your annual growth goal is personalized according to you it's not the standardized form so we have canyon school district granite school district ogden school district salt lake school district and uh washington um i actually spent some time with the title one principals and granite school district uh and to do the presentations and we had this really really great experience where um you know all the principals and the people at that meeting were applauding and grateful and uh really willing and wanting to learn with and from each other so let's look at the next slide i hope the transition doesn't make you nauseous it's it's a one it's a called vortex and so i always find it really interesting so why these uh criteria so i want to focus on the research just a minute the highest effect size in hattie's research is collective teacher efficacy so the way in which teachers work together whether it's by grade level or however it's organized really shows that the impact of teacher collaboration is critical the other one is teachers personal estimates of achievement for students without focusing on uh what some people consider as deficits you know about five years ago there was this amazing study that showed 90 percent poverty or economically disadvantaged 90 english learners and 90 achievement so the question ends up being how is it that teachers actually uh perceive the students that come into their classroom and you know kids are really incredibly insightful and can read people so the way in which they perceive that other people are perceiving them does have an impact on them and then the last thing that hattie says is cognitive task analysis which means you know uh one of those ways in which the way in which the curriculum and teaching is designed is challenging and engaging for students so one question to really ask ourselves is you know as individuals as people in the state agencies people at the district level as principals everybody in leadership and every educator so what are we actually doing i'm not talking about thinking about i'm talking about doing to mitigate discriminatory practices and policies that limit equitable access uh to close utah's opportunity gap now the reason the reason we're using opportunity gap is that the question ends up being who has access to which courses and why not looking at the student and saying well you know based on your achievement we don't think you're going to be successful in this ninth grade ap class right so that's the piece that's really important okay so then uh we have we added a special criteria so the next slide please so this is pretty important and the reason again it's really important is that elementary school students that actually reach weda composite of five we use that criteria of 10 or more of the students in schools of excellence we added this additional criteria and the state average for becoming proficient for students learning english is only six percent so we thought that ten percent uh was an incredible challenge because it's you know way above what the state averages so if you look at the schools uh the top school is backman in salt lake school district with 13 uh edison in salt lake uh monroe and granite odyssey in ogden spring creek and provo escalante in salt lake liberty in salt lake and mountain view in salt lake so again remember we're going to have this really interesting conversation with the principals of all the schools in salt lake and we'll do the invitation is out for other districts um with principals and teachers as well as um district people and even students so we'll have this conversation it'll be recorded and i'll send out a notice of december 2nd and when that will happen i think it's 10 o'clock in the next slide so i'm open to questions and comments and the next steps as i mentioned before will be school panels so we can actually hear from educators we're looking at other data sets one is to actually close the opportunity gap in high schools and we already have kind of a list of those top high schools which i think you'll really be surprised frankly but they're actually closing the opportunity gap which means that students are becoming better prepared who are english learners for post-secondary success so any questions or comments i'm open to anything one question from nathan marshall is the percent of ell students who reach proficiency ones that started it at a one or does it look at all students scores two through four and reach a five yeah it's all students learning english right because remember uh the whole point is whenever you come into our school system the annual growth goal is determined based on your proficiency when you actually come in you see the point the point is really important because why would we have a standard criteria for annual growth goal when you have an eighth grader who comes in and they're a refugee student right so that's really the point that's why this whole idea about personalized instruction and the way in which we differentiate and how we use the individual student reports which granite does very well i was at their title one meeting and they were really spectacular in relationship to helping use this individual student reports which actually came out much earlier this year than ever before like the first part of september any other questions and i might just add christelle that that growth goal is in the accountability manual so if people are not sure we can get that to you hey crystal this is annalise how are you i'm wonderful how are you annelise it's good to hear you quick question um would you mind sharing the list of the top high schools you're mentioning so we could learn from them uh yes we're working out the details uh so that probably won't happen until i'm actually thinking it'll probably be like right after the december second panel and then we'll actually release the recognitions the other thing we're doing which we found out which i'm sure you're all going to be really interested in is the transition between grade three and grade four is critical in relationship to whether the students actually uh become proficient by the end of sixth grade and then we're looking at the transition really effective transitions between elementary and middle uh oftentimes well i'll just say the pattern is you know when students leave elementary and their whole world is kind of transformed they don't really make that transition very effectively unless the leas and the schools really work together with for vertical alignment the same thing with middle to high schools so we're going to be doing that so we can all learn from each other so thanks annelise for your question perfect hey um christelle just curious there were a lot of salt lake city school districts are they do they happen to be in the same maybe feeder cone uh you know actually they don't which is really interesting because when we initially when we started doing monitoring and looking at classrooms uh and so this is what i learned because every day i wake up and i'm thinking i'm gonna learn so much today and i do and it's always unexpected uh we actually thought we would do monitoring of classrooms based on feeder patterns it didn't work out that way at all what we actually found out later was the tsi schools designated for english learners were like all over the place and so then we had to really rethink about what we would actually do to provide support for schools in tsi targeted school improvement for students learning english so your your question is like really critical we're hoping we can actually figure what figure out what that looks like in salt lake one of the strategies we're thinking about is identifying all those students who became proficient in those schools and then i identify which schools they're going to which middle schools they're going to because they're not going all to the same middle schools and then trying to figure out how we mentor them to make sure that if they're english proficient they actually have access uh based on counselors programming for more academic uh courses so i don't know if that helps your question you know you're wondering but i always typed out the same thing too but it doesn't work out that way so thank you for the question thank you any other questions for christelle so i just want to say how much i love working with title 1 directors title iii directors and visiting classrooms which i haven't been able to do obviously since the pandemic but the work that you're doing is really incredible and so this whole idea of recognizing that work uh is such a boost to everybody so and learning you know from and with each other so thank you for your dedication and your commitment to our children really appreciate it thank you christelle our next segment of the meeting we're going to spend a few minutes um talking to you about state reporting requirements we're going to break into some small groups and then we'll come back to the whole group to gather your feedback and then also just some ideas of some possible professional learning opportunities that we might engage in or not again depending on the feedback we get from you and then following that cheryl ellsworth our family and community engagement specialist is going to present some information to you about a family engagement initiative that she's been working really hard on for the past several months to um get that get all the paperwork and everything in our office in place so that she can move forward with that so first of all we want to talk to you about led reporting requirements for state programs we have had feedback from the state legislature saying that they're receiving comments from lea saying reporting requirements are burdensome there are too many of them some of our state board of education members have reported the same thing that they're hearing from lea saying there are too many reporting requirements now in 2019 the office of the legislative auditor general spent quite a bit of time at usbe several months actually working out of our office interviewing staff reviewing all of the kinds of reports and information that that is being asked of leas and then they prepared this report which is linked right here so this is a performance audit of public education reporting requirements so if you're interested in reading that whole report you can do that and in there the olag the office of the legislative auditor general made some recommendations back to usd to review all of the state reporting requirements and then work in conjunction with the legislature and the state board of education to make changes to state statute or board rule or both depending on what the report was to try to reduce and consolidate some of these reports to reduce the burden on lease so darren nielsen has been tasked with leading up this initiative for the state board and initially there were five there were 96 reports that they identified through state code and board rule that were required so they identified five and presented that to the education interim committee in october and said some of these are programs that are still on the state books but they're no longer being funded so we eliminate that or here's a program that had a lot of requirements in year one but now the program's been going for a number of years leas are already you know doing this work and have been for a while we could reduce some of the requirements maybe related to those programs excuse me so they identified five out of 96 and the legislature said no there has to be more than five there's you know you're still saying there's 91 what can we do about some of those so what we want to do is gather feedback from you that we can then use to get back to uspe leadership and then darren can take that back to the state board and to the legislature to talk about what things can be done to reduce some of that burden before we break into those groups i want to tell you about a couple of efforts that are taking place right now at usbe that are designed specifically to help reduce burden on lease and one of those is as you're aware many federal programs and a lot of state programs for example digital digital teaching and learning um some of the others maybe k3 literacy and others require that lease in schools sometimes these are at the school level there's a comprehensive needs assessment that's conducted and so then that work sometimes becomes duplicative right because you've already had to do a needs assessment for a federal program and then you may be asked to do another one for digital teaching and learning or you know idea or some other state or federal program so we are working with the region 15 comprehensive center at west ed and we have been meeting for a number of months and we do have lea stakeholders involved and some of you in this meeting today are involved representing your lease so we're in the development stage right now we have these five leas that have agreed to pilot this when it's completed and then provide us with feedback so all along the way we've had these lea leaders who've been saying you know we really want this to be part of it that part right there we don't think is is that important so let's not really let's not worry about that piece of it so the the further goal of this is once we have this comprehensive needs assessment developed that will cover all of the federal programs and all of the state programs will be the first state in the country to have a comprehensive needs assessment that does that georgia has a nice one but it primarily covers their federal programs arizona has one that's good and it just covers the needs assessment in their schools but this would be comprehensive would take in all the programs so eventually once that's developed the next goal and again working with our state board and our legislature would be to develop a con consolidated plan so rather than filling out a title one plan a land trust plan uh you know k3 literacy plan and you know there's so many plans out there is then to develop a tool a similar tool where we could consolidate the plans so those are two things that we're currently involved in right now the cna is under development when that's done and leas have piloted that and we have feedback about how it needs to be refined revised before it rolls out to everyone then the next goal will be to develop that consolidated plan and there are some other states we heard from the state of wisconsin yesterday in a meeting they're they're working on this the state of michigan is working on this so this is taking place across the country as state education agencies realize that there's a lot of work for least to do and we have small leas charter lease that are single campuses and some small rural lease that have you know limited number of personnel to do all of this work so it does become burdensome so we're working on that the other thing under the every student succeeds act there's a requirement for a resource allocation review so we we again are working with the region 15 comprehensive center west ed to develop a process um and we have two leas right now that have agreed that if indicated they will pilot this for us we have a charter lea and a district lea we'd like to have at least one additional district lea to participate in that so if that's something that you would be willing to do you could pop that in the chat box you can send me an email so that we can loop you in as we continue to develop this more we want to involve some lease in that actual process so it's something that you feel is valuable it's not a punishment it's not meant to be a punishment it's not meant to be an evaluation it's a way of looking at how lease are allocating fiscal and human resources and other resources to support schools based on the needs those individual schools have so not looking at equitable or not looking at equal distribution of resources but looking at an on a basis of equity so we have a really you know school that has a lot of needs and achievement is not doing well maybe that school needs additional human fiscal resources to come in and help support that continuous school improvement process so what we're going to do is and again we cannot rescind reporting requirements for federal programs so if you say you know we really hate the desktop monitoring instrument we really don't like filling out all of the paperwork in utah grants for the title one those are not things we can waive we you have to fill those out that's a federal requirement so what we're talking about are state funded program reporting requirements so the questions are what what reports from these state level programs does your lea find burdensome what suggestions do you have to help mitigate those reporting burdens on leas and we want to review those recommendations we'll give those back to darren nielsen so that he can go to the next time he goes up to meet with the legislature for this this agenda item at the legislature you know there will be additional reports that can be considered for either being you know changed so the the burden's not so great or possibly even removed if there's not a need for it so we're going to break into some small groups for about 15 minutes so that you have an opportunity to talk in smaller groups with your peers then we'll come back from those small groups and have some time for for you to share out with the full group and people from the team if if they get randomly assigned into one of those breakout groups they'll be taking notes and information back but um this is really important to us so if we don't know what it is you find burdensome and it's just general you know we just don't like it there's just too much there's no way for us to know what we need to do to help support you and help reduce those burdens so i'm going to um assign us into some breakout groups and that will take just a minute so you'll just be randomly assigned and while you're doing that becky i just want to thank lisa and aaron for volunteering to come and be a help as we move forward and lots of people giving kudos to the work and realize how important it is to release that burden and so we appreciate all that the state's doing to try to reduce that expectation on lease thank you tracy so you should be receiving that invitation if you'll just click and join that room and then i will bring you back in about 15 minutes so i'm going to set an alarm on my phone so that i remember to bring you back this this this this this i heard some really amazing discussions as i tried to pop in and out of some of the rooms so this time right now we'd like each group to kind of report back to the full group and um we have plenty of time to do this so i heard some really great comments i heard several people commenting about they felt the the consolidated needs assessment would be useful to them um we're really happy to hear that um but i'd like to hear hear what different groups have to say and then please put things in the chat box as well that we can capture and make sure we get this information back to darren nelson and others so that we can actually start tackling some of your concerns this is tracy i'm going to share first and then i can watch the chat but our group spoke about that there's just so many of them and that they just keep coming so there doesn't seem to be a break and you think you maybe have done one because there's so many of the same types of things that are asked that um you know it feels like you've already finished that one so recognizing that there is a lot of a lot of reports and many of them we might not be able to address because they are at the federal level but especially those in our group who are feeling the most pain was our charter school lease where they have not only all of the funding reporting that's required for receiving funding but then they have another layer of charter reporting that's above and beyond that and uh recognizing it it just is a really heavy lobe and we actually went through and listed some of the state reporting that we could think of uh and then we had one of our charter school directors share that if we could maybe even organize it by the kinds of funds that come or maybe by themes so if we had a location to go we could fill it out and it would all be all the different reports related to one like all related to after school programs or maybe one that's all related to curriculum or um and then the underlying agreement was if we could have one place to go to get access to all the reports so just trying to track where do i go to get into it or how do i find where that's at or i don't remember exactly what that link is that would be a big big relief for some who are trying to keep track and they're the only person that's able to do it so and then it looks like leslie has shared from her group it's difficult to be efficient when the qualtrics forms won't save along the way so if we can make those changes the folks then need to start all over and a master list with links and directions would be helpful that came from our group as well that would make a big difference on how we are able to even organize some people have been just keeping folders and folders and folders with each of them and trying to date them so that they can try to go and find them and that's just a lot of extra work maybe we don't have to have other comments that could come in the chat or if anybody's willing to jump in turn off your mute and share that way hey becky i um i know you you popped into our group but um i was kind of taking some notes there um a couple of things that came up was um having a masterplace kind of to keep the data whether if that's going to be data gateway that is the crosswalk for all things data but then um because this year gifted talented ap and concurrent that we were able to get that data but it it didn't match um like what where wherever the state was pulling that data from and loading it it it was not matching local lease so if we're going to do that that it needs to be accurate data and we didn't have a dashboard to to be able to cross-reference that um there also was a concern about qualtrics that it's all dependent on the usbe specialist and if the specialist changes how we're collecting data reporting that data is different and if the specialist knows qualtrics the hairier the application is and the requirements are um so if you have a usb specialist content area specialist that loves qualtrics it becomes um a little more cumbersome and tedious um and that there also seems to be a disconnect between um utah like filling out the form from the from the content area specialist and filling out the form um in utah grants and then like hitting submit and everything gets approved to finance and finance our finance guys in our lea are a whole different beast and there's those two are not talking and meshing so like we submit that grant and then finance takes over and then never the two shall meet and there's not that connection there um where and it just sounds like there's a general consensus that back in the day uca had all of the different grants and there is one due date and one needs assessment whereas now we just every week we finish one grant and then another grant is due and we have to pull that all of this data again um and that seems to be a little overwhelming and cumbersome okay yeah that has been that's something we've noticed too with utah grants is we have you know we have the needs assessment that was in there we have you know abilities for lease to look at all their funding streams and figure out how they were going to braid and blend those funds which in utah grounds everything is in a silo right now so and within the chat we have a couple of ideas here too you know a small district there's usually only one person who's doing that whereas if you're in a bigger district often that's distributed across a number of people um state years funding recognizing that's improved and the data selection for that actually is pretty critical looking at in group one there was a need for many reminders for due dates so if we did have one due date that would help also just sort of a survey fatigue on people's part is it possible to load also title funds in spring summer we realize that's a federal program but do is do we appreciate the simplified title one plan looking forward to consolidated needs assessment and maybe a common due date uh group seven talking about consideration for charter schools with regards to reporting requests being in line with their school's charter requirement uh group six saying the reports often come at once having extensions this year for various reports has been nice to spread the work and the due dates come usually at busier times of our school year and being able to have more than one owner for each grant could also be helpful uh group four is excited the idea of streamlining uh the applications and then group six as a whole uh nice to see becky's email saying what changed in dmi for instance having these notices in the grant platform if we have one platform would be nice also if the prompts are the same as our last year's response was accepted maybe having the la's response sort of pre-populated on something that's already been approved unless you know there's a major change so it would be the same or they can just make small tweaks yeah we did check one plans you filled those out last year for the first time on that new form so this year there's no need to create a whole new one if you want to tweak it that's fine but you could simply you know attach the one from last year if you're still working on the same things we just made sure that planning capsules all of the federal requirements and the assurances that were necessary so um yeah we're more than willing to do that the other thing about utah grounds that we had an ability in in the utah consolidated application to pre-populate more things we're in utah grants it doesn't operate that well which has been frustrating on our end i know it's been frustrating on your end but you know the only thing they can do is have forms that you have to download to fill out upload um and it seems like you know this year we had a lot of delays with utah grants where they had to keep going back to the vendor and having them you know make corrections so yeah we're aware that we're hoping over time they can they can do some things to streamline that we know that's an issue so what other things um what other concerns things that you want to point out this is really beneficial to us to have your feedback becky this is missy i just really appreciated having the um perspective from the charter and then i was able to chat with sherry who's a huge school district um and so seeing those different perspectives from a medium district it really helped shore that conversation up as part of that consolidated needs assessment so i think that'll be helpful when we go back to that bigger group sorry yeah thank you missy that's great okay i hope that we've caught you know captured all of your things i know that we're capturing things in the chat box and the usbe staff that were spread around in some of those groups i know were taking notes and i tried to jot down things as i had an opportunity to you know pop in and out but um becky can i make just one last comment absolutely um one one other thing that got brought up in our meeting was you know everybody doesn't necessarily use the same sis system and sometimes the reports require us to do some data mining that sometimes it's easier for some las to do because of the robustness i guess of their sis and sometimes aspire may not have necessarily the capability to run the reports that's being even asked and so sometimes some lease are required to go in and run their own you know pivot tables or you know additional data collecting resources outside of the sis itself and that can become labor intensive so maybe as part of this conversation i'm really evaluating not not only how is the data collected on the back end but what are the resources that the state has for um the the collection on the front end and the systems that are there to to consolidate make those things happen so this is my thought thank you that's a great that's great feedback um we appreciate that so um let me see if i can get back to the powerpoint let's see so anything else you want to share one more chat came in i just wanted to put out there lisa said qualtric surveys do not produce a retrievable report for those who submit them and so they're copying as they go but the resultant formatting just leaves a lot to be desired tough tough management because they aren't able to print that or you know save them okay that's really good to know and i haven't had quality training there are some people on our staff that have but that's good information to take back to the qualtrics team in our data institute statistics department and then carry on just shares if you forget the copy as you go you lose the qualtrics info just a reminder try to keep saving that as you're moving along or printing it out or however they're doing that copy okay so let me see yeah and i think um becky this is renee for my generous i think to to add on to that i mean i hearing you say our qualtrics team and our data team charters in small districts we don't necessarily have a qualtric stream or data team that's usually your principal or one of your admin staff so i think that's part of the challenge is there's people who are learning how to do it and are really good at it but then the user side it's um we're sort of getting the brunt of that yeah thank you um thank you for your comment i was really actually referring to our usbe data and statistics team in our qualtrics team we don't really have a data person on our team we've been begging for one for a number of years and and haven't been able to get a person haven't been able to get funding to get a person yet so um we have to rely on our data and statistics team as well i have a few we have a few team members on our team that have had some qualtrics training but um yeah i totally understand especially when you're um you know in a small charter lea or some of our small rural leagues you just don't have you know the number of people to go around to do all of this work it really does become become a challenge yeah i'm going to pull that power point back up and see if i can get it to display the way let's see there we go so please at any time if there's additional information that you want to email to us or if you want us to set up another opportunity where you can just meet in small groups and talk about these kind of things and provide feedback we are more than happy to set that up at any point in time your feedback really is important so and we're really excited to have some leas involved with us in that consolidated needs assessment work and the resource allocation work um the feedback's been really important so i'd like to talk to you a little bit about some some things we might consider for professional learning if you are interested and again these are just some things that i've i've been thinking about um but it may not be anything that you are interested in at all and that's perfectly fine but we're just thinking about you know what we might be able to do to support you and to support schools to support um you know maybe not even doing anything else this year because things are so stressful right now with the covid situation that we don't want to put any additional burdens on lease or schools by any means but maybe thinking thinking forward once we get you know the viruses under control a little better we have vaccines and treatments that will help in that arena but um william parrott and kathleen budge who are husband and wife team and professors at boise state university uh recently uh published the second edition of their book turning high poverty schools into high performing schools so the original edition of their book was published in 2012 and it looked at their initial resource research with schools across the country that were high poverty schools but really getting amazing um student outcomes so they've engaged in additional research and this this book came out in april of this year so that would be one opportunity we could purchase the book and do a book study with them or we could provide copies of the book and have them do some you know some training for us which could include you as title 1 directors we could extend it out if you wanted to include you know principals and teachers and others so that's just one idea usb has done two cohorts of teacher clarity and assessment capable learners jennifer thompson did a cohort that finished up last school year it was a two-year process and then title one started a cohort last winter and are continuing that this year our presenters nancy frye and tony faddis have been providing our training this year virtually and we shortened those meetings down to three hours so they do a session in the morning and then they repeat the session in the afternoon to try to meet as many people schedules as possible and then we're able to send out their slides and other things handouts to um to the school teams that they can they can use as they they have fit as they have time so that's another opportunity we could possibly pursue another thing that i know that um the the corwin team that has the rights to um developing assessment capable learners which is stock based on dr john hattie's work they have corwin owns the north american rights to those materials and they work closely with dr hattie but something that they were i noticed they were offering this summer was an opportunity for educators to go through a several day training to become certified as a trainer for teacher clarity which would then build capacities for districts and lease to um you know send a person that's been through either jennifer's cohort or a title one cohort and they have the basics of you know what that is and then they would be able to be a trainer back in their own lease so that rather than us you know providing training and bringing in national presenters you would have someone at your lea that could continue that work and you know work with teachers and coaches and others so that was that's another thing that we might consider and then a couple of others um this past summer we were going to have the summer systems conference and that had to be cancelled due to the pandemic but we had arranged for tara brown the connections coach and also dr linda cargus bone linda's the picture at the top and tara's the one the second picture i've noticed i receive emails and different things from them they've been doing virtual sessions with states and districts and schools across the country dr bone focuses on brain based learning especially dealing with students that have had adverse childhood experiences um so looking at those various aces that impact students um so we i've seen her present at the national title one conference she's excellent tara brown is all about making those personal connections with kids that we know make such a difference i'm sure we've all had people in our own lives that you know made those connections with us and made us feel like we were capable and confident we could do hard things we could achieve you know achieve things that we wanted to achieve and so she as well is very energetic um has excellent strategies um to reach out to students to parents to colleagues to families so that's those are some other things we might consider so um i'm to stop sharing again so i can see you and becky one question that came through i just want to confirm yes susie ramos asking is there a way to get involved in some of those groups that have already started or are you looking at having another cohort coming up it would be a new cohort yeah you have to start these things from the beginning um with the teacher clarity and assessment capable learners we're in year two of that so we've already had um i think let's see one two we've had four days of the seven days of training so our next day coming up will be in in march i think no january we have a date in january and then there'll be another date in the spring and that will wrap up that second cohort so we'd be looking to see if there are leas and schools interested in starting another cohort so i know some of you have are currently participating some of you participated in that first cohort that jennifer thompson organized and um the sense that i've been getting personally as a teacher i really love what they're presenting i've studied instruction since i've been a young teacher at the elementary level i always wanted to make sure that i knew what i was doing and was doing the best that i possibly could for my students and so as someone who studied instruction did my dissertation on comp reading comprehension instruction what they're doing is is just meaningful and i think it's easy for teachers to grasp but certainly nothing new it's basically about being clear in your intentions having you know a learning objective a language objective a social objective how are kids going to interact around that learning you know understanding how learning progresses so you take a standard and you look at okay if the kids have to meet the standard at the end of the year if that's the goal how do we scaffold that learning across time to ensure that every student is getting there so i see lisa flinders just posted in the chat box lisa is the principal at south summit elementary and she's also the title one director for south summit district she has a lot on her plate but lisa and her team have been participating and the sense i'm getting from participants is that they're really loving it and we were really worried this year with the pandemic thinking you know we don't want to put burdens on people we know teachers are at the end of their wits some days maybe a lot of days they're stressed there's so much going on um they're worried about their students they're worried about their own health they're worried about their own families but everybody said no let's go ahead we'll just do it virtually and let's just keep going and so i've been so pleased with that and so so privileged to be part of that and to interact with schools we also have a usbe facilitator that was assigned to each of the teams that it's been a little bit more difficult in the virtual setting but we have a usb person that's there to help support and you know be the liaison between those teams and usbe to see what it is they need so those are just some ideas that i have but like i said they may not be anything that any of you are really interested in but we just want to collect some feedback so as we look ahead to do planning and we're looking at our funding and how we can you know utilize our funding and and do something that's meaningful for you those are just some things that i thought might be possibilities so any discussion on that or if you want to put in the chat box tracy is so great to um to monitor out so again it looks like lindsay's saying yes interested so maybe what we can do is i hate to even say the word survey but if we put together just a simple form that you just would click on anything you're interested in susie's saying yes so i don't want it to be anything burdensome i see katinas responding yes it would just take a couple minutes amy mitchell from washington county has two schools from washington county participating in acl this year and they've just been just been wonderful and i'm so grateful for those of you that are supporting your schools and attending that and um you know providing that additional support so we will send out a very very brief survey i'm gonna call out leslie evans she's our she knows how to do surveys in forms which is it's not a qualtric survey it's very very simple it's like a little quick survey monkey and we'll just kind of list those options and you can just click yes i'm interested and then we'll kind of get data on which ones seem to be of most interest and then we can start reaching out as our team to these providers about you know what we can do to set that up and and some timelines that we could get back to you so thanks so much for your feedback on that yeah missy's saying qualtrics is the new sweater okay all right thank you missy yeah i don't use qualtrics i don't know much about it i filled out surveys but i don't know how to create them so i'm not going to add to your burdens with that personally um so let's see let me come back here to the oh yeah unless he's saying i won't use qualtrics he promises okay so moving on um cheryl ellsworth is working on a really great um whoops i went too far she's working on a great um initiative the belonging schools initiative so cheryl you back with us she had to step away i know for a minute yes i had a step away to speak to some years at salt lake community college i am here um i don't know am i the only one who sees it very small is there a way to make it larger oh i'm seeing it on my on my laptop in the small version but on my monitor i'm seeing seeing a full version i don't know what you're seeing we see your small version where you see the next slide coming up oh you that's okay let me stop sharing and see your display settings okay because i i'm i'm using it to drive my let me see if i can okay let me try this is that better cheryl oh yes that is perfect okay sorry yeah so i have no idea if i'm backlit or anything but hopefully i'm not um so as you can see um well let me actually start with the fact that some of you may not know this but um we i was part of a fellowship from the years 2018 to 2019 for 20 months with um jd talbot from granite district um jenny mayor clint mayor glenn who was with salt lake district now at the university of part neighborhood partners and dr paul cutner at the university neighborhood partners we wanted to do family engagement and look at it from a preschool to 16 approach so from early childhood to to college age and um if you could go to the next slide for me becky um when we when we um did this fellowship we had to do a really deep dive learning into uh family engagement here in utah and we learned from doing empathy interviews with administrators legislators teachers parents students all kinds of people um that there we got some research that families and um were were feeling like there were three processes um that schools can implement to help build a culture of belonging that extends to all families this was around welcoming and valuing reciprocal learning and impact and we call this the belonging schools initiative which is a plan for advancing equity through family engagement with utah schools and right now our slogan is utah schools belong to you you belong in utah schools um i just wanted to bring some of you up to speed some of you have seen this research but some of you are new um but finally after working on this for a very long time and then um a huge hiccup with with everything coveted related we were supposed to start doing this in april but we're going to be rolling this out starting in january if we can go the next slide so i just wanted to give you a heads up of what our partnership overview is to build belonging schools across utah we will launch two programs in 2021. we are seeking a broad coalition of educators and leaders representative of utah's diverse landscaping communities so i am talking i want my from san juan to box elder i want everyone i want reservations urban suburban rural remote i want all of you a part of our leadership roundtable so the first part of our partnership is that we will have a family engagement leader roundtable leaders of family engagement statewide will meet on a bi-monthly basis to discuss and catalog family engagement best practices for utah schools which means it's for your community versus saying cheryl what can i do you're going to figure it out for your community and recruitment kicks off january um uh 2021 um the first round table meeting will be held in february march 2021. um but the next part of this is the family engagement collaborative where teachers from across the state are invited to join the six this six session professional development series teachers will learn concrete research-based strategies for family engagement and will have the opportunity to share that learning with colleagues around the state at least two teachers per participating school must supply um recruitment kicks off for that april 2021 and the first session will be held in june 2021 so to give you a longer timeline we have a an 18-month contract a year and a half contract with flamboyant and so this first beginning part first six months of the year we're going to be working with the leaders and the teachers so that we can then in the next school year really be rolling out the belongings schools initiatives in schools um so i'm really excited about this um this has been something that i've been working on for a very long time and i didn't want it to just die on this like hill of like covid um i knew we had to be patient um and so the next slide just has my information if you want to go ahead and share that becky uh but essentially again um recruitment for all of that will begin in january so first it's for you leaders like for you to really understand um when i was meeting with the salt lake community college folks this morning a question came up i see that teachers are for the second part should we be planning on stipends for them the answer to that is absolutely yes these teachers are going to be meeting six times in the year after school and so we want to make sure that we're compensating them um our our contract work what we've paid for the state is to get the training the training materials all of the needed pieces for your um communities that's all that i have um are there any questions um i can we can start on stop unsh or we can stop sharing now becky that's all i have thank you cheryl it's always great to hear from you and yeah cheryl has worked on this for a quite a while and it kind of got hung up due to covet and he just kept dogging it and following it through to get all the paperwork in place so this will be a great initiative for leas to participate in and again build some capacity yep thank you and yes the all the other things that we're doing home visits related i will be reaching out to lea directors for stories and numbers and all of those pieces we were we're going to be highlighted in the national um parenting or parent-teacher home visit project for home visits because utah had some really high numbers last year so keep up the good work with the things that we're already doing but this will be a deeper dive into family engagement so be thinking if you're a leader that wants to be involved for our leadership roundtable great but even if it's not you and there's someone else at your district that you're like this person needs to be a part of this please please let them know thank you thank you chris thank you sharmless and you should notice a couple of months ago cheryl started putting together a one-page family engagement moment she calls it and we've been sending those out to you on a monthly basis where she highlights something that's happening and provides you know some information and ideas that are just quick and simple we hope that you're sharing those out with your title one schools so next we're going to turn the time over to um tracy vandiventer who is the director for the uspe center for continuous school improvement and she along with dr max lang and leslie evans who are two school improvement specialists they will be talking to you about school improvement thank you so much everyone we wanted to get together and answer a couple of questions especially regarding some of the funding and csi tsi related questions and becky i'm just checking in was this kind of an optional because it was more specific or did you want to just have everyone stay be part of this conversation well thanks for reminding me tracy somebody has to keep me on the straight and narrow yeah if you do not have schools and improvement and you're not wanting to spend these last few minutes of the meeting with us that's perfectly fine as i said the meeting will be recorded but if you do have schools that are in you know school improvement you may want to stay and listen to the updates that tracy and max and leslie have for you today so if this if you're going to sign off thank you for being here today we're grateful for everything that you do and um we'll certainly be in touch with more information thank you everyone very good and it gives people a chance to step away and those who are here will be focusing on uh funding discussions related to some of our federal funds i'll go ahead and pass it off thanks tracy i'll i'll start here so we've talked a lot about the impacts of covid today right and that's an ongoing reality we're all dealing with so i wanted to lay out um a reminder and then get your feedback and opinion about where we should go from here so the first uh cohort of csi schools both the low graduation rate and the five percent overall performance schools were identified in 1819 so we're calling that planning year zero in that year every lea with schools identified was issued a formula award and then subsequently a competitive award in and these decisions were influenced by and collaboratively decided upon in a group called the committee of practitioners that is run uh through dr donaldson's team so that was the advice we were given is let's give a formula award then a competitive and competitive ongoing so that schools in need of more funding above the formula amount could um taylor make their proposal to to win a competitive award for funding to implement their sip okay so 1819 status quo we give formula award competitive award 1920 um proceed with just the competitive award and that was our plan going forward so now when with cobit onset we decided in spring of 20 to only to administer a formula grant to ease that administrative burden of having to submit a proposal and ask schools to tailor their wishes in other ways so we use the same formula we used in 1819 and issued those grants so our question is would it be useful and beneficial and could schools spend their money by 9 30 2021 if we issued another competitive grant in this second implementation year so some of you hopefully those who have stayed have schools and improvement in csi and you could always get back to us with this but if you have input in the chat we'd be happy to hear just to jump in we we certainly are wanting to know from you what is the best way for you to be able to access this money but we just finished a conversation earlier today that talked about we are up to our eyeballs in you know reports and surveys and qualtrics has become a swear word um today so we know that competitive comes with its own challenges and extra duties on your end so we aren't necessarily swaying it one way or the other we wanted to just know that this is an option and if you feel that this would be beneficial for you we'll certainly open it up but if you're feeling like nope we're pretty good we'll we're satisfied with the formula distribution we could go that direction as well and i see uh jams talking about that possibly competitive could be good depending on the time frame of the competitive process so good feedback and jam feel free to put in your favorite time frame up there for recommendation and we can take that into consideration leslie let me ask a brief question and a back story so we have eight csi schools in granite in early december i'm pulling those folks together we're going to do a start a monthly csi cohort meeting for supports and planning towards exit um with that if i brought this up and kind of a cancel with those eight principles would that feedback generated be too late for you or when are you hoping to receive all the feedback i think it's reasonable to wait did you say beginning of december yeah let me look at the date one more time um we don't have a hard and fast we want this to be useful to you but also doable to get it out there um but i don't see a problem i'd like to get the input it's the wednesday december 2nd we're meeting that morning that's great okay i think if i had we had this input by december out of a hat that could be a sunday or something i don't know but mid december then i feel comfortable we would be able to get that out and offered in january the competitive i mean we've issued this grant before it's essentially the the asks need to align with the sip and be allowable expenses and within the budget of the whole the whole grant so okay okay i'll meet with them qualtrics application promise and jm's kind of put an update to her comment too that if the funding's to be useful to schools they would know it need to know or know in mid-january what their progress to date spend and this is the time that they may need additional funds to make significant impact so they need to have access to their funds in early february so tapping into trying to get that turnaround as quickly as possible so they can you know access those funds know where they're at and know what's left and be able to make some decisions understood so in case i'm sure we're not capturing all and especially the alternative high schools that are identified for csi so in this meeting so we will um i will write an email with the expectation that feedback is given by december but i'm kind of hearing that you might think you need it but mid-december and then we'll work our hardest to get it um out and in utah grants by january mid-january and if you don't mind i just want to check with jam are you asking that we would try to get to the schools what their progress to date has been on spending the existing funds or is am i misreading that what it is is that the schools we're we're monitoring we're working with them they're we're analyzing their progress and that halfway mark is a significant mark for them they we're talking about their progress we're seeing how they've used the funds what's the impact been to date so in that january time frame if they need to make some necessary refinements to adjust and get the results that they want by the end of the year they may need additional funds at that time to either do the pd or whatever they need to have done to make that shift that's what i'm talking about because they see because because they're monitoring themselves weekly monthly but that is that midway to see where are we at that sometimes significant refinements are made i see thanks for clarifying so looking at maybe that 90-day plan will be coming to a close and they'll be looking at creating a new plan and that would help them make decisions about whether those funds would be useful or not yeah we don't we don't use the term 90 day plans anymore in our district it's the action steps and it's midway during the year to see it do they have to abandon or discontinue some action steps refine others and maybe add new ones perfect thanks for clarifying just a little more context you know some of the reason we're asking this question is that it can also be burdensome to have to be able to spend all that money so there's been some hanging out their money from 18 that we've been having a hard time expending through the system but leslie i'm so glad you said that because that's why i say possibly i don't want to just throw money at them i want us to look mid-year to see okay what's been spent to date where is your balance is do you really truly need more money because if they need it then i want to have that opportunity to apply quickly for it and get it if they don't let's just utilize their resources more effectively because they have some that still hasn't been used i'll jump in i i agree jm that was my reason for pause i really want to talk to the principals and some principals might say yes give me more money but part of our workshop is a really critical lens of how have you spent your money and kind of pushed back or pushed the other way and saying that they could really utilize it so that i i i agree the um we don't want the money to be burdensome or wasted but if my forthcoming feedback is if it will be helpful in achieving these goals then then i'll i'll communicate that to you leslie thank you great got an enthusiastic yes from chelsea as well well we have a couple other topics um to throw out there but we did want this to be um somewhat informal discussion and if you have questions please feel free to jump in but i think we can move on to the next slide if everyone's okay with that tracy do you want me to take this one as well okay yeah go ahead thank you leslie welcome so again starting with you know with the lens of covet and things um csi and tsi exit we've gotten some clarification on um sorry this is just a reminder of what's currently in the federal legislation for csi exit that if you're a school that has uh was identified for having the lowest five percent you have to rank above the bottom fifteen percent of title one schools in 21 22. and just to clarify that's a one point marker so in 21 22 you have to be above um the 15 whereas low grad rate schools have to have a graduation rate above 67 in both this year 2021 and 21 22. and then schools in tsi will exit when for two consecutive years they have no student groups performing below the percentage of points associated with the lowest performing five percent of schools so just a reminder on that one on this slide in particular any questions here and the impact of not having assessment data in 20 1920 does not affect the exit for csi overall performance or grad rate just to clarify there um leslie and i i usually beat this drum but i know there's was where tsi the exit criteria we kind of hang our hat on the two consecutive years and we don't have that year so i'm i just um still assuming that the state board will kind of grapple with what the exit looks like considering the summit of waiver in 2020 yes those discussions are still pending as you can imagine there's lots of larger um issues keeping people busy yeah very good we'll keep bugging though because we really want to answer you one day and have a party lisa is asking a question about if there will be an update about tsi but i'm not sure lisa which update you were thinking of um whether there's like a new identification or just to clarify we did not identify tsi schools for this year so there wouldn't have been an update for that so the spreadsheets that are on the resources tab of the assessment and accountability website are the most up-to-date in other words there's the 18-19 list and or the 17 18 18 19 the two two years they're listed funny but it's just the two years that we've identified csi which was 1819 1920. perfect okay lisa says that we've answered it it's good okay good here's where i started to talk about changes um due to assessment irregularities so we're using that term quite a bit these days right an assessment irregularity was having a soft closure on all schools in utah and not administering end of year testing if that were to come to pass again at the end of this year the state board has approved some of our recommendations to have a plan in place in case that happens so if that does happen a state review panel will review eligible schools data and make recommendations to the utah state board who then approves or doesn't our recommendations and i've provided one example here for you and some of you are very familiar with this example that the schools that would have exited state turnaround or would have been eligible for exit for state turnaround based on 1920 results we had to come up with a a process for them to to do that and so they were eligible to present to this panel of 10 people if they were above the lowest three percent and then they addressed the following questions do you have evidence that you have have made substantial progress and growth in addition to the data in the accountability system or without that data and do you have more data from the implementation of your school improvement plan or turnaround plan so we wanted to give you a heads up that if you are a school in improvement or you have a school that's in csi that's collecting progress data and formative assessment data and any information that would be helpful to show that you've made substantial progress would be very wise to keep now i know some of your many of you all of you are keeping it as it is but like in a formal way that's easy to access in case you need to um we need to enact this special provision um open to other additions or questions and we did share that out with the cohort schools and let them know that this had been done in the past but it's recent that the state board approved that recommendation that we use that model again if there are additional irregularities so that there would be an exit panel that would review presentations and you would only be able to make those presentations if you're above the three percent and then highlighting those final two bullets so two questions that are listed on the screen there um so we just didn't want anyone to be surprised that those would be the kinds of questions most likely the exit panel would ask schools to present on and max has answered one question regarding csi and tsi that yes if your school was designated that way they have maintained that designation uh katrina walker asking have lower graduation rates due to pandemic been considered and katrina no unfortunately the federal government's been pretty um consistent about that 67 percent rate and i think the only piece that is being considered is the participation rates in the assessment but not in the graduation rates itself uh jams wanting to know um leslie if you'd be willing to share your powerpoint uh with them and i think that becky already has that included in the slides that was sent so this is this is just like the tail end of those slides there so and darren uh nielsen of course assistant superintendent shared this morning the struggles with looking at how does accountability and how do we you know move forward and and generally this year as we have data we are not expecting to have accountability tied into that which is why some of those dates are being pushed forward and as that becomes more clear and addendums and amendments and waivers have been approved by the federal government we'll be sharing that out with you other questions doesn't look like it nothing's popping up in chat right not um so much but i just want and i think i'm sure uh but i wanted to verify so i'm looking at slide 43 and kind of circling back to um the possibility that if there's testing you know continued testing irregularities and a csi school is above a whatever cut percent then it's not for sure but there's a possibility that they can provide an alternate portfolio that might mirror so i'm using all these ifs and mites and maybes that might mirror how kennedy junior high for example uh gave a portfolio presentation to usbe for exit so if i use all those m's and um ums and maybes um what i feel uh so it sounds like we're good to present that just so schools our csi schools start preparing did i understand that correctly the hardest thing about the csi um as as a piece of that panel um presentation is it's a one year only you know above that 15 whereas with the cohort schools or the tsi schools they're you know they've got a couple of years where they're asked to show above so i that one i'm probably less comfortable and saying yeah i think that's an option that's going to be out there we're all just crossing our fingers and toes that by next spring now this coming spring and a few months but a year and a couple of months that we'll have a solid piece um of evidence that we could have in our end of level assessments that would allow us to see that they are above the 15 uh because what in order to be able to present to the panel previously you had to at least to have some evidence that you were above that you know baseline score so i guess it could be possible that if a csi school has been above that 15 percent in previous years while they've been a csi they might be able to use that as evidence to say and we've maintained it but that that hasn't been addressed at this point in specifics to the csi okay okay oh i like you know the i'll cross fingers and toes that that would be uh it's very cool other questions comments no it doesn't look like it leslie okay that was our okay there we go oh go ahead teresa well i was just gonna say and of course we're always open to feedback from you if at any time you have questions after we've completed our meeting today our goal is to serve you and to support you and to make sure that you have everything you need to make sure your schools your title and schools are supported yes absolutely so i want to thank everyone for joining us today we're giving you back a few minutes of your day which is always great right about 21 mil you're getting back so we had a we had an ambitious agenda and we um managed to get through all of that so thank you so much for your attendance for your participation your feedback as i said we love working with you i think we have the best educators in utah um really bar none you're by far the best the best people we love you we want you to be well and safe and please stay in touch please let us know anything we can do to help you because that's why we're here and there are no no silly questions ever all questions are important to us so please send those our way and have a wonderful rest of your day and a happy thanksgiving bye everyone you want to wave you can turn your camera i love seeing everybody in the brady bunch squares wish we could see you in person it's just not the same just seeing your little names in a box is there any way you could put the screen back up there with the contact information for just one more moment you betcha let me just get to that yeah it's in your um you have it too in the slide so are you seeing that again it's there okay very good thank you very much oh thank you

Channel: USBE - STUDENT SERVICES

Views: 108

Rating: 0 out of 5

Keywords:

Id: FQEFsyjgCRg

Length: 161min 44sec (9704 seconds)

Published: Wed Nov 18 2020