Arizona Audit report presentation, Cyber Ninjas share their results of election audit

that is a true statement they were close now i find it ironic that our secretary of state and a few others have called this a sham audit that you can't trust it you can't believe it well the interesting fact is is truth is truth numbers are numbers and we've said that from day one what you're going to see is exactly what it is the truth and those numbers were close within a few hundred but what you have not seen and not heard yet which is what you are going to hear right now what you have not seen and you have not heard is about the statutes that were broken how chain of cons custody was not followed how we had a number of issues which is why people question the ballots and the elections so i ask that you please keep an open mind i please ask that you listen to this because the reality of this is what this is all about is making sure your vote counts senator peterson thank you madam president i would uh join you and welcome everybody here who's here in person listening or watching this historic event this is the first time in the history of our country that an audit of this scale and magnitude has ever been conducted it's unfortunate that it is an incomplete audit beca due to the lack of cooperation and the obstruction from the county however in spite of that this is still the most complete audit that has ever been done the goal here as the president has said is election integrity our citizens have demanded it the importance of our society having faith in the election process cannot be overstated the number she gave of people who had concerns was significant but even if it was much less it would still merit this type of an audit to bring reassurance to our citizens this report should help the senate understand what has been working what needs improvement and whether further investigation by law enforcement should occur so i look forward to receiving this report with all of you at this time thank you senator peterson with that we will start with our first presenter our first presenter is joining us via zoom because he lives on the east coast his name is dr shiva he was the one commissioned by the senate um and that was because we finally got the envelope ballots are the ballot envelopes so we could check signatures or lack thereof and this was his part of the audit dr shiva welcome to the arizona senate how are you today thank you very much for having me honorable president fan thank you very much absolutely also senator peterson thank you the floor is yours please proceed thank you before i start i have a presentation uh this is also accompanied by a report i just want to make one preparatory remark and that is uh as a follow-up to senator peterson we live in the age of what we call engineering systems complex systems taking an airplane using an iphone self-driving cars these are highly complex systems but what are known in the modern world is engineering systems our election voting systems are also engineering systems and i want to thank the leadership of the senate the stakeholders because they've taken a historic uh step here to bring the same level of engineering systems capabilities to election voting systems and one of those important attributes is that whether an anomaly is small or large or insignificant or monumental it all must be welcomed because from an engineering standpoint it can only do one thing which is to enhance the system so today as uh senator fan president fan said i'm going to be sharing with you a particular area that we looked at which was looking at the election early voting ballot return envelope images and that's what i'm going to share with you so let me begin i'm going to start up the powerpoint and i think i think i need to share the screen here let me see if i can scare first there all right let me start here so the title of this talk is pattern recognition classification of early voting ballot return this the screen is blank what's blank i'm sharing always a technical problem okay let me stop the share and restart it again let's try that again there we go thank you that worked okay um and so the title as i said is a pattern recognition classification early voting ballots return envelope images for signature presence detection we're not doing signature detail verification but we're doing signature presence detection and we're taking an engineering systems approach so let me begin i will go through this agenda we're going to give a little bit of background we're going to review the maricopa results we're going to go through what our analysis resulted in and then compare them and then report on some of the key findings and anomalies and then also raise questions for maricopa officials and then propose some recommendations so the executive summary of this to to sort of summarize the whole uh talk here today or the findings is that the early voting ballot the return envelope you really want to think about that's really the protected vehicle by which the evb which is the early voting ballot is transported and processed and the authentic the authentication the verification of the signature on the evb return envelope is critical to reliability of the entire process and our audit reveals anomalies raising questions on the verifiability of the signature verification process so let's begin so a little bit of background you can read more about it but um i was selected and i was honored to be selected because my background for more than 40 years has been in the field of pattern recognition and classification particularly system science and it goes back a long period of time but just very briefly you know in this field you look at reality in this case we're looking at the electronic uh the election voting uh the early voting ballot system and then you try to model it that's typically what science and engineers do for example for many years i looked at how babies were processing sleep and trying to understand their system of looking at sleep patterns same in the area of deaf-blind communications again you have the reality you have signals same in the area looking at a bridge and you get signals from it based on the signals you're trying to predict what's going on in that bridge and same if you look at an aircraft wing without having to open it up you look at these signals and you're trying to predict what's going on and for many years we did a project on looking at handwritten bank check normals and these different areas including email looking at what's inside of an email as a system these are all pattern recognition problems and in fact if you look at food trying to figure out will this food cause different kinds of diseases but the reality in this area is you're looking at the the particular real world and you have a model you can look at the heart you can look at the cardiovascular system of that and you can say hey i have some states of this system which are normal signals on some states which are abnormal signals so in this case what we're looking at is we have the early voting ballot system and we have the early voting ballot return envelope images these are the signals that we received to try to figure out what's going on inside that system so you have the reality of what took place and you have the expression of that which is embodied in those images and this is what one of those images looks like you have a image you have upper left the name of the voter and here you have the signature and we were asked as a part of the scope of this audit not to look anywhere else but to look in the signature region and to look to see if there was a normal state a signature there or the abnormal states blanks split into two and they'll talk about more detail a complete blank or likely blank or what we call a scribble a scribble is something trying to get into the direction of hey was this signature even valid okay and it's a very very rudimentary step we took not full signature verification all right so early voting ballot system and you have these images so before we head into the actual process we did let's look at the reality of what took place so the this is on the left side the actual reality what was reported by the maricopa county in their voter education report in fact what was called their canvas report and what we see here if you look carefully here here are the numbers that they reported from 2016 but more importantly we're concerned about 2020. and in 2020 one of the lines in their report is called the early ballots verified and counted which has 1 million 915 487 and the rejected early ballots which are these numbers are the ones that were not included in the final count bad signatures which means they went through a signature verification process and out of that 587 out of the one million nine hundred and eighteen thousand four hundred sixty feet which all of them were deemed to be bad signatures and the way we get this number is you take all of the ones that were verified and counted the ones that were bad signatures there were some which were no signatures which were blanks and then late returns so to be clear according to the report this was a total number of early voting ballot return envelopes received by maricopa county to make it a little more clear we have 1 million 918 463 total unique ebb return envelopes of which 1455 were found to have no signatures and at the end there was the the and an amount of 1 million 917 0008 was ready for signature verification and as a result of that 587 were found to have bad signatures now in signature verification it's a detailed process you can read the report but counties vary and maricopas considered along with florida apparently have one of the uh better signature verification processes but according to that out of the entire lot of one million nine hundred eighteen thousand four hundred sixty three 587 after this verification process where someone looks at the signature and they look at the voter registration signatures and they're and they're doing typically what's called a 27 point analysis so 587 were found to have bad signatures 934 with late returns for total amount which matches with what was reported of one million nine hundred fifteen thousand four hundred eighty seven okay so this is maricopa's reported results to the public the scope of our audit just to i'm gonna go through the scope of this audit by also going through the process so they'll be somewhat educational when early voting ballots which some people which is a subset of those are called mail-in ballots are submitted those ballots are imaged which means scan converted to digital images so the evb return envelopes we're talking about the outside the covering that important covering within which is contained the ballot is scanned before they're even tabulated so the ebb return envelopes are opened and scanned okay and those create images in fact there are six different kinds of evb return envelope formats let me walk you through some of them and here's sort of a little bit closer but let me walk you through some of them so this is a majority of them we call we've called them the standard image files this is for our lingua franca this may not be the standard language but for the purpose of this um presentation so this is one way that they look there are also another set called ukava image files and there's three different kinds here you have the signature here the date here the name of the voter here another version of that is a little more complicated it's more of an affidavit the signatures here the name of the voters here and obviously we redacted stuff for privacy and the third type of the yukaba is type c so you have uif a d and c all right then finally we have large print uh image files lis and these are for people who have trouble seeing so the instructions are everything are larger and then finally you have the braille format so again six different image formats that we had to process let me walk you through the counts that we got we got a a a drive with all of these images on them the sis which is the first one i show you showed you there's one million nine hundred nineteen thousand five hundred ninety eight which is about ninety nine point five percent the uifa's eight thousand eight hundred forty nine point four five nine percent the uif's 277.014 percent and we go to the uifc's which are the affidavits which a small set which were 0.01 and then finally we had the large formats which were 475.024 and the braille about 29 of them 0.02 and that added up to all of them so for the audit we were uh provided by the arizona state senate 1 million 929 240 and these were apparently all the evb return envelopes that maricopa county got okay what we noticed when we looked at this which was fascinating our first interesting uh i guess wasn't an anomaly but something we were a little bit surprised because we thought the duplicates would be remotely removed but there were duplicates in here um so we had to remove duplicates so out of those set there were duplicates so these were the duplicates we found about not about 16 934 voters who had submitted sixteen thousand nine hundred thirty four two copy duplicates which means each of those voters submitted two ballots okay two return uh early voting ballot return envelopes interesting enough we also found another 188 voters unique voters who had submitted 376 duplicates so the total image count was 564 once you removed the duplicates right because you want to take one of them and then finally we found also four individuals who submitted 12 duplicates total images being 16. so the total duplicates was 34 448 total images of which 17 332 or 322 were duplicates from 17 126 voters this was by the way we'll get to it was not reported in the report so when you look at the first level of analysis on our the data that we got 1 million 929 240 ebb return on lows received we subtract out the duplicates and then we have 1 million 911 918 000 okay so then the process that we were really commissioned for was to do a very uh you know basic analysis to see is there a signature there or not signature presence detection so in that process again the goal is is there a signature there in the signature region interesting enough some people don't follow instructions they write all over but it wasn't our job to go scan everywhere our job was to look in that signature region so again specifically we looked in that signature region one category of the signature region has a signature another category is blank likely blank and scribble and just to keep it real simple there's a very simple classification if it's zero percent non-white pixel density which means nothing in that area it's a blank if it has a little bit zero percent or greater but no more than point one percent like this example here we put it into another category just for early classification purposes we called it a likely blank and then if it was greater than one percent density greater than one percent or uh that was denoted uh you know a scribble and what's interesting in anything over greater than one percent by the way this is actually this should be point one percent to one percent slight mistake here so this is greater than one percent okay so let me repeat this should be greater than point one percent less than one percent please correct that and this is greater than one percent all right then what we did was we did a distribution curve we looked at all of the ballots and we did a distribution of pixel densities just to give you an idea of how liberal we were in accepting something as a signature so if you look at this it was only this little area here which is between point one to one percent that's a definite non-white pixel density that's a scribble everything else we were quite liberal and accepted those as signatures even in our analysis so again we were not hired to do signature verification all right now after this typically signature verification would take place we'll do what the county did and then they would also open up them the ballots and then to open up the envelopes and tabulate them so let's go to our signature presence detection so we took it we took a ballot or it could be any ballot first of all we classified it into one of these six categories that's what the system did and then for each one of the for four of those we removed the duplicates as i've shared and then we looked at the ballots once we knew which one it was we identified what kind of region we extracted the region if possible though this was not part of the scope we also tried to extract the name we were able to do that for close to i think 99 of them so we also use different types of classifiers which is in pattern recognition lingo for each one of these different kinds of ballots to ultimately put them into one of these four buckets so let's look at the results so we're first going to look at the non-duplicates which means all those early voting ballot evb to be simple return envelopes which were non-duplicates and here we see predominantly 99.77 percent we denoted a signature again anything greater than one percent pixel density these scribbles we found were 0.13 okay 2420 and the blanks fell into two groups the definitive blanks of 1771 and the likely blanks at 101 this adds up to 0.1 percent okay then we also looked at the counts so of these you had just to be clear we had 2 420 scribbles and we have 1 872 blanks we put both of these together in our sum total similarly we now went and looked this is a little more complicated at the two copy duplicates because we had to deal with those there was a substantial amount 34 000 of them um 34 000 and more so the duplicate recognition splitted into three in fact three groups but we have subgroups so there could be you know you have the if you think about it the mother and the son or the mother and the daughter okay or the parent and the child relationship so one of the two one could have been a signature one could have had a signature one could have had a signature the other could have had a blank one could have had a signature that could have been a likely blank one could have had a signature and c denotes scribble if any one of them had a signature we said okay we call it a signature alternatively in the scribble area one could have had a scribble other could have been a blank one could have been a scribble other could have been likely one convenience scribble in a scribble these were denoted as scribbles in the duplicate case and then finally you had ones that are blank blank blank and likely blank and likely and likely blank those were denoted as blanks all right so let me just to give you some feeling of what these look like so here is one from the same person again for the purpose we're not allowed to share the names of voters etc but you can see here that here's one there is a signature here and there's a signature here okay that um but they are both the same so these are both signs so two ballots both signed another example is one is signed and the other is a blank a complete blank here in fact in the signature region is what we're concerned with another is signed and the other one is a likely blank it's got some stuff here so echo male categorize it as a likely blank here's an example where you have a signed one but you have a scribble over here okay and then here we have the here we have a scribble but here you have a blank but it has been verified and approved and we'll talk about this later we'll come back to this here you have one that's a scribble a little bit some stuff here and a likely blank and the one with the scribble was a brood here you have two scribbles and one of them was approved the exact same ones and here you have one with a blank and a blank and this blank has been approved and here we have one which is a blank and a likely blank and a likely blank with the little dot over here has been approved and similarly over here we have a signature and a likely blank and again the likely blank is being approved and here's an example of a three copy one we have three copy duplicates all are blank and we'll come back to showing how if they were approved or not and here's an example of three copy scribbles all right so that gives you an understanding of the flavor of these the analysis we had to do of these two copy duplicates and so on those two copies we categorized these ones in green as signatures the ones in red as scribbles and the ones in blank as uh sorry black is blank once in red is scribbles so again you have for two copies 16 934 and these are the counts on that so you have 155 scribbles from the 2 copy 45 blanks from the 2 copy interesting enough we also had three copy and four copy analysis we had to do i'm not going to go into all the details of this but there were also three copy and four copy uh duplicates there were two sets of three copy which were blank um i'm sorry uh two sets which were scribble five which were blank so now we do the totals calculations so if you add the non-duplicate blanks 1872 two copy duplicate blanks 45 and then two three copy duplicate blanks you get two so 1919. similar if we do the scribbles 24 20 for the non-duplicates 155 for the two copy five with the three copy we get 2580. so our nat results state that we received 1 million 929 240 the 17322 were removed to get this many unique return envelopes and then we subtracted the non-signature and what we call scribbles to come up with a total of one million nine hundred and seven thousand four hundred nineteen so this is what we would have sent to in this analysis should have gone to signature verification now let's do the comparative analysis which is looking side by side what echomal uh uh uncovered from the audit and what maricopa had so how many evb return envelopes were received well in the disk drive that we received this is a number but we don't know in fact it's not reported in the maricopa report how many actual return envelopes they receive the duplicates we have 17 322 which will be subtracted this is unreported so when you compare the unique edb return envelopes what you find is that maricopa has 6545 more than we have in the possession of those image files after your real duplicates now we go to the signature presence detection we discovered one 1919 no signatures they reported 1455. so in this case we have 464 more blanks in the scribbles they don't have a scribbles category and we'll discuss that if we start thinking about these scribbles as potential bad signatures we'll get some insight but at the end of the day what went to signature verification of maricopa was 1 million 917 08 which is 9 000 from 89 589 more than we would have sent pursuant to this analysis they also had bad signatures and late returns as we talked about okay now i want to go to the key findings and anomalies you've seen the process you've understood the methodology we've gone through some detail let's go to the highlights here so these are the key findings and i'm going to go through each one of them one of the key findings is it's unknown how many evb return envelopes were originally received by maricopa we had 34 488 duplicates from 17 126 unique voters it's actually duplicate images maricopa reported no duplicates in the canvas report we have 464 more no signatures between blanks than uh identif that were identified by us versus maricopa maricopa has 6545 more unique um unique edb return envelopes this is what's interesting if you consider our scribbles again a very very low tolerance and pattern recognition having done this for over 40 years you know we could have used like 36 features we used one single feature pixel density we would love to use more features by just using that one feature at a very low threshold threshold is key here we've identified 2580 scribbles which would have assumed they're all bad signatures maricopa identified 0.031 percent which is what that 587 represents this actually represents four times that we'll come back to that finally maricopa has 9589 more net ebb envelopes that was submitted to signature verification versus what we have and we're also going to see shortly is that we're going to see that we also saw through further analysis of 25 scourge of duplicates in the last six days between uh november 4th to the 9th we also saw some very interesting other anomalies where blanks of duplicates were being stamped verified and approved we also saw stamps of verified and approved in blank signature regions and i'll show you though share with you those what's more uh interesting i would consider potentially a critical anomaly is that we saw the verified and approved stamps appearing behind the envelopes and i'll show you this it's almost as though it was imaged on there or i don't want to say you know photoshop but put on there but it's quite fascinating i'm sure there's some explanation for this and then finally we have cases where we have two different voter ids having the same address same phone same name with matching signatures so let's go look at some of these anomalies a little more graphically anomaly one maricopa reported only 587 bad signatures to give you some idea that's 0.031 percent and for people who don't didn't really like math i thought i'd make it a little more pictorial so this would be one bad signature for every 3 268 early voting ballot envelope so if you think about 1 million 918 463 early voting ballots the unique ones that maricopa said they got and for each paper by the way the size of the paper is 0.1 millimeter and you were to put them up you would get 630 feet that's the size believe it or not it's of the saint louis arch that height when compared to the bad signatures would be only about 2.31 inches so just to give you a pictorial understanding a very very low percentage was considered that so again that would be to be specific 0.0306 percent of all edb return envelopes were deemed as bad signature okay in our case the anomalies we discovered 2 480 bad signatures scribbles again very low tolerance which is 0.135 and to give you an idea just to state it again we were not commissioned to identify we were commissioned to identify the president blank scripts and signatures not to perform signature verification so scribbles alone were considered bad signatures and echo mill itself identified 335 more bad signatures than maricopa did from its entire signature verification process okay very important point here and by the way if we go look back at the state of arizona 2016 general election out of the two million ballots that came in they had a signature mismatch rate of 0.13 percent close to what we would have had if we just included those scribbles all right again just to restate the scribbles are a very very low threshold we were not asked to do signature verification again anything between 0.1 to 1 pixel density the third anomaly is this is again focusing on signatures we did a randomized analysis of just uh supervised review which is human review just randomly looking at signatures their legibility again this is a wonderful analysis that can be done in handwriting four weeks before the election and four days after we can't share with you this obviously but we found out four weeks before 95 were legible signatures and only five percent were illegible but four days after five percent were legible 95 percent were illegible if i were to do a heat map again this is a representation it would look like this where the red represents eligibility and the green represents legibility it would look like this if you wanted to visually do this and if we had more if we were commissioned to do this we could do this but you can see a marked difference between legibility there could again be a explanation for this fourth anomaly as the edv which means the electronic voting ballot i couldn't put the word envelopes but i think you get the idea increased by 53 percent in the general election from 2016 to 2020 in maricopa bad signatures decreased let me explain what i mean by that so in 2016 which is you know in the previous election to 2020 1 million 257 179 return envelopes were submitted 1 456 were considered to have a mismatch with a rejection rate of 0.116 in 2020 we have nearly 56 percent sorry 52 more return envelopes but the signature mismatch rate goes down by nearly 56 percent so it's gone down by a significant number here okay nearly four times so this went up the other two came down a very interesting inverse relationship which would be again one of the questions we have for maricopa officials anomaly number five is there's no mention of duplicates in maricopa canvas report again if you look here here's the canvas report there's no mention of duplicates here we in our case we found 1726 voters sent in two or more ballots as duplicates as you've seen before a anomaly 6 is 25 plus or more of the duplicates came in during november 4th to the 9th essentially on election day and after so how did we find this well here's a plot again let me walk you through this on the y-axis is a number of early voting ballot return envelopes and we're plotting it by day and again this is based on the drive we got which were time stamped by particular days so it almost looks like an interesting heartbeat here so on this day 10 14 nearly you know 200 000 envelopes came in on this day nothing came in so you get this interesting heartbeat signal then what we did was we said why don't we on top of this layer in the blanks scribbles and duplicates again since the the axis for the eb b's is higher that's on the right axis on the left axis i've done the totals for the scribbles blanks and duplicates what's fascinating is you notice in the early part things are following the heartbeat okay but somewhere along here particularly on october 26th this heartbeat starts sort of separating particularly the saffron line is the duplicate so you have the duplicates stop matching in many ways is heartbeat so here everything stops so what we did was we did further analysis on the blanks i'm sorry the duplicates and the evre and this is what's interesting so what we're plotting here is the duplicates as a function of the daily percentage of ebv's so here's a signal but what you see is you see suddenly this what you may call a significant growth in percentage in fact in several of these days there's 96 of the ballots that came in on two of these days are um are duplicates so there is a serious number of duplicates in fact the area under this curve is close to 30 percent 25-30 30 of the duplicates came in between uh november 4th to november 9th and that motivated us to also on the same plot drop the blanks and the scribbles and you see the same phenomenon there anomaly 7. i'm sorry i'm pretty gone through that the next anomaly is we also noted that the ev33 system which is the system that is the one that contains all the early voting uh ballots had 932 voters who submitted duplicates versus the 17 126 that we identified but what's fascinating is when we match those 932 voters against r17 126 only 2138 voters match let me repeat that again the eb 33 as we understand we're not experts that this is a system where all the early voting ballots are stored when that was gone through and found that 9382 those were voters who submitted duplicates clearly this should be a subset of ours because we're supposed to have all the early voting ballots we found out only 2113 matched and let me just walk you through some of these duplicates where the duplicates are stamped and also approved so you see the duplicate and here it's being stamped nothing on here it's being approved duplicate one duplicate two this one is being approved and again all of these came in after november fourth same here another duplicate being approved and a blank here's another one example three two duplicates the blank being approved same here fourth example and there are many others but these are pure blanks and they're being approved in fact this is a three copy duplicates where two are approved so one voter sent in three copies and two got approved another one is three copy duplicates where one is approved all right the next anomaly is verified and approved in the blank signature region so what do i mean by that so this is blank but they're verified and approved is appearing right in the blank and this is process issues which we'd love to get answers to so same thing here same thing here the verified and approved is right there and then finally this is an interesting anomaly where we have two evbs from or two evbs where people have the same voter id same name address and phone number with matching signatures with two different voter ids okay so it's we were had to redact this but imagine if you could see this there's a person's name here an address which is the same as a name and address here very similar matching signatures same phone numbers but they have two different voter ids so repeat again two different voter ids same name matching signatures as you look if you looked at the visually same phone numbers and another example here actually there's three examples here another person here we call by the way this is there's not a person called john doe it's this is a um just to prefer for to protect the innocent um and this is jane doe again matching signatures same address same phone number but two different voter ids same here two different voter ids same address same name we don't have the phone number here etcetera so we have three examples of that then finally the last anomaly i want to show where we saw something fascinating is where the verified and approved stamp and you have to look at this carefully it's occurring behind the envelope triangle let me explain so if you look at this carefully this is an image of an envelope here is a triangle which is pointing people to print here now you would think if it was stamped this stamp should be over this image but it is actually behind the triangle all right and you see it here again close up here these are all different ballots which were approved post november 4th predominantly where the image of the verified and approved is behind this now maybe this is done for a good reason maybe it's an imaging technology but typically you could you know if you use photoshop you'd have layers but i i don't want to even accuse that but i just want to say that it is interesting that the verified and approve is behind the envelope here the envelope triangle all right so again there's some examples here so questions these are the questions we have for maricopa officials one is did maricopa receive any duplicates again i've gone through we received we have in our possession 34 448 images representing 17 duplicates from 17 126 unique voters to copy three copied poor copy the word duplicate does not show up as a keyword in their report but it would be interesting to know duplicates exist the second one is there a reason that echo male has no more more no signatures than reported by maricopa um is it because we solely analyze only the signature region and if not why the next question is why did echomal detect more scribbles in maricopa's reporting of bad signatures again this comes to the point which i um probably emphasize enough here but if our scribbles at that less than one percent point one to one percent pixel density were considered bad signatures that is significantly more three to four times more than the maricopa's 587 bad signatures and the other questions are the date stamps and the directories that we have the date on which the evd return envelopes are received by maricopa officials we have assumed that when we did our time temporal charts and then finally why does the approval stamp verified and approved appear to exist only on a small subset of the ebv return envelopes out of all the 1.9 million the that verified and approved we find most of them exist after 11 4 after election day but very few sprinkled so out of all those envelopes our initial supervised review reveals maybe 10 percent have the stamp on them the other thing is did maricopa stamp some edb return envelopes as ebb approved even though signature even though signature is black since they found a signature elsewhere and that would be good to know how did they do that what is the adjudication process finally all of this leads to a very important set of things from an engineering standpoint what is a standard operating procedure we call it an sop for the evb processing if in any engineering system you have the sops because again we're relying on this very very important process of the signature and its verification what is the sop for the signature verification and what is the sop procuring of questionable signatures and finally what is the chain of custody talk a little bit more about that and the last set of question is why was there sudden surge of duplicates during 1104 to 1109 which is incongruent with the trend we were seeing the early voting ballot return envelope counts finally why is it verified and approved stamp envelope appearing behind the printed envelope triangle how does that happen is there some imaging that's done are the envelopes printed i mean it's a very i just have a question from an imaging standpoint and the other question is can two voter ids can two different voter ids be associated with the same person the same address of matching signatures is that allowed and then finally why are blanks being stamped as verified and approved and then more importantly why is the stamp verified and approved appearing in in a blank signature region so those are our questions in conclusion as i started this conversation the evb return envelope is a container of the ballot it's a very important thing i mean if you think about the human body as a system your skin is what contains you that envelope is what contains these bowel it's a very very important part so there's significant opportunities to enhance precision verifiability reliability audit ability and reproducibility in the world of engineering systems we call these properties and these anomalies give us a wonderful opportunity to enhance these at least these five attributes so we believe what needs to happen is that the signal our conclusion is a signature verification process um no pun intended is unverifiable okay we can't really verify this process and there's a lack of systems integration and reporting example the ev33s which should have all the uh early voting ballots just on the duplicates issue we haven't had time to do a full systems integration that wasn't the scope but even on the ballots we have far more duplicates than what are even in the system and currently for to us because we haven't been able to get access to the standard operating procedures that is opaque and non-transparent so the future research we believe that is absolutely necessary is we need to do full signature verification audit which means do the full 27 point analysis we have the capability right now we have everything imaged if we have the the if we acquire maricopa's sop for signature verification if we can get their 27-point analysis algorithm we can replicate all of this and using the algorithm that they have define an actual false positive false negative error rate what i mean by this is this would be a profound opportunity for improving u.s election processes because this has not been done you read literature on the left or the right everyone complains that the signature verification process has significant issues we have an opportunity right now with this data and the opportunity here to do a 27-point analysis and really come up with an actual rate which would give us a scientific metric of the confidence value of the entire edb system and finally we need to review the chain of custody today what happens is when a signature is there's questionable signature people call the person they contact them and then they have some conversation which is then verified where are those conversation records are those tickets stored anywhere and do we have access to that that's it thank you everyone thank you very much i'll take questions if there are anything dr shiva just before you get off here um i think it's really important that we know your credentials you kind of just breezed through that slide can you go back to that real quick and just go go over your credentials for everybody so we that's important for us as we're reading your report sure um i appreciate the opportunity to share that let me bring it up so let me just give you a little bit of my background i have four degrees from mit a phd in biological engineering and what's called computational systems biology which is all about doing computation recognizing patterns my masters is also from the mit department of mechanical engineering where i did computational weight propagation to look at a very important area of pattern recognition called non-destructive testing where you're looking you don't want to actually open up a bridge or you don't want to open up the aircraft wing you're sending signals in and you're classifying them my other degree is from the mit media lab in scientific visualization my master's where i also use these same techniques to do some of the earliest complex visualizations for classification and my bachelor's is in uh electrical engineering computer science also from mit where i built for that one of the first cardio cardiology systems for doing pattern analysis cardiology signals but beyond that my focus for 40 plus years has been in this field of pattern recognition and classification in biology and medicine engineering aeronautics civil electrical banking and finance military across a range of areas handwriting recognition on bank checks email analysis in fact i as a graduate student uh i won i was only graduate student asked to participate i won the white house competition for automatically categorizing the white house's email this is 1993 when email was becoming a consumer application prior to when it was a business application of the email that i created back in 1978 in 1993 when email actually became a consumer application the clinton white house was getting tons of email they wanted to automatically analyze it i ended up winning that left mit took a 10-year hiatus and built echo mail pattern analysis recognition recognition emails did a lot of work for many years as an undergraduate helping deaf-blind analysis of signature pattern analysis there currently i work in a company called cytosol where we're looking at analysis of signals biomarker signals to figure out the right combinations of medicines that company actually got a multi-combination therapy allowed by the fda for pancreatic cancer um i've written a number of patents books anyone wants to go to go to vashiva.com and you can look at my biography over there i've published in the leading journals in the world nature neuroscience is one of the eminent journals in the world cell biophysical journal ieee these are high impact peer review journals [Music] the u.s copyright office delivered me the first copyright for the invention of the email i'm a fulbright scholar a lamelson mit finalist i won one of the earliest westinghouse science honors award i was a nominee for the national medal of technology and innovation and i've been invited to give distinguished lectures at the national science foundation nih fda in fact in november 19 i delivered the prestige lecture on the immune system and invited lecture at the nsf where we discussed the immune system and several years ago mit had me delivered their presidential fellows lecture thank you for sharing that you've raised a lot of very important questions for us to get answers to thank you dr shiva your report was extremely insightful and considering the fact that we only got the envelopes just a few short weeks ago we appreciate you dropping everything all of your other responsibilities and jumped on this right away so that this could be accomplished by today's hearing date so we appreciate that very very much and i also want to thank uh doug applegate and phil evans two of my colleagues at echo mail and the echo mail team in particular all the stakeholders and again i want to thank the courage of the leadership of the arizona state senate this will go down in history as one of the most important engineering events not just an election event it will go down as a very important engineering event for the engineering systems of election voting systems i really appreciate the opportunity and i'm very honored to support this effort thank you thank you dr shiva and for everyone um in tv land or whatever um these reports all will be made available in their entirety and they'll be uploaded on the website as soon as we can get them uploaded so you can see all of these personal information and the personal yeah just make sure that the rejection that you saw was any personal information we tried very hard to make sure we complied with the court order and make sure anybody's names addresses that were on there was not available to the public however the unredacted version will go to the attorney general's office so he can seek further investigation on these anomalies thank you dr shiva thank you very much all right thank you so much and just also a note it was interesting we had received a lot of um emails affidavits you name it from people that actually worked at m tech and at the polls and that they had told us that when it all started what is it 27 or 29 points of signature verification 27 thank you that that's what you're supposed to do is 27 points of signature verification and at some point it went to 20 it went to 10 and towards the last few weeks we were told that they were told just stop checking signatures we've got to get this done so what he's showing us here does in some sense correlate with the things that people had told us so for what it's worth okay let's go to doug logan cyber ninjas you were going to give us your report and would you please give us just a tad of your background sure so i'm doug logan i'm the ceo of cyber ninjas i have done cyber security work for a lot of major organizations including bank america and j.p morgan chase done decent work in the federal government as well i hold the certified information systems security professionals or cissp i also um have gx web application penetration tester and gx certified instant handler i am listed as i am actually an expert on the antrum election case um associate and i have a report that's actually published associated with that that's publicly available from matt deperno's website and i have of course been running this audit for the last roughly five months mr lone could you pull the mic or could you get a little bit closer to mike please is that better okay awesome it's not clicking there we go so we just want to start with uh an overview of what we actually accomplished here because this is this is an audit like we've never had before and involved as you mentioned earlier madam president i re involved over 1 500 people and based on our calculations it was actually more than 100 000 hours put in place as you can see up on the photo right now this is an example this actually happened in in the evening one of the days so this isn't even a full group of everybody and we're going to go through each role that was out there what what functions they were performing but you'll notice that everyone is located based on colors that helped us keep track of where people were and make sure no one was out of place and help us make sure that we both secured and maintained custody of all the ballots at all times custody of all the ballots at all times custody of all the ballots at all times of custody of all the ballots at all times over a hundred fifteen hundred people and like a hundred thousand hours now security was something that was extremely important with everything that we did here we had a multiple layers of security we actually had an external perimeter that was maintained by the arizona rangers so as you came in that would validate that you're on a list of individuals in order to be able to get in we also had an interior checks so that as you walked through the door you came up to a desk where you were both chest checked for covid valley to be on the list made sure you had a badge and and all those things to make sure that you're someone that was supposed to be there to even be in the building all the ballots and election equipment were stored beside um within these cages and anytime they ever left the cage they were actually signed out by an individual so we have a complete signed record of every individual that came and picked up every box so someone you know someone took over custody of that box and we actually had individuals called runners and they would run it from the ballot corral and they take it over to whatever table they need to go to um and that individual would then sign the box over to the table manager at the table that would then utilize it to process it so at no time was a box of ballots or individual ballots outside of the care of someone's specific authorized care who has signed off of it in fact we actually had 24 7 video surveillance on everything at all times in addition you'll see this lovely police officer they were actually there 24 7 as well and they were always within sight of the ballot corrals or where we had the election equipment stored we had we had police officers at both locations um always maintaining and always making sure that that the ballots and the equipment was 100 secure and that nothing could happen to them so this is one of our tallying tables what you can see in the middle is actually a lazy susan you'll notice that there's three counters around the table there's someone who's actually at this table they're loading and someone who's unloaded so all counters were maricopa residents who specifically voted in the last election we wanted to make sure that if anyone was involved in this very important action that they had skin in the game um and they were they were local we didn't use anyone out of state for any functions having to do with with the actual italian of the palettes i should i should say actually telling the ballots some of the table manager from out of state the people actually counting the ballots were all maricopa residents there were other functions where we had volunteers from other states though as much as possible we tried to keep everybody local in maricopa county we think it's very important for our local community to take ownership for their election process and we wanted to facilitate that as much as possible all these individuals were background checked and validated we had one individual that that had slipped through the process because he'd been on the ballot versus specifically um that does not something shows up in standard background checks so we started doing additional checks comparing everything on things to make sure that never happened again it was very important to us to to make sure that we always um had things that we're going to make sure that were beyond reproach transparency and everything we did was very very important now all three counters were blind that means they were not allowed to talk to each other as that ballot went around and was in front of them they would tally on a sheet of paper and they put a little marcus based on whether for both the presidential and the senate race they put a mark to to see whatever it was roughly every 50 ballots they would compare the numbers with each other actually the table manager they can compare them and if two out of three of them agreed and the third person was no more than one count off for for the race they would proceed and they would move on if there was any more discrepancies beyond that they actually had to stop they had to find the ballots in question that they're in question they had to recount them so there's absolutely no way to to have speed here without having accuracy and we found that when we had a brand new table it it was relatively slow but as soon as they've been doing it for about two shifts or so their speed greatly increased and it was amazing how quickly they could count ballots in the very beginning of the process in our first three weeks our fastest i think that our most balanced we ever counted was roughly 30 000 when we came back and had a more of a full shop i think at our peak we we did over 150 000 ballots in a day and we're routinely doing over a hundred thousand um so the the speed like i said uh greatly increased um especially as we work through this okay so the way the ballots would actually work is we have we had a ballot corral where the ballots were originally in and they would be signed out of that ballot corral and they'd be taken to a tally table after they were tallied they'd go back to a ballot corral whether which was specific to being in progress and then when we had our paper examination tables they were ready for a box of ballots that's where it came it came out of the in progress so at the paper exam examination tables we had dslr cameras those dslr cameras took pictures on the front and the back of these ballots and they gave us very high definition images that allowed us to see all sorts of intricacies of the paper and what's going on after it was done with dslr actually excuse me sir i'm sorry the clicker's not working there we go um let's go back there really quick so once i went into the microscope stands we had four microscopes they were taken images of of the uh magnified image and that was over a court number of different places that included the presidential oval to take a look at how that was filled out and with the way we had the lights we could actually see when it was filled out with a ballpoint pen you could actually see the ridges of the person pressing down on it so you could tell the difference between something that was filled out with a sharpie you could tell something was print out by a pen or you could tell actually if something was computer printed um you could tell the difference between all of them we also had a microscope over certain parts of the the ballot that would allow us to look at the paper fibers to timber to help determine what type of paper was utilized now when we were done with this we actually had over 140 terabytes of data just from the paper examination alone so we had massive amounts of data in fact with our camera footage and everything that ran the operation we ended up with close to two tera sorry two petabytes of data to give you a comparison that is vastly more data that your average large thousand person company has um so we were running a very impressive network here that was completely air gapped wasn't connected to anything outside of the floor in order to support this in order to have all the data requirements associated with it it was a very complicated operation so just i want to give a high level status of where we are right now so first of all we've completed the hand count out of all federal races we've done all the image and microsoft capture of all the ballots we've reviewed and did a comparison of the official results we've done analysis of the voter rolls we've done the vast majority of the analysis of the actual voting machines and voting equipment in progress right now we're hopeful to soon with the settlement that the senate put together to be reviewing the logs and routers based on the settlement terms we're also hoping that the completion of the the paper analysis that's being done will be done very shortly now from a scope standpoint stopped again um so no longer in scope um was the canvas in um that was decided that was removed we have the tabular configuration to check internet conf configuration with something that was not provided we believe with the other information we have we'll be able to get similar data but it will not be the same exact data um we had requested review the voter roll system but a lot of the systems are used for checking in for the cyborg system were not provided as part of the equipment that was given us we had hoped to look at the review of the icx devices but again those had not been provided we had wanted to look at the provisional ballots the ones that had not been counted to to count the sealed envelopes to make sure they matched up with everything and made sense and those again were not provided and we'd also hope to take a look at the undeliverable ballots um to see how many of them were bounced back or what happened to them and again those were not things that were provided and so there were things that are no longer in scope and we're not able to take a look at okay so now we'll get into the fun part and get more into the the actual talents and results in order to understand some of these findings i want to make sure that everyone has a clear picture as to how this process works so if we have ballots that are actually damaged or otherwise can't be run through the tabulators for example the braille ballots won't fit through the tabulators some of the yukava ones are in formats that can't they need to go through what's called a duplication process now this is a different thing than what dr shiva was talking about when he talked about duplicates he was when he talked about duplicates he was talking about more than one envelope going to the same person so we're going to talk about the word duplicates here but we're actually talking about when there was an original ballot that couldn't be run through the scanner and they created a copy of it which was run through the scanner and counted so when you have duplicates when you have the rituals and you have the duplicates only one of those counted and it should only be the duplicates now specifically the originals are often referred to as damage sent to duplication or dsd and we're going to use that notation throughout things that dsd means the original ballots whereas the actual duplicates are referred to as dupes so there should be one one dsd per dupe and there should be a unique serial number on every single dsd in every single dupe to match them up so that you will know for sure that this ballot right here was duplicated to this other ballot and if you want to compare and make sure the duplication process was done correctly and that it actually represented the voter intent you'd be able to easily match them up and it'd be easier to handle from an audit standpoint duplicate ballots also should be stored separately from the original ballot so there's not confusion they don't get mixed up so let's talk a little bit about those findings so our duplicate ballots were coming commingled with the original ballots not all of them there were in at least one box case which is something that they are not supposed to do according to the epm duplicate ballots had incorrect and missing serial numbers if you take a look at this table that's showing up you'll notice the the serial number on the left is what was actually on the ballot um for the duplicate ballot i know the serial number the board one hand duped to 118 on 214 on the right is what was actually on the original damage um you'll the you'll notice the numbers are actually different on the left-hand side it says dupe board three hand hand zero two one fourteen on the right is his board one hand dupe 214. we were able to match these up because of the ballot characteristics and the precinct that it came from but it's a very painful process and it has to be done manually in order to figure out what matches to what else when they're not actually stamped with the same exact serial number on them we also had a number of uh serial numbers that were printed like this one that you see on the screen where it's you can't really read it it's not legible um so it theoretically has a serial number on it but there's no way to match it up because you cannot read what the value is on it in addition to to having ballots that flat out did not have a serial number anywhere on there whatsoever so there was no way to match them up with originals now we also had duplicate ballots that reuse serial numbers you would expect that a serial number in order to match it up without being unique but you'll notice there's two examples up here of two different pairs that are not associated with each other but had the same exact serial number utilized in this case is i believe it you know we've got a large print and a damage standard damage pad we also have a few others in here again same exact serial number but these were not the same ballots and specifically if you ever take a look at a large print ballot they're huge they're very very large so it's not just that something was photocopied or anything like that they literally use the same serial number on otherwise unrelated ballots and this is probably one of the more interesting parts is that we had more duplicates than original ballots um so according you know to our accounts from our audit we had 26 965 original ballots and then we had 29 557 um that were duplicate ballots and those numbers should be the same based on the numbers received from maricopa county we should have had 27 869 of both originals and duplicates and they should have matched up perfectly now these extra duplicates did appear to to favor trump and jorgensen um if you take a look at the original ballots we've got trump has nine 995 and 400 404 and if you take a look at the duplicates um there's there's a some number of more of them um so if you take a look at the percentage all the way over to the right the expected percentage per candidate and that's based on if we take the originals and assume the duplicates should be at the same exact percentage um so we had 48 percent of the originals were trump um 50 of the of the originals were bought in and two percent of the originals were were jorgensen um you can see what they actually have a difference in reality trump had 58 of the duplicates 33 and 3. so both trump and jorgensen gained slightly um with the duplicate process same thing when we take a look at the senate race um it favors slightly to to mcsally not necessarily as much as it does in the other case both of these percentages are within the realm of error of just human error making mistakes so if we take a look at our final tally results and we look at this at the senate race it does look like kelly still shows up as a head and we actually run into that we have 541 less for mcsally and 60 less for kelly in the ballot totals if we take a look at the presidential race trump actually loses 261 votes from the official votes biden gains 99 and jorgensen loses 204 votes and again these are all you know very small numbers when we're talking about 2.1 million ballots these are very small discrepancies so we can say that the the the ballots that were provided to us to count in the coliseum um very accurately correlate with the official campus numbers um that came through so we did have at least a batch of 50 ballots that was run through the tabulators twice this is specifically when they took the dominion tabulators they had the same batch and it was run through more than one time we found this because through our various counts we had very clear confirmations of how many ballots were supposed to be in a box and when we compared our results against the cast vote records we were roughly 50 off and that made us go take a look at the dominion images and we when we compared it to nearby ballot we actually found um that there was a set of 50 ballots um that that had been run through i said again twice as far as we could tell with it only happen in once there's no indication as to whether that you know we don't have no clear indication whether it was human error or whether it was intentional we assumed it's human error because it wasn't happened in very small frequency this is an example of a ballot from two different batches it's probably relatively small text for people to read but if you read all the way down on the bottom it'll tell you that it's tabulated on the left-hand picture tabulator 6004 um btc which is the batch 288 and that's the image 154 and then the right-hand side we have again same tabular but batch number 287 which is one different and it's image number five and we had roughly 50 of them that were in between the two of these we found something similar from the ukava where the ukava for those who don't donate ukava ballots are for military and overseas personnel it's a way that they can actually cast votes even though they're not physically here those ballots get turned in either via an online portal they can be submitted via email they can be mailed in i think they can fax them in there's a whole bunch of different ways these can come in as a result they're in a lot of different formats and there's a lot of variance among them but on the left-hand picture you can actually see that it's board number two hand dupe 573 is what approved it on the right hand side it's hand dupe 574 and it's probably too small for you to read but if you read the serial number circle down on the top left you'll notice that the time stamp and the seal numbers are exactly the same which means that someone likely printed out the yuakov ballot twice and both of them made it into these things we did not find this in in a lot of quantities but the only way we had to look at it was by hand and there were there's over 10 000 yuakava ballots because every single one of these yukava ballots is slightly different form it's actually even difficult to do it in automated fashion the only good way would be to to go through by hand without some sophisticated processing okay so we're going to talk a little bit about the official results that are actually turned out by the county and this actually connects quite a bit with some of the stuff that dr shiva was talking about so from a definition standpoint we have the official canvas the official canvas is the official certified results that are put out by the county it has the tallies of votes for candidate per precinct and provisional it tells what the turnout was for a given precinct and all of that information we also have what's called here in maricopa county they call the vm55 file or the final voted file it is a list of every single person that showed up to vote and that's counted both for you know for in-person whether it's early voting in person or whether it's mail-in and all of those are kept track of separately so we have a number of all those different categories of who showed up to vote on any given day we then have the vm 34 volt full voter file this is also referred to as say your voter rolls this is a list of everybody who should be eligible to vote and the county in maricopa seems to to make these available roughly on a monthly basis so that the the full voter rolls um so that is your theoretical full list of everyone that may be able to you know may show up to vote on a given election then we have our ev32 files or ev32 early vote incense every single time a mail-in ballot is sent it's supposed to have an entry in an ev32 file that corresponds with that mail-in ballot that's sent out likewise we have an ev33 file which is when any type of early vote is returned now this both includes um when a mail-in ballot is received and an in-person when someone comes and votes in person ev33 includes both of those you may remember at the herein that we had a while ago we had mentioned that the ev32s do not match the ev33s we are doing a quick analysis in order to justify canvassing i mean there were 74 000 that were off of the vast majority of the 74 000 were from early voting in person and that is why there was not an ev32 associated with it we have this clarified in our report as well that was not a purposeful discrepancy it's just something that that you know that was not immediately clear at that point so just to give an example on how these systems work in order to match everything up so if we have 10 people who mailed in a ballot and we had 10 people who voted early in person and we had 10 people voted on election day in person you know we have roughly we have 30 30 votes that are out there so what this means is we should have the official canvas with 30 votes and it should be allocated per precinct accordingly it should be allocated it should be allocated based on the candidates and you should have your official tallies associated with it you would also expect that if you went to your vm 55 file you should see 10 people who voted via mail-in because it has different codes based on that you should have 10 people who voted early in person and 10 people who vote on election day and every single one of those entries should have the name and address associated with the person that matches up with their voter rolls likewise you would expect that your ev32 would only have 10 ballots mailed in it because that was what was sent out and your ev33 should have 10 people in it from the mail in ballots and 10 people from the actual eevee in person for a total of 20. this is what you would expect in a balanced system this is not actually what we found when we started comparing all these numbers all these numbers were different in very different ways and it's something creates quite a few discrepancies um so none of these systems actually balanced we're going to go through these so our official canvas has 3432 more ballots cast than the list of people who show as having cast a vote in the vm55 file now i do want to specifically interject in here that we we finally heard back from maricopa county um because we asked him about this discrepancy i think it was at least a week ago but it was a couple weeks ago so the day before we presented our results they decided to tell us that those were actually for for the protected voters who don't actually you know who either judges or battered women or other individuals who are concerned about publishing their addresses that that is the reason why that discrepancy is in there i can't validate whether that's that's accurate or not accurate this is information that we just just received what i can say is that this sort of stuff is exactly why with audits um usually the organization you're in the process of auditing um cooperates and works with you um and um it would have been you know these yeah would have been extremely helpful in order to to get feedback um and work through them through this entire process now we do have 9041 male and voters shown return ballots more return ballots in ev-33 than they were sent ev-32 sounds like dr shiva found something very similar to this as well so specifically we found they were mailed one ballot but somehow um two ballots were received which i do not know how you would have one ballot sent and two received the assumption would have to be that it's a clerical error or there's something else going on it's not clear how you can have that happen 277 printing show an official canvas as having more ballots cast than people should up to vote for a total of 1 551 excess votes again the county has explained to us that the same reason for the vm-55 difference they're saying is a reason there we have not had a chance to validate that there are 2 472 ballots shown in ev33 that don't have a corresponding entries in the vm55 and only 2042 ballots show is rejected in the official canvas for discrepancy of 430. so let's walk through this really quickly so if something is in the ev33 that means that an early vote was received and we have the individual's name and the voter id associated with it so if an ev33 was received you would expect that if it's not in the vm 55 file of who voted then it had to be a rejected ballot just makes logical sense it has to be rejected ballot but there's 2472 that show in the ev-33 but there's only two thousand i'm sorry there's only two thousand and forty two ballots the show is rejected um so there's a discrepancy of four hundred thirty again just another place where they show that they don't seem to match up we also have 397 mail in ballots show is received that were never shown as sent so they were we know that there are in the vm55 they're mail-in ballots and they were received without somehow ever being sent a ballot now we also have two thousand five hundred two hundred fifty five thousand three hundred twenty six early votes shown in the vm-55 that do not have a corresponding entry in the ev-33 and just be clear this is not you know when you were looking at ev3 33 entries um ev33 entries um are supposed to happen when a ballot is received but it's not you know the actual tracking of the ballots are under you know the the actual official canvas or the vm55 um so we had in the vm 55 individuals who voted we had entries for early voting they were in there where in 255 000 cases of those they were not actually included in the ev33 file in reality all of these you know all these systems to be audible and to be verifiable all these systems should be able to be in agreement with each other i mean even if we have protected voters there should be some way to know the number of protected voters who voted in order to match it all up so that you have a system that balance balances okay voters who are moved so we took a look at the entire list of individuals who voted specifically um that was from the vm55 final voted file and we ran it through a commercial database called the put out by melissa called personator and personator is a best-in-class identity system that helps check addresses to make sure they're associated with the user it'll show prior and current addresses it'll track move dates and i'll actually also track date of birth and date of deaths so we went and took the voter rolls and we compared them against this to see how many people might have moved and based on statute should not have necessarily cast a vote so the first thing we found is that 23 voters who voted via mail-in ballots even though they showed in melissa as having moved from that address and we we wanted to make sure we accounted for the circumstances where a college student might have moved away from home or family member might have moved somewhere else so we actually eliminated all the chances all the cases where someone was still at that residence that had the same last name and that's how we came up with 23 344. so if your mailing ballot is sent to an address that you no longer live at there should be no way for that for you to receive that mail-in ballot um at least there you know it's generally not except a way for you to reset mail ballot because mail and ballots are legally not allowed to be forwarded so it can't be forwarded to your new address so the only way this situation could happen legally is if you know the prior you know if you know the current resident and you're able to meet with them and pick up your mailing ballot or somehow have some other arrangement to pick up your ballots were there but still 23 344 people voted when they should no longer you know have access you know would not normally have access at the given address we had 2 382 voters who voted in person even they showed melissa as having moved out of maricopa county prior to that date we have 2081 voters who moved out of state in the 29 days before the election and appeared to be given a full ballot which should have been you know if anything if they voted at all it should have been a president-only ballot where literally the only option on the ballot would been president that is something that is by arizona statute let's talk a little bit about the voter rolls registration dates um do not generally change on in your voter rolls so this is your day of registration unless there's it's to correct a mistake and this is something that we received out of the the actual recorder's office they told us that those dates should not generally be changing your date of registration should be your data registration they'll and it says there at the end the only time a voter may have two dates of registration is if the registration has previously been canceled and the voter registers again the original record would be canceled for provided reasons and then you know they'll have a new record with a new date of registration so the old record should not exist and it should not have a data registration change likewise we have this thing called afseek and afc extent is actually a unique identifier that is a reference to a transaction so for example if you went and you needed to change your address you would fill out a form and when you filled out that form you would turn it into the recorder's office and they would image that and they would process it and when they processed it they would assign it a unique identifier that unique identifier actually gets stored in your voter rolls as the latest one and it is specific to that change that that change request so this should be unique they should you should not even have it twice in your voter rolls it should only happen once and it should not be shared among multiple individuals and this is something again we confirmed with recorder's office now we also have a statute that says complete names should be used and they'll get into our findings that we have here in a second so we had as many as as 5047 individuals who voted in more than one county for up to 5295 additional votes if these are duplicates now i will tell you that there's these individuals had the same name first middle and last name and the same exact birth year because that's what's in the voter rolls but if you have an extremely common last name which can happen there's some of these may in fact just be individuals with the same exact name in the same exact birth year we have no way to validate that 100 percent we had 393 voters with incomplete names that voted in the election this is include individuals with last name only last name is just an initial there's no last name our first name is just an initial and again there are some individuals who in some cases this could happen but this is not a frequent thing that you typically see we also had 198 individuals who registered after the october 15th cutoff and yet still voted in the election and we had 2861 voters who have shared an absence number with another voter at some point in time and that does not we don't know exactly what that means but based on the descriptions of everything that's happened in the system it suggests there may be some integrity issues with the data when an impossible situation is happening in a system and if you've got integrity systems in in something as important as the voter rolls there would be a concern we have 282 potentially deceased voters in this election i know that there's been some much wider numbers that have been circulated on the internet we tried to validate this stuff very very precisely it can be a difficult thing to match up voter rolls to individuals from our test in we believe that all the ones we have in here accurate but there are potentially additional ones as well we have 186 people who potentially have duplicate voter ids that both voted i think that dr shiva was mentioning that we haven't had individuals with the same first name last name and the same address and seem to have the same signature um that is something that we have also seen in the voter rolls with people who seem to have literally the same exact name we have 186 people that's the case where they actually the first name and last name and the address you know all match up and year of birth because assumption would be you might have a junior in in a different place and so they'd have a different day of birth associated with it but not the same address and that is the end of the presentation right now i'm gonna with that i'm gonna hand it off to ben cotton who's gonna go over our digital finance thank you mr cotton when you start could you just give us a little bit of your background software the maker of incase and incase enterprise software they're now owned by open text i have 25 plus years of doing digital forensics incident response and examinations in support of both government government as part of my service uh prior to my digital forensics background for the last 25 years i also served 21 years as a tab qualified special forces soldier serving this country and defending our our freedoms i recognize how critically important the voter integrity is to this nation and this again madam president i agree with you this is not a left issue this is not a right issue this is an american issue and as i as i talk about this you know i would hope that the findings of this audit will be turned into actionable legislative um meaningful uh product that we can move forward and secure these elections moving forward so as i talked today people may have heard some of my previous testimony and this is not working stand by everybody pulling up the powerpoint we had a few of these will be redundant but it's important to reiterate these findings in the course of this we had a few of these will be redundant but it's important to reiterate uh these findings in the course of this final hearing so we'll talk about the withheld devices and data and how that impacted our ability to provide a complete report to the senate here today we'll talk about the cyber security issues that we have found the hardware configuration control issues atypical anonymous logins that are present on systems they're listening reports and attempted connections on boot up and internet connections and internet history that was found on these devices as part of the part of the course next slide please so let's talk about the withheld devices for a moment here as with any audit access to information and the right information in the timely manner is absolutely critical to finding a complete result for an investigation in the case of this audit we were never provided access to the routers and network-related data and that becomes very impactful when we start talking about validating and confirming unauthorized accesses to the election management system itself and to the other devices i would like to sit here today and tell you that i had fully ruled out any unauthorized access but given the lack of access to this information i cannot do that at this time it is our understanding that there has been an agreement reached with the maricopa county and i look forward to getting access to this data so that we can complete these findings we were not provided the poll worker laptops now these are the laptops that poll workers use at each precinct to validate the voters to and to interact with election related functions at that particular precinct we were not provided any of the icx devices so the icx devices are used for handicapped and other graphically required interfaces with the voting systems we know that the county had a number of these based on the historical video from the mtech but we were not provided any of those we were not provided with the icp credentials to validate the configuration settings or the advanced administrative settings on the actual scanners the icps this was critical and a significant short finding in that i cannot sit here today and tell you whether or not the wireless modems were enabled and connected to the internet at the time of the vote and i cannot sit here and tell you today what the status was of the land connections that we know were inherent to those devices as part of the purchase from or the least from dominion okay let's let's go a little deeper into the cyber security issues as as we found them um actually you've uh you've skipped a slide i don't know i'm sorry um so let's talk about the uh go back one slide please let's talk about the cyber security issues now uh the department of homeland security has a division called csa and sisa has a recommendation that's published on the internet for how to configure and manage election systems that is freely available and it's recognized as kind of the gold standard for securing an election system i will i will tell you that every item up here is part of that recommendation from from csa i will tell you that in maricopa county they failed to perform perform basic operating system patch management functions now let me explain that for a bit so if you have a home computer you realize that every thursday or wednesday depending on your cycle microsoft will release a security patch to correct vulnerabilities that have been discovered since they released the operating system and it was installed on your computer the last time that the operating system was patched on the maricopa county election systems was the date that they installed the dominion software which was the 6th of august of 2019. okay so at the time of election it had been over a year since that system had been patched we found that that was also the same case with the antivirus definitions so we know as part of this world that we live in that people are coming up with new ways to hack a system to to exploit vulnerabilities and get unauthorized access to the systems and they're doing this continually all major security vendors update their antivirus at least on a weekly basis to make sure that we can protect our systems from these newly originated vulnerabilities and exploits the last time that the antivirus had been updated on the maricopa county systems was the 6th of august 2019. okay now maricopa county did release a statement saying that if they had patched the operating system or if they had updated the antivirus that that would have invalidated the eac certification for the voting system itself obviously there are a couple of issues with this position and the first one is is that we are relying on a certification system that would impose obsolescence instead of security in the very act of trying to secure a voting system that is nonsensical and it should never occur so if that is in fact the case we need to take a very close look at what we're relying on to validate and certify these election systems and software to ensure that we're not certifying guaranteed obsolescence of the system now let's assume for a moment that what maricopa county said was true that they could not update those systems because of this certification issue there are a couple of problems with that as borne out by the artifacts on the actual ems server itself so if that is true then that would mean that no new executable files no new dynamic link libraries could be created or modified on that system after the date of the software installation which once again was six august 2019 what we found is that there are four executable files that were created after this date of dominion install there were 45 executable files that were modified after this date of install there were 376 dynamic link library files which were created after the dominion software installed and there were 150 or 1053 dll files that were modified after this date so if we assume that what the county represented is true then in fact that voting system would not have been certified at the time of the election let's talk a little bit about law management here so there is a federal statute that requires the preservation of election related materials for 22 months after the date of the election that applies not only to paper but that also applies to digital artifacts maricopa county failed to preserve the operating system security logs to cover the dates of the election they provided security logs early in the audit process but they did not provide the windows security log itself when we examined the ems server we found that the the dates covered by the security log only went back as far as the fifth of uh february 2021. now there's a couple reasons for that and we'll go into those uh later but the bottom line is is that they they failed to preserve those those logs or at least those logs were not turned over to the auditors and so i'm assuming since they were part of the subpoena that those should have been provided had they been present next item there is credential management this is probably the most offensive item on this list to me because it carries such a huge impact on the securability of a system what we found is that for the election management system the adjudication systems the iccs and all of the voting related systems they all shared a common password for both user accounts and for administrative accounts and just to be crystal clear it was the same password for all those accounts so to complicate matters as well those accounts had not been changed since the installation of the software so they were established on the 6th of august 2019 and never changed there furthermore was not an individual accountability of the users who access specific accounts so that you could tie a username and action in an individual when you discovered something that was an anomaly okay we did not see any software or any effort to establish and monitor a host baseline of of programs and and processes furthermore we did not see any log aggregation or methodology by which to establish and monitor the network communications for this system there simply was none of that software present on any of these devices next slide please we also didn't saw and detected that there was a failure in hardware configuration within the voting system now what you're seeing right here is the system the acquisition photos for the system that was identified as the adjudication ii workstation okay now you will see in that picture that there are two hard drives that came out of that system it's not uncommon to have two hard drives in a system for data storage etc etc however both of those hard drives are bootable okay so what that means is that you can boot from a hard drive that is not part of the election configuration and have access to the election network okay very important okay it's clearly not an approved configuration it was the only system that we found a dual boot uh situation now to further complicate matters a little bit and i'm not going to make judgment as to the legality of this or whether or not this deserves further action is that on that second bootable hard drive there appeared to be non-maricopa county data so let me let me define that a little bit with on that hard drive there were dominion databases that appeared to be one demonstration data but also data that may have originated from washington state and south carolina and i derived that not from an in-depth investigation of the data that was outside my scope but from the naming conventions of the databases themselves okay so once again and approving the configuration of the of these systems is critically important to preserving the integrity of that election system i would also note that neither of the two audit next slide one of the challenges that we had was was actually the accountability of deleted items we talked about deleted items before but let's be crystal clear about this from the ems which is the election management system server on the c drive there were 865 directories and 85 673 election related files deleted between 10 28 and 1105 they also included some log files so what is difficult to determine is that i know they were deleted what i don't have is any accountability or any ability to track from a evidence management perspective how those deleted files were treated and what happened to them if they were archived or not okay there are a dvd file and those are actually the results of the of the election totals off of each uh tabulating device and those were part of the deleted files that were removed from the ems from the so the the ems actually had two hard drives well they had six hard drives configured into two logical drives so the second logical drive was called so the second logical drive was called the d drive and that contained all of the election database both historically and should have contained the information for this 2020 general election as well you'll note that there were thousand five hundred and seventy one directories and one million sixty four thousand seven hundred and forty six election related files deleted between the first of november twenty three of those on the high pro 1 there were 304 directories containing 59 387 files of election data that were deleted from the hi-pro scanner one on three march 2021. now this becomes kind of important because part of our analysis was to look at the interaction of these scanners these systems and how data flowed you'll notice that that's about a month and a half before they turned that over to us next slide you see a significant amount of deletion on high pro 3 on that same date 100 1061 directories 196 463 files containing election data deleted on on that particular date on 3 march as well next slide and on high pro 4 you see the same high volume of deletions on the 3rd of march now once again this may be part of a normal process with how they handle votes but the timing of this becomes a bit suspect as well as the fact that we didn't see these deletions on high pro number two and once again i don't have a chain of custody for what happened to these votes after they were deleted it may be a plausible explanation i simply don't know at this point okay next slide let's talk about the the fail to preserve the operating system logs as i explained earlier the earliest time frame that was covered by the ems logs security logs was 5 february 2021 that clearly does not cover the election time period now there's a very good reason why it didn't and if you go down to that last last bullet there you'll see there were three dis last bullet there you'll see there were three distinct there's a user defined setting that you can define how much quantity of logs are retained before they get overwritten okay in this particular case the ems security log setting was set to 20 megabytes of data so the windows operating system will preserve all the security logged entries up until the point at which it reaches 20 megabytes of data at that point it starts following a first in last out approach to log retention so as you create a new entry an older entry is deleted and overwritten inside of that that log file now first and foremost we need to remember that we do have that 22 month federal mandate so it's clear that at least what existed on the ems when we received it as part of this audit we did not have the time period covered by that federal mandate as it was supposed to be covered that security log was not turned over as part of any other documents that we have by maricopa county so i'm going to assume at this point that it's not available for us to look at or else they would have turned that over to us now if you look at that last bullet that first in first out approach all of a sudden becomes readily apparent as to what happened on these distinct dates so on each of these dates an individual executed a script and that script repeatedly looked for a blank password for all of the accounts on on the system depending on the system there were only about 16 accounts that were present on a given system okay so this script was run multiple times on 211 462 log entries were overridden by this script on the 3rd of march 37 686 log entries were overwritten by this same script on the 12th which is the day before we received the system there were 330 log entries overwritten by that script now the challenge here is that i know that this occurred i know which account did it it was the ems admin account if you reflect back to what i just said about the lack of accountability of assigning that that username to an individual it now becomes extremely difficult to prove who did it okay now luckily we happen to have some historical data from the mtech video feeds and so we leverage that data to backtrack and align these times and we we have captured screenshots of maricopa county people at the keyboards during those time periods okay now we've identified that into those individuals but we will not release their names because we understand what the scrutiny is and what the impacts would be to those individuals but i just want to tell you that the very point that they did not have dms now remember the lack of log retention at this point we could not find any logged entry that corresponded to the uh to this activity okay from the security logs those windows security logs only went back to the fifth but everything was purged on the context of the election to be audited but that clearly was not the case in this instance that's right just to clarify so this is a log file specifically from report report um italian i'm sorry results telling and reporting which is the dominion software that entry says that someone went into the program and clicked on something that said i want to purge all the results for this election that goes through and that deletes all of the we have redacted specific elements uh to include the host name and the ip address and and some some of those types of things but the fact of the matter is is that those items are recorded as part of that normal anonymous log activity you will have the host name that logged into it you will have the ip address that originated the request in most cases you'll have the username as anonymous but that username is then validated against the access control lists and the user authentication mechanisms and validated so the very next log entry in a normal anonymous activity is a validation of that user's credentials to access that particular device or process or whatever they were accessing what you see on the left is something that we discovered in the logs which is what i call atypical anonymous login you will notice that none of those items that are captured by normal activity are present in this log you don't have an ip address of the originating device you don't have a hostname of the originating device and furthermore when i look at these in context of the actual security log itself there is no validation of a user's credentials immediately following this there are hundreds of these types of anonymous logins in the security logs that we do have i cannot tell you right at this point if this same type or pattern of activity occurred during the election cycle because these logs don't exist for that cover the election okay but i can tell you that without access to that router data and the network data i cannot validate whether or not these were legitimate accesses you'll notice that it is a login type three so it was it was a remote access i cannot tell if this is a legitimate access or an unauthorized access at this time next slide please so we also took a look at what happened when when the ems was booted up and and let me walk through the methodology uh here to to kind of assuage everybody's concerns so within the uh when we actually imaged all these processes and these systems by the way we we image 770 devices and we gathered over 114 terabytes of original forensics data okay and we preserve that in a forensics image file that i could then leverage without fear of modifying or changing anything on the original device so we took that that image file and we turned that into a virtual machine i created a enclave that i could boot that virtual machine up into and that i could then monitor the boot processes without connecting to the internet without exposing any voted data to unauthorized users and i actually booted up the ems to see what happened what it was listening for and to identify if there was in fact any zero-day malware in the in the memory what what we did discover is that as you would expect there were a number of ports this is a normal part of an operating system ports are used to establish connections and provide functionality to the operating system we've discovered 59 of those that were open and while most of these things were what i would have expected there were some unexpected high port activity specific to the when it exe which controls your accesses and your logons and your dns which controls your domain name service so if you go and you type in yahoo.com the computer will use the dns service to actually determine which ip that is so that you can connect and then the dhcp server all of those are normal valid windows processes but for example a dns the default port on that is port 53. okay in the case of the ems we not only had the port 53 but we also had dns monitoring on an on a high port and i won't i won't list that port number here in this open forum but that was a little bit unexpected i will tell you that on the ems that you were utilizing both ipv ipv4 and the newer version ipv6 enabled so these services had dual functions with dual listening ports and and things of that nature there were ports and there were services that were enabled that allowed remote access okay so the rdp protocol for example as well as the terminal services were enabled on the ems okay next slide as part of that memory analysis we did a complete um check of the call outs and the attempts to connect from the ems out to the internet now once again the county has repeatedly said that these were isolated systems etc etc etc the ems attempted to connect to those ips most of which are are normal i would take a look at the level three parent the edge cast connections there and i would probably [Music] request from the maricopa county the documentation for those functions that that rely on those connections and to determine whether or not those are certified or not we were not provided any of those that certification document but those are the those are the two items up there that i would ask maricopa county for some further clarification on as part of this analysis i was able to determine that there was no zero-day malware in the memory at boot up of the ems server next slide let's talk about internet history and connections okay as you will recall maricopa county commission two independent auditors to come in both of those auditors had a finding that there was no internet connection at the time that they conducted the audit okay when i initially did the analysis and i was only searching the um what is called the allocated space so the allocated space is what you as a user see when you open up internet explorer or when you open up file explorer when you see that directory structure that's allocated space when i looked at the internet history of the allocated space i had the same conclusion as the auditors did however i took this one step further i actually carved the unallocated space in the entire file system for internet artifacts okay and when i did that the history was significantly different than both the representation by the auditors and the representation by maricopa county that these systems had never connected or were exposed to the internet and we found internet activity and multiple visits on the ems server three of the ems client workstations one adjudication workstation and then the reweb 1610 and the regis 1202 now before i get into this i want to kind of walk through my methodology a little bit so it's very common for operating systems to have default urls compatibility caches things of that nature that may have an internet url as part of that part of that artifact so in order to to ensure that we didn't get any of that default data i only reported on internet artifacts that one the date occurred after the installation of the dominion software so after the 6th of august 2019 and that had multiple visits to the same site okay with dates after that time frame so that would eliminate any of the default url artifacts that may have been on the system next slide please okay so from the ems server you'll see that there's actually three visits to the same site on the same day okay that clearly is not a private url or a private ip address and so what i can tell you is that the ems server at least on that date was connected to the internet okay now i'd also like to uh point out that relying on the uh on the unallocated space for these artifacts i don't have a complete history of all the internet connections because things get overwritten things get changed things of that nature but the importance of this is is that at some point in time specifically those last visited dates device was connected to the internet now if you look at that date there's also a correlation to the purging of the database it's the day before the audit okay on the same time exactly almost the same time exactly okay obviously this requires an explanation okay next slide please when we talk about the ems client here are the here are the connections now the the nine connections at the at the top there you'll see that as far back as uh february 2020 there were four connections made to the uh to that url to the microsoft url and then on the 22nd which coincidentally enough is during the time of the audits there were five connections to the microsoft url i included that that lower set of findings to illustrate that the importance of some of the items that were not produced to us so not all of the election connected devices were produced to us for analysis and that ip of a 192.168.100.11 is one of the private it's a private uh network that was the election network and you will see that it accessed a bunch of web pages off of that device indicating that it was a configurator or it was a file server or something of that nature i want to draw your attention to that very last line the m underscore network underscore wireless lan dot html that was accessed on the 19th or excuse me on the 30th of october 2000 2019. now we have not received any information about any wireless lan configurations but yet here you have someone accessing it from the ems client to to access what i can only surmise was a wireless lan configurator on that date and you can see that the ems admin 0 1 account was used for that once again who the actual human was behind that account i cannot tell you because of the shared passwords and the shared user accounts but i can tell you that not all the devices were produced to us that would have shed significant light on our findings next slide okay once again this is ems client 3 and you can see that the last date last there are six visits to microsoft.com and the last visit was on the third of february 2021. now keep in mind that those previous five visits were obviously before that time frame and we don't have a record by nature of this artifact when each of those visits occurred next slide okay now re-web 1601 is kind of an interesting case now once again we did not receive any network configuration diagram we did not receive any functional information as to the network uh you know it was one of those things that as an auditor in most cases i can go back to the person being audited or the entity being audited and say what is this okay how did this function how did this interact in this particular occasion in this particular situation there was extreme resistance and quite frankly in my opinion obstructionistic actions taken by the county to prevent this type of exchange now this system clearly was connected to the internet now whether or not that was by design or whether this is one of those isolated and protected systems that the county has indicated never touched the internet i cannot tell you but i can tell you that it had significant uh internet access and this is only a something that would fit on the screen right there's literally thousands of of connections to the internet by this system based on the naming convention i would assume that this is some form of web-based server that was used in in the election system because it was produced under the subpoena which that was one of the requirements for now the other thing that i will tell you is that this device was produced to us on an external four terabyte hard drive and originally it was represented to us as this was a forensics image of this device when we actually looked at it we found that all of those devices that were produced on the external four terabyte hard drives they were simply an operational system clone of that device and it was not preserved in a forensics manner what i can also not tell you is what steps were taken on the part of the county to ensure that the unused portion of that hard drive not occupied by this device i cannot tell you what steps were taken by them to ensure that those were wiped or zeroed out so that we would not co-mingle data okay so i do want to caveat these findings and the regis findings with those statements but clearly these devices had continual and repeated access to the to the internet next slide this is the regis 1201 and that by the way is the host name not the function of this device and once again you see repeated access that ip address is actually the public ip address for the maricopa.gov public public url next slide so i appreciate your your diligence and your patience on this as as we think about what i've talked about today it really boils down to accountability right and making sure that our election systems are secured i will tell you that they were not based on any measure that i as a i.t professional performing countless vulnerability assessments and incident responses that i have occurred had a had a client that engaged me had this state of a of a network it would have resulted in a failure on our audit so at this point i uh i would like to remind people that from a totality of what these findings are there simply is no accountability by anyone accessing these devices you had shared passwords you had shared user accounts you had remote access if someone could get access to this system they wouldn't need a zero day exploit the systems were so far out of date from a security compliance standpoint that it would have taken the average kitty hacker less than 10 minutes using metasploit to hack this system okay and i would like to remind everyone that's listening to this that when you have a network of computers like you have in these voting systems it only takes one person bringing in a little hockey puck with admin access to provide external remote access to that voting system and in the situations where you don't have accountability you have shared usernames and you have shared passwords you simply cannot guarantee the security and the accountability on those systems and i thank you very much for your time and i'm available for questions if you haven't thank you sir we appreciate that um we're not gonna we're not doing questions at this time since it's just a presentation we will have committee hearings that will give everybody the opportunity to ask questions in the future so thank you okay um we are going to go back to mr logan uh for those that are looking at your watch uh where the bulk of this is is done but there's still some more important things to do um mr logan is going to quickly because we are running overtime go through his recommendations of what improvements the senate might be able to help do through legislation um and then mr pullen is going to quickly give his report about the independent ballot count the senate did and then we have ken bennett standing by who was our senate liaison who will be giving you the observations that he noted when he was there working every day so can doug could you do me a favor and go through those recommendations quickly please is it working are you clicking okay just click a bunch of times still tell on the screen we'll talk through it because if i can't control it's going to take too long okay and pull your mic up closer please everybody's texting me saying they're having a hard time hearing you guys i have a lot of transitions on it so if you just click and show it all up at once you should be working maybe i'll look at that okay i'm just gonna melt oops so legislation should be considered that links voter role registration to changes in driver's license we saw a lot of indications that there was old potentially old information in the voter rolls and it's very important that our voter rolls remain clean if they don't remain clean it would facilitate you know it makes it easier for almost any type of way that someone might want to take advantage of the system if only the people who are registered to vote and you could show up to vote are in the voter rolls that makes it more difficult and already when you go into the dmv you can you can register the vote but specifically if you change your license to another state if you change your address which are things that people are usually pretty diligent about taking care of that should also update your voter registration details so that when you're out of state and you're in another state it's not possible for your old voter rolls to be used by you or anybody else specifically we recommend so the ncoa is a natural change of address but something put up at the u.s postal service i view a ballot basically like money i mean we should not be you know just mailing money to people who are not necessarily still at the address or have moved um so check in the ncoa before you mail out ballots will help make sure that the currency of the ballots are only ever received by individuals who are legitimate voters who are still living at that location so 90 days before the election in addition they should check it right before mail and ballot so we are not ever effectively mailing currency out to an address where someone has moved to another state i understand that the the epm and guidance from the secretary of state suggests that the eric and social security's masters deathless and others should be checked regularly against voter rolls but i believe it needs a little bit more oomph to it and that there should be a legally required frequency for counties to do so to make sure that the regular being maintained rather than just a guidance that it should be done at some stage let's talk a little bit about election software both from the report i have out of antrum and specifically with what some of the inconsistencies and oddities that we're finding about the voter roll system both here um and quite frankly across the country i highly recommend that we pass legislation where that requires that these applications that are extremely important are built up to a higher standard and specifically are making sure that they're ensuring the confidentiality and integrity of the systems the open web application security project is is known as as a leader in the application security space and they specifically have something called the application security verification standard or the asvs it has levels one through levels three level three is for a sensitive system and i would highly recommend that would be a requirement for anything associated with voter rolls or voter systems that they would build up to that standard um doing so make sure that the application itself has a necessary standards built into it in order to to make sure things aren't altered and that there are appropriate logs if so to in order to take care of that specifically with that it's always a good idea to to run the asvs assessments on a regular basis usually every three or four years and specifically that the vendors should be required to attest that the asps standard was fully applied i further recommend that the the vendors you shouldn't be able to use the same vendor over and over again for any type of certification activity it's to it creates too much opportunity for if there was some some impropriety by a vendor that you could continue to pass so rotate inventor vendors at least every three years and putting that in the law would help ensure that not only are these being assessed but they're has set up to a higher standard and this goes beyond specifically what the eac is requiring because quite frankly based on everything that's being said the eac is not requiring anything close to the what is necessary in order to protect our election system and hopefully that will get better over time but in the meantime arizona can be a leader in this area okay voting machines specifically legislation should be considered that requires fallen of all the susa guidelines for election systems and equipment and any variances against those should be documented and there has to be risk managers that are signed off and should be public for by the appropriate for any derivations from those guidelines now those guidelines cover pretty much everything that mr cotton was covering today everything from setting up baselines on the systems everything from baselines and network processes making sure that user accounts are are handled properly and all those details are all covered by that so it's a very simple legislation that's follow basically says that guidance needs to be followed legislation should be considered which requires assignment invincibility usernames and passwords uh mr cotton's talked about that a lot today legislation to be considered that requires real-time network monitoring of all collection equipment even on the airgap networks um so there is some indication of what occurred there it may seem odd to to to have that on an air-gapped network but as mr cotton was mentioning there's very small devices that you can take in and if you can plug them into a network port you can effectively give the entire network internet access i mean they're they're physically very small they can be hidden easily and if you've got real-time network monitoring on their systems especially where the ems is you can prevent you can identify that and at least have a log of it and potentially prevent it before anything happens and legislation should be considered that would prohibit internet-capable election management system servers or equipment for being utilized there's a number of devices that when you look at the serial numbers from the pro vnv and sli audits they show that they had wi-fi cards and stuff put into them anytime the capability is within a device there's the potential for it to turn on for someone to turn it on and activate it or use it to connect to some other um other device like so it's it's highly recommended if the equipment's not even there then you can't have a failure to configure create any issues so legislation that far not just preventing internet access but preventing any type of capability in the device would help ensure the integrity especially as we keep being told that things are not connected to the internet from the voting machine standpoint county employees should have all the administrative access on all election equipment sufficient to independently validate all configuration the fact that maricopa county said they did not have the hardware tokens necessary in order to see if their election equipment was connected to the internet or not is extremely extremely alarming the accountability the county needs to be able to hold vendors accountable we may use subcontractors various different actions but the responsibility always falls on the county and therefore they need to always have all access that should not be something they can relegate to somebody else in addition election voting machines should have a paper backup of all ballots which can be used to confirm the votes where cash is intended and these machines must be regularly maintained to vendors maintenance schedule one of the things that that was found in some of our examination of the paper ballots is a lot of things were miscalibrated or otherwise not following general manufacture guidance if we want to to make sure that we get the intended results out of things always um we should make sure that whatever goes through logic and accuracy testing whatever it is that's a standard equipment used is the same thing we're using on election day and if you're testing a system with something different than what what has happened in the real world it's not a very good test and legislation should be considered that uh requires that paper stocks use size and election day conform to again to manufacture specifications and that it's been tested properly now we think that you should have legislation that should consider creating an audit department that should regularly conduct audits on a rotating basis across all the counties in arizona to the best of my knowledge nobody is currently doing this but based on the the audit we performed there was a lot of processes and procedures that were not conducive to effective audits the way that stuff gets better is by regularly checking it and regularly validating it and now the whole world is looking at our elections i think it's very important that we take advantage of that and make sure that that what is done in our election departments is brought up to the same standards that financial industry uses and other critical systems it should not be an area that's lagging when our elections our voting equipment helps choose the most powerful individual in the world um there's a lot of adversaries that would like to take advantage of that and we need to treat it accordingly and make sure that that it is being audited to so that those standards are maintained legislation to be considered that requires batches of ballots be clearly labeled separated from each other in a manner that cannot easily mix together and easily connected to the batches run through the tabulation equipment there was a lot of hoops we had to get through jump through to even connect a box of ballasts to what was run through the software in order to match those two up and that is something that should be simple to do again because it facilitates audits and that those audits facilitate accountability it well a full audit like what we did this time it cannot always be done the better the record-keeping is the easier it is to do partial audits to confirm things and that's something the audit department can do on a regular basis in addition to honest uh you know sometimes doing full audits but it's just it's not cost effective to do that every single year legislations be considered to penalize purposely inhibit a legislative investigation or an officially sanctioned audit of an election i think why that's in there is a little bit obvious audits are really effective when you have the cooperation of the management who controls things and it's very very difficult to to to manage them that's why financial services if you're your typical financial audit if you don't if you don't comply with the audit you can literally be put in jail at times okay ballots legislation to be considered that will make ballot images and cast vote records artifacts from an election that is published within a few days the results being certified for increased transparency and accountability of the election process these are things that we think is important for for the arizonans american public to be able to see and validate and see with their own eyes currently in arizona it's we had a judge that stated that we could not make the ballot images publicly available there should be nothing that links once a ballot comes out of its envelope there should be nothing that links it back to a person and there should be absolutely no reason why it shouldn't be able to be public legislation's further be considered which require all battles to be cast on paper with security features such as watermarks or similar technology with a detailed account of what papers were used with our paper analysis we wanted to be able to say that this is legitimate paper and valid and real and we wanted to be able to say this is not legitimate paper but with with this many wide number of papers that were used i think we're estimated over over 10 different copies different types of paper it's very difficult to make that but if there's official paper that's that's kept track of it'd be much easier for an audit when it's conducted be able to say without a shadow of a doubt whether it is in fact printed on legitimate paper mail and voting should incorporate an objective standard verification for early for voter identification similar to the id requirements for in-person voting it seems likely that that mail-in voting um will continue to increase um from a security standpoint i out i advocate no more mail-in ballots but that's not probably realistic um so that being the case if it's going to continue we need to have good identification requirements and that is then thank you for your time madam president thank you mr logan i appreciate that very much we'll take all those into consideration i'm sure we'll have more um let's go now to um senator senator sorry randy uh mr poland randy paulin was um our co-legislative um liaison there as they saw the work building up quite so much mr pullen could you give us just a tad of your background and um as to why i selected you to please do this independent count thank you president fan as well as chairman peterson thank you for your commitment to this forensic audit and your resilience over the last six months as we went through it thank you okay just real quickly about me personally a graduate from arizona state university undergraduate in math and chemistry in an mba in 1981 i sat for and passed the cpa exam in 1980 became a cpa and i've been a cpa since then employment wise i started out working on my mba as a i was working at a engineering company writing software for them and then i joined deloitte haskins and cells and began working in their audit department and helped them develop and test the first statistical sampling software system for doing audits i became a partner panel curve forester and then again i went back and became a partner at deloitte and touch where i focused on financial auditing uh specifically for bank savings and loans in hospitality industry i actually did get involved in forensic audits so i understood how they were functioning and what you had to do in order to complete them i started my own company back in the 90s and i still do consulting and accounting services and i also started an i.t company in 2001 wage watch which still exists this day and we developed uh software uh that's still considered some of the best in the industry that we're in background wise we the senate decided they wanted to do an independent count of the ballots in order to confirm the count by the county as well as to count by cyber ninjas on the forensic audit and so that was kicked off on june 28th they selected me to run that machine count again it goes back to my experience and knowledge i immediately got brian blim who was the attorney who had worked on the floor of the coliseum and was the legal counsel and had dealt with any forensic issues that came up on the floor during the hand count and by that by the end of june he was no longer a contractor with the cyber ninjas he was independent and he agreed to assist in the machine count we also enlisted a lot of former volunteers who had worked on the floor doing tally counts as well as working in the corrals taking care of the ballot boxes and so we put together a pretty good team we went out and found the equipment for doing a machine count and looked at several different varieties and finally decided on i need to stop pushing this uh and found the equipment equipment we found interestingly enough that was highly considered around the country as very good equipment and it's all this is in the report and we selected two bantam one counting machines that were made by u.s paper counters and we also got two paper joggers they're called and what they do is they help align all the paper before you run it through and do the count on the paper and again these ballots are very heavy paper so when the technician came in and set up the machines and we were testing the paper he had to make adjustments to the machines that could handle the quality of the paper that we were running through the system so once we had that figured out then we had the technician uh train our volunteers who this worked for us and then the ballot counting started on 7 14 and we completed it in 12 days i will tell you that the volunteers worked incredibly hard on this as well as the observers we had overseeing what they were doing and we started working at eight o'clock in the morning and we went until midnight on those days and we had two teams that were working that very very solid work so well here's what we found the maricopa county official canvas you can see it's has it's been reported earlier and we did the machine count and it was very close we were 121 less ballots than they had counted which again you can see the forensic election audit that count on the ballots was a little bit less but all of its was within a thousand of what maricopa county did not surprising and one of the things we did learn on the machines when we were testing them and setting them up to operate is when they did was a miscount and we did do recounts based on that but when there was a miscount essentially it was an undercount okay so being a little bit less than the official canvas count was not surprising now here are some of the things we found and this kind of confirms some of the things that were talked about earlier in these reports we did find missing batches in boxes where you would open a box and it was supposed to have seven batches and there would only be six batches in the box okay and then sometimes we found there was eight batches in the box but only supposed to be seven so we found these kinds of problems am i causing this oh okay uh so we were finding miss uh missing batches and boxes we were finding uh uh batches and boxes not listed on the boxes so we were kind of like a ghost batch and then when you open some of the boxes the batch counts that went with every batch which was typically supposed to be about 200 in the batch they weren't with the batch they were on top of the batch or they're on the side and so we'd have to go through and figure out uh manually which which batch sheet would with which which batch which again goes back to uh if things were done more properly this would be much easier it could even been done faster uh as that so that's pretty much what we came up with uh with this and so it again independently confirmed the numbers that the county and cyber ninjas found in the ballot count thank you thank you well thank you mr poland we appreciate that very much we had those who didn't know we had the ballots and before we gave them back we said well what do we do if maricopa county and cyber ninjas counts don't match how do we know which side to go with and so we decided let's get a couple of machines of our own and do an independent just in case there was a difference we could have an idea of which side to go with thank you all right mr bennett are you still on zoom with us i am adam president thank you thank you for waiting so long sir i apologize for the delay um mr bennett would you give us just a tad background about yourself and tell us what your observations were as the senate liaison and they i've provided it i have provided a powerpoint to your staff and so i would ask that they um prepare to bring that up but in the meantime um i served as the 21st secretary of state from 2009 to 2015 that's the chief elections official of the state of arizona i also served as the president of the senate for four years like yourself and another four years as a state senator so i kind of bridge that those two domains maybe i've also i also have an accounting degree from arizona state university and have worked in numerous businesses usually as the ceo or cfo chief financial officer of those companies um and i see the screen so i'll jump in here i know our time has gone long today but you asked me to observe throughout my days there at the audit areas where compliance with our election laws and procedures was accomplished and maybe where they weren't complied with the as you know elections in arizona are governed by election laws which are adopted by the legislature and the governor and then a secretary of state's procedures manual which is promulgated every other year by the secretary of state's office but has to have the consent of the attorney general and the governor as well and between those two documents there's over 1300 pages of laws and procedures i'd like to just briefly say that no election can be conducted perfectly because that is administered by imperfect human beings but that doesn't mean we don't try because it is through our elections that that we the people give our consent of the governed as is identified in the declaration of independence and every citizen deserves to know um that they are being treated equally under the law as required by the the constitution so every legal vote has to be accounted accurately and not canceled out by unlawful votes this report is intended to identify where maricopa county failed or may have failed to comply with some of these statutes but having said that i believe that the majority of our election officials in arizona are honorable well-intentioned people so i intend this report in the spirit of constructive improvement but also maintaining appropriate accountability let's go to the first slide um as we've heard in previous testimony and the slides aren't changing on my screen but i hope they are at your end first slide or observation is related to the missing signatures on ballot envelope uh affidavits and there's some statutes there that i'm not seeing a change on your slides but i'll assume that the slides are changing at your end you'll see the statutes 547 16 547 and 48 that early ballots have to be accompanied by an affidavit uh and by seven o'clock on election day um there is a cure period in section 550 which is not noted there but ars 16 552 is very clear that that the election board is to check the voters affidavit and quote if it's found to be sufficient the vote shall be count allowed if the affidavit is insufficient the vote shall not be allowed and equally prescriptive in the election procedures manual if the early ballot affidavit is not signed the county recorder shall not count the ballot as you heard in previous testimony the scope of this audit did not involve comparing signatures with the voter registration files but you'd you had people dr shiva look for and identify a number of missing signatures on ballot envelope affidavits which to the extent that ballots from those envelopes were tallied would violate the above statutes and procedures next slide i hope you're seeing is because i'm not seeing it original and duplicate balance without matching serial numbers uh ars 1621-a is very specific that all duplicate ballots created pursuant to this subsection shall be clearly labeled duplicate bear a serial number that's recorded on the damaged or defective ballot and the epm says something very similar and gives one of the reasons which is to tie the ballots together the other reason would be to make sure that the votes recorded on the duplicate uh correctly reflect the votes on the original or damaged ballot there were approximately 2500 duplicated ballots where there were no discernible serial numbers recorded on either the original or the duplicate ballot obviously this does not comply with those statutes and procedures the next slide i hope is missing chain of custody ars 1621 says that the accountant will maintain a chain of custody for all of election equipment and ballots during early voting which is kind of the beginning of an election through the completion of provisional voting tabulation or kind of the end of processing balance the senate requested this chain of custody the county provided a very detailed chain of custody of the ballots that we did receive but we never received a chain of custody all the way back to the election period as the statute requires next item uh is insufficient ballot paper thickness this is not a major item but ars 16502a says that ballots shall be printed on a paper of sufficient thickness to prevent the printing they're on from being discernible from the back and there can be debate as to whether this is just for the pruning on the ballots or when the ballots are marked with their votes but [Music] there were as mr logan noted multiple uh the audit found multiple thicknesses of paper stock used in the printing of ballots uh some of which um would allow kind of that bleed through effect which would not be in compliance with the above statute our next one is common usernames and passwords as mr cotton noted the election procedures manual specifically says that applications within the ems system should use separate usernames and secure passwords for each user or station and as he noted he found common use for names and passwords being used which is inconsistent with this guidance from the election procedures manual another the next one would be missing serial numbers on electronically adjudicated ballots an addendum to the 2019 election procedures manual specifies that tabulation machines may be programmed to out-stack or to print identification numbers on the ballots with write-in votes that are electronically tallied this process is often known as adjudication there was um well maricopa's system dominion does not use an out-stacked method um but it does not appear that these identification numbers were printed on the electronically adjudicated ballots as required by this part of the procedures manual and the last slide would be well i guess there's a one after this but the last item is possible ineligible voters several articles which include multiple statutes within irs 16 as well as many aspects of the election procedures manual identify arizona's requirements for an individual to be considered an eligible voter and therefore allowed to cast a legal vote the audit identified numerous questions regarding possible ineligible voters however these determinations were as you heard from mr logan made from comparisons between the county's final voted data and private data sources not the official voter registration data so further investigation with the cooperation of the county hopefully is necessary to determine whether ineligible voters were allowed to vote in the 2020 election and mr logan went into some of the possible categories of you know people had moved or deceased or other categories that i think you saw in earlier testimony and the last slide is simply a thank you for me for allowing me to work on this project and i will conclude i guess by saying this that i have already started to hear from people saying that um well if the audit failed because it didn't prove that the election was overturned or that there was a different result well as you noted when you began this process madam president and when you began this this uh hearing today there was no predetermined at least in my mind and i know in yours and warren's and everyone who worked on this audit there was no predetermined outcome that if we didn't find this or didn't find that we have failed because it's exactly the opposite if we identify strengths and weaknesses in our election procedures and statutes and if we confirm that in this case an election was conducted uh where the hand count of the ballots matches the electronic tally of the election precision the election system used by the county i don't consider that a failure at all in fact maybe mr logan and his company have identified the the most accurate hand counting process that has maybe ever been used in the country and that's directly in opposition to many uh within our state and across the country who said that their procedures were terrible or this was that and nothing could be further from the truth um i you know there were a lot of good honest people that gave a lot of time as you noted in this audit i offer these uh as you requested as areas where we could make constructive improvement and i'll leave it at that thank you thank you mr bennett and we're sorry you went last and you had to hold on the longest but uh thank you so much for um doing this for some of you may not know mr bennett did this uh on his own time on his own nickel and he lives in prescott but literally drove down almost every single day to be at the audit with the rest of the team to make sure that everything was done accordingly and chain of custody was followed and everything was always documented so thank you mr bennett that was quite a yeoman's job on your part all right closing comments senator peterson i'll let you start um thank you madam president and i'll be fairly brief here i just first of all want to thank you um for what you've done with this you have faced incredible opposition and hostility and you have handled that with grace and dignity so i want to thank you and recognize you for that again the goal here is election integrity and making sure our citizens have faith in the process so as we wrap this up what are our next steps what's where do we go from here and i think there's legislation and i think there's law enforcement that needs to be involved so i'm going to name off you know eight bullet points here that i think need to be handled by my colleagues and and by our attorney general first of all and what i have found perhaps the most unsettling through this whole process is the obstruction that we have seen from the county the failure to comply with the auditor a brazen willingness to violate a legal subpoena our attorney general said that was against the law it's it's truly alarming and furthermore to their willingness to expend significant resources human capital you name it to block and to stop this audit as i recall it was like 18 000 or something like that for one of their audits that they spent money on how much money have they spent trying to stop our audit has to be in the hundreds of thousands of dollars but that would be an interesting number to see number two the numbers don't reconcile as you've seen through this is the theme throughout all the reports all of us as citizens we should be able to pull up these election results and we should be able to every direction reconcile the numbers if i subtract you know early ballots or if i subject track if i add the rejected if i you know no matter which way you should be able to come up with a reconciliation so that everything balances that needs to happen it appears that they broke the law with duplicate ballots and that's that's a huge deal that needs to be resolved we need to get to the bottom of whether that law was broken how to prevent it in the future hold people accountable that did it this time churning of locks the churning of locks we need to find out why that happened who did it what was the motive what what and what was there we need to we need to get to the bottom of the logs that were there chain of custody issues number five chain of custody issues number six a failure to preserve data files and number seven cyber security weaknesses that were so that were shown by mr cotton so evident here those are not going to get any better we're seeing people being held hostage via cyber security every day and it's constantly becoming more sophisticated we have to definitely need to up our game there and number eight the envelopes with blank signatures we have a lot of questions there that need to be answered so with that madam president i look forward to working with my colleagues and with the attorney general in any way to resolve these issues and to improve our elections and to increase election integrity in the state of arizona thank you senator peterson senator peterson is our jude chairman this uh started in the jude committee back in december and uh only appropriate to end up there and my closing remarks first and foremost let me tell everybody that all of these reports are up and posted on our website for you to access all of them that website is a z senate republicans dot com azsenate republicans dot com you can all go there and see all those reports second of all i have already transmitted a letter to our attorney general's office with all of those reports that not only includes everything that we have noted here but also everything that senator peterson has concerns about the attorney general has that we are asking him to open up a formal investigation so that he can pursue and get seek additional information additional facts perhaps get some of these missing things that we were never able to get verify this all this information and take the appropriate actions of anything that is necessary to do i have very every confidence that he will be doing that i also want to say thank you to my senate republican colleagues when we started this it was we had a caucus and every single one of our republican members said this is important our constituents have questions they want answered we didn't think it was going to be this long we didn't think it was going to be this expensive or this difficult but we did it we hung in there we did it and as you can see we do have some work to do here we have a lot of work to do because quite honestly if we don't follow our rules don't follow our elections this is how problems can happen we also know that 18 different states since sent representatives here because they have constituents asking the same questions and the very least what i think that we can all come out of this is that we need to do audits to some extent we need to do bigger audits on every election just to make sure that everybody's following the rules so with that i want to say thank you to mr logan mr poland mr cotton um mr bennett all of you present mr shiva dr shiva thank you for all your hard work i know this was very difficult on you and your family mr logan you've been here for many months and you've left your wife and 11 children um back at home and he has another one due in next month in a couple couple of weeks so we gave up a lot for this we appreciate it so to everyone thank you all very much thank you for being so polite up there we appreciate that and prayers and blessings to everybody and let's uh let's move our elections forward thank you we are adjourned for this we appreciate it so to

Channel: azcentral.com and The Arizona Republic

Views: 1,633,392

Rating: 4.7777843 out of 5

Keywords: arizona news, arizona republic, azcentral, arizona audit, senate, trump, biden, election results, 2020, cyber ninjas

Id: sAAu6O33rNE

Length: 181min 35sec (10895 seconds)

Published: Fri Sep 24 2021