Security of Data on Disk - Computerphile

while we've had a pretty good discipline around hard disks and scrubbing them and this is a well understood problem in the in the industry um there's still a little bit of a you know a problem around the way in which we use solid state memories and you know the best security approaches towards those so you can completely delete all your contents and eventually the microcontroller will erase all the blocks but it's not happening under the operator control it's not happening because the user says that it's happening at the whim of the microcontroller today i thought we'd talk about information in permanent storage try to understand some of the differences interesting some of the security issues that are different between them let's look at magnetic disks so a magnetic disk it is literally like a a record if anyone can remember what records are the final record final record just remember those and it has a little head little magnetic reading head that moves in and out of the magnetic disk and there are concentric circles here that hold the data the physical movement of these drive heads adds latency and so you often find associated with magnetic drives comments like the seek time is 10 milliseconds seek times have been 10 milliseconds for a long time going back to the 60s and 70s magnetic seat times were 10 milliseconds and 10 milliseconds in those days was blowing fast compared to the processing but these days of course it's a vast amount of time and 10 milliseconds your processor executed billions of instructions sometimes though there may be a flaw on the magnetic media it may be manufactured that way or one of the classics being that if you shake a magnetic drive when it's working these little heads that sit very close to the the surface actually will ding the surface and knock some of the oxide off and you get a fault and that means that that whatever block that is in has now failed some systems the operating system is told about this and handles it by never using that block and in others the drive hides the fact by magically if someone accesses this sector it says oh no you don't want that block i know i've actually remapped that to another sector somewhere else and that bad block in direction or redirection is very common but of course if we if all the blocks have previously been contiguous we've now got a chunk here and then one block somewhere else so you notice this in the performance so one of the arguments as to why to tell the operating system is it might want to move the whole file um so that it can get a nice contiguous set of good blocks for performance reasons so now we move on to flash memory or solid-state drives names are used interchangeably here we're using semiconductor memory and the semiconductor memory while it has one property that's very similar to disks which is its best addressed as blocks rather than as individual bytes um we can access any block in these memories at the same speed there's no difference between what the last block we read and the next block we read it can be anywhere but these memories have one particular property that is slightly irritating in order to write a block we must first erase it and there's an erase stage whereas with a magnetic drive you can write a new block over over one that was already there in one go it takes the same amount of time as it does take a read in the case of flash memories we must erase a block before we can write it so we have to come up with a slightly different means to actually assign these blocks there's one other thing that magnetic media as long as it gets refreshed every so often and actually is able to remember for a long time flash memory wears out it actually does wear out over time magnetic drives will wear out over time because being mechanical they will just fail but in flash memory inside the flash memory is a little microcontroller what it does is a thing called wear leveling it sets its mission to make sure that each of the blocks is used uh with the same frequency and it does this by dynamically remapping the blocks so i may ask for block 72 and it goes yeah whatever i will remap that internally it maintains a table internally that remaps all these blocks so let's let's just look at a simple example let's have our nice simple file it's composed of contiguous blocks and then i update one of the blocks so i want to update block number five and the microcontroller down here the wear leveler will say in order to reuse this i would need to erase it first and that takes a long time so rather it looks at someone else on the flash memory and it says i'll make that number five it now marks that block and says when you get around to it erase that block so now the process of writing a block for a file is decoupled from the erasing of the old contents that's great so it provides wear leveling it can make sure that all of the blocks are used equally hence the lifetime of this device will be the lifetime of you know when all the cells start to fail rather than there being one particular cell that happened to get used a lot because it was a block that got rewritten a lot there are a number of consequences of this on a magnetic drive if you actually look at how this little magnetic read head works what it reads is a constantly varying magnetic field strength and we translate that into a series of ones and zeros by thresholding the signal we say well if it's above this strength it's one if it's below that it's a zero this very complex signal is not just dependent on the bit pattern that was last written to the drive but it actually depends on what was there previously if we had previously a zero and then we write a 0 we may have a field strength that's down here if it was 1 and then we write a 0 it may be here so still below our threshold so we still read it as a 0. if it was a 0 and then a one it might be here and if it was a one and a one might be here now if we built a different circuit instead of just discriminating zero and one and we said we want to find out what was there previously we could set multiple threshold levels and we could read what was there previously before we'd overwritten it with the new blog and this has actually been used by people to recover information from hard drives and it's one of the reasons why whenever you dispose of a hard drive you are advised to run one of the many algorithms that rewrite the blocks multiple times over and over again with very defined patterns in order to essentially leave random noise behind rather than trace of your actual signal and the whole industry in terms of people securely erasing magnetic drives and then of course if you're paranoid you then physically destroy it as well but first of all you've got to destroy the contents now who uses these sorts of techniques well you'd have to say some of your national security agencies might use these sorts of techniques to find things if there are big secrets but people have also used this just for commercial criminal activity as well to find out information but we have a solution to this which is algorithms that where we rewrite the blocks over and over again it erases all trace of the data but oh dear our flash drives let's remember what they do here's the file i want to erase so i run this algorithm and it says right right all over these disk blocks but the microcontroller is going to go oh no that's going to take too long i'll write a whole new set of disc blocks and it puts these on the please erase these sometime in the future list as you run the algorithm multiple times it will continue to rewrite this file in a new area of memory leaving behind the actual data was there previously now the little microcontroller down here is eventually going to get around to cleaning these cells out and erasing them but until it does that that data has not gone away you quite happily delete all your files you run this algorithm to rewrite all the file contents and erase them and lo and behold all the contents are still there and this has been used quite a number of times the classic example of camera thieves who had erased the flash memory but hadn't quite sufficiently erased it to the point at which the police were able to recover the files over there previously but it does mean that that little usb drive or ssd drive has to be continued to be plugged in so if you delete all your files and unplug it then most of the contents has not actually been erased and is still available for someone to rediscover you buy yourself a four gigabyte remember flash memory you've actually probably got a six gigabyte flash memory because the most amazing thing you then discover which through me for a while because i'd rather forgotten it works like this

Channel: Computerphile

Views: 209,656

Keywords: computers, computerphile, data security, disk drive, semiconductor memory, ssd, Disk Storage

Id: 4SSSMi4X_mA

Length: 8min 45sec (525 seconds)

Published: Tue Sep 17 2013

Reddit Comments

[u/R4D10Active]

SSD's or (Automatic Evidence Destroyers) as I call them in digital forensics are a pain because not all manufacturers do wear-leveling or garbage collection (TRIM) the same.

[u/Reygle]

I just spent about an hour watching videos on this channel- which I'd never heard about before. Definitely a great find for me, I'm going to enjoy and learn a lot there. Thanks!

[u/johnbentley]

So is there a solution to securely erasing SSDs (something that forces the microcontroller to blank all the blocks)?