My Storage Workflow for Video Production - 2021

so whether you're brand new in video production or you've been doing it for a while one of the things that you will constantly tackle and try to evaluate on a regular basis is the storage situation files from cameras are not getting any smaller they're only going to get bigger and bigger and trying to manage them all and keep track of where they all are and be able to somehow do it at a reasonable price is something that can be very tricky very confusing and for the last bunch of years as i've run a video production company i've changed things around quite a bit but at the core it still is pretty similar to what i've been doing for years and it works pretty well for us a few years ago i did a video all about my actual storage setup and how i manage all the files that we create here at scaling creative i figured i might as well update it and tell you what we're doing now and if it's working and if it's something i'd recommend so let me walk you through our entire storage workflow for everything we do as a video production company so my company scaling creative is basically a content creation company like so many video production companies but one of the things that we do very often is work with ongoing clients on a regular basis so we aren't usually just importing footage editing something and kind of getting rid of the footage and hoping that they hire us again we're working regularly with companies on a regular basis monthly weekly and we're always pulling footage from things from a while back sometimes we're pulling things from years back and so our situation may be a little different than yours if you are not someone that has recurring clients but for someone like us that has a lot of recurring clients it's important for us to be able to access that stuff as quick as possible but not have it be taken up by a lot of costs internally but also having the safety of it being externally on some sort of server so it's a little bit confusing to maintain but we've kind of come up with a solution that works pretty well for us though it's not perfect it's somewhat affordable and it works really well for most of the stuff that we do it really only consists of three things portable little ssds that we go from computer to computer when we're actually editing the second thing is a raid type of system that is not server based but it's just a large hard drive system that we can easily access and pull stuff from and transfer to the ssds that we're doing editing from and then the third thing is an online server based thing which we're currently using a business version of google drive which we simply archive all the footage that we shoot on there so let me break it down so each and every time that we film anything whether it's a small project or a big project our process is the same we try to keep it as much of a formula as possible so that things don't get missed footage doesn't get lost and cards don't get formatted that weren't supposed to be formatted yet every single time we come back from a shoot we bring it to this little location here this is kind of like our community computer no one works on this computer on a regular basis but this computer is basically connected to all the raid systems and all the larger in-house hard drives that we use for all the footage the reason we use this as kind of a community computer is because number one it's connected to all the large hard drives so no one has to constantly plug into big hard drives to be able to access footage but more importantly since no one's ever here we have like a big sd reader here that has a lot of sd simultaneously that you can import footage on so if we get back from the big shoot that had three or four cameras plus audio sources and all that you can plug them all in at the same time let them all import go back to your desk start getting to work on other stuff while everything's importing to a library depending on where it's actually at this makes it much easier instead of doing one card at a time on a personal computer and having to wait for it to import this is where we can just come plug in ssds or just have the hard drives already plugged into it import everything and we're good to go as soon as all the footage is actually imported and safely copied from all the sd cards to a hard drive of some sort we then reveal in finder so that we can see all the original source files and we upload every single file that we create in a very organized structure onto google drive we currently have the google drive business suite and as a business suite customer it basically means that you get unlimited storage on the google drive that is super useful to us because we currently are up to 70 terabytes of footage that is actually on the google drive now we do not use google drive to be able to actually edit from nor do we need it to be a certain speed to even have that capability it's simply for archival sake all the stuff that is stored on google drive is basically all the same stuff that for the most part we have locally with different hard drives but in the crazy event that all of our hard drives failed or there was a fire or whatever happened and we lost everything we actually have backups of it all on google drive now sure if something happened crazy we wouldn't necessarily have all the project files that we're working on but we're hoping that doesn't happen but worst case scenario we're not having to actually re-shoot anything we would just have to maybe do some re-editing on a couple of projects we like google drive or that type of system much better than just something like back blaze or one of those crash plan things that is basically backing up a hard drive itself because it's very easy to just go to google drive watch files in there in real time share a specific file with someone or just download a certain file really quickly and be able to access that file again if you use one of those kind of automated backup systems for your actual hard drives it's kind of more of a cumbersome process and it's not as easy to just pull directly a file you're looking for or a folder you're looking for it's definitely a little bit more bulky and annoying we really like google drive because you can just access it so easy on any device and get files so every single file that we've ever shot for scaling creative is backed up on the google drive in an organized folder structure where you have the actual client you have their exports you have their original media you have any sort of assets that we need for them like logo files and all that is all stored very organized on a google drive every single client is set up exactly the same so any of the editors that is working together on any project we instantly know where we can go to find clips and be able to get them very quickly if they're not on a hard drive that we're currently using then we get to a local aspect which are the hard drives in-house so there are a couple main hard drives that we use where everything is basically pulled from we have a drobo over here the drobo is about 36 terabytes i've actually had the drobo for a while that is now dedicated solely to my wedding production company which is 4lc studios where we house all of our weddings that are basically within the last year or so so any wedding that's happened within the last year still stays locally in case we need to re-edit something or pull footage for something like a doc film or whatever and then after a year those come off the drobo entirely and they go on google drive and they remain there and they're not local at all so in the event that we had to pull something from an old wedding we would just download the entire thing again from google drive i will say i've had some issues with the drobo in the past so it's not really a system that i actually can recommend anymore it's had a couple of hard drive failures it has times where it just doesn't want to connect to the computer at all so it can somewhat be frustrating not a huge fan of it anymore but because it still works and it still operates like normal most of the time it's what we use now for 4lc studios but like i said we're always backing everything up including weddings on google drive as well so everything is stored there in the event that we ever have a hard drive failure we can always get things back the drive that we are now using for scaling creative is this owc mercury elite pro quad usb c drive this drive is actually a 24 terabyte drive but it's about 18 terabytes usable because when you are using a raid system it's not full capacity available because everything's backing up like onto other drives and if you don't understand raid you can research it more but in a nutshell this is about 18 terabytes of usable space now 18 terabytes of usable space is for the most part usually pretty filled up on these hard drives because the fact that like i said we do a lot of ongoing work where we need to pull stuff regularly so we're not always deleting everything off of here but the only thing that really stays on these drives are clients that we really are currently working with or have worked with very recently if we work with the client for a bunch of months and then we stopped working with them and we haven't worked with them for a bunch of months after that we will delete everything off the hard drives we still have access to it like a set in google drive that's kind of one of the perks that we talked about with our clients is that as a recurring ongoing client we always have access to continue to pull stuff from the previous things that we've filmed so it makes it easier and quicker for us to edit those things rather than having to re-pull them off of google drive to kind of start back over with that this hard drive so far has actually been awesome to us it's a usb c drive and it actually has pretty decent speeds so we don't edit off of this drive on a regular basis but in the event that we had to edit something quick that was on there and we didn't want to have to transfer everything to an ssd as i'll talk about in a second it is plenty fast enough just to be able to work on it's much faster than the drobo was and the actual transfer speeds of footage from one drive to the next to the ssds is much much faster and it works really well i've been really happy with these drives so far they've been super reliable i've had this one for over a year and a half and i've had no issues and i've also bought this same one for a previous client had them use it and they've had no issues so these are working really really well for us and i'm very happy with them and then these two drives you see up here are just basically drives i've had for a long time but they are g drives they are four terabytes a piece they are just usb 3. they are 7 200 rpm drives so they're nothing crazy fast but all these are actually for are direct backups which are about weekly backups of the actual ssds which i will explain more about on a regular basis so these are kind of just backing up our working drives on a regular basis where these are more archival and things that we're not editing on a regular basis but we want to still have access to and then the ssds are what we actually edit off of this whole hard drive system is only running to a very basic mac mini so it only has eight gigs of ram it's only a six core i5 processor so it's only about a six 600 machine but like i said because we're not really worried about the processing power for this and really this is just a unit that all the hard drives can plug into and be accessible this computer does not have to be anything fancy it's plenty fast enough for the everyday edit to just do something but obviously doesn't compare to the computers that we use on a regular basis for editing and this just stays here all the time it's always on it's always accessible it's hardwired to the internet so it's plenty fast enough for everything that we do has wireless mouse and keyboard directly connected to it and then we all have our own mouse is that we can just come over here and press a button on our mouses and connect our regular mouses to this computer if we want to work but for the most part we're not usually working here there's usually no one here and one more very important note about this specific unit over here this is running into a battery backup system i highly recommend that your computer that is actually running your hard drives or your raid systems or things that are always running should absolutely be on a battery backup system if you don't know what a battery backup system is basically everything in this entire desk is being powered into this and then this runs into the wall in the event that you have a power outage this continues to run and none of the hard drives will actually have to be turned off that's super important because if all your hard drives are working and running or doing tasks of any sort and you have a power outage and it completely shuts down everything that is not good for hard drives at all you really are supposed to eject hard drives correctly and when you suddenly have them lose power and go off it can permanently damage drives so i definitely recommend you have a battery backup for any of your main hard drives that are plugged in if you're using things like ssds or a laptop you're okay because obviously if power runs out of those or power goes out the computer itself still has a battery and so it's still going to operate like normal but these kind of drives something like a mac mini something like these hard drives that are not bus powered meaning they have to have a power supply i highly recommend that you have a battery backup once everything is imported and once we are ready to actually get to work editing that's where these come in the samsung t5 ssds now they make even faster versions of these now but these are still the usbc versions that we've had maybe for about a year or so and we have five of them they're absolutely identical and each one of them is two terabytes a piece so i think these run for about 350 a piece 400 i don't know something like that but this is what we use for all of our editing we have five identical ones and each one of them is specifically labeled sc ssd and a number we use a trello board to be able to simply keep track of what projects are on what ssd and that way if you were assigned a project to edit you can go to that trello board see exactly what ssd it is on and then you grab that ssd and you can actually get to work normally we're doing our best to make sure that whoever is working on a certain project on a certain ssd has all the stuff in that ssd that they would need to work on so that you're not having a situation midday where a certain editor needs to grab footage from a different ssd that they don't have if we're trying to assign someone multiple projects during a day we make sure that all those projects are living on the current ssd so there are definitely multiple times a week that we are sometimes rotating things from one sc to the other or from the main hard drive to another ssd we're making it as quick as possible and as you know efficient as possible to be able to make sure that once someone plugs this ssd into their computer for the most part they can use that ssd for the majority of the day unless they absolutely need to change something around or if a task gets assigned to someone else then they can grab the ssd that corresponds to it since these are two terabytes apiece most of the ssds have at least two to three sometimes even four or five clients on a specific ssd now some of our clients that we've been working for for years that have a ton of footage might have one whole ssd just for them because they have up to two terabytes of footage and if we are always pulling from that footage then we want to obviously make sure that it's easily accessible we don't want to have just the bare minimum on this drive and half of someone need to go to that main community computer every single time that they're pulling footage so if we can keep everything on a certain ssd that one person can use and that's nice and easy in the event that we have a specific client that is so large they can't even fit on a two terabyte drive we usually will break up the actual library itself to contain different things maybe we have one that is nothing but b-roll for that client one that is more of like certain projects that are currently being worked on whatever but for the most part everyone fits on their own ssd and we can usually fit a couple of clients on each ssd now these kind of to me are on like a life cycle of about two to three years before they usually end up getting replaced not because they aren't fast enough anymore they don't really slow down we haven't had any issues at all with these ssds but naturally space gets cheaper and cheaper so where these right now are about 350 or 400 bucks for two terabytes in about a year or two it'll probably be 350 or 400 bucks for eight terabytes and naturally as files get bigger and bigger on computers and on the actual cameras the more space you can have that still does the same job is better so i usually will try to get newer drives every couple years if the space ends up being cheaper for the price now the important thing to note about the way that we're doing things is that if this is the main library that an editor is working off of then all of the edits and all the things that they're doing on a regular basis with all that footage really is only stored in one place at that given time so if an editor spends an entire week editing tons and tons of stuff for a specific client on this specific ssd this ssd needs to be backed up as well because if you don't back up this ssd doesn't matter if you still have all the original footage you can lose all of the edits that someone actually worked a lot of time on if something happens to this one ssd so remember i was talking about these drives specifically these four terabyte g drives that's where these come into play so as you see when i plug an ssd into this computer it shows up on the desktop here sc ssd number three now you see these two drives here ssd backup one and ssd backup two they're four terabytes a piece so each one of them can back up two full two terabyte ssds so ssd backup number one backs up ssd one and two ssd backup two backs up ssd three and four now what works really nice is we use a program called chronosync we've been using this for many years and what chronosync allows us to do is basically clone that ssd identically to another drive and synchronize all the changes and all the deletions between the two that's really helpful because it doesn't make us have to actually like copy and paste and replace everything that was in the drive and figure out what changed and we need to delete this or empty the trash here it basically can be set up to just synchronize all deletions between the two so when we back up ssd 3 to ssd backup number two for example it's copying every aspect of the folder structure directly that is mirrored on the actual drive so it's not like a backup copy where we have to dig into folders to try to find like where the backups are it's basically mirroring it and what that allows you to do is when i go into the actual ssd in chrono sync i can set up all that stuff in advance so you can see here how i can say synchronize deletions and it's going left to right so the root folder of ssd3 which is the entire folder in its exact form gets backed up directly to the folder that i have labeled ssd backup number two right on here so everything that happens in this drive including deletions gets removed from the backup as well when it syncs over so then all we have to do is plug in ssd number three to this computer open up chronosync and simply run this selected task right here synchronize it gets to work and it basically mirrors it identically now what's cool about chronosync is the fact that when it actually is syncing from left to right to that it's only syncing up anything that has changed since the last time it backed up so you're not having to every single time back up two terabytes worth of stuff if you basically only used one specific client project over the course of the week before you maybe back that up again it only will back up and change that one specific client or maybe you only touched one or two files inside of a different client it's only going to look for what has changed from the last time you did it so you're not having to wait hours on end to transfer two terabytes every single time you plug it in it might only do 150 gigs it might do 50 gigs it might do every bit of it depending on how much has changed but i really like the setup because it's pretty easy to just plug in the hard drives synchronize them get them and know that in the event that one of the ssds failed all that hard work that you put into all the edits you're only going to lose at max a couple of days depending on how often you actually back that up it's our goal to try to back it up at least once a week but if we do a ton of edits on a specific client sometimes i recommend that we do it even sooner because you always have a backup like i said we're never worried about losing the actual footage that we shot but the edits are really what takes the longest and we'd hoped to not have to redo all the edits we worked hard on because one specific drive failed now if you only have one computer and all those raid systems and all the ssds plugged in simultaneously you can actually use a program like chronosync to automatically back up at certain times so let's say you have one computer a desktop where the ssds are always plugged into you're running off of them and you have your raid system as well you could make it so that every day from 1am to 5am if those drives are present it will do that whole backup process entirely by itself so you're never having to worry about it i used to do that regularly when i only worked by myself because i always had the drives connected the way that we do it now the editor would obviously have to come and plug it into this and then run that manual sync since it's not always in there at a certain time but that is convenient you also can make it set up so that every single time you plug it in it runs the task right away as soon as you plug it in now we don't have it do that because we might not necessarily want to be backing it up and we might want to actually pull something that we messed up on and grab it from the ssd backup first before it syncs so you can set up however you want but that application chronosync super useful so i think that basically covers our entire setup three things google drive to archive everything big raid systems to archive all the stuff that we probably will use in the next three to six months and the ssds to use on a regular basis for all of our editing the workflow is pretty simple and easy to understand but i think it really works well for the type of work that we do where we have safety of backups but speed and reliability for the edits that we're doing on a regular basis and we have backups of everything so so google drive is somewhat backing up all the raid systems the other drives are backing up the ssds so you kind of have redundancies redundancies is the name of the game when it comes to anything in storage because no matter how reliable you think your computer is or a hard drive hard drives can fail they will fail inevitably and being able to have a backup of everything you do is so so important i cannot stress that enough so a lot of people have asked me in the past why don't i use any sort of server based hard drive system so that we're not having to transfer ssds back and forth and we basically can access everything at all times and there are kind of a couple of reasons behind that the first most important reason behind it is the fact that a local base drive that would really do that really really well that i've gotten priced out is very very expensive but beyond the cost it kind of becomes a little tricky if you are not always editing locally at this location so i got a couple of things priced out to do something like a jellyfish or some sort of server setup where we'd have access at all times to everything no need for any additional ssds and the system would work amazing the problem is we don't always work from the office there are times that i'm gone for maybe a week at a time and so i might not even have the editors work here at all they may work from home covet was a good example of that as well and if you don't have unbelievably fast internet at all the locations that becomes a little tricky it's also a little tricky with our situation because we currently use final cut pro 10 and the way that final cut pro works is it has libraries and everything lives inside those libraries and you can only open one library simultaneously so it wouldn't work very well if we had multiple editors trying to work off the same client because you can't open that library in multiple locations at the same time the way the final cut's set up they don't have a really good collaborative workflow so because that's still the editor that we use right now because we still like it the best for the work that we do the server thing while it could be really beneficial in certain aspects it just doesn't seem like it justifies yet the cost for what it would actually be for us once they can really maybe figure out their collab workflow better it might be consideration but as of now the setup that we currently have just makes more sense so thanks for watching as always i hope this video was somewhat helpful to you comment below if you have any questions i'd be happy to try to answer them in the best way possible and i always like to know what systems are you using are you using one similar to us are you thinking that there might be something that i'm completely missing that would make it even more efficient for me let me know i'm always looking to learn and grow and continue to improve in everything that i do especially efficiency so thanks as always for watching keep crushing it

Channel: Scott McKenna

Views: 14,229

Keywords: video editing storage workflow, best hard drives for video editing, best external hard drive for video editing, video editing storage, best ssd for video editing, hard drives for video editing, video editing storage setup, video editing workflow, video editing, raid 5, cloud storage, best way to organize video project files, organize video projects, external hard drive, scott mckenna hard drive, scott mckenna storage, my video production storage workflow, scott mckenna

Id: AXb2yZtF2wU

Length: 24min 1sec (1441 seconds)

Published: Thu Mar 25 2021