I'm a Mess, so I'm Making My Own File Organizer [TagStudio]

hey where you been at man we're all ready for game night yeah sorry I'm late I was feeding the dogs and they bumped into me and knock spaghetti sauce all themselves oh wait I have the perfect image for this where did I oh oh no no back in 2021 I started really looking through all my options for how to get this sorted out was there a magic folder organization method that would help me find things quicker could I take advant AG of some hidden file properties to add in my own extra data would I just need to find some other piece of software that would help me organize my photos and files did anything that could really do what I want even exist after all the options out there just really seemed not to fit me I decided to take things into my own hands and slowly started working on my own piece of photo and file management software over the past few years it's gone from a simple python script that connected words with photo file names on my hard drive to a full-blown file organization Powerhouse this journey hasn't been without its twists and turns though from fundamentally rethinking how I go about storing and retrieving files to rebuilding this program from scratch more than once after running into technical limitations to even questioning the entire project after a recent discovery today I'm going to take you through all that and show just exactly what I've been trying to build here this is the search for the perfect file organizer the development of my own program in the process and everything in between folders kind of suck at its core this project has been about sculpting a great system first and then using or building software to bring those ideas to life unless you were born in the late naughty aies your first idea when it comes to organizing files and photos on your computer is probably going to be folders they're the simple classic way to organize your stuff into distinct categories they work great for a lot of things but have some serious shortcomings oh hello there here this file in particular is a photo of this cat this is important it will come up later folders can basically do one of two things they can either contain files or contain other folders actually if you use symbolic links you can actually link to other files virtually no I'm not talking about Sim links in this video actually I'm not trying to scare off my new audience they're probably already about to click off once they realize it's not an obsidian video that's why I'm cracking jokes and waving pictures of cute cats on the screen uh yeah uh where was I folders are about as Bare Bones as you can get you can use them to create categories and subcategories but whatever you want to store in them has to fall under one of those categories if you're trying to store something that falls into more than one then you're pretty much out of luck you could start duplicating folders or even start duplicating your files in order to force things to work for you but having to resort to any sort of duplication like that is a sign that there's probably something wrong with the system you're using but what if we took the idea of categories and flipped it on its head what if instead of trying to place a photo or file into a series of siloed categories we brought the categories to the file this is where the power of tags come in don't try to place our picture of the cat into separate cats or creatures folders but instead the tags build a natural description of it it's tagged both a cat and a creature no need for crazy subfolders or wasteful duplication let's go ahead and tag this cute too oh look at the little guy tags not only make it easier to classify your files but they make it even quicker and more intuitive to find them later just search for the tags you want in the first place instead of sifting through different possible folders now we're getting somewhere but what if I told you the tags can have even more power than this some programs out there like obsidian for example support what are called nested tags this enables you to put some tags Under the Umbrella of others in a sort of parent child relationship when searching for the parent tag you'll also get results for any child tags under that but not the other way around this lets us cut down on the number of tags needed in order to fully describe a file in addition to reducing confusion between tags created with a single name but are meant for different situations now while Nesta tags are already in improvement over regular tags they could be even better nested tags have some limitations of their own one is that they can only have a single lineage of ancestor tags if a tag could fall under more than one parent tag you're basically out of luck again because could we fix this the same way we fix this folder problem yes we can and as it turns out I'm only at least the second most insane person alive to dive this deep into studying data organization systems like this without getting too lost in some hardcore data science by using something called an attribute Value method of creating tags where in this system tags are treated as both attributes and values we can basically tag our tags with other tags and as much as we want see this guy he's from One Piece the anime or well the manga technically well both see if I'm using a nest tag then I kind of have to choose is one piece of manga or an anime when I really could be talking about either or both but with attribute value style tags were no longer limited to having to choose a single parent category for all my programmers out there you can think of this as a type of multiple inheritance the child tag inherits its values for multiple parent attributes lovely are you getting it by using these kinds of tags we can create a single tag that makes up the bulk of our categorization with e paired with a powerful search engine and finding our original files will be just as easy even if I forget some of the finer details of the image or file I'll still be able to narrow it down in a way that folders could never offer so we've got the base of our system that we want now let's take a look at what's out there then my helpless maker attacking dreams come true back when I first started searching for good solutions to this my standards were a lot lower I just wanted something with any kind of tagging something that was crossplatform and they didn't want to host all my stuff in their cloud or rub their eyes all over my files in order to train their AI algorithms this was back in 2021 too mind you before I jump straight into looking for any dedicated software or Services I wanted to see if there's any sort of Industry standards for keeping track of extra information or metadata along with regular files this way I might be able to stick to using a simple file manager increasing the portability and future proofness of the system as far as standards go there's two main schools of thought when it comes to this storing data inside the file or storing data alongside the file first there's the idea of embedding custom metadata right in files themselves most popular media file types have some sort of way to do this and typically rely on one of two easily hard to- remember acronyms xif and XMP I really won't go into too much depth with these but here's the gist of what they do and why they fail to please me exf is most commonly used by cameraman manufacturers to store information like the f-stop exposure time ISO speed location data and more in the file it's great for well exactly that but not a great option if you have your own custom fields that you want to add XMP on the other hand gives you some more freedom when it comes to user created tags and Fields the biggest shortcomings of these standards though are their limited file type compatibility and their ease of use or lack thereof XMP is probably the closest there is to an existing standard that would fulfill what I'm looking for but with everything it would take to use it I'm kind of just better off creating my own system for managing metadata without worrying about making sure it was backwards compatible with some photographers aging copy of Photoshop on their Windows XP laptop from 20 years ago luckily there's a way around files that don't support this kind of metadata being crammed into them sidecar files are little text files that ride along with a main file and store all of its metadata for it how consider it they typically name something similar to the main file and are meant to be kept in the same directory at all times XMP actually supports keeping data inside of sidecar files instead of baking it directly into the files themselves this has the added benefit of keeping your personal metadata separate from your files in the event of sharing them with other people the big issue with sidecar files though is that they can get lost easily when you manually move your files around for me personally just this awkwardness alone is pretty much a deal breaker when it comes to these things oh my God just pick something so XF is out and I'd rather use something better than XMP Sidecar files just lower your standards Heck if I even did go down the XMP route I'd still be limited to whatever file manager or specialty software I'd be using so while keeping that nice taging system that we want in mind let's take a look at how the big file manager and photo gallery programs stack up Windows File Explorer Pros I'm already using it it's crossplatform if you consider folders to be crossplatform it's compatible with all files cons no native tag support whatsoever ever the search is slow it keeps glitching out on me it keeps doing the green load thingy all right this it's just a bit Bucky in my experience Mac OS finder Pros again folders are crossplatform I guess and it technically works with all file types plus it actually has some native tag support but the tags are Global and the tags are only crossplatform between iOS editor's note here just wanted to add I don't really want every tag I create to be in a global pool but rather separate by something like a vault or Library System I know it's another Wrinkle In The Fold but I just don't like the idea of sensitive tags just showing up everywhere I don't want to see my tax information when to tag in Taco recie you know dolphin the Linux file manager from KDE same folder benefits works with all files also has its own native tag support but like Mac OS the tags are Global the tags aren't crossplatform at all and the tags can be lost if you're not careful the give and take of Linux okay what about programs just for photos then Apple photos and iCloud Pros it's crossplatform it has albums and it has folders but for some reason you can't add existing albums to folders on iOS I don't understand why they made it like this at all I'll mention this now but depending on the program albums can behave strictly like folders or sort of tag- likee categories allowing for the same photo to be in multiple albums at once for the purposes of comparing these in my own sanity I'm going to assume that you can add the same photo photo to multiple albums in all of these cons include it only works with photos and videos gee didn't see that coming no tags crossplatform functionality requires their Cloud doesn't work great on Windows or Linux can't put existing albums into folders and these days AI scanning could be applied to your library at any time without your consent Google photos Pros include crossplatform is it's web based it has albums which is honestly more than I remember it having cons include pretty much requires a subscription it used to be free in that regard Cloud only AI search only no tags no folders no focus on user organization whatsoever I used to use Google photos but then it got me so frustrated that I started this project in the first place all right reeling it back in Samsung gallery Pros include it actually has tags but not in the secure folder it used to have this and then they kind of took it away out of nowhere I feel like I'm one of three people in the world who noticed this anyway it has albums it has album groups or folders cons include well it's a not crossplatform pretty much uh just Samsung phones really not a uh desktop app oh and I hope you don't lose all your tags in an update that would just be tragic wouldn't it Adobe Lightroom turns out there's a whole organization side of this program that I didn't even know about Pros include it has folders it has albums it has tags it has tag groups it has custom metadata and it's crossplatform and cross desktop cons include it's photo focused understandably it's a paid subscription and it's Adobe thanks Adobe here's an interesting one that came up early in my research tag spaces Pro I'm also realizing now that I may have come up with a similar name for my program Pros include it has folders it has tags I mean it better with a name like that works with all files it's crossplatform on desktop has an open source tagging format allows for custom metadata and it has a freaking Web Clipper how cool is that the cons include it is paid either a subscription or a one-time purchase and uh well the tagging system is literally just putting the text of the tag in the file names themselves and if you weren't aware file names on operating systems usually have a pretty small character limit although to be fair they do have an alternative system to this that uses sidecar files instead although you know how I feel about sidecar files at this point oh wait where the other half of the graphic go did it get lost in the transition dang it while some of the previous options seemed decent nothing I could find out there completely checked off everything from my list and if I was going to put all this time and effort into switching to a new program or service to essentially organized my life I didn't want it to be suddenly shut down one day or start charging me a subscription or do anything nefarious with my personal data now as a programmer myself this is when I began to have my Thanos moment and started just doing things myself thus began the construction of my dream solution it all began as a humble python script I knew from the very beginning that I wanted a central database that held all the tags and additional metadata for files inside it no sidecar files and no trying to embed the metadata into files that could even support it this would allow me to have greater flexibility when moving files around outside of the program as well as gaining the ability to have multiple vaults or libraries that contain different sets of tags for different purposes up to this point I've been using python along with tiinder to create a simple GUI I chose these because I wanted something that was crossplatform and JavaScript plus electron was giving me some headaches when it came to Native file access I've already used tkinter as a front end for some small projects in the past so I just went ahead using it for this as well it was during this time I developed my attribute value tag system this is when I gave the app a new name to better reflect its Growing Power and potential tag studio also tagbox was pretty generic and apparently taken by something similar around the time I started implementing the custom tags though things started to get sluggish tkinter went from being a tool to an obstacle the draw times were just becoming too slow for a photo library with a dynamic interface and the consensus online yeah that's just teer use something else instead so I did what every programmer dreams of doing refactoring and starting over from scratch oh the joy I just went to work building a proper GUI using QT this time around now I developed the metadata system into something unlike I'd ever seen before let me show you exactly what it's like now besides seeing some relevant metadata built into the file itself you can add pretty much any kind of userdefined metadata you want to the entry you want a title for the image sure you want to add a description of it of course some additional notes oh yeah an artist how about two might be a collaboration and this hasn't even gotten to the tags yet tags themselves are made up of a full name a shorthand name a list of aliases a list of subtags and a color for style usually the names of the tags are displayed as their full name followed by the first parent tag shorthand in parenthesis for clarification against other tags with the same name both names as well as everything in the Alias list are used to help you find the tag without needing to remember the exact name or spelling of it this is in my opinion a GameChanger when it comes to being able to search through your media you may have also noticed that instead of directly tagging the entry I've decided to create these groups or categories the tags can then be applied to by default I had groups for metat tags and content tags which in my experience helps separate the different circumstances of tagging for Clear organization actually let me show you an example of a metatag you're probably familiar with some sort of favoriting or liking system from different programs and websites and tag studio is no different instead of making a favorite act as another bit of information though marking an entry as a favorite actually adds the favor tag Under The Meta tag group so why the heck would I do it this way what's the point of adding this extra layer when thinking about what a favorite is really it's just a category that you're applying to a piece of content just like a tag and if a favor is just a tag then why not give it full access to the power of our tagging system now you can have other tags and inherent from the favorite tag or apply whatever aliases you want to it the same goes for the archive tag both of which I've given special easy access badges on every thumbnail as I've alluded to or tried to brush over this whole thing isn't exactly finished yet I've been working at a slow pace on purpose for it but also feature creep has slowly set in causing me to keep pushing off a final release there's a few little things left that need to be in here for a public release and then some big features that will make the program much more usable probably the biggest feature I've yet to implement that I know I want in the initial release is a cleaned up and Powerful way to add tags in bulk right now I'm planning on adding a series of tag focused side panels that can always be open or out of your way if you wish these include a tag search panel a recent tags panel a top tags panel and a pin tags or tag pallet panel tag pal yeah no way I'm calling it that combined with a good multi- selection system I want you to be able to bulk add tags based on any or all of these common metrics you'll also be able to copy and paste data from one entry to another or multiple others all that is probably going to take the longest to implement though and I came up with the ideas before switching to the whole tag groups concept so I'll need to Workshop some of the finer details over time all right while doing my research for this video I came across something new something that as a programmer or a YouTuber you never want to see that is the exact Pro that you've been working on already done by someone else how did I not find this program before Color Me surprised when among some YouTube results for tax spaces Pro I found a video for something called Eagle app or just Eagle eagle. cool I'll just call it Eagle here at first glance it's extremely similar to this tag studio app I've been developing in the background it's got the grid of images and files in the center it's got the right SI panel with all the metadata fields and worst yet it's kind of just better than mine in a lot of ways it's got this Web Clipper plug-in that you can use to instantly import posts and files into your library it has native support for audio files and waveform previews it's got this fancy flow layout option in the middle that I'll be honest I wanted in tag Studio from the beginning but it's pretty difficult to do that in a performant way in QT and python so that's when I realized I needed to stop take a step back and think over my options was this just the endgame of everything that I have been planning should I just throw my program aside toss over the $29.95 onetime purchase it's not even a subscription and embrace my new Eagle life honestly I could Heck if you're someone who's looking for what seems like a great file organization solution I'd even recommend giving this thing a look somehow there's only what less than like a dozen videos on YouTube covering it no wonder I wasn't able to find out about this thing earlier however as I started thinking over it more while eagle has most of what I was working on for tag Studio it doesn't have everything from what I can tell it's using plain old traditional tags without any of the subtags or aliases that I've come to embrace in tag Studio sure it's not a deal breaker but it's a big difference nonetheless not only that but I was curious just how Eagle is handling storing all this metadata well I was pleased to see it was in an open format it's all in sidecar files under a specific directory structure meaning whenever you add a file to your Eco Library it actually duplicates your file and puts it under a new directory to keep track of and plop a sidecar file next to this is kind of bizarre to me as someone who's had no trouble managing libraries of files located on different parts of my hard drive one great reason to not move your files around is to provide compatibility with other software such as obsidian you thought this wasn't an obsidian video gotcha I mean it probably would have helped my SEO if I put that in the title but gotcha with my vault-like approach to creating libraries in tag Studio Plus the fact that it doesn't move your files you can effectively create a tag Studio library right on top of your obsidian Vault allowing some supercharged photo and file management there's actually a lot of untapped potential here huh I could easily create a way to import taxs from obsidian heck I wonder if I could create some sort of plugin that would integrate the two even better all right right uh maybe in another video okay so there's more differences than meet the eye but what about use cases Eagle advertises itself as being a way to organize reference images although clearly it's useful for a whole lot more than that while managing reference images is also one of the fundamental reasons why I started developing tag Studio I didn't want to generalize it just that even though Eagle can handle images and videos and icon files and audio files and dang they can even do PSD previews h even though it can do all that due to focusing on references for designers they probably don't have the incentive to add additional support for any file types outside of that space wait they're working on a plug-in system oh come on if there's one sure thing that my program has over Eagle is that it's my program there's nothing more customizable than your own app and to me there's enough I want to do Beyond Eagle's capabilities to where I'll continue working on TX studio for the time being if anything eagle has been an inspiration for additional capability in my own program and maybe as a cautionary tale for things to avoid I don't know though they seem to be on a pretty similar wavelength is me a lot similar of a wavelength in that tag space Pro app oh wait they updated their logo oh come on thanks for watching this unusual project video while I'll continue to work on tag Studio I'm real curious if there's any other features that you guys might want to see added to it or if maybe you're considering taking a look at Eagle and seeing if it's a good fit for you but if you've enjoyed coming with me on this organizational Journey or if you just like seeing me delve into random technical projects then consider subscribing I been Travis and I'll see you in the next one what the hell is this it's your dogs covered in all that sauce Travis that was three years ago [Music]

Channel: CyanVoxel

Views: 293,443

Keywords: Organization, File, Explorer, Obsidian, Notion, Notes, Software, Development, Dev, devlog, tutorial, guide, photos, pictures, gallery, album, tag, tags, adobe, lightroom, ai, aestetic, cloud, sync, database, chatgpt, chat gpt, generative, openat, css, snippet, vault, library, mobile, organizer, data, windows, mac, linux, icloud, google, reference, zettelkasten, python, tkinter, qt, electron, javascript, visual studio, tag studio, eagle, eagle app, eagle.cool, tagspaces, pro, samsung, iphone, ipad, game dev, gpt3, gpt4, gemini, js, copilot

Id: wTQeMkYRMcw

Length: 23min 32sec (1412 seconds)

Published: Fri Apr 12 2024