Replacing the Unix tradition

so this is going to be a rant about how the so-called UNIX tradition the system of a hierarchy of processes and file permissions and all that sort of stuff that comes down to us from the original unix all that stuff really needs to be scrapped it's well past time that we effectively entirely replaced the UNIX user land not really the kernel that's all fine we can keep most of like say the Linux kernel but some of it will have to be changed to present a different kind of abstraction for user land and if we're going into details I should be clear I'm picking on UNIX here specifically but windows and other alternatives have all sorts of their own similar problems they just made the different variants of the same mistakes I should also say this is part of a larger rant about just general software quality and why it's so hard to make good software these days I think one of the very major reasons is because we're building on top of really shoddy platforms and that goes for UNIX but it also goes for Windows and basically everything else whether you talking about the web is a platform it's really unfortunately where we are today because everything is this huge kludge is this huge giant mess where yes of course there are very good reasons economical and some technical why we don't just scrap the whole thing why these things have hobbled along for so long and why the path forward usually taken is to just pile more stuff on top certainly at each step along the way there are very good reasons why that has been done but you add that up over the decades and what we get is a bigger and bigger mess and that's what all these platforms units included feel like they feel like this giant mess where is like a work area it sits like a desk where you have all your stuff piled up and you don't want anyone to touch it because hey you know where everything is but all this mess has it created over decades now and you the person who owns a desk or in the analogy to the community of people keeping track of this mess you know where everything is you're comfortable with this giant mess and it would actually be a huge burden to you if if order to be scrapped and replaced with something much more sane and that's unfortunately where we are as the UNIX and other platforms are these giant messes where the people in charge the people doing the actual work like maintaining Lewis Carroll and maintaining UNIX user land they're all comfortable with this giant mess but is extremely inhospitable to newcomers and it creates all sorts of burdens for people who aren't experts in the whole giant mess but have to work with it tangentially you know I have to deal with some parts of it as as you do if say you're building user software or actually if you're just a regular user in many cases what the general programmer experienced feels like these days and in fact in many cases the general user experience feels like is that all but the simplest pieces of software are all rat holes unto themselves where you know something doesn't act like you expect it to and figuring out how to fix it is this giant tangent that leads you on to 20 other tangents and those 20 other tangents probably each have their own twenty other tangents that's what the modern user experience is often like for these power users and it's what the programming experience is almost always like even just the act of building software and keeping track of your your code with version control all of those tools have gotten so complex to build tools and version control systems tools have gone so complex I think in many cases because the underlying platforms are abstracting over have also gotten really complex and if that underlying platform were simpler then this wouldn't filter up to the software above it so what's so bad about Unix well first off almost anything to do with terminals and shells I should qualify that right off the bat by saying clearly the idea of a programmatic user environment where you have complete control programmatically over the system that's totally sound but everything from there about terminals and shells is really a disaster and it doesn't have to be this way the core problem here is just the sheer complexity of the whole arrangement sure in the simple case of hey you bring up the terminal you have a shell there and you type commands and does what the command says it seems really simple for the the trivial cases but what we end up dealing with inevitably in a complex system is all the edge cases when anything goes wrong the whole complexity of the thing tends to rear its head and it's this huge drain on all your time and your attention just think of all the moving pieces and the conceptual complex that I involve with a very simple command like say LS - LA redirected to foo to really understand what's going on here you have to know like 15 different things so first off there's this terminal thing which is like the sudo device in the Linux kernel so it's like a pseudo hardware device that is interacting with this shell process text type that the terminal gets to the shell process it looks at that command and interprets it as invoking the LS program which should be on your path right otherwise it won't know how to find it and what the is path well it's this environment variable thing which has to do with process hierarchies of data handed down from processes spawning other processes and so anyway the path variable lists a bunch of directories where LS should be found on one of those and so the shell process Forks itself and execs LS and the fork the text - la that's passed into the parameters of the first function by si conventions passed into the the char array their char pointer array I should say and so what - la means is entirely up to this particular program which we're invoking the LS program the angle bracket foo however that's totally different that's something the shell itself recognizes as part of its syntax and so it is okay this before I fork an exec to run the LS program I need to open this file foo in the current working directory which is a parts of the shell processes state we're going to open up that file for writing and in the fork where we're going to exec LS we're going to swap out the usual standard output file descriptor which is 0 or is it 1 I forgot it is I think it's I think is 1 anyway we're going to replace that state file descriptor with the file descriptor for the open file foo which we're going to write - such that when we exec the LS program it has the file foo as its standard output and so the the the command output doesn't go out to the terminal it instead goes out to this file foo and because we didn't run this command in the background the the shell itself is actually waiting for the LS fork to return to give an exit code and then it will collect its the finished child process and then continued on its way and I think I've even left out some details but that alone what I've just described that's an extremely complicated story and you could say it well sure as just a casual user of the command-line you don't have to use that but the problem with the UNIX environment and really just all of our software is that inevitably you're going to hit that point where you do have to know all these moving parts because there's going to be something you just don't understand about why something is going wrong until you understand that whole complete story and so I think it's very important we ask the question does the story really need to be that complicated and I say no it does not and the only even just the complexity of all that almost all the particulars feel like this giant clutch where just in like the naming conventions make no sense to any newcomers just this jumble of names that has been piled up over decades and sure there's a lot of people out there are very comfortable with all these and familiar with all these names but if you're not familiar with all these names they just don't make any sense they're the namings old strange there are inconsistent conventions for both the naming and syntax of command parameters and the standard unix shell and all its variants whether bash or - or zsh or just all the variants now all really terrible they're just really bad variants of dynamic languages over decades they've just piled on all these convenience notation features which can save experts a few keystrokes but are just not worth it in the grand scheme of things they just put a huge burden on everyone else I don't think it's an exaggeration to say we'd be better off just having a commonly used dynamic language whether Python or Ruby or JavaScript even who cares as the standard shell language even though be more verbose to run commands you just run functions instance' like run and the first argument is the name of the program you want to run and then a string for all the remaining arguments that would be better that because you'd have a simple consistent language relatively simple notationally language the shell languages are just these couple together messes of ad hoc conveniences which is why perl for example is a shitty dynamic language that no one uses anymore because it takes Holt's influence from the shells it was a worthy experiment at the time but we learned better and the better dynamic languages that have survived move away from that direction I should be clear though that simply replacing the shell language with something more sensible and somehow removing that layer of legacy Crofton which is the the whole terminal concept that all would be an improvement but it wouldn't really solve the root problem which is all that inherent complexity I described in what's going on behind initiating a process behind a simple command like LS - L a and also be clear that all that complexity really just can't be sidestepped it's really kind of tethered together with the heart of most Linux distributions where everything is glued together with a bunch of shell scripts that situation has kind of improved over the years because like system V came in and replaced a whole bunch of old startup scripts but there are so many places and Linux systems where you really just can't avoid the shuttle it's it's just tied up with the whole thing it's it's a central glue that makes the thing go and of course if anything goes wrong on your system and you want to fix it and you investigate internet forums to find the answer the answers you're going to get are almost always going to be the form of some kind of shell business that you're going to have to deal with okay so the show and terminals suck what else sucks dependency management package managers are in concept a very good idea it's just that well sure very often software install what the package manager does work very often but very often it doesn't and the question is well why are theay systems supposed to solve all those problems if a piece of software works on one person's machine on the developers machine let's say why is it such a headache to get it working on other people's machines if you use the PC and the DOS error you may remember that used to be as simple as simply just copying all the files of a program into a directory on the target system and then it should work on that system assuming it works on other systems with the same version of DOS barring of course issues with the low memory barrier and hardware support but what we didn't deal with back in those days is one installed program messing up another because they have conflicting dependencies if the package managers have managed dependencies simply by ensuring that all the static pieces are there on your system that are needed then package management should work brilliantly but they do more than that that's the problem they also will install many packages they end up running scripts and other programatic code that actually manipulates stuff that reaches in and modifies parts of the system that MUX with configuration and that's where everything goes wrong in other words these package managers yes do a reasonable job of ensuring that all the packages that you think should be on the system are actually there but then what it doesn't do is actually verify that those packages are in there virgin state because the way we write our software is you end up just recklessly mucking with the state of all of our pieces all of our packages so of course the package managers can't give real insurances because they don't really track all possible configurations of these packages they just track the packages themselves but of course in many cases whether or not your software is actually working hinges on the configuration state I think it's really terrible about these sort of software failures is that they're not really considered bugs these software packages are written to certain expectations about their running environment and when that environment doesn't meet those expectations the authors of those programs don't really treat that as a bug they just say oh you need to fix your environment we have this tangled system where the pieces are supposed to work together or at least live side-by-side peacefully but there are many many scenarios where no one's really in charge or feels responsible for when the pieces don't really fit together and don't really cohabitate well so how could we fix this well I think first off you need to have a much stricter notion of package management where there are strong guarantees that separate packages can't really mess with each other and also the packages can't really get into states where suddenly they aren't really the package expected by other packages if a depends upon B B really shouldn't be able to get into any state where it doesn't suffice what a expects so how do we ensure that well I think part of it is at least just having higher standards for our software in terms of how much configuration with deemed reasonable for a program to have but aside from just having higher standards we could also I think minimize configuration state we recognize that most configuration a huge huge chunk of it concerns just two things it's about resolving references of just hooking up a is supposed to find B and then there are other huge half of configurations all about security it's about permissions those two things right there resolving references and permissions I think describe probably the root of 80% plus even 90% plus of all the stupid problems I've had to deal with on my computer things like oh that thing that's supposed to be in that directory wasn't in there or is miss named or whatever or have the wrong permissions all that sort of garbage so if you could somehow get rid of all or at least most of this configuration State concerning resolving references and permissions you would be eliminating I think the primary cause of stupid errors okay so another question is what would a system look like that solves all these problems or at least mitigated them well first I'm going to do a broad overview of my proposal before going into the details so first off for package management to really make sense I think it actually has to be something done at the kernel level there is a kernel level notion of what packages are on the system bare with me I know it sounds strange but I think the rationale will be evident fairly soon so within the system all packages meaning mainly all programs but also all shared libraries and then also all files and all directories all the things basically that the kernel identifies are known by both a UUID and then also hash IDs with the idea that the UUID identifies something through all of its versions whereas the hash ID both verifies cryptographically that this is what we say it is and also it identifies the version and the idea of these IDs is they really should be machine independent they're not just some artifact of one particular system when you copy a file around when you distribute a package that you these IDs are really going to be the same on every single system and so when we resolve references in configuration in the system it's all through these IDs what we're getting away from is hooking up references to the reference in the form of file paths that's extremely error-prone that's a huge source of all of our headaches that's what we don't want to do unlike file paths these IDs are going to be the same on every single system now of course there are details here well how can you enforce UUID is being uniform across all systems you can't really do that but I think as a matter of all practical purposes it will work well enough for reasons I won't get into quite yet now conventionally directories on UNIX and all systems like Windows they're a listing of names mapped to actual behind the scenes file IDs so in a sense conventionally directories are actually what give files their names and you can have hard links so-called to the same file and from multiple directories but then they'd have different names in different directories that's not how things work in the system in this system each file in directory has a set of metadata optional metadata attributes associated with them including possibly a name but also things like the creation date and whatever else you want but all of those things including any name in fact are all optional you don't have to give your files and directories names they can just exist because the true name of anything is really just its set of IDs the idea here is that we want to move away from any sort of conventional file hierarchy there's no notion of root directories there's no notion of mounting partitions any of that sort of business is just you have a bunch of files and directories known by their IDs and rather than thinking in terms of browsing up and down neatly organized hierarchies because in actual fact they're never really neatly organized instead of even attending that what we're really doing is just relying on search search of the metadata of the files and directories so as I mentioned we want to make the security system as reductively simple as possible and so we actually will only have two privileged levels those noble concept of user account privileges per se it's just every program running has either admin privileges or non admin privileges or maybe you call them super I don't know maybe it'll just be called super user privileges or whatever now I also understand there's no process hierarchy there's just your installed packages and programs when installed are given admin privileges or not and so when installed program runs and hasn't been privileged is it then can do certain things like in fact install and remove packages on the system that's one of the privileges so each installed program each installed package and each user account including whatever we call the admin account whether it's super years or whatever all of these things have their own individual file space and the rule is that the min programs can see anything on the system they do anything they want of course but normal programs can only access their own file space and the non admin user account spaces and understand there's there's a notion of a logged in user but there's no process hierarchy right so when a user program an on admitting it can see actually all of the user accounts whether those people are all logged in or not we're not trying to really protect the the normal users of the system from each other because I think in modern computing that just doesn't make any sense it's it's an out of date notion it certainly made sense back in like the the 70s when a bunch of people log into terminals and of course you want to actually protect those users from each other because there are a bunch of strangers all sharing the same machine I don't think it just doesn't happen anymore that strangers are really sharing the same machine and sure you have people like spouses sharing the same laptop and so forth but the notion that we're really going to keep the one person's files on the machine secure and truly private from the other user of the machine just it's a total fanciful notion because one of those people is going to have admin access right so it just becomes totally meaningless this barrier between users messing with each other now when it comes to IPC inter-process communication there really should be one primary mechanism like the default way of initiating all sort of contact between programs and what that should be is something like a request response mechanism any program can send any request to any other program and that program then comes back with a response to my understanding what I'm describing is actually quite like what KD bus is going to be on Linux though I think that is complicated by all concerns about the conventional UNIX permission system but we're dispensing with all that so it's a much simpler model here do understand though the implication here is that each program effectively in some sense has to be like a server it has to listen for requests and respond to them sort of like the way that Windows programs always have to have this message pump going now what requests a program will respond to is entirely up to that program it's just a matter of each programs external public API for many programs the set of requests that response to is going to be very simple like a lot of user-facing software like a game for example there's not much it really needs to do when in relation to other programs it just needs to be started maybe paused and maybe stopped so that might like be a good bare minimum that every program would implement because understand in the system we're trying to minimize as much configuration state as possible and what programs happen to be running at the moment that itself is configuration and so it's a source of error where you try to talk to that program but oh wait it's not running and so the idea here is that with this IPC mechanism of request run response a program should be in the state of ready to respond to these requests at any time now maybe the program is dormant or maybe it was never loaded into memory and started executing at all but the idea is that the kernel when your program receives any sort of request then it spins up and it should then be ready to give a response so again there's no process hierarchy here there's no programs managing other programs really as much as possible each program to external appearances should be this stateless always available thing it should be more like a service basically now this request and response mechanism is a good baseline for meeting most common needs of communication but then of course there are special cases where you have a lot of data you want to send back and forth and so this request response may not be most suitable so between processes through this request response mechanism we can share handles to things like files pipes and chunks of shared memory and that way you can have higher performance mechanisms for IPC but understand the general pattern here is everything is initiated through this request response I'm not totally certain about some of the details here like when you hand off a handle to a file to another program should you have to then also do some kind of system call that validates that file handle so that it gives that other program permission because maybe maybe these file handles that you share between programs maybe they're actually system-wide IDs instead of processed specific ones so they're details like that I haven't totally worked out of course and lastly again we're trying to minimize as much configuration here as possible but there's going to be some legitimate amount of configuration that our programs are going to need so where should we put it a traditional UNIX answers that hey we'll have this Etsy directory and a bunch of well-known file paths to certain config files there that's really error-prone because have a bunch of different programs all mucking with the same state they're the same configuration that all sorts of other conventions for storing configuration and files now I'm not positive about this but I suspect it actually is a better system to have a central registry the problem I think with the Windows registry is that has a complicated permission system it also is kind of reckless in terms of what programs can modify what part of the registry and then a whole lot of programs totally abuse the registry because they they're not really storing config in the registry so much as they're storing data that it's being abused as sort of this general-purpose IPC mechanism instead of every a place to store config so the registry I have in mind the idea is that would just be a free-form storage of key value pairs but every user account including the in mint account and then every program should have its own separate namespace for this key value pairs and of course non admin programs could only touch their own namespace and not the namespace of other programs as for touching the user account namespaces I'm thinking maybe only admin programs should be able to touch that stuff like do you want regular programs to fight over what your language setting is or what your user preference for Mouse speed is I'm not sure that's something that should be possible maybe maybe the non admin programs can read user account preferences but they can't modify them maybe even something more complicated than that I'm not certain it might also be best if perhaps there were a separate namespace for every pairing of a user account and a program so that when a program stores settings that might affect their currently logged in user they would have one place to store that's separate from where they would store settings for other users that might be a thing I'm not sort of maybe there's a simpler solution to that this whole registry business is probably what I'm least confident about I'm I know the Windows registry has become a huge mess I'm not exactly certain why though and perhaps if we just avoid the extra complications they introduced and turn out fine it's an open question so now consider how much the system I just described consider how much it simplifies we're getting rid of any notion of a process hierarchy so there's no real fork exec there's no environment variables thank God you don't have the file handles which are child inherits from its parent process so there's no notion of standard that stand out there's a no inherited permissions or anything like that there's no exit codes no program arguments none of that we're also getting rid of or I guess you could say so radically simplifying the user and file permission system there's no notion of groups there's really no file ownership per se there's no exec bit there's no mountain of partitions none of that stuff and as I mentioned that the star roles are ditching basically everything about terminals and shells so there's no sessions there's no job control you know foreground versus background there's and know crappy language is only really proper programming languages that are meant for general purpose work and that brings me to the question of well okay if we're ditching terminals and shells what does the replacement look like well first off you don't really need a terminal to have an interactive prompt you could just do Sabol like what the JavaScript console in the browsers does where you you're presented with a prompt and then you get back to response just like you would for an interactive session of a dynamic language at the terminal so whatever language you use for your command prompt that language itself would provide its own interactive command prompt these command prompts however would have a call response format it would be like a terminal where it's this basically this stream of text input and string of text output that actually can be written to and read from in an interleaved fashion by different processes we don't want that at all we just want the shell itself presents user with a command prompt and then they enter something and they get back a response and that response is displayed in a self-contained text area it doesn't just constantly stream and interleave with whatever comes after if you follow that terminal model then you have to introduce all sorts of complications to build questions like well this process is run in the background but we don't want it to actually spit stuff out to the standard output so we have this notion of sessions and so forth and just we want to sidestep all of that so now it's just call response call response like you would see as I mentioned in the JavaScript console in the browsers now for certain commands that you issue you want to get back a streaming response like maybe it's just this ongoing updated log that you're seeing the output of so the idea then is that in the response area you would have this scroll a response doesn't have to be this static thing it can be this live constantly updating view of data being streamed in from the request that was made so for example if the command you issue sends an IPC request to some other program perhaps it expects back as a response a file handle and then your command prompt your shell as we whatever you want to call it takes out file handle and then constantly reads from it and then updates the response to that command but understand the scroll of your shell itself always shows the commands and the responses one after the other not interleaved as it might happen in the terminal I also think would be really nice if that when you're typing your commands you could have all the full features of a proper text editing environment it's just really noxious how like say the standard cut and paste shortcut keys don't work in terminals because of legacy reasons basically that's just really really terrible so in fact depending on the shellings but in what I have in mind you very often would want to format whatever code you want to execute across multiple line so you want to use Enter key and not have it actually execute the command so they can actually shift enter or maybe control enter should it be what you do to actually execute the command if you just hit Enter it should take you down to the next line as what normally happens when you're writing text so whatever the show language looks like exactly one thing you're going to want to do a lot is issue requests to various processes on the system various programs and so to make that happen basically what you want is for your shell to provide some standard library functions for making such IPC requests lastly one thing I would really like to see modernized about the command line experience just the output should be much prettier it should be readable in a way that output of traditional commands and you know traditional logging to terminal just isn't because terminals come from this earlier archaic error with you know no real notion of text formatting so I think the solution is that many responses in the show should actually display HTML output or you know I don't like HTML and all sorts of issues with it but something like HTML at least some sort of response that is formatted text and having HTML or something like it would actually also be a way to imitate the behavior of a lot of terminal programs these days where it's actually sort of interactive where you issue one command but then it prompts you for the next step well if the response that comes back in our show has HTML with some kind of form then like you can click on a node or use keyboard input or whatever to then submit another command and so you could get a very similar effect like maybe some sort of command wants you to hit yes/no like are you sure really sure you want to do this and so in the response it could present a form with a button yes and a button no and you type Y or n or or you click on the button and it submits the command again this time saying yeah rule I'm really sure and you could do other interesting things like present a table or a list where you can click on various elements to get more information or maybe is a convenient way to execute a command on some element of the list you could do all sorts of stuff but of course past a certain limit you probably should just start writing an actual proper interactive program if that's what you really want okay I hope that gives you some idea of how we could have a command line environment that's more modernized and also ditches a lot of legacy complications of the traditional terminals and shelves another big remaining question you might have though is well what the development look like on this system because as I mentioned for a program to run on the system it really has to be an installed package and so what that implies then is we're going to want to in the course of development have an easy way of installing a sort of temporary package and so I think what we'll need is reserved half the the address space the ID space of our package is for private use so that when doing development you can choose just a temporary ID for your program just so you can run it on your own local system but then of course when you want to actually publish your program then you're going to want to use a public ID so anyway you're developing the program you have all these source files that you then compile into running program or whatever the process is with your language and you want to actually then install that well you're going to need what we might call a manifest file that's going to describe your package and its dependencies and understand that we consider the target system API and API ABI combos as part of the dependencies here because we want to make sure that when you install a package that it's targeting the correct platform and then with that manifest file and then also probably just a single directory that lists all the files they need to be part of the package there's some sort of system call that takes that specified directory and manifest file and installs that as a package and then you have your program installed as a package to which you can issue IPC requests and thereby kick off the execution of the program I suppose you could argue this sounds more roundabout than the conventional way of just compiling your executable and then running it now of course even that's not so simple because you have to set the exact bit before you can run the file and of course if what I described is more complicated well of course you could just automate it as part of your build process probably not a big deal you may still be wondering though what about the case of dynamic languages or rather languages with runtimes where you don't necessarily have a binary executable how does that work well conventionally what we do is we invoke the say the Python interpreter and there's argument we pass in the name of the the script file that is the the kickoff for our program well what we could do instead is we could have a small binary which doesn't have like the full runtime but it drags in the bulk of the runtime from a shared library and this way you could take your Python script or your whatever dynamic language script and package it up into something that fits in basically the same package mold of natively compiled programs and whatever the details in complications here I just think it's really important that whatever we do we need to arrive at some solution where every language isn't inventing its own package management system on top of the system-wide package managers we're supposed to already been using if our system package manager doesn't accommodate all of the things we want to install in the system that I think we failed because then for these special cases we'll be dragging in the extra complexity which is exactly the sort of thing that's so terrible about the systems we have already so I hope now you have some notion of what using the system might look like I'll end here by addressing just a few miscellaneous lingering questions first off what about network sockets the problem with network sockets is that they effectively represent a kind of configuration state and you have all these different programs that you might want to install in the same system and there's potential there for conflict cuz you might want to use the same the same ports and what we can't seem to do is just resolve these conflicts as they come up on the system automatically because external programs stuff on other systems is expecting certain programs in our system to run at certain ports so I have some notion of how this problem might be addressed you could basically have the system automatically assign ports rather than letting programs choose which ports they want to use and therefore the pro system can ensure no two programs are using the same ports and then you could have running on a well-known port a lookup service that tells other systems when they want to know hey what what port is that program running on it can tell them so for example another system asks hey the program with such-and-such public UUID what port is it using this would be a workable solution perhaps though it doesn't introduce a little overhead and having to look up these ports but of course probably the bigger flaw is that it presumes that every system in the world has some notion of the packages I described which of course will not be the case another question that comes up is well what about plugins and modular programs how exactly do they fit into the package system because I think what we don't want is for this package to be installed but then to bring in all these plugins or modules and then the package has this totally different configuration state than what it started with in fact more than that it's not just more configuration that's more code and so this program which was understood to be one thing when it was installed is now something quite different and that violates the whole spirit of this package management system as much as possible we want to install things on our systems to be just what they say they are and not something else I suspect the solution here is simply that any plug-in or module to a program gets installed as an extra package and then the existing program that depends upon this new plug under module it's manifest describing its dependencies gets updated with this new module I think something that simple may actually be a workable solution another question people might ask is what I described like plan nine which wouldn't be a favorable comparison because plan nine was a total failure it didn't replace UNIX at all there are a few ways in which what I described sounds a little bit like plan nine like actually getting away from conventional hierarchical file systems I was sort of an idea of in plan nine but I don't think there's actually major overlap because actually what plan nine was really all about it was an attempt to take UNIX and somehow make it network transparent to do some sort of rethink of the system that made it them somehow supposedly more suitable for networking I think ultimately plan nine failed because that vision didn't really coalesced it it wasn't really clear if it was any benefit to be Network transparent in that way I think network transparency like that may actually really just be a mistake I think what we want out of our platforms is to take care of the local system and then let all networking concerns really should be handled at the application level in fact in a way UNIX itself I think suffers from thinking too much about the networked environment because you know originally it was developed for many computers with a bunch of terminals and so already it was designed to accommodate a sort of networking and so it brought in this complicated notion of user permissions and process hierarchy permissions and all that stuff which I want to get rid of so I would understand people making a comparison with plan nine but I think actually my system goes really in the other direction going back towards a network agnostic platform the last question I've been thinking about is well how do we implement the system so ideally of course as I described I think at the start is that you would take say the Linux kernel and you'd modify it to present a different set of user land abstractions that of course would involve a lot of work and require a lot of expertise that I don't have I know I'm not a kernel hacker I'm sure it would be interesting working and I'd like to see it done eventually but I think the the shortcut to getting such a system as I described up and running at least as a proof of concept is to do what's called application virtualization this is basically what things like the so called containers like docker have done where it's not hardware virtualization we're not embedding an operating system within another operating system is just we're presenting an abstracted environment for programs to run in so the thought is I could define a set of ap is such that programs written to this API could run within a sort of abstracted environment within Linux but with all the appearances of the system I described so for example instead of using normal system calls for dealing with files you would use the ones provided by my API and then you would be working with files in the manner as my system prescribes now I imagine such an implementation would not be ideal in certain ways like for example the hole request response IPC mechanism I don't know exactly how it might implement that within Linux in userland terms but it could be done probably though with overhead relative to what you would get if it were properly implemented at the kernel level so this implementation wouldn't be ideal in terms of performance though probably wouldn't really be all that bad either and I imagine there's something in the security model that couldn't be enforced properly so it may not provide the the proper security guarantees that a real implementation of the system would still despite all these drawbacks I think that actually would be the way to proceed certainly as a means of getting a proof-of-concept of the whole system out there but also to work out kinks in terms of certain issues and use cases which I'm certain I have not properly accounted for so in fact if you have any thoughts about how the system I described won't work in some key way or has some major issue and perhaps you can propose some solution I'd really like to hear that sort of thing and I hope that maybe if you didn't agree with everything I said you at least are starting to think about how our platforms could be better how they could be greatly simplified and causes less pain

Channel: Brian Will

Views: 79,965

Rating: 3.8876112 out of 5

Keywords: Unix (Operating System), Software (Industry)

Id: L9v4Mg8wi4U

Length: 40min 45sec (2445 seconds)

Published: Wed Sep 23 2015

Reddit Comments

[u/deux3xmachina]

I've got no idea who this guy is, and it's true that *NIX systems have a lot of baggage, but I think the only bits of this rant that didn't make me want to stop him and ask "Just how is that an improvement?" Were "language-specific package managers represent failure" and the general concept of using message based IPC universally.

Most of what he described may only reduce complexity as perceived from certain perspectives, possibly from the end-user view, but Android, iOS, and MacOS are all built on top of the same abstractions he's ranting against and are widely considered some of the simplest UX.

I'm not entirely sure what his experience is, but it mostly feels like someone complaining that your interactive $SHELL isn't a competent programming language, when that's not what they're meant to be used for anyway. It's like saying you can't tow a U-Haul with your Prius or that your hatchet isn't useful for changing tires.

Edit: if anyone happens to know how to get in touch with this guy, could you PM me or direct him to this comment? I'd really like to discuss his ideas as invited by the video. I'm no rockstar or anything, but I do work with *BSD and MacOS for fun and profit, and I'm genuinely confused what experiences would lead someone to think that these are the appropriate solutions for *NIX systems.

Edit 2: I'm just now seeing that this was published in 2015, so it may no longer accurately reflect his views, and be difficult to recall why he held these beliefs. I'd still be interested in discussing the proposed solutions, if he still holds these beliefs.

[u/Oxc0ffea]

Use Linux for another 5 years then come back and revisit. Most (every one?) of these ideas don't make sense or are mis-directed.

[u/[deleted]]

I think he wants AS/400

[u/icantthinkofone]

I thought he could have made a fool of himself in a lot less than the 40 minutes this video runs. It only took a minute or so for me to realize it wasn't a joke video.

[u/pinacoelho]

Does he seriously think that a user typing that command needs to know about argv/fork/exec???

Yes, any useful system is going to be complex underneath.

[u/strongAmerica]

The art of abstraction is what he is describing, and *NIX doesn't do abstraction well. A computer is a massive state machine and what I hear Brian saying is why should program A be modified with program B's config, which has often happened to me with *NIX.

[u/[deleted]]

So he wants TempleOS then

[u/attero_]

So the closest things he describes here are and some thoughts of mine:

• interactive shell: ipython / javascript console / Powershell?

• Package Management: nix package management with UUIDs for a namespace for each package / appimage / flatpack?

• IPC: Androids Intent system

• Permissions: Sandboxed applications each running with no access, have to request permissions from the kernel to gain certain capabilities (no real need for a true admin account) Look at iOS / Android's permission models

• Configuration: I'd like to see something like .plist on macOS for configuration with small independent files, clearly defined schema and location. Putting settings into a relational database/registry/huge xml structure for no apparent reason seems unnecessary. looking at you gconf

• Kernel talks to hardware and provides process separation/ access control/scheduling. It has afaik no real concept of files and can be seen as a Service Provider programms can interact with via systemcalls. Why would you put the complexity of file retrieval/organization into the kernel?

• File metadata with just xargs / flat filesystem hirachy --> a lot messier. Than the unix permission model / ACLs IMO