Hey, I'm Dave, Welcome to my Shop!
I'm Dave Plummer, a retired operating systems engineer from Microsoft going
back to the first days of Windows NT, and today you can watch me do something you never
thought you'd see an old Microsoft developer do:
First you'll get to watch as I build a custom
Linux kernel right before your eyes, in real time, for installation atop the WSL2 subsystem on
Windows 10, and it's all with a goal towards the ultimate desktop hybrid system:
Because did
you know that with the Windows 10 Hypervisor you can use the Windows shell as your UI but actual
Linux distros as your command line and kernel, all at native speeds, side by side, on the
same box as the same time? I'll reveal how you can get the best of both worlds with the new
Windows Subsystem for Linux, Version 2.
[Intro]
The nice thing about the way Windows NT was
architected is that it was always intended to support multiple simultaneous subsystems,
all running at the same time. Way back when, there was the OS/2 subsystem, and of course the
Windows subsystem, and the little-used POSIX subsystem.
In each case these subsystems
are written to run atop the Windows NT kernel. That means a Linux distribution like Ubuntu
actually sits atop the Windows Subsystem which in turns sits atop the Windows Kernel. This
extra layer of abstraction is one more than you'd normally have on a native Linux system and
as a result, the performance was not optimal, because any Linux kernel calls are effectively
translated into Windows NT kernel calls. That overhead of the translation layer slows
things down somewhat.
To solve this as well as some other issues, Microsoft introduced WSL2,
or Windows Subsystem for Linux Version 2. Under WSL2 the NT kernel gets out of the way
entirely and the Linux kernel talks directly to the virtualization hardware via the hypervisor.
Except for the presence of the hypervisor itself, which has little to no real impact, you get
essentially the same raw bare metal performance out of the Linux subsystem on Windows as you would
running that same distribution of Linux natively on that hardware all by itself. The overhead
is usually quoted as less than two percent.
WSL2 runs as a Virtual Machine, but one that
is entirely managed behind the scenes for you. It employs the full native Linux kernel and
provides 100% system call compatibility because it IS the actual Linux kernel: it's built from
the same code.
With WSL1 if you had code that did disk IO, the Linux layer would talk to the
NT kernel and the NT filesystem and Linux IO calls would be translated into native NT style
calls. Contrast this with WSL2 where you are actually running the ext4 filesystem natively,
but inside a virtual hard disk. Linux talks to the disk directly much as it would in a standalone
installation. If you've used the original WSL, you'll notice the disk IO with WSL2 is much,
much faster. Running a Linux distribution under WSL2 is very much like running it on the
underlying hardware without Windows in the way.
To make it more interesting, we're going to
build our custom Linux kernel under Linux on WSL and then swap out the kernel from underneath
ourselves, without even rebooting Windows, and then verify we're running our new custom kernel
when we're done.
In other words, just as you might compile your Windows program under Windows,
we're going to use Linux to build a custom Linux, then run that Linux. And we're going to do it
all on a Windows 10 foundation.
We're going to rely heavily on our CPU's virtualization ability
regardless of whether its AMD or Intel, so the first thing we need to do is reboot the machine,
go into the system BIOS, and make sure that CPU virtualization is enabled. This settings hides
under a variety of names, such as VT-x, AMD-V, SVM, or Vanderpool. Once you've found and enabled
it, start your machine up under Windows 10. There are too many variations of different PC BIOSes
for me to cover them all here but suffice to say you'll need to turn on virtualization in the BIOS
before any of this can work. You'll also need to upgrade to at least build 1903 if you aren't
there already. For what it's worth, I'm doing all of this on Version 20, Second Half, Build
19042).
We're going to use the WSL environment to build the WSL kernel, which means you need
WSL on your machine in the first place. If this sounds like a chicken and egg problem, it's not.
It's just an egg problem. You need the WSL1 egg to grow the WSL2 chicken, and you can pick it up from
Microsoft simply by running
[wsl -install]
To build the Windows Subsystem for Linux,
Microsoft uses one of the latest generally stable releases of Linux. We're going
to use a copy of Microsoft's enlistment, but I've also done it at least once recently with
the original kernel GitHub repo right from the fingertips of Linus himself. It's all the same but
starting with the Microsoft tree ensures we have the same view of the world as their developers
did when they built their version.
We do this in two simple repo steps: first we grab the original
Linux source, then we grab the adjunctive source from Microsoft that allows the Linux kernel to run
against the Windows Hypervisor.
We're going to go with version 4.19.84 of the Linux kernel, but you
can generally use any version that you know has been successfully adapted by Microsoft. That means
they have provided a separate enlistment that will let the Linux sources compile and build against
the Windows Hypervisor. That's not even really a requirement, as you might get away with being the
very first person to run a particular new version of the Kernel, but it's less work in general if
someone else has already done the heavy lifting. I'll use Git to retrieve first the kernel itself
and then the Microsoft custom code around it.
I'm using Ubuntu 20.04 LTS, but I've also tested
these instructions under Debian at least once, so they should work somewhat broadly. The
next step is to get the essential tools you need to compile the code. We can do that
with the Advanced Package Tool, or APT. It makes installing a piece of software and
all of its dependencies largely trivial. It's time to head on down to the desktop and open some
consoles. Are you sitting comfortably? Then we'll begin.
[Enlistment Segment]
Our first step will
be to install the toolset that we require in order to actually compile and build the Linux kernel's
source code. I'll put the text of all of these steps in the video description so that you can
accurately copy and paste as needed rather than trying to write stuff down from a video! How am
I typing? What, do you look at your hands?
11 Megabytes? Well, I believe I
can spare it. I'll allow it.
And the final step for tool installation is a
second list of APT packages that might actually be optional or only required if you go the
X/Windows route down the road. I know the build works with them, though, so I'm not going
to mess with a successful recipe at this point.
Once this is done, we can pull down the actual
source. In terms of where to get the source, you have a couple of choices. I've done it once using
the code right from Linus's personal GitHub repo, just for maximum authenticity. For maximum
simplicity, however, we're going to start with a copy of Microsoft's repository primarily
because I know it works well with the WSL.
Actually, a smarter man wouldn't put all his
source code in the root folder, so let me make a subdirectory here...
Based on what I THINK I know
about git, it will first pull down the initial version of every file in the entire project
and then every change that's ever been made to each file. It will apply the diffs and in the
end wind up with today's version of every file. That's the current source.
How the kernel is
built and with what options it is compiled with is heavily influenced by a config file
provided to the build process. Rather than trying to make a config from scratch we'll
start with a copy of Microsoft's configuration, as we'll assume it's pretty close to being
exactly what we need for our own purposes.
Now if you don't at least have a giant neck beard
you don't want to change too much in here for fear of breaking something, but I am going to modify
the description tag so that I can see when I am actually running my own custom version.
Then we'll save this file back out...
If hacking on configuration text
files seems like a bit much, relax, because all you need to do is type make
menuconfig and it will create and launch a handy GUI wherein you can make your favorite
modifications to the kernel build options.
When it launches, we can see under the version
options that our configuration is plainly marked as DavesGarage, meaning if nothing else my changes
to the config file didn't break it in a way that the GUI can't parse it. That's a good sign!
With
our options set we can launch the build. I love it when a build is simple, and we don't actually need
ANY options here at all, but I am going add the -j option so that it runs a multiprocessor build, as
I have 64 cores and it's cold in the shop today.
I'm also going to drag a copy of Task Manager in
here so we can monitor the load on the old 3970X. When there are abundant files to compile at
once, it will peg things out to 100% cpu, but duing times of archiving and linking, which
are much more about single core performance, we will see many cores sitting idle.
For the cynics who think I would stoop to deplorable editing tricks like speeding up the
video to make my machine seem more powerful, be sure to watch the uptime monitor in
Task Manager and note that it's running at a regular rate.
Wait a minute, did I just
see a warning fly by? They allow warnings to be checked into their OS? Eww! When I was a kid I
spent a long time fixing every warning so that our code would compile at /WX /W4, which meant highest
warning level and all warnings are errors. It kept you honest. So did walking to school in the snow,
uphill both ways of course.
OK, it's linking! If all goes well, in the end we'll be left with a
file called bzImage that contains our freshly baked kernel, hot from the compiler's oven.
To use our new kernel we need to copy it over to the Windows side of the world so that it can use
it to launch WSL. To do that you'll need an admin Command Prompt which you should use to make a
folder to hold your kernel. Call it wslkernel and then copy the kernel in there via the C drive's
mountpoint.
After we've copied it from the Linux side we should see it in that folder, so once I've
confirmed that the bzImage file has indeed made it to the wslkernel folder, we can shut down
WSL entirely with the wsl shutdown command.
All I need to do to hook things up is to specify
our file as the WSL kernel in the WSLconfig file, which is stored in the user's home directory.
When you edit it, keep in mind that you have to use C style escaping for backslashes.
bzImage is likely not the best name for a kernel so let me rename it something more
befitting of its role, like kernel.new
Once I've confirmed that wsl is in the shutdown
state, all we need to do is restart it. As soon as we do we can run neofetch to
get the banner info for our kernel.
Aha, the Windows flag. And we can confirm
that we're running my custom version of the kernel because it's listed as DavesGarage. You
can also see that we allocated potentially all 64 processor cores and a quarter of the
available ram to be used by WSL.
Thanks to CPU virtualization, remember that
everything running in the Windows Subsystem for Linux, including our custom kernel, is now
effectively talking directly to the hardware through the hypervisor and not going through
the Windows kernel to get its work done.
How can that be, though? Isn't it a bit like the
Highlander, in that there can be only one? Whereas I've made it sound a bit like Windows and Linux
are BOTH running natively on the same hardware at the same time, and that'd be silly... or would
it? Well, it turns out not so silly after all. As long as the Windows Hypervisor is running as
the bottom layer, you actually CAN have multiple different operating systems running effectively in
native mode at the same time. Both can access all the hardware, devices, and memory that has been
assigned to their hyper visor session. They truly are running natively side by side on the same box.
And that means we have really achieved what might be the best of both worlds.
If you look at my
desktop, you can see a weird mixture of different environments: PowerShell, Command Prompt, and two
Linux shells, one from Ubuntu and one from Debian, all running natively on the hypervisor
simultaneously. I have my Linux shell configured to launch over my Windows filesystem
mountpoint so that I can use Linux and the standard Unix-style command toolset to operate on
my Windows installation as well, since I find the Linux tools a little more predictable for me than
the long and verbose Powershell command set.
Let me take a second to get both the Windows and
Linux environments engaged, perhaps both listing the contents of the entire harddrive. We can get
all four rolling away at the same time, and then drag Task Manager in to see how they are faring.
From a quick inspection, you can see a couple of single-threaded tasks taking up the majority of
a core, and those would be our directory tasks.
Short of running X/Windows, which is also possible
but just not something I'd want to inflict on myself, you can run pretty much anything
that requires a Linux kernel, like multiple distributions at once as we are here. I've
even got so far as trying to boot a Mac image, and I've got to the Clover bootloader, but not
much further.
I imagine by now you can see the potential here - you get the full Windows desktop
user experience, but the computing world is opened to the entirety of everything that can be done
with the Linux kernel, including all of the various flavors and distributions. You don't
need to worry about finding a distribution with an acceptable user interface, because you can use
Windows with its excellent desktop shell as the host for all of the others.
You can also run
apps in each of the subsystems at the same time, so you can be running Linux apps at the same time
as Photoshop or Premier on the Windows side.
If you enjoyed this whirlwind look at the Linux
kernel on Windows, I'd be honored if you took a moment right now to subscribe to my channel.
This episode is a bit of a leap into the Linux space for me and your subscription serves to let
me know that I'm going in the right direction. I'd then make more like it and if you turn on the
bell icon you'd even be notified of them when I do. It's a win-win.
As always, remember I'm not
selling anything, and I don't have any Patreons, I'm just in this for the subs and likes, so if you
did enjoy the episode, please be sure to leave me one of each before going. YouTube apparently
really does care whether you care, and they measure it no small part by subs and likes.
And don't forget to head on over to Dave's Garage at the end of the month for a Livestream on
Sunday the 28th of February at 10 AM Pacific, 1PM Eastern. All questions will be answered,
all inquiries addressed, and you can help me plan for future episodes! The more the merrier,
so bring a friend. Our first ever livestream had a 1000 folks show up and it was a lot of fun,
so please do stop by on the 28th.
Thanks for joining me here in the shop. In
the meantime, and in between time, I hope to see you next time, right here
in Dave's Garage.
He's a pretty interesting guy; original programmer for Task Manager (the one piece of Windows software that basically always works).
Now do the reverse. Oh wait
"get the best of both worlds with the windows ui and the linux kernel"
But the windows gui is quite possibly one of the worst aspects of windows!
microsoft's commits to FOSS are kinda pretty good ngl, even though most are self serving