Tim Beale: Linux Kernel Development for Newbies

hi-yah so I'm gonna be going through the kind of beginner's guide to Linux kernel development so if you've ever been curious about contributing to the upstream Linux kernel but haven't quite known where they get started hopefully this will be useful to you so my background is an embedded software development so I've been a bit of time making around at the Linux kernel level so but despite that I've only got a handful of upstream commits to my name so I kind of I know enough to almost know what I'm talking about but I still make a lot of very dumb newbie mistakes there I'm gonna go into in more detail so the kernel development process revolves a lot around the Linux release cycle and upstream maintainer have got into a fairly good rhythm where they release a new kernel version roughly every 10 weeks or so and that process what happens is there's an initial merge window where basically all the new development gets applied and it's coming in the form of get merge requests from the over various subsystem maintainer and so each kernel version has roughly 13,000 commits going into it and they the bulk of him get applied in this two-week window and in the rest of the time spent producing release candidate builds testing me in raising regressions fixing bugs and but while this is all going on there's still new development continually building up ready to go into the next merge window so it's a bit of a balancing act they do we're a phase been too long fixing the regressions they may just get more pain the next time around with merge window so once all the very worst of the revisions are fixed in a normally branch release the colonel vision and create a stable branch in a separate stable repo it's looked after by a separate maintainer and bugfixes will continue to go into that stream there will be a new stable version every week or so in each of those releases has like 100 or more bug fixes going into it so all this is really possible due to the subsystem maintainer in the Linux community in the exceptional job they do so they've been some very smart people working on the Linux kernel and they've come up with some very clever and elegant design solutions which means that the code is fairly independent in modular and so you can divide and conquer and split it up and have different people looking after different areas of code so when you seem to patch upstream you're not sending it directly to lowness you're not gonna get trolled on the mailing list you're sending it to a subsystem maintainer and they might seem that on to another maintainer who looks after a larger subset of code and eventually it makes its way into the master kernel branch and so all up there's about 300 of these different kernel repos there and used over various maintained errs some of them have two so one for bug fixes going into the stable branch and one for new development going into the niche next merge window in the details of all of us in the kernel source code and there's a maintenance file that lists all the mailing lists of maintainer x' and those get trees they use the other one thing to point out is there is a repo called Linux next which isn't really a traditional get repo but it sounds like a logical place to get started but it's a it's a bad idea to use it because it's gets recreated from scratch every single day by emerging 300 odd get to giver and it's mostly just used for integration testing and finding problems that hit the merge window so the basic kernel development process looks like any open source project really it's pretty straightforward so I'll go over these steps in a bit more detail and pointing out all the embarrassing mistakes I've made so you don't have to repeat them one thing I won't be covering is embedded software development which follows the same process but some of the steps are different like so you have to cross compile when you're building and yeah it's bit too complicated to give them today so the first question is which one of the 300 get repos do you actually want to use certain doubt it's best to use stable repo so that will give you a fairly solid and reliable base from what from which to work from so if you come across any like weird problems in there probably of your own doing rather than someone else's weird problem and they picked up the excellent if you're just interested in testing the latest kernel VIN news Linus's repo and if you're doing new development when you hopefully know what area code you working in and will be a relevant maintainer subsystem tree that you can use so once you've cloned the linux repo you've just got 25 million lines of C code and on any given laptop or phone you're not gonna be running all that code you gonna be running if you know a few million lines of it so the system there allows us to exclude all the drivers and features that we don't care about is called K config and K configures a series of configuration options which they end up as see if gifts and they get used by that make files to work out what drivers to compile and which ones to ignore and it compile-time the configuration options that are used by you current build determined by a plain text file called config and their lives at the top of the source code tree but it's not and source code control so you have to generate it yourself if you don't hit that file prison or it's out date being as soon as you hit make the build system will prompt in you of questions do you want to enable this feature you snow for each of the config options that's not sure about so there are over 16,000 different k config options and the kernels so if you don't have a file prison and speaking from experience you'll be hitting like your snow prompts for quite some time so the simplest way to generate this file is just to use the same config options that were used for the kernel that you're currently running so there's still prison in a /boot directory on your system so you can just copy those and reuse those the problem with this is your the kernel that you're running has already update so there's already been a bunch of new k config options added so you have to initialize those as well so the simplest way to do this is run a second command make mini config so it brings up a menu based configuration system which allows you to toggle the various different options but you don't have to do anything if you just save and exit then it's initialized all the defaults for you to something sensible so that's the most complicated part of whole process once you're done that you're you know you're good so then you just hit make there's a lot of code to compile so it's best to use make - J with the number of cores on your system one pitfall here is don't use make - J with no arguments because they're just Tunes build into a giant full pond which completely except your machine and there are some bill pick packages that you need to install before the build will succeed via pretty straightforward and then much of compiler to install the new kernel it's just a couple of commands to run make install and in the next time you reboot it will start running you a new kernel so part of reason this is so simple as if you were running a distro like Ubuntu it comes with the install kernel script which does some of work behind the scenes that's grub to use your new kernel if you don't have that present mean you might have to do some steps here manually to to update your bootloader yeah so you when you do a make install you haven't you shouldn't have overwritten your your current kernel image your old you know when you new can all exist side by side so the its with from familiarizing yourself with how to to rollback of something terrible goes wrong with your new kernel so boot loader know how to select your your old kernel image it doesn't boot up and yeah be nice to your IT staff it work and don't do it on a quick machine that's what themes before so if you're using a theme in all the the process is just the same so your cloning the code inside the VM and do the make inside the VM and in make install replaces image that the VM will use next time reboots so testing it if you're using a VM be generous it needs a lot of RAM in disk space yeah if you run out of these things in the kernel just it doesn't boat it doesn't doesn't really tell you give you a nice message why and you're giving the VM multiple calls will speed things up the other thing is your when you're running the new Conal you're suddenly jumping from saya 4.18 cone or 25.2 and all your use of space packages and EPS and libraries have been built for a much older kernel and so they won't be particularly happy with it change usually you are just locking up so a bunt oh here's some upstream some mainline kernel packages they attract against the latest master colonel and also all the visions in between so you can install those feeling and they should sort out problems with the with the UI but it's worth knowing with a shortcut keys to drop to the shell and get your second shell on that sort of thing yeah so that's basically building and running the kernel so if you've got a pitch that you know I wanted to get up streamed so there's a lot of documentation I happen on how to do a bit it's with reading up on it that's all that's all online or you can find it in the source code itself so it's good detailed information on how to the coding style and how to post pictures up line up stream yeah the maintainer z' get a lot of get a lot of pictures so some mailing lists have 500 emails a day so it's worth taking the effort to make sure your pictures is clean and tidy and correct as possible before sending upstream there's a couple of Handy scripts that you can use at this point so one check pitch top pill just you can run it on your pitch file and it just tells you if you've made any dumb mistakes they forgot a signed off tag or you're for meetings I'm not quite right so I thought that I followed the process to the leader but every time I run check pitcher always finds something something I've done wrong but one at Sandhya's get maintainer which tells you noticing you senior patch off - based on the code that you've changed what mailing list to use and it's useful is checking the tree that your maintainer is actually using and chicken pitch applies to there and it still compiles and everything before before you send it off so you can only see in pictures to the mailing list using send email you can't see those attachments or anything I've never used gets in email before and found a bit of a pain we asked to use so the first problem I did was i I didn't seal it up properly so I skipped those some of those configuration steps and it still looked like the email was getting through to my gmail but it was completely and also indicated in mailing list just completely ignored it the second problem was when gets in the emails he talks about the ELISA means that Chile's in my mail server was using two different ports for that so when it was trying to connect it was trying to use the wrong port and nothing was happening and it wasn't giving me a useful so what one thing I thought was going wrong was the some of the documentation on lines is there if you don't specify your password and get config which I didn't want to do because it would be stored in plain text that gets in the email will default to a password of an empty string which seems like a terrible default to use so yeah that's not the case will if you don't specify your path will prompt you that does something sensible that for a while it was confusing me yeah so that's that's the basic process if you want yeah if you're interested in contributing to the Linux kernel as we're reading all the documentation is there's a lot of good documentation near on on how to do development how to contribute just software development in general there's a free online book Linux in a nutshell written by one of the funeral maintainer and they kind of goes out over similar sort of material to what I've covered probably explains things a lot more clearly and there's a newbie kernel newbies site that has some tips on where did it get started there's also challenge that some of the kernel people came up with a few years ago where they they see a series of tasks where you got if you completed all twenty tasks and you'd be pretty much a fully fledged kernel developer so it's not running anymore but you can find the other tasks online so you can just complete them in your own time yeah but the general advice rather than trying to find typos or white space problem to fix is to just build and test the latest kernel and report any bugs that you find in it's the best way to contribute and if you do that you'll sooner or later you'll find a bug you'll be able to fix yourself and your hands dirty that way the other thing is there's a lot of new development constantly happening with the kernels so you can just find a area of code it interests you and yeah it started that way so I had a couple in the it was you Ethernet port so when you do shutdown doesn't actually physically link report down so there's new new kind of support added to actually power off at the port so the LED goes off and it's sort of things so babies bug net so I fixed it and another one was related to the IP stick so the RF support didn't work quite right for a particular type of UDP traffic sorry you yeah I don't really know I guess you'd fight it out on the list probably the the problem is you you Wow you're probably going live the same maintainer that would be cases where you've kind of go and fire different maintain errs and different get repos and so you wouldn't find out about the conflict until later when you actually go to merge it's part of why they're Linux next Rico's here to kind of pick up things like that immediately what's that sir yeah would probably happen what I mean the maintainer would probably make the decision and be like I like this pitch better I'll play there I mean you have to go off and rework you of change too yep yep to 200 money a kept repos those four images I'm not sure what the how have it works with scripting and stuff no idea sorry

Channel: WOSSAT

Views: 18,266

Keywords: open source, linux, kernel, kernel development, wossat

Id: OkhOoBqLb3Y

Length: 20min 51sec (1251 seconds)

Published: Mon Aug 12 2019