Let's port Diablo to SerenityOS!

well hello friends welcome back to the program today we are gonna have some fun so um when i was a kid one of my favorite games on the pc was diablo i played diablo for hours and hours and hours and i recently learned that some people have reverse engineered the original diablo executable and recreated it as an open source project and um they've ported it to all kinds of platforms already as you can see here and it's called um devolutionx or divolution x and i figured today we are going to get this thing running on serenity so um basically how to play is that you download the thing or build from source so that's us and then we need a mpq file from the original cd uh which i have already prepared and then we just need to install some sdl and sdl add-ons and then we should be able to run the thing now it's probably not quite that easy but we'll see um we have sdl-2 but we don't have sdl-2 mixer or stl-2 ttf so we're gonna have to port these um but i figured we'll just we'll just start by checking out the code and um we can build it on linux to see how that works and um then we will try to build it for serenity and and get it going so um it looks like a cmec project which is convenient for us because we already use cmake so um i guess we can just build here we'll use ninja and yeah so it already found all the things here and if we build it i'm not really expecting any trouble because i'm assuming the people working on this project probably i use linux all the time so um yeah we have our executable here and how do i run this um data dear diablo yeah okay so that just comes right on um hello buddy so yeah and i don't know if you can hear it but uh sound works as well so really really easy to get it going and um i think basically we just have to get started so if we make our own serenity build directory and then we'll run cmake again but this time we will specify a different c make tool chain file um let's see make tool chain um certainly arch not set okay and the compiler is not in the path sure sure we can fix that uh huh okay so could not find sodium what is sodium sodium libsodium crypto library um is this strictly necessary though if this static libs if not no net i like the sound of no net [Music] um disable network yeah um i i don't care about network um functionality right now because um i don't have anybody to play with at the moment so i think we can just uh we can just build it with the no net and then uh wait let me let me start that over okay so now we error out at cannot find sdl cannot find sdl okay so maybe we um oh maybe i don't have sdl here okay okay so we'll just build the stl port nice and easy okay so now we have sdl and let's run cmake again it still doesn't find sdl do i um how does it find sdl does it use like um let's see stl how does this thing work i am not good at reading cmake i'll tell you that right now but um [Music] find the sdl2 headers and libraries sure so i would assume that they use like um pkg config or something like that pkg config found sure okay so they look for that with a good config and uh this is unreadable to me so um [Music] pikachu config yeah so there's the pikachu config thing here so we just uh maybe we just need to um set pkg config path to uh root uh user live pkg config oh right and let's try again that didn't quite work [Music] um wonder why so [Music] let me see what our tool chain file actually says anyway so we specify a handful of things including system name what happens if we say serenity or os here i mean probably nothing system is unknown to c make okay okay so it has to be known by cement i guess that's why we were generic um okay what if we don't say anything if we don't say anything then it progresses all the way to stl to ttf okay i don't know why that helps but um let's ignore that and just move forward so let's get stl to ttf and make a port of that so [Music] maybe we can go directly to the boarding process and let's see here so our sdl port um you know what let's let let's not get carried away here let's let's look inside of it first okay so this is auto comp based [Music] so all we need to do is run with this host triplet and usually you have to edit config sub i like to put it after ultrix for some reason okay um so i don't think that it cared this was too easy because you can see here that it's using gcc instead of the actual compiler it should be using so that was not successful also it's like doing linuxy things here like it's finding my host free type and my host lib png and stuff um so let's try that again let's put this compiler in the path it looks here like it finds all the serenity tool chain stuff um okay and then he's upset because this thing is missing maybe we can um like whenever i build something i like to i like to always disable everything that i don't know that i need so is there something here we can disable just test programs and that sounds harmless um so what did we have with we could specify where sdl is because it seems like he seems to not find sdl i mean actually here look it's it's using our host installation of sdl so um it's also picking up the host png which i don't like i wonder why that is i don't really understand why but okay so with sdl prefix um we'll just point it to the serenity build root user and now it's built with the serenity compiler and everything is good except it can't find the free type because we don't have free type [Music] we can get that blah blah blah what is the latest version here 210 4 is that it um october 2020 that's that's good enough for me so let's just uh let's get a free type port going here okay and another configure so we can reuse the command um let's see this it doesn't look like the regular auto cough stuff so it throws me up a little bit but uh it's complaining complaining about the host triplet so we know that it's a config dot sub issue once again so builds unix config.sub um look for all tricks and after boom um wait did i do something stupid oh oh forgot to put the pipe character so again it's picking up stuff on my host because i don't have lube png or harf buzz or broadly um this is something that's frequently an issue when building ports is that i'm not sure what we need to do like it's always different from package to package how do you convince the build system of export to stop looking at my host machine and only look here in the serenity build environment like um porting software is kind of frustrating in that way when you're cross compiling like if we were in serenity doing everything in serenity this would probably just work well but you know um but since we're outside there's always this confusion between like my host linux system and the serenity um cross compilation environment that i wanted to build for but this thing is just detecting way too much stuff so is there a way to disable without with yeah right here so with zlip no with b zip two no with png no with harf booze and no and finally with bradley you know and look at that now this is much much more likely to work since we don't have any um dependencies hopefully we'll see because with these uh like free desktop type projects like it's just a web of dependencies i bet you that whatever you pull in like it's going to pull in something else and something else and eventually you're pulling in like lib gtk or something and then you're screwed because we don't want to be building gtk for surrounding but that actually seems to have worked so let's try to install it okay see it even installed the um free type pkg config stuff so uh we needed we needed this to build sdl2 dtf so do i still have that here i guess it was here okay what whatever um sdl2 ttf conf let's try to build it again okay so now we are missing all of these um undefined reference to various serenity symbols um that's fine so you can usually add libraries by just using either libs or extra libs um let's try extra libs so all the serenity stuff would be lg l graphics l um core lipcl compression uh that didn't quite work let's try just libs then i don't know like now i'm just guessing because okay that really didn't work wait now it's finding the host sdl again oh because i have to say with sdl prefix right i forgot about that okay so it builds let's install that and try to continue the evolution cmake okay so it found stl2 dtf awesome now we need sdl to mixer uh the website looks exactly like stl2 dtf which is nice i guess look at that that is some consistency in web design right there maybe they're using the same css anyway let's see so sdl2 mixer so this is the sound mixing library for sdl which for some reason is not not part of the main sdl distribution i i don't know why steel is split up like it is but that's neither here nor there so let's just look for config sub right away that looks like what we want okay and configure make and okay so with sdl prefix might be relevant actually let's just use the exact same configure command as we had for still two ttf uh complete with libs and everything um wait what so now this doesn't work because well it looks like it's making a library here yeah and then what wait what's actually failing though play wave so it's building some like command line utility but i don't care about that i don't i just want the library um can we just can we just um install this because it built the library so maybe we can just install it without yeah yeah yeah so we can we can allow that part of the build to fail and then now we have everything we need for evolution x so um wait you need this file but we don't have that file why is that we do have that file but it's a dev sim link i guess because we don't build sdl um we probably don't build it as a shared library i don't know we don't build any of our ports as shared libraries we should probably figure out why that doesn't work and start doing that you can see here we have static versions of sdl um also i don't know why sdl goes in user lib and other stuff goes in user local lib that's a bit mysterious um but what if we just uh what if what if this so file this misleading sl file would just disappear you know like would anybody miss it if we just cmec once again and we were actually able to build for a moment there there's a lot of warnings um so let's let's see what we can find so we're failing the build sure we built a little bit let's see how much builds if we just run with ninja k0 there's a lot of warnings about deprecated string copy functions stuff like that well that's not not the end of the world but then we have some some problems that apparently we have to deal with so let's take a look um error what are the errors here q is not declared uh-huh key t does not name a type all right well that sounds like a real problem okay wait so kitty is not defined if in death no net but we are in the no net so we don't define a key t and then you still have q kitties here hmm so i guess we can just hack this by doing like okay did that actually bring us past some of the problems i guess it did um and then oh large file was not declared sure we don't have a large file um and then stat 64. yeah we don't have these um so linux has these like 64-bit versions of things like f-stat64 um and we don't have that it's not actually in the man page f-stat 64. hmm but what if what if we just oops is that 64. what if we just um do a little let me just do a little thing here and um do this so stats f-stat 64. f-stat who's gonna know who's gonna know that we did that and that was oh large file stat64 [Music] off 64 tlc 64. okay so we'll we'll just um we'll just do this because we don't have 64-bit variants of these syscalls but or these types either but we can we can pretend that we do uh oh large file zero um yeah what if we just did that f truncate 64. and we'll just f truncate so that actually removes that problem well it bypasses the problem well you know that's okay um and then call to overloaded tostring is ambiguous all right where was that on line 20. so this call right here is ambiguous okay well i never uh don't normally use the stl so i have to look up what this is a numeric value to stud string sure so we have this thing right here converts a floating point value to a string with the same content as as as pertinfo would produce for a sufficiently large buff right but why is that ambiguous so in basic string we have all these candidates but none of them are good but we should have like that exact float variant thing two string um float it's right here sure but it's gated with this thing which i assume we don't set senior nights to die oh uh please relax feature check um let's see just uh this sounds like uh i mean it's a long time ago but it sounds like somebody being running into a similar problem here that i think so when you build gcc they do feature checking on your c library and we provide our own c library right like we don't use the new glib c so this guy here is using uc libsy which presumably also fails whatever check causes gcc to enable this thing so uh we don't get this feature i think because we're missing something in rc library so we could either figure out what's missing in our c library and add that or we we can press ahead by just hacking this a little bit so let's see value sure so we'll just um we'll just do something a little a little dirty here um nobody needs to know uh just between you and me and i guess we have to call this something else because value is a member and then same file line 42 set value all right so yeah this is the same thing again so we'll just uh we'll just take this and like set value f value uh paste that here and wow we are really editing the code here today wait are these hard tabs is gross but who am i to judge really i i shouldn't i shouldn't judge people for their choice and tabs it's none of my business okay um binding reference of type float amp to cause cons flood discards const qualifier presumably um because it's a const float here and this guy takes a float amp for some reason well you shouldn't take a float amp if you're not going to mutate the thing anyway so it seems like an easy fix um or is it wait wait wait wait this is a cbp file there's a header somewhere here hpp look at that what kind of library is this anyway radon never heard of it radon well we are messing a little bit with the key class of the radon library whatever that is and okay now we are missing something called copy sign and ooh exceptions they're throwing exceptions we don't have exceptions important serenity i i didn't even [Music] consider that that might be a possibility but maybe we can um maybe we can work around it somehow we'll we'll take a look but let's first do this copy sign thing so what is copy sign um returns a value with the magnitude of x and the sine of y uh huh so [Music] you take the sign from y and put it into x so if oh man um i guess it goes in math.h right yeah so um copy sign let's see if we can just do a naive implementation okay come on c-line you can do this um okay so um we want to copy the sign from y to x so if x is less than zero and y is less than zero return x just fine um and then if x is greater than or equal to zero and y is less than zero return minus x and then if x is less than zero and y is greater than or equal to zero return minus x and otherwise return x uh is that right i we'll we'll we'll put it into our javascript interpreter function paste um wait what was wrong with that like pasting into the javascript interpreter isn't not the most reliable thing like oh it's because it says double uh yeah so let's get rid of those and do that yeah why are you this way like why can't you just work um can can we make it work i just want to experiment with this function okay sure yup seems good um okay good enough for me there's probably some like uh some some clever way of doing that with fewer comparisons or whatever but i don't know um oh yes and then of course we have to rebuild at least rebuild our liv-m i guess we have to or does that show up if we just no it's not there yet um copy sign i've never seen that function like one of the funnest things about uh working on serenity and like uh building our own c library and stuff is um it's really showing me all of these things that are there on every unix system that i had no idea about like i did not i did not know about this function and i didn't know about next after either um or or just so many different things it's kind of fun yeah okay so this takes a moment for some reason i guess because a lot of people include math.how [Music] so far though this is pretty straightforward i just a little bit confused about the hacks to get get the cmake thing to work like removing that part where we say like this is a generic cmec platform or whatever like why does it work if we remove that i don't know but we're not gonna pursue answers to all questions today we're just gonna we're just gonna try to make the game run and then you know later we'll we'll have plenty of time to figure out why why things work or don't work so did we yeah okay so the copy sign thing is fixed and then we have throw statements so how bad is it oh well so we have third party asio throwing some stuff although not all of that was actually okay yeah so this looks more like a real breakdown so we have some throws in azio and then the majority of throws are in dvl-net and i'm actually building without net support so i don't know why this stuff is even why it's even building this but maybe maybe they build some of the netcode even if you're building with no net so yeah so they have like they check known at inside ddlnet so so probably like some of it is required even if you're building with no net so how about um we just take these throws and we just what if we just comment them out like uh who who's gonna know um this looks like we can't just well it's inside it from death definite actually so who cares um but we should probably do something like wrap it in one of those so that we don't break the if statement or we could you know we could even put um assert not reached or verify not reached sorry yeah that's what we'll do we'll just put verify not reached that's fine and um wait packet.h136 verify not i think i think we're probably never gonna hit this anyway because um because we're not going to use any networking features so but it will be good to see if we actually do hit it okay all right now we are failing to link so we're we need to link against uh you know libgui sdl ttf needs free type so how can we specify libraries here like libs can you say extra libs to cmake scenic shared linker flags um do they care about extra lips i don't know if they do that would be nice if they did i you know i guess we can just uh we can just hack it in here because they do this for various platforms already so we can just add our own stuff like if serenity also we should probably set serenity in our tool chain file um target link libraries right what am i trying to do um pre-type gui graphics core ipc compression also i'm not sure why why lib compress is called lid libcompression.so but for some reason it is oh and i did not set serenity in our tool chain file which maybe we should it linked it's here it's a 32-bit elf built for serenity look at that that is cool okay let's um install it i guess okay there's a high chance that this is not going to work but we'll see okay so we have an error at sourcex display line 216. sure um 216. create renderer fails uh sdl creates render with render flags well we don't support accelerated rendering that's for sure i guess they they always request that but let's just uh switch it up for renderer software and try again because we don't have gl or anything like that but um that still doesn't work wait what were they asking for more render flags i guess this thing here vsync what if we just don't ask for vsync because i mean we can't really we could probably figure out a way to give you vsync fake vsync but not nothing real so let's just um wait did i actually rebuild i don't think i did yeah we don't have vsync um because we use a very simple uh graphics driver at the moment ooh okay um so some issues cannot open this font okay um where did that go so we uh i mean it's not my responsibility to go and look there but what if what if we were just put that here would um okay the scaling looks a bit like ass but we can just press alt enter to get it into the windowed mode so also we have some some issues here with the um [Music] i think alpha blending is broken over sdl windows for some reason um right so we need the mpq file so moment of truth uh we will put this in my home directory and try again um okay so first off let's copy the font um oh right then i have to say like data deer on and we have a sec fault well that's progress so what happened failed to allocate anywhere oh look at that that's a huge vm allocation so it wants to allocate 493 megabytes of contiguous virtual memory and i bet you that aslr is just making that impossible that is my bad um yeah so if you look here at the address space because we've randomized the placement of all the dynamic libraries there just isn't a 493 megabyte hole anywhere in the address space so we cannot map what is presumably the mpq file right yeah it's 493 megs so they're trying to map the whole dang thing and we can't do that because of aslr so let's let's just disable aslr for a moment we can just ignore the randomized flag so we'll just uh save false don't randomize and rebuild our kernel and then copy the font um and then devolutionx databeer dot okay dude um okay the keyboard does not work but okay yeah here's the networking code presumably so there's a loopback connection so you can play with yourself maybe that's why they still build with net support so you can play the multiplayer game by yourself for whatever whatever reason i guess if you want to like make a character and grind it locally i remember doing that in the late 90s actually okay so why why can't i type a name though because like up and down uh works enter works but i can't type characters um let's figure that out so keyboard no wait what was that get keyword state keyboard controller is keyboard control button pressed but um where did they call that controller uh that sounds like like not like typing so sdo keyboard input uh so they have keyboard events okay so who does something with those nobody um steel key down maybe uh what are these things we have the credits the diablo ui i mean what is the diablo ui key down ui text input that is interesting so backspace or left seems to remove a character from the right side of whatever ui text input is i wonder if that is it's i mean it sounds like it might be the thing that we are editing the name with p item ui edit man this is some kind of old looking c plus plus ui text input sure so what is pi what is this thing anyway ui edit okay so the ui has a um list of items which i guess is like their cons the widget concept and ui edit is one such thing ui edit is a class or struct was it you i edit yeah here sure maybe i should have opened this in um sea lion actually i don't know why i didn't do that new project wait how do i open an existing project i should know how to do that evolution x c make list ui edit your item okay sweet sweet sweet sweet so here is their ui edit class which might be the text edit that i'm typing into um this doesn't really tell me much though so where is this used cell game cell hero so i guess this is like select game ui and this is the select hero ui so we are in the select hero ui i would guess new single player hero okay this is the class selector that we're building sure ui edit hero info dot name yes i want to edit the name but then what do you get events somewhere no do you get keys pressed somewhere no any sdl stuff anywhere okay okay calm down um ui edit so we're just adding it to this vector and then we're calling ui init list so this is the thing we looked at so for any ui edit item in the current ui they um do sdl start text input okay i am not familiar with that api what does that do start accepting unicode text input events um sdl text input i don't think that we generate these events in our scale port of serenity um we probably just make key down and key up so maybe we are supposed to generate sdl text input events let's get let's look at our port um start text input uh wait what start texting but no matches oh right because this is not a git repo um okay so haiku x11 coco they all do some start and text input stuff so what does the generic thing do stl start text input it will enable these two events text input and text editing sure and then if we have some text input thing um wait so what is the what does the event look like though like like let's look at what haiku does text input okay so this is the handle key handler and the haiku port of sdl so if they get an sdl pressed and the text input event is enabled then they will send keyboard text in addition to sending the key down event i guess so maybe we just have to call send keyboard text this will be like a little bit a little bit hackish but let's try it so in the serendi port we do everything in the video directory it's not a very well factored port and it's also really really far behind mainline sdl we should probably probably spend some time bringing it up to speed at some point key down event so here what what if we just send keyboard text or we can check do this check actually um if the text input events are enabled so if this then um i guess they just want the text so in serenity we have on key events we have a text getter which will give you sort of um um like a utf-8 string with a text so we can do something like this and um send keyboard text text characters i guess if text is non empty okay so like that let's rebuild the port and see if that works and because everything is statically linked we also have to uh rebuild the evolution x of course there's so many so many directories going on here that i um losing track a little bit ninja install let's say oh wait i don't think that actually rebuilt [Music] diablo man boom dude look at that we are in very good boys and girls let's talk to griswold good um okay so surprisingly smooth for what it is i mean there's like a little bit of um a little bit of laggishness to it but i can't tell if it's actually laggy or because it reminds me of the way it was back in the day please listen to me the archbishop lazarus the sound works perfect i'm here to find the lost prince the bastard led us into a trap now so the memory usage is pretty high and the sanctity of this place has been fouled this is awesome it is totally playable um the cursor is annoying so it's not it's not supposed to show you the mouse cursor obviously and also the full screen kind of looks really bad i don't think this is a scaling i bet this is sdl scaling um although what if we just turn down the resolution here and then go full screen yeah so this is sort of the way it's meant to be played [Music] and now it looks kind of ass because if the aspect ratio is all wrong of course but okay let's let's not get carried away playing the game um that cursor thing is irritating so let's see how does that work how do they hide the cursor normally um right so they're probably calling this api yeah so they're disabling the cursor somewhere um and i bet you that we just straight up don't care about that api show cursor let's look at what haiku does um no video okay wait where is the thing then dot c okay all right show cursor um so if mouse cursor shown changes then they call sdl set cursor which is also in mass.com sure [Music] um and then i guess you need to have something on your where where do we install the thing for this like show cursor they have haiku show cursor let's see how they've implemented that okay so let's do something similar in our port um um just do that and serenity and then um [Music] what do we really have like set cursor we have her window cursors in serenity that's kind of the closest thing on gui window cursor cursor cursor set cursor right so we say set standard cursor hidden and then that should hide the cursor i think so let's try to do that um so if you want to see the cursor wait what's our window is it g app how do we find a window um gui window m window the serenity platform window hmm wait i don't know exactly how to get to where we want to go also how did they install that thing show cursor mouse in it okay all right all right so they also have a mouse in it thing i should have been paying attention to that mouse in it so we'll make our thing here and i guess we can just um like stash a pointer to our window somewhere and knit video in it where is that it's right here yeah i'm just going to put these way up top ask kling about being able to query this from g window this comment looks pretty out of date okay well we're just gonna like motor forward here and and hack something together because i just just going to get rid of the cursor [Music] so cur cursor depth cursor i don't know what to make of these things video net sdl mouse in it sure we'll put that down here somewhere and you know what i'm i'm just gonna do something hackish like um gooey window g window and then g window set cursor gui standard cursor um because none is the default cursor and otherwise hidden at least that's what i thought oh it's gfx standard cursor um okay so that builds let's rebuild um devolution and then reinstall the system um wait segmentation violation again what is it this time moldy reference oh shh i guess i messed that up um how did i mess that up though uh show cursor so presumably they're calling here and g window is obviously null because i never set g window what was i expecting what was i expecting um so after we create this we'll just do this and window putter yeah right so that should work and then rebuild that and reinstall that and the cursor is gone nice cursorless man dude that is so neat so now it's just me and my hand cursor the way but the way it was meant to be played and it's a little bit laggy um the hand cursor i mean don't have any cpu usage really to whine about here it's like really quite low um so i don't know i don't know exactly what would be causing lag it might just be like um the way that they drive display updates or something would have to look into that separately but basically the game is now playable listen to me which is awesome so [Music] i'm very happy about that as you can maybe tell this place has been uh and i'm probably going to play a little bit but i'm not gonna i'm not gonna make this video about that so what did we really do here like we had to port um we had to port um sdl ttf and sdl mixer and then we didn't do much in serenity we had to implement copy sign in the math library and then we had to disable aslr i don't love that one um but to make it work we have no choice so that kind of tells me that we might need a way to um to launch an application with aslr disabled [Music] or somehow build something that can opt out of it which is a bit uh weird but um as long as we're on 32-bit like um if you want to if you want to map 500 megabytes and you have spread libraries all over your address space that's just not going to work so it is what it is and then the tool chain change right just setting that thing yeah and then of course we had to do a bunch of hacks in the devolution code base as well so um fixing this radon thing because of stood two string not working right and then aliasing a bunch of 64-bit file system calls adding a linker invocation help sure and then turning off accelerated rendering and replacing all of the exception stuff with uh verify not reached and adding a typedef here just to fix the build so that seems fine um that's pretty cool so i think this is probably gonna be it for today's video so there's a lot of stuff here to clean up but i'm not gonna i'm not gonna sit here and clean it up in the video because i need to take a little break and then i can think about uh what to do with everything but um this was fun so i hope that um you saw something interesting here today and thank you very much for checking out this video i certainly had a good time making it and um the coaster less man um ah what a great game this is and and i think i never actually beat the game at least not a single player i probably cheesed it in multiplayer but single player i never finished it so definitely something i need to do now that i can play it on serenity uh anyway that's it for the video so i guess i'll see you next time bye

Channel: Andreas Kling

Views: 60,224

Keywords: os hacking, os development, c++, c++ development, operating system development, operating system programming, c++ programming, serenity os, serenity, serenity operating system, hobby os, hobby operating system, programming stream, os programming, c++ kernel, osdev, alternative operating systems, operating systems, diablo, devilutionx

Id: ZOzZ8R4gphE

Length: 79min 39sec (4779 seconds)

Published: Fri Mar 05 2021

Reddit Comments

[u/AbleZion]

In an hour.

[u/ChrisRR]

Damn Andreas is a beast

[u/BrokenHS]

Now do TempleOS.

[u/Mgladiethor]

Any new gpl os?