Native Simulation Mode for My Language (Porth Ep.30)

all right looks like we are lives hello everyone and welcome to another zoizin session how about that but you didn't expect that to happen so uh let's make a little bit of an announcement and start the stream so let me bring up my discord server and uh the topic of today's stream is simulation mode for port force that's the topic of today so uh red circle well red car let's use red car then live on uh twitch and uh simulation mod for port forth and we're gonna give the link to the uh to the twitch channel which is the two slash setting and we're gonna pin everyone who's interested to be pinked there we go so um for the past several i don't know for how long but i think for the past month we've been actively rewriting um we've been actively writing porth in itself right so um port by the way is a programming language that we've developed developing we've been developing for two months um it's it's moved to gitlab so let me actually bring the gitlab thingy uh so yeah and it's come a pretty long way i would say so you can find the source code of this thing in the description if you're watching on youtube and if you're watching live you can find the source code in the chat i'm going to post it in the chat hello everyone hello camel hello i could do hello lonely little lucas say hello hello welcome welcome welcome so um yeah uh the original implementation of this language was in python well was it it's still in python and we recently like a month ago started to rewrite it in itself and uh we actually made a huge progress on that because now the the compiler is capable of compiling itself it doesn't support everything the original compiler supports but at least it can compile itself one of the features the original compiler had is a simulation mode it's more like interpretation mode where instead of compiling the source code it would interpret it well it would technically compile it to uh to basically bytecode of its intermediate representation and then interpret that byte code right so um and our uh port fourth implementation right the original implementation of port is called ports.pi and the new implementation is port forth right the port port implementation lacks the simulation mode it can compile but it can't simulate and i decided to leave that topic for a stream uh usually i work on the compiler like off screen if there is like some uh very like not interesting to stream topics like you know things to clean up things to tide it up uh so i usually don't stream them because it's not really that interesting to watch i tend to stream like major features that are important for people to know so and yeah one of those features is going to be the simulation mode so let's go ahead and implement the simulation mode for port port right how about that hello jess bella hello young beans and lean mana soimer poly insecure alex couch uh arch lure miss i hope i pronounced your nickname correctly hello hello bitstream welcome welcome welcome really glad to see you today how are you guys doing uh how are you guys doing cheers by the way [Music] all right so let's go um i'm going to actually fish delete this changes i didn't think i actually fished the latest stuff um i don't quite remember uh get clean fdx let me actually clean everything and i'm going to fetch everything i think clean everything was a bad idea um you will know why soon you will know why soon because i'm not going to spoil anything right so i probably have to recreate that that program yet again i forgot i was supposed to actually commit this source code as an example to the uh to the code base of porth but i forgot and now i accidentally killed it but i mean there is a way i can recover it so nothing actually lost um so yeah people already guessing what it is it's a thumbnail generator and port right so and the video about that i actually recorded that video offline uh right because it's what it was a rather quick video so and it's about to actually release automatically i scheduled it to release like in in two hours right so it's gonna be on the youtube channel in two hours right well i i can basically recover the source code from that video anyway and then commit it later so that's it should be fine uh so yeah okay so we have some some changes in here that we probably want to merge and let me see what's going on in here so uh here's the original compiler right the original compiler looks like this we can write a simple hello world let's actually check if our language still works because because who knows maybe i broke something while working on it offline uh and i don't have a highlighting in vim disgusting why don't i have a highlighting in vim for my language this is absolutely disgusting all right so i do have a support for this language uh for this um for this language in vim um so here it is and what i need to do i need to copy parse.beam here right so this is basically where i have to copy it so let me quickly try to do that i'm gonna copy him uh so editor ports dot vm into veeam uh syntax right do i even have any of these uh directories maybe i have to create them so i try to copy this kind of stuff it will say that such directory does not exist okay let's try to create such directory we're going to do make dear minus p meaning that uh if in the path any of the directories are not present they will be created so basically it creates par and parent directories as well uh there we go so if we're gonna try to come uh copy it right now so everything seems to be fine uh all right so now let me try to do hello the fourth right and it still doesn't work i think i have to do something like uh porth and it still doesn't work that's really strange did i copy paste it incorrectly i think i just copy pasted it incorrectly i think i'm straight up the pega um so what was that uh and i even lost the uh okay i see i think it's supposed to okay uh i should have actually created okay and i see i just forgot the tilde or something right yeah damn it uh i'm so bad at uh console at linux console uh so let me now try to copy paste this entire thing it's gonna be editor ports dot theme and it's gonna be veeam syntax maybe that thing exists in there uh all right so that means i accidentally accidentally created this thing i forgot to let okay so i just have a little bit of a performance anxiety for some reason even though i've been streaming like for five years already uh right now for whatever reason i just have a little bit of a performance like that uh so um now if i try to do something like this and a set ft porth yeah finally it can highlight something i'm not sure if it's a good highlighting but it's at least something right so it's gonna be hello uh world right hello world and then puts and yeah so maybe this is because it's a console beam if i used something like gvm it would be a little bit better uh yeah it's a little bit better but i cannot scale my text right here we go so we have a hello world if i try to compile this hello world uh with our python compiler it will generate assembly it will uh feed that assembly into nasm and then it will link it to the final executable and if you try to run it here it is it says hello world but it doesn't have a new line i keep forgetting the new line every time uh so let me quickly just fix that i have no idea why i'm using beam by the way so i could i should have just run emacs i even have a emacs running so what's what's wrong with me i don't understand in any case here is the hello world so um yeah and the rewrite of the compiler is located in pors.porth so here is the rewrite of the compiler it's a relatively big file uh almost 3 000 lines of code and all of that in porth right let's try to compile that compiler with ports.pi right for that pi i'm gonna try to compile it fourth uh the fourth uh and uh it will take some time right uh there we go and if i try to run the final executable as you can see it created a native executable for the compiler right so it's not a script anymore and if i try to run it it basically says that you have to provide some sub commands we can try to now compile it with itself right i'm compiling the compiler with itself and as you can see it is uh successfully compiled it uh you know with itself right one of the things you can't do right now you can't simulate it right if you try to simulate it it will say that some of the operations in the simulation mode are not implemented specifically function calls right so there we go so and here it will say that it tried to call this function but since the function calls are not implemented in the simulation mode it couldn't do that right it could not do that and our goal would be to just like go through all of these like missing features so we have basically a huge switch case right so we have an immediate representation and we have a byte code of intermediate representation so what we do we just go through the byte code of the intermediate representation and we have a huge switch case if the byte uh represents this separation we do that if it represents another operation we do another thing and so on and so forth right so we have a huge switch case for all of the possible operations of intermediate representation and as you can see in the simulation mode some of them are they're missing so what we have to do now we have to just try to run the simulation and every time something fails we have to fix it and we have to keep repeating that until we implemented the full simulation mode so that's basically the workflow that we're going to be using today does it make sense it makes sense hello hello everyone welcome welcome welcome so does anyone have any questions maybe about what we're about to commence all right so in the meantime uh i'm gonna just go to the place where i was supposed to perform a call and i suppose we need to perform the call as far as i can understand the call is rather simple so the operation the operation has the place where you have to jump the execution and on top of that you have to save the current place the current address of the execution into the return stack so when you hit a return you know where you have to jump back right so we can even take a look at how it's implemented in port that pi so def simulate a little engine linux and so op type call right there we go so we have some sort of a return stack we append uh the current instruction plus one so we're going to return to the next instruction after the call and then we're just simply jumping to the to the address of the procedure that we're calling so we only have two things the problem is in the simulation mode of fourth port we don't have a return stack so we'll have to implement it uh super quick super super super quick let me see where is the simulation of our nation simulate ops so here is the simulate ops and here we have like a section of the code uh and here we have a stack but this is the data stack right it's a simulation data stack we need some sort of a simulation return stack that's what we need uh simulation return stack so do i want to create like a separate capacity for such a stack maybe maybe not all right so maybe yes let's actually implement separate capacity right so uh simulation uh return stack capacity and how many uh elements of the return stack we want to have usually by default i just say 1024. if we need more i'm i'll just prolocate more [Music] okay so the next thing we need to do we need to allocate the memory for the return stack right so let's create simulation return stack and we're gonna store u64 integers and sign 64 integers on it and multiply by the the capacity of the return stack there we go we also need to keep track of how many elements we have on the stack right so this is going to be simulation return stack count size of u64 there we go and we probably need two operations like similar operations like in here simulation stack push and simulation stack pop but for the return stack so i might as well actually just copy paste that stuff right so i think this this is gonna be the easiest thing to do uh at least for now uh i'm gonna just query replace a sim stack uh right so this is gonna be sim red stack uh there we go we have to be a little bit careful because here we also have to do sim return stack uh return return stack overflow in the simulation mode there we go so that's the operation that we're going to use to push the return stack on uh to push the return address onto onto the return stack so we need to do the same thing for the pop right we need to do the same thing for the pop i'm gonna query place sim stack uh sim red stack there we go uh i don't think i have to update any capitalized constants in here so here i can just do return stack underflow in simulation mode there we go so let me see if now we still compile if we didn't make any mistakes to be fair i want to compile with the python compiler because python compiler type checks things right so por that port does not type check anything right now so it's not really suitable for like serious development uh but poor's that pie is actually type checking things okay so are you going to keep the compiler source in one file besides std or will you eventually spread out different parts into different files i have no idea it's like asking are you gonna use uh six letter variables in the future or you're gonna stick with the five layer variables depends on what i needs it's just like so why is that important anyway does anyone have any other questions um [Music] um so we got a some sort of a donation i didn't quite and see it something with the stream lapse to be fair uh a second so arch lua or oramis donated five dollars and z i don't know what they are but thank you so much for five dollars and z uh thank you for everything i really enjoy watching your streams all the best thank you thank you so much for the notion um as well so will you support shebang for using port as a scripting language here's the thing it's not up to me to support shebang right uh so she ben she bank doesn't work like that essentially uh i don't know who exactly interprets she bank does anybody know who exactly interprets she bank um well the only support that is needed from my side is probably uh like a supporting hash sign as the uh as the comment if that's what you mean right so so essentially um it is not me who's interpreting shebang it's the either shell or the colonel or something like that right so the only thing i need is just these things i don't have to do any special so support right so i don't have to do any special support like it's just like it's already supported unless like uh but adding the hash sign as the comment is such a trivial thing that like it's it's literally five minutes like why do we even have to discuss that um i i don't understand the question and i'm sorry i think i'm being stupid but it's it's too trivial i don't understand um [Music] [Music] does anyone have any questions about non-trivial features of of the language um all right so let's actually continue developing simulation all right so let me let me see proc simulation simulate oops so uh all right so we implemented the return stack and the first thing i need to do uh all right so let's actually try to run uh you know pors.porth with the simulation right so this is going to be for the port uh and the first thing i want to do i want to grab the current ip and i don't quite remember where the current ip is located so is that a simulation ip yes it is a simulation app so i can take a simulation ip uh read it uh as a number and then increment it by one right and then i can try to push it onto the return stack so simulation returns stack uh i can't see and dismissed return stack uh push right so this is gonna be push we pushed it on the onto the return stack after that i need to take the uh simulation operation and i need to take its operand right i'm taking its operand and i need to read it as a number so then i can save that operand as the current simulation ip there we go so that way i believe right we push the next address to the return stack and then we jump to the whatever address we have in the op call right so let's go ahead and just run this entire thing and it still says that this thing is not implemented this is probably because um okay so let's actually set up the following thing we're going to recompile for the port right then we're going to run it uh right so it will take some time and then we're gonna try to simulate it with itself right that way we're constantly recompiling the compiler and trying to simulate it with itself so okay so we went pass up call uh now we need to do the preparation as far as i know preparation is only needed for for the compilation right so because you need to put something like switch up the stacks and prepare the return stack and stuff like that in simulation mode i didn't think we need to do anything in here and in fact import.pi uh import that pipe um so oh yeah this one is really interesting so we're allocating some memory uh in the local memory region all right this is actually very interesting huh that's pretty cool um so that means we need to have some sort of a local memory thing game [Music] all right so what's the size of the local memory that we use for the uh for this thing simulation local memory capacity so we use 640 kilobytes in the python right 640 kilobytes in python so probably have to use the same size or maybe not i don't know we'll see we'll see um so this is the return stack and let's do something like simulation um simulation local memory um local memory capacity right so let's actually use 640 one two three there we go so let's allocate that memory simulation local memory and it's going to be just like this capacity usually i don't call them capacity usually i just call them cap right because it's a little bit shorter [Music] when you finish the compiler in port and remove the python implementation how will you distribute the compiler binary i think i explained that multiple times throughout several videos already and i think this is the last time i'm gonna actually explain it so the way porth works it generates an assembly file right so we're gonna basically commit that assembly file uh to the to the repo and if you need to bootstrap compiler the first step you do you compile that assembly file yourself manually with nasam then you get a compiler executable which can basically bootstrap itself after that so i actually explained the whole process i even remember the video where i specifically even showed i simulated that process here's the video youtube actually listened to us youtube knows what the what is the video i'm talking about right so i'm not answering that question ever again every time somebody asks this question redirect them to this specific video this is where i explained how the porth will be distributed how it will be bootstrapped so i'm going to actually put that in the description as well so i think let me actually see i i think i can even find a specific uh a specific time code where i even showed that uh yeah so here it is and that library so i'm gonna copy paste the standard library and standard library consists of a single file basically std.port right so roughly you're gonna have these things uh right and you don't have to have the port compiler so the first thing you would do you would compile the port.osm ff64 uh pours dot asm right you will generate the object file uh so it will take some time okay so here's the object file we're gonna link the object file into the final executable uh pors.o uh there we go and we've got the porth compiler right uh we can try to uh run that pause compiler right there we go so here it is and after that with that port compiler will be able to recompile it with itself and there we go so we just bootstrapped the compiler right so this is roughly how we're gonna this is roughly the workflow that you're gonna have in the future for this specific project when we get rid of the python okay so i think it's a pretty good explanation in my opinion so i think i need to even save the time code right so right so roughly you can so it's around a single file basically std.port right so and what's cool about youtube what's cool about youtube is that you can uh click right click this thing you can right click and copy a video url at the current time all right so and you get a url that you can just like you know share with somebody uh so i'm gonna go and just put this in the description how the um how port is going to be bootstrapped after complete removal removal of python and here is the link so um do we have something like a bootstrap command let me actually see bootstrap uh we don't have i'm gonna create the command at cmd bootstrap right so i'm gonna put this link in here so every time somebody asks how you're gonna bootstrap the compiler without python give them this specific link please okay thank you very much cool uh so cozy white bam thank you so much for 11 months of twitch from solution thank you thank you thank you uh did i do the it up okay so it's fine uh thank you thank you thank you and welcome to our epic ports club um all right so i think everything's okay [Music] all right so let's continue let's continue so i'm gonna save the description [Music] i'm gonna save the description um all right so this is the simulation local memory or something like that um uh so we also need to allocate some sort of a variable that keeps track of the um of the rsp basically the top of the stack so essentially uh in terms of local memory in the simulation mode we just grow the local memory towards the zero the similar way x8664 works because it just like makes it similar like makes it easier to um to keep the environment consistent between the compilation mode and the simulation mode right so that's why we need like an extra variable to keep track of this kind of stuff okay memory uh simulation uh simulation local um memory rsp size of u64 um so here is that but when we start the simulation we have to initialize the rsp with um the well you basically have to set it to simulation local memory capacity right so let me actually see let me actually see uh but to be fam we actually have to set it to simulation local memory uh plus the simulation local memory capacity because this thing is supposed to be a pointer that's right so this is actually particularly so it has to be a pointer so we get this value then we take simulation local memory rsp and we save that pointer there there we go so now we keep track of the uh of the rsp pointer it's a pointer that points at the top of the stack um [Music] so we need to go into the preparation uh preparation procedure all right uh so and in here so the cool thing is that i can always look into the python implementation and just steal something from there uh op type or p preparation is it uh or p it's actually not prefixed with a p prep yeah there we go so essentially i need to take the uh operand of the current operation so i have to do sim op op operand right and i have to read it so here is the operand then i have to subtract that operand from rsp so that means i have to take the local um actually simulation local memory rsp i have to read it as well there we go so here is that value here is the second value and i subtract this value from that one and i have to save that value back into rsp right i have to save that value back into rsp like so but here is an interesting thing i want to have some sort of a check um [Music] and you want to have some sort of a check that this thing does not uh sort of overflow the local memory stack if you know what i'm talking about so i think this is the only place where we're gonna like allocate this kind of memory so i'm not gonna extract this into a separate procedure because of that so i think i can put that uh check right here so essentially if um not really here simulation local memory capacity rsp i can read that value right i'm reading that value uh and then i need to compare it with the simulation local memory and if for whatever reason this thing becomes uh less than this one actually we have to read this one as a pointer right so it stores the pointer and this thing is also the pointer if at any point the top of the stack becomes like lower than the beginning of the memory that means we hit an overflow uh right so and let me actually print an error for specifically for that error uh so local memory um local memory capacity exceeded in the simulation simulation mode it's kind of interesting right in the compilation we use return stack for both return addresses and the local memories but in the simulation mode for whatever reason i decided to actually separate those addresses i don't have any specific technical reason to do that it's just like it was easier to do that all right so it was just easier to do that [Music] okay so and in here i probably want to exit uh with like non-zero exit code in case this kind of stuff has happened uh all right so maybe i also want to um show where exactly this error has happened uh right because we know uh we have an operand right we have an operand and inside of the operand we have a token uh right so we can do sim op i can take the token from which the operation has originated and then from that token i can extract its location right and i can print that location uh to the standard output right like this and then i can say this happened like here um note the error has happened here right and then we're going to do it like this [Music] all right so uh looks good so we pre-allocated some memory and uh then after that uh then after that uh prep proc right prep proc uh we just go to the next uh instruction right so we just simply increment uh the sim ip hopefully hopefully that will work hopefully that will work [Music] okay so let me try to come well i mean okay so i want you to compile that [Music] rv is not implemented wow oh okay all right this is actually pretty um pretty interesting so do i really want to jump into like argue right away maybe we should try to simulate like simpler program first right so let's not try to simulate the camparo itself uh in the examples we have a lot of like different programs like bubble sort cat checkers fibonacci i think we even have hello yeah so let's actually start with the hello world right let's try to simulate hello world first um because a similar to the compiler itself it could be actually a little bit overwhelming so hello world okay all right so now we need to implement push str so here we just have to push the pointer to the string so what's interesting is that the operand inside of the separation contains an index in the array of string literals right ester lit where is that array so i think it's uh somewhere in begin str lit is is it a string where is it i don't i don't see it begin str and it's missing so there is the str buff okay so there is a str buff but where is ester elite cstr elite oh because it's capitalized right emacs for whatever reason couldn't find str lead because it was capitalized i thought it has a like case insensitive search anyway so essentially the operation of pushing a point or on the stack pushing a string on the stack contains an index inside of this array so essentially what we have to do we have to take that index and get the string literal by that index um right and then we have to push that onto the stack so let's go ahead and do that uh so this is located here and this is what we're trying to do okay that's cool um so i'm gonna take this uh simulation op op operand here is the operand we read it and we got the index right so then i need to multiply it by the side of the string because the uh the array of string literals contains uh strings right so that's why i need to multiply by that and then i can need to take esther lead i think it's called ester litz right and i add this thing and now on the top of the stack on the top of the stack i have a pointer that points at the element inside of that array so here it is here's the element inside of that array so and now i just have to this one is interesting right i have to put that string onto the stack right after i put that string on the stack i have a pointer to the beginning of the string and the size of the string so now i have to take these two values and push them into the simulation stack right so it's quite interesting so i have my own data stack and i also have a simulation stack of the program i'm simulating uh right so that's actually very interesting i suppose like and if i just do something like simulation stack uh push right that will push a pointer onto the stack on the simulation stack and then it will push the uh the integer right and it will work uh now we don't even have anything wait a second it doesn't accept oh okay so the simulation stack actually um actually accepts only integers not the pointers so before you can do that before you can push the pointer um right you have to cast it to integer right so let's actually cast it to integer there we go so as you can see now we consumed all of these things uh but when you pop elements from the stack and push them into the other stack they will be pushed in in a different order so we have to do before pushing them i think we have to swap them right so in here uh we'll have the order of pointer integer right but before doing that we can swap them uh and now it has integer pointer and then we're pushing them into the simulation stack and there we go we consumed well and because of that i have to cast it in a different uh place there we go uh okay that's pretty cool so for this who doesn't know uh this is a special operation i think it's just like a stop um stop operation basically what it does it instructs the compiler to stop the compilation here and show the current type stack the stack that it uses for type checking right so and it's actually very convenient for like inspecting the program like at any point of the program you just you can just say what is the state of the stack here can you tell me what's going to be the state of this stack here without actually executing it just show me the types and where they originate and it will stop the calculation and will just tell you which is rather convenient right so it's basically says okay so at this point on the stack you only have an integer and that integer was pushed by this separation well it originates from the multiplication uh right so you can now put this thing in here and as you can see before the multiplication you had two integers and you can see where those integers are coming from so this integer comes from a read operation from the memory and this integer comes from this thing so basically by uh you know placing these question marks you can trace different data datas in different like um you know execution paths uh which is rather convenient i think it's a pretty cool invention that they came up with force doesn't have anything like that right so as far as i know in fourth there is really weird philosophy in fourth is that if you need any tools to debug or inspect or analyze your code in fourth your code is bad something like that and somehow that justifies not having static typing not having a debugger not having like any tools that help you to analyze this thing it's it's really weird like i briefly looked into the philosophy of forth and it was really weird right so if you need any tools that actually help you as the developer your code is bad and you should feel bad you should throw it away and that justifies not doing anything well i guess it's some definition of a bad code so i've seen worse definitions of bad code throughout my career but it still sounds really weird to me maybe i misunderstood something right maybe somebody could enlighten me on the fourth philosophy but i couldn't understand that um did they make a confusing programming language do you do the dynamic typing no it's the programmer who's wrong exactly um alrighty so i think this is fine now um right [Music] this is this looks readable look at that uh concatenative languages were a very good idea this is absolutely readable i love it uh anyways so let's actually try to run it once again [Music] okay so this thing is still not implemented uh push weight what did i implement then i wait a second did they really implement sister i don't remember implementing sister excuse me uh all right okay so i think i actually missed a little bit and i implemented okay so sister is supposed to be implemented slightly differently okay the implementation has to go there all right i'm really glad that i have like extensive amount of checks that constantly crash the camper and stuff like that because it saves my ass so many times um okay so i think this should actually go here okay data stack overflow in the simulation mode cool i think this is a classic who can guess what i forgot who can guess what i forgot any ogs in the chat who was who were actually following the development since the beginning what did i forget [Music] iplus one exactly i so here is the thing uh like ap plus one is such a simple problem you just do ip plus one at the end of the iteration right at the end of the iteration but you have to do iplus one in each of the branch of this which gains in a switch case unless you don't so there is like a like a five or six branches where you don't have to do that and it's just like uh i don't know so and it's kind of weird so uh let's go into the proc sim uh simulate ops uh op uh push str um op push str and in here i suppose i have to do this thing right so i increment the simulation i i p and i also have to yeah okay so in the preparation i uh also increment it [Music] okay she's called three oh my god our simulation mode tried to call a cisco holy and unlike in python this is actually super easy to implement in here right so essentially we have to just use this c school three uh you know intrinsic right so basically in the simulation mode we can just reuse the intrinsic of the language itself of the language that we're simulating and that way simulation will be able to do all of the things the compilation can uh which is kind of cool and it's probably going to be even like as fast as compilation maybe not no it's not going to be as fast it's going to be like a little bit of the overhead of simulating everything and stuff like that but it's definitely going to be faster than the python simulation that's for sure that's for freaking show [Music] zelen zero hello welcome to the stream so okay so let me see how we're gonna do that here is an interesting thing so essentially c school three accepts four arguments right it's it's kind of counter counter intuitive but it does accept four arguments so first three are the arguments of the cisco and the fourth one is the cisco number right so yeah so in even cisco zero accept at least one argument because it accepts the uh the number of the cisco was the name port inspired from porn yes the answer is yes so this is the official lore this is a canon uh porth comes from porn anyway so um cisco what we need to do so we have four arguments on the stack we have four arguments in the stack we have to move those arguments from the simulation stack into the data stack and just call cisco three right we can do something like sim stack uh pop right and we have to do that uh four times right we're doing that four times uh but we can't just do c school three we can't just do that because in the simulation stack there they are in one order um and then one once we pop them into the data stack they're going to be in a reverse order right so and i'm not even sure how we can sort of reverse them let me see so in the simulation stack uh right it's going to be a 1 a 2 a 3 and then the number right so this is how they're going to be in the simulation stack sim stack right and then data stack my data stack once i start doing pops um so i'll pop number first so here goes the number then a3 then a2 then a 1 there we go so and i can't call cisco 3 on that because like this is not a correct argument so i need to somehow reversal of that i wonder if there is any way where i can just actively reverse things like actively keep reversing things but i don't think so i don't think i can easily do that maybe we'll have to introduce some sort of like a third step maybe it actually depends i have an interesting idea so we don't have an arbitrary access to our own data stack but we do have an arbitrary access to the simulation stack because the simulation stack is just like a region of memory where we can access things at random what we can do here instead of using simulation stack pop we can directly get the elements from the simulation stack and put them in the whatever order we need right so i think that would be a great solution that will scale to even six arguments right basically check if you have enough arguments on the simulation stack and get those elements directly from the stack just directly get them and then just pop them out of the stack or something like that um [Music] okay so i'm thinking maybe we could implement some sort of like a simulation stack operation which accepts how many arguments uh do you want to pop off this stack and map them into the uh into the data this is actually a pretty cool idea actually god dammit our type system won't allow us to do that right so i won't be able to create the procedure that will return a variable amount of the arguments based on the input right so here is the thing here is the thing so porth what i want you uh so the thing that i want to do requires the following thing uh simulation stack uh i don't know pop and it will accept an integer and it will return the variable amount of integers based on the input if the input was 5 there's gonna be five of these things in here our type system isn't capable of doing that yet in the future we're going to have a lot of interesting things about variatic input arguments and variatic output arguments we're going to do all sorts of things that can be can be checked at compile time once we implement everything in port of course but for now we have to live with a very simple and very dumb um and very dumb type system so uh i'm just saying so i think i want to make a small break because i need to refill my water and after the break uh we're going to continue and we're going to try to implement this thing where we can just like take three elements from the simulation stack and put them in the same order onto the data stack so then we can do the cisco three right and whatever we developed for that will be useful for cisco four cisco five cisco six and so on and so forth right that makes sense all right let's make a small break um all right so let's continue uh let me think how we can access all of those elements right so simulation stack pop i need to find the procedure simulation stack pop right here it is uh so here we take the the count and count points to the element after this entire stuff right so this is essentially how we have it a b c d and count uh points somewhere here right if i subtract one i get that argument so we had we got some close 76 um subscribed uh for five months with tier one subscription thank you thank you thank you and welcome uh back to our epic port club uh thank you really appreciate it uh all right so uh now let me think um so if i do minus one i will access this one so essentially what i need to do uh i need to um do minus four and if i push this thing uh i'll push it yeah so essentially i have to start with minus four and uh push the elements like stat going forward or something so simulation stack but before i can do that i need to check that i have enough uh you know elements on the stack in the first place right so the first thing we want to do we want to take a simulation stack count we want to read it and we want to check that we have at least four elements there so this is gonna be four uh greater or equal right uh actually if it's less than four we're gonna say the uh data stack underflow in the simulation mode that's basically what we're gonna say data stack underflow in the simulation mode yeah that's fine i was still thinking maybe i can come up with a better solution but i don't really see it uh okay so this simulation on the floor um all right so and in here uh what i have to do is just like i have to take this thing uh 64 and then i have to subtract uh subtract four if i subtract one i get that one uh one two three four so i just have to do four minus then i have to multiply this entire thing by the size of the element of that stack so this i need to find memory simulation stack all right the elements have the size of u64 so i have to multiply it by u64 and then i have to add the simulation stack in here so that way i get the elements in here and then i just read that thing all right so then i just read that thing uh interestingly enough right i can sort of like straighten it up and duplicate this entire thing four times right so and then i can switch these elements so here i'm uh copying the fourth element from the from the simulation stack on the top uh then the third one and then the second one and then the first one and they will come out at the correct order hopefully um yeah that's basically how we're gonna do that so after that after that after that i will perform this cs call right and the cs call will uh essentially return the cisco will essentially return the the code the return code from the kernel so before i can do that i need to remove all of these four elements from the simulation stack and i think i'm gonna do it like this so it's gonna be simulation stack drop um and we're gonna also drop it like that one two three four right and also let's delete training white spaces uh okay and then uh i will be able to push the return thing into the simulation stack right let me actually double check so here we should have only a single element on the stack well at least on the top of the stack we should have whatever we returned for whatever we got from simulate uh cisco 3. uh okay so this one should be called pop uh right this one should be called pop and as you can see on top of the stack we only have an integer that originates from cisco three right so and uh the only thing we can do with that thing is just like push it into the uh into the simulation stack right so and that's pretty much it i think uh all right so i think it performed this simulation look at that it already performed this syscall but it still failed because we didn't remove this uh you know this assertion right so essentially i think i think it works i think our port compiler is capable of interpreting hello world about that that's actually pretty cool uh almost it managed to print hello world right but then it needed to return from the procedure that printed it right so it actually went into the right cisco uh which performed this is called three but now it had to perform the end uh ip yeah thank you for reminding me about a p plus one that's right so let's not forget ip plus one uh increment 64. there we go so this is the way we're gonna implement the six calls i think it kind of makes sense to actually put the drop somewhere here uh if that makes any sense oh my god what the is going on emacs is going freaking crazy uh okay so let me delete trading white spaces and what i want to say yeah so essentially we're gonna get those things from the stack we're gonna pop them right away and then we're gonna call the cisco and then we're gonna push the result of the cisco onto the stack onto the simulation stack all right so um [Music] so and then we can copy paste like a similar solution to all of the uh cisco uh implementations i think it's gonna be pretty straightforward uh anyways so let me recap one more time and something went horribly wrong what did i do wrong what did i do wrong yeah okay so i should have actually dropped those things right you have to drop them right away because pop actually pushes it onto the data stack all right so and so that's really funny return stack on the phone uh all right so what is going on holy am i going crazy what the is going on um so i did ip plus one then op return where is op return uh let me actually find that because it's supposed to i think i accidentally removed it or something already oh boy um oh peer return right it's supposed to exit when it tries to return but it kept doing this thing and cisco three uh cisco three where is the cisco three uh we do increment this thing right so everything should be fine is that because i moved this thing before this is called is that what's happened i don't quite understand why does it act like that so i made this small mistakes this small mistake and now it just like keeps doing this stuff and i don't understand why did it accidentally removed the oh my god i think i think like i pressed something in emacs and it deleted something that it's not supposed to delete and now i don't know where it is so i have to look at diff uh just to see what the hell is going on uh all right so here is this kind of stuff um so here is the sim ip plus one uh and in a call in a call it's supposed to have sim okay so i think it removed sim ip did it even have one i don't remember if i actually incremented the instruction pointer at the call well it's not supposed to do that why is it why is it keep repeating i'm so confused it's like what did i do wrong um so and so i also get messaged right now so all right so let me see oh boy uh i love software development um so what is going on why is it keep repeating this thing it's not supposed to keep repeating it right it is not supposed to keep repeating it so if i put something like here uh to do blurp right to do blurp uh he puts right he puts he puts one exit oh boy all right so um let me see okay so here is the blurb it didn't really do that properly uh okay so then um so we did one hello world and after that it will probably go right it will probably go into the op return right it will probably go into the op return and in op return where is op return i don't see it let's find op return you know peer return it's supposed to fail but it didn't [Music] instead of doing that instead of doing that it keeps repeating hello world right until return stack overflow in the simulation until a return stack overflows so it keeps calling that thing so i'm so confused okay so uh cisco three and what's funny is that it just worked and it failed at the op return we saw that like a minute ago or something like several minutes ago and then i just switched something around in here because i wanted to restructure things and it stopped working and i don't understand why like it it failed at the right place it should fail at op return but now it doesn't this is so weird uh and i don't understand it is it's just like it's not supposed to do that um so like where did they forget okay so in a in a push string okay i do see my ip in a preparation i increment everything in the c school i increment everything in a call i'm not supposed to increment everything anything because we're supposed to jump into that thing right we're supposed to jump there we keep jumping there like alright so uh let me let me actually remove this entire thing right so i'm gonna just go ahead and uh remove this implementation right so we're gonna start over uh almost right like kind of start over uh and then we'll see what's going on all right okay so it fails at cisco three that makes sense right so that makes sense we're starting over uh then i'm taking the simulation stack um simulation stack count all right i'm reading this entire thing so this is going to be 64. uh and i'm just checking that this thing has at least four if it has less than four right so i have to do the following thing here um so sim stack pop right so i have to fail with this thing in here i have to fail with this thing in here um so here is the simulation mode cool uh then i take the simulation uh stack count i read this entire thing right so i'm just reading this stuff uh then i subtracting four right i'm just subtracting four after that i want to multiply it by the size of this thing and i'm gonna add the seams stack here right so that's the um the point that i want to try to achieve and here i'm reading this thing yet again um okay so that makes sense i think all right and i'm gonna do that four times so this is gonna be three two one uh and if i take a look at what uh what happened here right we have uh four integers that we got from the simulation stack right so all of them are in here right all of them are in here uh so then i want to perform the c school 3 and the cisco 3 actually returns that integer in here okay so the next thing i want to do i want to do simulation stack pop and i'm going to drop it four times uh right so after that i'm still going to have that um you know integer and then i'm going to do sim stack push right and i'm going to have no any elements on the stack so then i can simply do simulation ip increment 64. so i just recreated the entire thing i just recreated the entire thing uh it should say hello world one and uh hello world and then it will fail in here uh so now if i remove this thing did it hit a return or did it not did hit return or did it not i'm extremely confused um i can just go ahead and implement the return i suppose but it's so weird like what the hell so why does it not hit the return why it does not hit the return [Music] can we trace the operations somehow so it keeps repeating everything why would it keep repeating so if i uh remove this thing if i don't increment the ip it hits the return and hits it in here who increments the ip the is going on uh excuse me uh what is going on like why uh cisco three right what it's it doesn't make any sense i don't understand like why um you don't have to increment it you have to increment it don't you i just played myself i hate that iep plus one problem i absolutely hate iplus one problem okay so here's the thing uh in the intermediate presentation we have operations different kinds of operations okay so let me actually show you these operations it's it's so dumb holy the entire problem was extremely dumb so here are all the possible operations of the intermediate presentation you can push an integer push the local memory global memory strings if else's blah blah blah and one of the operations that you can have is intrinsic and intrinsic is basically hardcoded function so here's the thing intrinsics by definition don't change the control flow of the execution so because of that they never have a situation when they directly assign ip so for any intrinsic for any intrinsic once you executed the intrinsic you can just increment by one and because of that for the intrinsics i actually moved ip plus one outside of the huge switch case and yeah so basically you have to do iplus one for each branch of the switch case unless you don't and for the special operations you never have to do that it would have been better i think at this point it would have been better if i just like updated ip on each individual branch regardless of something of something being compressible or not right regardless of something being compressible so uh yeah all right so i have to it doesn't really matter whatever um but it's so annoying this is such an annoying problem like it produces the code that is not compressible it's a like right because you always have to do that unless you don't right and it's always like that right so it usually you may think that a particular repetitive code is easy to abstract away but it's always like every instance of the copy paste code is slightly different it's just like slightly a little bit different and it makes it almost impossible to abstract it away and you just have to copy paste it so not all of the code is easily compressible and this is something that you have to deal with like all the time right and since the cs grad students are so afraid of duplicate codes when they encounter this situation when you have to duplicate code but slightly differently they get stuck because the professor told them that you're not supposed to copy paste but in the wild you're expected to copy paste and they don't even know what to do like the professor says one thing and the industry says another thing and and that's how they get stuck right and it takes years and years of the experience to actually get unstuck from that mindset that you're not allowed to copy paste and actually start copy pasting things and getting done uh so it took me like 10 years to actually get to that point it's really difficult right um it took me 10 years to get rid of the fear of copy paste imagine that that's how long it took me right because like for the 10 years i was afraid to copy paste code because the code has to be compressed don't repeat yourself keep it simple stupid don't repeat yourself and other like hundreds of other like you know solid like uh abbreviations and it takes so much effort to get rid of that mindset and finally start doing things uh all right so anyway um so you don't have to do that thing in here and um yeah i'll have to keep that in mind so here are the intrinsics and um so here's the op i don't even know where the intrinsic start okay so here they here here's where they start and then okay um now let's go so it should fail at op return right okay so let's go ahead and implement op return um uh op return op type uh return so here we just have to take the uh local memory rsp and add the operand right so local memory it's actually simulation local memory rsp we're reading this entire thing then we take in the operand uh op uh it's actually simulation op op operand right and i'm reading that thing and i'm summing things up and then i'm saving the result back into this variable right i'm saving this entire stuff back into the variable uh all right so that way i deallocated all of the local variables in here so the next thing we want to do we want to pop the element from the return stack so it's going to be simulation a return stack and then i want to assign that value into the simulation stack right there we go so that's how we're gonna do all of that and is it gonna work is it gonna work now yo look at that we simulated hello world look at that so now i can do that without intermediate recommendation or something like that i can just take the port compiler and go ahead and try to simulate hello world and i couldn't do that because i forgot the sub command there we go so as you can see here is the simulation of hello world right we're basically interpreting it so and it takes us uh 64 milliseconds to simulate that with the port port and if we compare it with python it takes 200 milliseconds almost 200 milliseconds so this is the python simulation and this is the port port stimulation which is uh four times faster almost ah a little bit less than four times but i mean it is faster what's interesting is that i didn't do any optimizations to the fourth implementation i didn't optimize anything and you're gonna be shocked you're gonna be shocked right now i'm gonna tell you the like a little secret that i didn't tell anyone and i implemented all of that off screen so people don't yell at me while implementing that and so it's very important for me because otherwise i won't get any done so essentially in the language we have to have tables that map and name to a particular entity because we have a lot of constants right so for instance if you go into the standard library uh there is a shutdown of constants the shutdown of procedures and stuff like that and to keep track of all of that you need a table that maps a name of the constant or name of the procedure to you know its metadata you know what data structure i used for these tables you would think that i used a hash table no i use the linear array so when i look up a name i look it up linearly one by far kin one one by white and this is still faster than python it's still faster than python i'm using the wrong data structure and this thing is faster than python so imagine how fast this is going to be when we switch to hash tables i'm going to do it like a separate stream where we implement a hash table in porth we implement the hash table in porth and uh we'll measure how much this thing will speed up all right it's called assassin for race okay so you can use whatever name you want [Music] don't want to be toxic but is that really such a big achievement being faster than python yeah you're right i'm just like making fun of tyson at this point yeah you're right it's not that big of an instrument i do agree with you uh so it is true um so yeah anyway um so let's actually commit to whatever we have already [Music] pause dot force make [Music] how can we say that [Music] fourth simulation simulation mode can simulate can simulate hello uh hello world hello the fourth examples hello the fourth right so it now can simulate that i'm gonna push that right into the repo you can find find the source code in here and then let me see so if you're watching on youtube you can find the source code in the description uh alrighty so let's go ahead and pick a different example that we want to simulate so what's going to be the next thing we could we could simulate uh i think we could simulate name right it's actually pretty cool program so if you do something like fourth simulate example name dot port right it says it asks for your name you say your name is alexi and it says hello alexi right so if we try to simulate this entire thing with force.fourth it will fail saying that yeah this thing needs to be able to push global memory right so uh let me think so that means we'll have to allocate we'll have to allocate global memory stuff or something this one's gonna be interesting uh porous dot ports to the um so let me go ahead and do that thing uh pause that by uh compile force.force dot force and then if i'm gonna run this entire thing and i want to simulate examples name.fourth do i really want to use the compilation what i'm sure because the compilation what cannot does not allow you to enter the values so but maybe that's fine because uh i'm gonna just go through the errors and just fix them one by one uh okay that's that's actually really funny right what is your name it managed to print what is your name right but then it couldn't uh you know push the global memory or something so it's kind of funny this is not what i want there we go so global mem global mem push the global mem so for the global memory you will have to just like have a buffer for the for the global memory um so the most logical thing would be to dynamically allocate uh the global memory because after the compilation right uh global memory capacity right after you compiled the program into intermediate representation right after you did that uh this variable holds how much of the global memory is expected for your specific program right so basically before the simulation what we have to do we have to allocate the buffer of this specific size basically malloc it uh but i'm thinking uh i don't want to implement my luck right now and i don't want to mess with m map right now uh do i want to do that so i'm basically gonna pre-allocate like a fixed buffer ah it's not gonna work it's not gonna work because we're allocating a lot of like global memory and stuff so we probably have to call map okay so that's probably what happened what we have to do [Music] all right so let's allocate a little bit of memory using a map does it sound good sounds good sounds to my gucci let's go so proc sim simulate ops so here we're gonna have memory uh global memory global memory memory and we're going to basically store a single pointer in here right we're storing a single pointer and we're going to obtain that point by actually calling mmap right so let's go ahead and do that uh man m map uh i don't remember how to allocate memory with m map but i'm going to try to remember so here we are initializing the same uh like local memory for the simulation and here we'll have to initialize the uh the global one so first thing we have to provide is an address then the length then the protection then the flags fd i don't remember what kind of d you have to provide if you are uh basically allocating memory but we'll see okay so address we don't know where we're gonna locate the memory so i'm gonna just put mo in here right and we'll just choose that for us so the length right we know the length uh it's a global uh a memory capacity right so after the compilation the uh expected global memory for the program is located in here so we're gonna just use that so in terms of protection protection uh basically tells what you can do with it with the memory read ride and stuff like that so it's gonna be prot uh yeah read write so i think this is the only thing we need from the memory so i'm gonna do something like uh prot uh read brought right and i'm gonna or them into the single mask so in terms of flags i don't remember what kind of flags we need to have so private create a private copy and write updates to the mapping are not visible to the other processes i think we'll need map private at least and another thing we'll need i actually don't like to remove this thing another thing we'll need is probably map anonymous right map anonymous uh so the mapping is not backed by any file and that's precisely what we have in here right so and we're gonna order this entire thing so in case of fd okay so uh if the argument is ignored however some implementations require fd to be minus one uh okay so here is an interesting thing porth dot port right now does not support like negative numbers properly so if i want to have minus one in here i have to do it like that right so this is called the art of getting done look people i implemented a self-hosted compiler without negative numbers this is the art of getting done all right don't judge me because i get done anyways so the offset i suppose the offset is going to be zero while you're constantly rat holding over what kind of syntax you want to use what kind of keyword you want to use for your procedures is it going to be proc is it going to be funk is it going to be fun is it gonna be something else while you discussing all of that i'm getting done without negative numbers so the same the same [Music] and it's not like it's not that hard weird flex exactly it is supposed to be a weird flex it is supposed to be because it's not the flex the point what of what i just said is not the flex because anyone can do that if you just stop bike sharing you can do so much if you just stop bike shooting and focus on what's important it's not the fl it's not meant to be a flex because it's not just like a gut coder or something like that anyone could do that it's just like stop bike shooting and what that's good there's no bike shooting is a pretty important term that you have to learn when you're doing software development all right so um if you never heard about law attribute law of triviality law of triviality three be out jesus christ i don't know how to speak english i really recommend to check it out uh so yeah check it out it's really cool uh all right i'm gonna put that thing in the description as well um and yeah it's not supposed to be a flex because anyone can do that it's not a flex you just have to believe in yourself just have to believe in yourself all righty uh so uh hopefully that's it i think we'll have to define some of these constants probably right uh to do to stop watching start streaming everyone can do that exactly uh so and after that i'm gonna do sim global memory and then i'm gonna save it in here so what's interesting is that uh this separation can fail so we probably want to check all right if it didn't fail so global memory we're gonna read that thing and map failed if it's equal to map failed we're gonna say something right here uh error could not allocate so and i think we have to do it like this right so this is going to be put uh it puts could not allocate uh global global memory capacity global memory requested 64. uh he puts you bytes for uh of global memory for the simulation uh for the simulation he puts and then we're gonna do it like that so by the way but that doesn't mean that you never have to implement uh negative numbers so that doesn't mean that uh if you ignore some trivial problems for now you never have to go back to them you eventually have to go back to them and fix them right so i'm just currently trying to get to the point where um you know it's mostly done right and once it's mostly done where it's at the right point we can start to uh you know cleaning things up and tidying things up uh and implementing all of these like missing features and whatnot right so it's more of like a again like i'm i'm comparing that with the illustration right so when you draw an illustration you start with a sketch with a rough sketch which is not finished which doesn't even look like a final illustration that you're going for but it just gets the shape right and then you start the line work right so you line over your sketch then you fill in the colors right solid colors and then you like cell shaded or just like some kind of a shading and like gradually slowly you you bring the illustration together right so you don't try to draw it right away usually you start with the sketch and it's the same thing with the programming actually it's the same thing right all right so unknown word prot right uh all right let's let's take a look at the standard library prot uh we have a pro to read but do we have a pro to right and what is uh what pro to write is supposed to be so let's actually try to find it in here uh okay so i'm going to grab our n define prot right so it is defined somewhere here so plot right is supposed to be two it's lib r it's a radar missing std porth so leap or right is supposed to be two and then we also have to do anonymous right anonymous is actually twain is that the correct thing in here so private is to privatize to here as well anonymous is 20. uh okay so this is going to be map anonymous uh and 20 in hex i'm not quite sure what is 20 in hex 32 all right don't know hex by heart i'm sorry i'm not a gut cooler so let's continue uh okay gum so um let's try to compile the entire thing um okay a non-word map failed i do remember actually creating map failed oh it does not exist okay well it's kind of weird why do i remember actually creating map fail so map failed uh it's basically minus one but cast oh i do remember why i do kind of remember why um because it's a negative value right so but we can quite easily do the following thing right so let me go to the port port uh right and basically i'm gonna put null and if this entire thing for whatever reason is less than no that means we couldn't allocate this entire stuff okay that's cool uh map uh expected to be point uh what is expected to be point argument zero is expected to be pointed oh all of that has to be in a different order uh all right so let's put it like this stack based languages am i right uh all right so there we go this is how i'm going to be reverting with this this stuff uh okay so argument uh so this one is not a point uh um oh map returns an integer okay so i can actually do it like this then hopefully uh-huh so if you try to simulate everything uh okay so this separation didn't fail right we can even confirm uh that everything went okay so i'm gonna do something like this uh it's gonna be puts and then i'm gonna take the global memory that we allocated ptr and i'm gonna just try to print it and see if it looks like a pointer all right i just want to see if it looks like a pointer uh two two two yeah it kind of does look like a pointer um okay uh cool so now when we're pushing the uh uh global memory um [Music] op push global memory so essentially what we have to do we have to push the pointer um relative to the global memory that we just allocated right oh and okay this is very important the global memory is expected to be zero initialized right so after we did this entire stuff what we have to do we have to mem set it luckily in the stand library we do have a function to mem set stuff right so here we have to provide the size we know the size it's a global memory capacity where we this is the size and then we have to provide the value that we have to fill it with so this is going to be zero and then we take the simulation global memory and read it as a pointer right and then we mem set it memset returns this pointer so i don't think we need it so i'm going to just do drop so this is quite important so this is quite important [Music] okay cool op push uh global memory right global memory uh porous dot pi uh let me find push uh i think it's just called push mem in here so when we're doing here we take the memory buffer pointer like a base pointer and we append the operand okay that's cool uh simulation global memory uh i read the value out of that i think i think it has to be a pointer right and then i take the simulation operand p operand and i read this thing as a value and i sum it up and here is the pointer within the global memory buffer which i want to push uh into the uh simulation stack right but before i can do that i have to cross this entire thing into integer i wonder if i have to do that so let's actually check the type of this thing uh right so what is it talking about oh yeah i forgot to do plus okay if we go to the plus and this is the pointer so what i have to do here is just i have to cast it to integer uh huh so it's this integer so everything's fine [Music] okay guy so data stack overflow and oh okay everything's fine i'm a professional software developer i know what i'm doing uh okay what is your name and it's asks us okay so let's actually see if we can answer anything uh all right so i'm gonna do port alexi and okay so now it fails saying that at eight is not implemented so it's a basically simple ins um intrinsic that reads one byte from the memory so that's basically what we need to implement in here so let's go ahead and implement that um so it fails at 16 27 right 1627. oh boy um and as far as i know on the stack uh usually how do you read that so it has a pointer right so we have to sim uh stack pop this thing and since it's a pointer we have to cast that to the pointer right and then we read that byte and we have to push that byte back into here right since it's an intrinsic we don't don't have to increment it and to be fair this is basically it so this is how you implement this in the intrinsic i think uh so let's go ahead and see if i can recompile the internet stuff can i type in the compilation mod i'm curious yeah so here's the problem with the compilation mode you cannot type in that compilation mode so you have to do it like that okay alexi and okay we can simulate name that's actually pretty cool so yeah hello let's see uh go you can in fact put anything in here and it kind of works [Music] that's pretty pough fourth fourth simulation mode can simulate uh examples name.fourth now there we go i'm gonna push that right at the river cool uh so let me see let me see so i'm gonna take a look at the examples and what other examples can we have quine did you guys know that it implemented quine imports right i in fact did it so this is the quine and port actually didn't stream it but it is legit it can actually print itself if you never heard about quine i really recommend to check it out so it's essentially program that uh can print its own source code right and if you think it's easy to implement just try to implement it yourself right trivial situations are relatively easy to implement you can just like you know do the empty file trick or use the language that just has a command that prints itself sure but if you're up to a challenge you can do that properly and just try to do something like this you'll quickly run into a problem uh that while trying to print its own source code you introduce more code that you need to print which in turn introduces more code which you need to print and you sort of go into this uh like infinite loop and it becomes very difficult to break out of that loop and actually implement coin so this is a very good exercise in fact um after i implemented a self-hosted compiler i would say that self-hosted compiler feels like implementing a coin in fact i think coins are an example of simplest self-hosted compiler so it's a compiler of the language with a single command and that single command is print the source code of your own compiler right so and while implementing the compiler for that language you're basically implementing coin um so yeah at least this is how i view coins maybe some people view them differently but anyway so i'm gonna put the link in the description and in the chat for anyone who's interested so uh yeah where's my description uh so quiet quine quite quite quite here is quite in fact i am going to tell you a little secret uh the name of the operation of the language that prints the source code with so on compiler is a question mark right so actually i think this language has uh multiple commands right if something that is not a question mark you just interpret it as it is if something is a question mark you just print your own source code and that's basically how you implement it and that's the compiler that basically compiles this source code uh right so we can actually uh demonstrate how this uh you know coin works right so uh i didn't want to create a separate window i just wanted to do example squine uh dot fourth and uh i'm gonna just compile it i'm gonna just run it okay so after i run this program it printed its own source code that we're gonna save uh here out fourth right uh yeah there we go and then uh we're gonna take a div between the output of the quine and the coin itself [Music] oh okay so well i mean for that thing to be a proper coin right for that thing to be a proper coin you kind of have to skip the output of the compiler itself right so you have to put like minus s flag right which suppresses all of the outputs and stuff like that so out.fourth and uh then if you do the diff uh whereas the give out dot fourth and examples quine uh the porous there's no difference between these things okay so this is equine can we interpret the coin so let's actually find out so let's simulate uh quine and i think we should be able to uh push sister okay this one is interesting so we're pushing c style strings uh pors.porth [Music] to push um system so i think it's the same as pushing string but instead of pushing like size and um pointer we only have to push the point so it means i have to only do something like this so i can copy-paste the entire thing right and in here as you can see we're pushing the size of the string in case of the c strings we don't need the size of the string because it's not terminated right so we have to do in here i think is just drop this entire value um right so let me see so this is the point ah okay i think this is the thing that we have don't have to push right okay so at the top of the stack we have an integer which we uh can just drop okay so we're gonna just drop this thing and we're only going to push the pointer in here all right so hopefully hopefully we'll be able to do uh okay here we interpreted the name this is not what i wanted can i interpret the uh the quine okay so the quine requires writing into the memory so this separation writes 64 bit number into the memory so let me see let me see 58 1658 and uh let's go ahead and do that [Music] 1658 so on this stack we're having a value that we need to write and the pointer right so if i do something like simulation stack uh pop on the data stack i get the pointer and then another time another one and uh then i get an integer so i'll have to swap them so after the first one i think i want to cast it to the pointer because that's what it is uh right and then after i pop them all i just need to swap them right let's take a look at the final result in here so here i have a pointer and i have a value that i want you to write okay so after that i can just like write that value in here right uh and that's it so i don't think we need to do anything else in here we just like perform the operation and that's it uh so that should be okay that should be okay um just thinking just thinking just thinking so everything's fine everything's fine um [Music] okay so i keep actually doing the wrong thing in here so uh now we need to implement a reading from the memory as well so that should be pretty step forward because we already implemented that for one byte right so we might as well just copy paste the implementation for eight bytes and just replace this entire thing with 64. right so there we go um [Music] so maybe i'm going to actually replace it with quiet now okay so now oh it needs if star implementation okay that's cool oh as you can see it already started to bring things look at that it already started to bring things so uh the only thing we need to implement in here is like if star 1457 1457 and as far as far as i know it's like it's not different from if right and even in python uh right if i take a look at if star uh if star yeah there we go so here's the simulation yeah so essentially if the operand is if or if star uh we just do the same thing so in our case it's kind of difficult to express this kind of stuff so i'm going to just copy paste the echo implementation into ifstar [Music] okay so uh now we need to implement uh writing single byte into the memory 1636. uh and i think here i can already start copy pasting things around right because it's pretty straightforward [Music] so what was that writing eight writing eight and where is the store 64 right so here's the store 64. and let me find store 8 uh-huh and here i just can do like like this in fact i can maybe straighten it up and straighten up this kind of stuff so all of the separations are like pretty step forward uh might as well just go ahead and copy paste implementations for all of these things right uh 16 bit right so here is just like 16. um it's kind of difficult to mess it up uh so maybe i should not try to do it like that well let's not try to do it like that because i'm kind of well you only we only need like two of these implementations for 32 bits right so here's the 32 bits and uh store 16 for 32 here as well 32 i hope i didn't make any mistakes here so that's pretty much all of the memory access intrinsics hopefully let's go ahead and see if it's going to work it worked so yeah we managed to simulate the quine i managed to simulate the coin let me go through the memory access intrinsics one more time just just in case so this is 18 16 16 32 32 64 64 and that's basically it that is basically it um i'm not sure how i'm going to be implementing the rxc or v in and p so this one is going to be a huge pain in this although for the for the end of p i would imagine that it's as simple as this uh right because that's basically it um but for the rxc and rv the arguments of there is a little bit of arguments that are consumed by the um by the compiler itself right so we'll have to keep track of how many arguments were consumed and give not consumed arguments to the simulated program right so this one is going to be very interesting to implement i think uh but apart from that did we implement like everything so we don't have a max in here yet but yeah well we will soon hopefully so push local memory is not implemented which is kind of weird but anyway i'm going to zero sim push um so this one has to be cast to point i'm gonna go uh expect it i mean it has to be integer of course okay so let me do a committee committee forest.forth simulation mode can simulate you can simulate examples uh coin plus all right so uh let me see can we already simulate ourselves right do we really need everything to simulate things yeah we do we need uh rxc and rgb at least and probably shut down of different um different c scores and what not i want to make a small break super quick and after the break we're going to try to continue and the goal of today's stream is to actually make the compiler to simulate itself right that's the main goal uh if we can manage to do that that'll be actually pretty power um not gonna lie all right let's make some break and all right let's continue the implementation evaluation okay so what's going to be the next thing that we have to implement uh i have to interpret uh we have fizzbuzz so how about that let's try to interpret these boss i feel like it will be able to interpret fizz buzz without any problems right because i didn't see anything special in here so i think it will be able to interpret fizzbuzz uh yes it is possible to implement fizzbuzz in our language so that language can already be used for coding interviews right which is already pretty pope so fees buzz porth okay um so is it simulating yeah there we go so i think it went really well uh i think we should try to simulate port with itself right so i think the time has come let's see what needs to be implemented in here uh okay could not open file well because there's no such file obviously uh i'm a dummy [Music] arc v is not implemented okay so this one is a little bit sus this one is a little bit sus the problem is the way our parser works it actually stores the arc v uh into a separate variable and it advances that variable so what we need to do we need to get the access to that specific variable and use that okay here we go that's basically what's going on so we have args right and we save the current arc v into args and then as we consume the arguments right as we consume the arguments we basically advance that pointer and in arc v intrinsic we just have to give the value of arcs so to be able to do that i think what we have to do we have to make this global that's what we have to do right so and the proc simulate i'm going to put this thing in here right so this entire thing becomes global right and the main function consumes some arguments but the rest of the arguments are available here for the simulation right so this is the rest of the arguments so let's go i'm gonna take args and i'm gonna read the pointer out of it and i'm gonna put that into the simulation stack so this is gonna be push uh and i think that's it do we need to do anything else in here i don't think so um so yes so so the only problem is rxc now essentially every time we consume a bit of args we have to decrement rxc as well uh right and then use the decremented arc c for the implementation of intrinsic so that means we'll have to keep track of more things that i expected [Music] okay so this is gonna be push and we have to cast it to the integer right okay so now it will complain about okay so it complains about push local memory oh it probably doesn't need rxc for the for the compiler because we don't use it okay the compiler doesn't use rxc that's the that's the point um that's pretty cool so that means we don't have to implement rxc for the compiler to be able to interpret itself that's actually pretty cool uh okay anyway uh so here's the local memory uh how do we interpret the local membrane def simulate little engine push local mem so what we do we take local memory rsp and we add operand okay make sense uh simulation local memory rsp we read the value out of that thing well we'll probably have to read it as a point but i mean whatever then we take the simulation op op operand right we read it as an integer we sum it up and we get the pointer that we have to push onto the uh simulation stack all right stack push and i'm gonna cast it to integer because push expects integer and then we also have to do some kp increment 64. right because this is not an intrinsic it's an operation of our nation okay so that seems to be all right do we need anything else do we need anything lco yo bro it partially simulated itself so it simulated itself enough to print its own usage that's actually pretty cool so it still needs cisco one right so uh it wants to have a cisco one okay uh so let me copy paste this entire thing uh cisco one uh and a c scroll one essentially uh we're just gonna have that right so this is gonna be just one this is gonna be just one there we go so i'm gonna copy paste the same code to all of them so all of this is called implementations all right uh not enough argument for projects for cisco ah damn team all right so we'll have to take these two things in here and uh this is gonna be cisco one okay looks fine do we need anything else almost there okay would you look at that is that it is that everything that we need to have in here okay so here's the porth i simulate porth with itself okay that's very interesting okay so it took some time to compile the program but can i now simulate holy so i simulated ports with itself and this simulated version said that i didn't provide file to simulate now i can simulate the hello world okay so for the hello world for simulation of the other things we have to have cisco for okay let's go ahead and implement cisco for um so let me see cisco four for the cisco four we have to have five of those things right uh so this is the c score four so this is going to be five uh this is gonna be uh five and this is going to be four all right so let me try to do this thing so the fourth assumes arc v is not terminated i wonder if that's defined by the standard i think i heard somewhere on the internet that posix kind of says that it has to be no terminated kind of sort somewhere i don't know we need to double check that okay cisco ii uh let's also implement c school two so for this is called two we just have to have three of them so i can just like use uh so this is five uh huh so this is going to be three i'm gonna remove this thing and i'm gonna remove these things c scroll two uh let's recompile this thing the main page says okay so man page won't lie uh c is called six okay so i suppose we have to implement all of these calls let's just go ahead and implement all of the ciscos why not uh syscall zero was the easiest one to implement look at that uh because it doesn't like it it expects one thing on the on the stack so it's pretty straightforward so cisco five okay so this is the c school five it expects six elements on the stack all right so here's that one and here is the sixth one and for that one we're going to have seven of them the art of getting done the art of getting done don't judge me chat i'm getting the done uh wait okay so this one is cisco six right this one is c c school five this is the fourth one third one and there we go so here are all of the c schools that we need anything else um okay let's try to simulate this thing holy it is slow okay so maybe something is wrong in here and i forgot to do plus one somewhere i'm not sure but it feels like it's stuck that's what it feels like uh wait a second all right so it expects the input file uh but it couldn't do so let me see [Music] okay i wonder if it's doing anything or is it just that slow uh let's trace it that's like actually a good idea we can try to stress it maybe just to see what's going on there [Music] well i mean yeah it only goes not that deep it doesn't really go that deep so uh it does not estrace some of the other things um okay so for the um for this stuff did you forgot like for this for the intrinsics we don't have to do plus one right we just don't have to do that [Music] all right so let's actually try something simple right let's try something simple uh how about full porth that takes 69 and just brings that thing right so we can try to simulate uh food.porth right and couldn't allocate zero bytes of global memory of the simulation i'm really glad that i decided to handle the output of a map who would have thought that you can't allocate zero bytes of global memory did you guys know that you cannot do that i didn't know that um so uh let me try to do that i could not allocate uh we can do something like i don't know um so global uh memory capacity right we're gonna read this entire thing and only if it's greater than zero only then we can try to you know allocate some right i think it does make sense right don't try to allocate it if it's right not um not available because some of the programs don't have any global memory so what's the point wait a second i think i'm being stupid yeah i have to put it like this even i would even say like this yeah that's right so i like this entire stuff all of these separations have to be under under the condition [Music] we're almost there it's just like really unclear what's hanging what's hanging in there okay so here's the 69 nothing special so then we can do poor the port port port or portfolio so then we provide the simulation and it says no inputs priority for the same sub command and then if we try to do something like this uh okay so that was pretty cool so if i try to include the standard library maybe the standard library the compilation of standard library is super slow maybe that's the point so if i try to include uh std porth maybe that's what slows everything down probably that's probably what's going on [Music] so we're simulating this simulation python speed re we finally reached the python speed finally my god ah that's very weird that i can just like simulate that though i can simulate without this stuff like i don't know if there is a bug in the simulation or it's just that slow that's the problem like i don't know um i really don't um [Music] simulate simulate simulate simulates maybe it is that slow who knows so let me take a look at some of this stuff uh that's at the div specifically so this is just a memory nothing special so push local memory i do increment sim ip all right so ah no no it's it's an intrinsic okay so that's fine our gui is fine cisco one so everything that is intrinsic it is totally fine uh to not increment by one [Music] okay [Music] so let me let me think what we can do to actually confirm that this thing is just being slow what can we do because if i start sticking simulation into this thing well okay so we can try to just log what's going on why not um so proc uh simulates here this let's try to simulate this thing [Music] so here is the op okay that's cool so i take the simulation op and i take the token where it's originated that originated that op then i take the talking location right so i add the location and i'm gonna print that location so it's gonna be put lock right so well that will slow it down even further right so but this one maybe we're gonna see if it's stuck somewhere right so maybe that's something that we can do and in here we can just print the operation right and do we even have anything like for converting operations into strings op as op type as oh there we go so we have that so i can just take um um sim op op type plus and read this entire thing and then i can say op type as a string and then i can print this thing and then the new line uh there we go so this is going to be something like that right so then like this like this and there we go so we're just printing it so we basically tracing the operations that we're currently interpreting uh right so i'm gonna try to compile the entire thing right so i'm recompiling the compiler recompiling the compiler um and i'm going to try to simulate the 69 example but i'm going to not include this thing so i'm simulating full ports right as you can see here we have full porth uh push integer and then intrinsic so basically tracing this entire thing uh if i try to simulate for the port the amount of operations here is going to be way bigger so here are all of the uh yeah here's all of the operations that we simulated that was kind of weird because it didn't bring them right away if you know what i'm talking about right so it waited a little bit for whatever reason uh so let me try to simulate it maybe in here um so that should be pretty straightforward okay so that's fine um let me see so if i try to now simulate the compiler with itself right so this is going to be full.porth and well that kind of okay so that worked and it finished right so that worked and it finished uh so if i try to simulate this thing now uh if i try to simulate this thing now is it repeating itself uh you out commented yesterday yeah yeah so i have to actually bring it back so uh let's simulate i'm actually a little bit uh tired uh skip rocks keep rocks everything seems to be fine uh for that fourth uh let me i'm really tired i can't think anymore uh but to some extent what if we try to compile it with itself right so if i try to simulate this thing right and then uh i hit the compile can i at least compile uh you know a simple program i can at least compile a simple problem uh okay can i at least compile a simple program and uh let's just um i can't think anymore uh all right so this is gonna be like that all right [Music] [Applause] so the hmm so but one layer of simulation works i think i'm gonna follow the consensus of that it works is just being super slow uh right because that simple example works perfectly right this simple example works absolutely perfectly so uh we're simulating the compiler which simulates the program and the program is being super simple and that works so the simulation of the simulation um so yeah and how how fast it is because uh simulating something more complicated may require like a lot of like you have to simulate like the whole machinery of the compiler actually um compare it to python python is going to be faster by the way uh right if i just do uh python fourth dot pi uh yeah so it's definitely gonna be faster yeah there we go so it's time eight times slower all right so i think today was a pretty good progress i would say uh we are finally able to make the compiler to simulate itself to some extent right so let's do something like ports.porth port.forth and let me see how should i call that the compiler the compiler can kinda simulate itself but it's super slow it's super slow uh and i'm gonna push that right into the ribbon all right does anyone have any questions does anyone have any questions so before i go oh my god matus f777 thank you so much for 35 months that's a lot actually thank you thank you so much for 35 months of uh tier one subscription that's that's a lot of month holy uh and welcome back to our epic parts club the real og of the channel thank you thank you so much for sticking for so long thank you thank you thank you uh i'm really glad that still there are people who've been around for this long no matter what this i don't deserve you guys really thank you thank you so much um all right so today was an interesting stream at least for myself uh right we implemented this simulation i'm i still don't really know if it's just that slow yeah okay so you know what i'm gonna do uh off screen i'm gonna actually leave this example right so basically simulating the compiler uh with itself and trying to simulate something with a standard library i'm gonna just leave it for like an hour or maybe two hours and just see if it's gonna ever finish right so and yeah we'll see um right that's it for today thanks everyone who's watching me right now i really appreciate it have a good one and i see you all next time i don't know what's going to be next time right so i still haven't decided uh usually i just like continue developing porth and if i see something interesting i basically schedule that for for the stream so so far i don't really know what could be the next topic we'll see we'll see the winner gets t that's for sure in any case thank you everyone for watching uh i can go i could go love you

Channel: Tsoding Daily

Views: 8,486

Keywords:

Id: 6tl7YmTnFQ4

Length: 134min 54sec (8094 seconds)

Published: Tue Nov 16 2021