LSP: Building a Language Server From Scratch

one of the best ways to grow as a developer and to learn a subject is to build your own version of something from scratch in this video we'll be building a language server from scratch to learn the language server protocol we'll start off with an empty typescript file with no dependencies and we'll end up with a working language server still with no dependencies we're going to build the features like initialization completion and document synchronization and we're going to be doing this all character by character line by line so fire up your text editor and come along with me on this journey and at the end we'll talk about how you can take your language server even further we'll start by getting things set up in vs code but don't worry this isn't vs code only you can happily follow along in whatever text editor you prefer just create an empty typescript project and then follow your editor suggestions for connecting to its language server implementation I'll have instructions for neovim in the description Visual Studio code is the most popular editor out today it's just one place where you can run a language server but if you're following along at home it's probably the editor you're using if you're trying to wire up a language server in vs code for the first time it can be a little confusing to make this easier I've published a minimum viable VSS code language server extension repo you can clone down this repo and have everything you need to create an extension in vs code it consists of extension code that will launch and coordinate communication with a language server process you can bundle this all up and publish it in the vs code Marketplace since this is so from scratch video we'll be deleting all the server code and really just using this as the glue to connect our language server to visual studio code we'll start by cloning down the minimum viable language server extension and we'll save it as LSP from scratch now that we've cloned that we'll CD into the directory and we'll do an npm install these instructions are in the readme and then once we're here we'll do one more more thing which is we'll touch TMP example.txt the reason we want to do this is just so that we have a reliable file placed that we can use for testing our language server and since it's a text file uh it's very unlikely that you have any other extensions that will be running on text files so it's it's going to be the most pristine experience that we can get having done that we're ready to open vs code and once we're in here we will go ahead and close the welcome menu I'm also going to close the sidebar with command B uh you can bring that back any point with command b or control B on Windows if you hear command and you're in Windows just press control and you should be good to go um and what we're going to do is do command p and open server. TS so this is the existing content that is in uh the minimum viable language server extension and you see it Imports a number number of things from vs code language server node quick plug if you're developing a language server uh for for real use not for Scratch as a toy this library is absolutely the way to go it's first class it's built by the people that maintain the spec itself so it's just the way to go you're going to get great types you're going to get great uh functions all the code completion yummy goodness that you want so a huge shout out to that but we are going to be writing this scratch so we won't be using that the reason this is all bundled in the MVP is that you probably will want to use it there and also we have one behavioral thing wired up here which is when the document's content changes we're going to send out an information message that says on did change content and then has the document URI so we're going to use this to prove that things are wired up before we clear everything out and truly start from scratch so I'm going to do command shift B here to start our build and then I will do command shift d to open the debugger so I'll click on the play symbol here this launches another instance of VSS code and it is the extension development host the extension development host runs any client code you have in your extension and then in our case it also will spawn the language server process as a little sidecar and the MVP has these things wired up to talk to each other so what we can do here is open our uh I did command TMP example.txt and we'll hit enter on that and so now we see down in the bottom we have onid change content for our example file that's great now that we've proved that this all works we are clear to close this and delete all of this code once again going to close the sidebar uh we'll do command p and open up package.json for the server and just to make sure we're truly from scratch we're going to delete our dependencies here so we clear that out and save close that and then we'll hop back in our terminal real quick and just do an mpm install and now if we do a tree of server node modules we're going to see that there's nothing in there so we're truly starting from scratch let's just go ahead and we'll edit our uh git config to remove the origin I don't want to accidentally push over this uh then we can actually commit our changes scratch all right cool so I'll have this repo uh pushed and available down in the description all right so we're ready to start from scratch and we truly have nothing here if we launch this uh we're going to get an error so let's just let's just do that to see what it says um whoops that was the wrong one we'll press play here and we saw some messages show up there and then immediately disappear if we look at the output of the server here we see the initialization failed so the client code the extension is trying to talk to it and it's not working and that's because it just doesn't know how to do anything yet um so we'll do oh actually let's leave this running so I'm just going to uh go back to our original rather than closing it because that way we can just hit this button up here when we want to restart it we'll close the sidebar and we need to figure out how to make these things talk to each other so before if we go to the extension TS again this is the code for vs code that launches the language server so here we can see how that works it basically uh gets sets up the server module as server.js this is the um JS version of the typescript file that we're writing in um and all of that compilation is set up as part of this MV MVP also uh and then we have a transport kind here of IPC which is interprocess communication for node um so what's happening basically is in the default we're talking to the server process over IPC and that way the the that's the the channel that all of our Json RPC messages are going to and from the client and server for uh we don't want to actually use that in this case we're going to use a different transport we're going to use uh standard in standard out so we'll go ahead and change this here and then I want to take a we'll close this file that's the only change we're going to make in the client and then I want to take a brief t tour to look at the uh LSP docs so let's do that now all right so language server protocol specification we're going to hop down to miscellaneous and then implementation considerations so if we scroll down just a little bit we see servers usually support different communication channels for example standard and standard out and pipes Etc to ease the usage of servers in different clients it's highly recommended that a server implementation supports the following commandline arguments to pick the communication channels so ideally your language server uh has a command line argument to opt into one of these protocols you don't have to support them all but ideally you let the client pick because we're just building this from scratch and we don't want to over complicate things with the communication channels we're just going to do standard and standard out the other nice thing about standard in standard out is that it's easy to get running with neovim and other clients so we'll be using just standard in standard out and ignoring the others so the question then becomes how do you speak uh standard in standard out in node how do we do this in our typescript and the answer is actually to use process.st standard in.on data as we can see here we want to pass in a call back we'll name the variable chunk and we'll learn more about why in a bit and so in here is where we could actually act on data coming in over standard in because we're talking over standard in and standard out we could actually write using a console.log you'll see in the uh explanation here that this prints to standard out with a new line however you're probably used to using console.log for debugging and uh communicating directly to a user so we're going to actually use process. standardout write uh I find that this is just a clearer name just to keep in my head I don't have to confuse it with uh you know logging it's not logging it's writing um we're speaking to the client so you may be asking yourself well if we can't console.log how are we going to have debugging information don't worry we'll get to that right now so since we're reading over standard in and we're writing over standard out we can't use console.log we need another approach so we're going to make a new file and we're going to save it as Source uh log. TS and then in here we will import Star as FS from FS we'll set up a log which is fs. creat WR stream we'll write that to TMP lp. log so by creating a right stream at the start of our app booting this will make sure that the log content is empty when the app starts we're going to appin to it as the app goes and then we'll shut it down when the app ends um and so what we'll do is we will say export def fault and we're going to have a write function here write is going to take a message which is either an object or unknown if the type of message is object we will uh log. write json.stringify our message makes it a little more easy to read and also it won't crash the log uh and then otherwise we'll just log. write our message and then for readability we'll also do a new line after all of that with that in place we can import log from log and now we will log out our chunk log. write chunk okay let's give this a spin and see if it works so our file opened up over here and uh we we relaunched our extension host so now we want to do command o and open TMP lp. log okay so we have a buffer that's not super great um H would have been nicer to get something useful here so let's actually log out two string we may want to rethink our logging approach later maybe we need a two string and then Json pars things but uh we'll think a little more about that let's restart and now looking at the log we have a Json RPC message excellent so it has a Content length of 5,752 uh it's Json RPC 2.0 which all messages in the lp spec will be um and then we have ID of zero this is the first message being sent and then in a method of initialize so the client is saying hey server uh invoke the initialize function and give me the result and we know that it wants the result because it's passing an ID there's two types of messages in the language server protocol there are sorry let's look at the base protocol there are requests messages which need a response every processed request must send send back a response to the of the request and those have an ID a method and prams we just saw all of those things and then we have to send back a response message there's also a notification message which is the same as a request message but it doesn't have an ID so those are just like hey this thing happened don't need to reply from you but this thing happened you may want to act on this thing that happened but you don't need to reply so we have a request message and we have it has ID method pams everything's is also going to have this Json RPC prefix um so let's go ahead and just copy this so that we have a type and we'll throw this over in our server the nice thing about the language server spec is that you can copy a lot of things you're going to probably end up deleting a lot of comments because uh while they're very helpful in the the protocol doc itself they're not going to be terribly helpful uh in your code so we can mostly copy things and delete comments but occasionally you do need to um actually we'll make this an unknown occasionally you do need to change some types just because the uh integer over here is not something that exists in typescript and neither is just the plain array um if we click on Integer we can see that it's actually just a number we could declare that type first and then use it in our types but we're not going to go that far uh and then we also want to Define message since we're extending message message so we'll paste that up here perfect okay we'll use these later I swear we're getting this request to fire the initialized method we have some pams the pams are basically telling us about the client what's the client's name what's its version uh what sort of scenario is it running in and then the interesting part is the capability so this tells us what the client can do and you can use this on your server to do a little bit of negotiation about well can I off offer these actions based on what they can and can't do we're going to ignore that for now uh and instead we're just going to look at the spec and see what the initialized method is supposed to look like and what it will return this is under life cycle messages initialize the initialize request is sent as the first request from the client to the server until the server has responded to the initialized request with an initialized result the client must not send any additional requests or notifications to the server so what we have right now is we're starting up the client sending us this initialized request we're not responding yet we don't know how so the client's just sitting there not further talking to us so let's go to uh initialize result momentarily but first just to note uh the way that these docs are structured you can see the request here it tells you uh what the method is and then what the params are and so you can click into those and get the type but we're going to go ahead and jump ahead to the initialized result if I scroll up a little bit you can see that this is defined as part of the response all right so initialized result describes the capabilities of the server so the request tells us about the client the result tells the client about the server so we're going to copy this and the language server protocol is all about methods so we're it's Json RPC we're invoking remote methods so what we're going to do is open the sidebar here and we're we're going to make a new folder this folder is going to be called Methods because we're going to keep adding more and more we'll make a new file in here and we'll call it initialize dots and now we can paste in our type again we'll clean up comments here okay we don't really care about server capabilities yet but we should at least know the shape of it so server capabilities is an object we can just give it a record for now excellent and so we should export our method so we'll Define it as export const initialize it's going to take in a message which is a request message then it's going to return an initialize result and this code is a little grumpy because we haven't defined either of those things but that's okay over in server we will export this and now we can safely import it over here and that makes that part happy so now we just need to make our response our our return value of this initialized method conform with initialized result and so to do that we'll just return uh an object with capabilities of an empty object for now some server info we'll give ourselves a name and we'll give oursel a version of 0.0.1 over in our server we can now import this method and if feels like we're ready to respond except when we think about what we're reading we're not just getting the Json payload here we're getting our content length we're getting the carriage returning a new line that happens after that and then the one that happens after that and then our Json message so to be able to invoke initialize with a true request message we need to parse out all of this other stuff first over in server. TS we'll do that now you might naively want to just split the chunk based on the new line characters and then this would be position zero this would be position one and this would be position two the problem is as the name chunk implies here we don't know that we have a single message that we're working with at a time we could have part of a message we could have the end of one message and the beginning of another message we could have several tiny messages uh it's just not something we can know ahead of time so we can't just naively split and then parse Json instead we're going to have to accumulate the data in a buffer and then process our buffer don't worry this sounds scarier than it is so we'll let buffer equal an empty string and then as things come in we're going to buffer plus equals the chunk so we're accumulating things and if we just left it like this this would just read and continue appending anything that we got to the buffer that's great so here's where we're going to process things we'll do a wall true we're going to process every message that we can in this buffer every message if any so the first thing we're going to do is check for the content link line and we'll do that with a match so we'll say link match is buffer. match content length colon then we're going to have some number of digits and then the RN carage return new Line all right so if we don't have a length match then we're not ready to process anything yet we don't have a full message uh so there's nothing we can do if the content link header isn't there there certainly isn't a message body so we'll break we'll continue accumulating and adding to our buffer if we do have a length match though we can extract the content length as a number so this will be length match one uh the zeroth place is the entire r x match the one is our capture the digits only and we'll use base 10 and then we also know that the message actually starts um after the carage return new Line carage return new Line so we'll say buffer. index of car return new Line Car return new Line plus 4 or to get past that content so now we can check to see if we have an entire message because we know where it begins and how long it should be so uh we'll continue unless the full message is in the buffer so we'll say if buffer. length is less than message start plus the content link we'll break we'll continue reading awesome so if we've gotten this far we know that we have the full message in the buffer and we know the size of that so we can extract the message we'll start off just getting the raw message this will be the string content uh this is buffer. slice message start and then the ending will be message start plus message sorry plus content link and then we can now parse this because because we we can feel confident that we have only the Json object so we'll say message is json.parse raw message oops raw message and then we'll write something out to our log so we'll do log. write we'll give this uh ID of message. ID and uh method of message. method we won't care about the prams for now and so at this point we could act on our message so we'll do to-do um call method and respond uh but we want to go ahead and finish up our Loop here so the last thing we're going to do is to remove the processed message from the buffer this will prevent us from processing it again and again and let us just move on with things so buffer is going to be buffer. slice message start plus content length okay and so we should be able to start this and see the ID and the method for our initialized method being called so we'll do that we'll restart pop back to the original window and we do see that here that's awesome now that we've isolated the Json message we can respond so we could do this as if message. method equals initialize respond which we haven't implemented yet with initialized message but as you can imagine as we had more and more methods this is going to end up being a lot of else cases or maybe we turn it into a case statement I prefer to just go ahead and make a method lookup so we'll get rid of that and let's imagine what the method lookup uh usage would actually look like so we can say const method is going to be method lookup message. method and then we can just say if method we will in respond with the respon respond with the results of our method being P the message and I think that's cleaner because then as we add new things all we need to do is add them to the method lookup and as long as there request responses this will work fine uh to make this work we do need to Implement both method lookup and respond so let's hop up to the top here and down here we will say const method lookup is going to be just initialize it's the simplest thing that could work but it actually needs to be a little more complicated than that because we're using an any to dig into that so let's let's just change this real quick we're going to say it's a type of record string so we're looking it up by the string name the method name and it's going to be a method and then over here we'll say that type method is going to be a function which takes a message of request message and then returns some object that we can serialize to Json so it's our initialized result for initialize but it will be other things in other places um since we named this method but we're actually using request message let's just go ahead and name this request method and we'll do the same thing down here it's a little clearer so now we're down to one error which is that we haven't implemented respond fortunately we know how to talk over standard out so this should be pretty straightforward we'll say respond is a function it's going to take an ID which is a request message ID type so it's going to be a string or a number and then it also is going to take some result which is an object uh we need to turn this into a Json RPC message so we'll have to create that content link header um to do the content link header though we first need to know the length of the message and to know the length of the message we need to convert it to Json so let's do that const message is going to be json. stringify ID and results uh we can also check this against our response message over here so it has an ID um it has a result if it was successful so in our respond here we're assuming success we'll handle errors differently so result could actually be all of these things um so let's let's just change our type to actually be an unknown here uh so we'll hop back over to our editor and we will change this to be an unknown perfect uh so now we have our message and as we know we need to also have have the content length header so we can say header is going to be content length message link we'll Define that in a moment and then rn rn so what is message length message length is going to [Music] be buffer. byte length of our message with UT f8 encoding so now we can uh log. write our message whoops our header plus our message and then we can process.st standard out. write our header plus message so now we should be working with a full loop here we need to correct one thing which is we know that respond is going to need the message. ID now that's Happy um so let's just talk about what we've done we've built a buffer reader we're reading on standard. in and accumulating our chunks into a buffer then we're processing any full messages that are in that buffer by checking for the content link header breaking if we don't have it yet keep reading otherwise making sure we have the full message if we don't have that we keep reading if we have everything though we can parse out the raw message and then convert that to Json and then if we respond to the method if we know about this method we can invoke it and send out the response over our standard out then we remove the message from the buffer so that we can keep going um respond is going to write a Json RPC message out to the client so maybe this just works we'll hop over to our log and we'll restart and we'll flip back and this looks really good guys we have our initialized method being sent from the client it's got ID of zero where replying with Json RPC with a Content length of 96 we send back our ID of zero so it knows hey this is the reply to that thing I sent we specify empty capabilities and then we specify our server info and this is very cool the client fires a notification that says that they're initialized and we know this is a notification again because it doesn't have an ID so this is super cool we've built a functioning language server it's able to boot up uh talk to a client and initialize now it explicitly doesn't do anything yet because we haven't taught it how to but this works and we can be proud of it but of course we can't stop there we just got going let's add in some capabilities when I think about capabilities uh I think language features so we'll hop down there the first one I like to implement is always completion proposals completion proposals is is the completion request we know that since it's a request that it is going to be sent to us with an ID and expects a response you can also uh see this in this Emoji which is kind of like a U-turn so it's coming to us and then we're sending something back um so this says the completion request is sent from the client to the server to compute completion items at a given cursor position completion items are presented in the intellisense user interface so basically this gives you all the goodness of uh code completion so like the user type's less than b and you know aha I'm in HTML I can uh suggest body if we scroll down we'll see that uh there's a lot going on here there's client capabilities we're going to ignore those for now uh there's server capabilities which are specified by the completion provider property and it can specify completion options all of these are optional so to enable this we can actually just specify completion provider as a blank object so let's let's do that now we'll hop over in our initialize and under capabilities we'll throw this in um and I can actually show you what this looks like immediately so if we redo this we'll hop over and look at our log this is what we saw before except we have the completion provider specified here and of course our content length got a little higher uh but check this out once we start typing we get a text document completion request we also get a cancel request uh I believe this is because we didn't respond in a timely manner so it just went ahead and canel uh that completion request but uh the important thing is that we have this method to respond to let's learn a little bit more about that in the spec we'll scroll down okay so the request comes with some completion perams we'll worry about those later and the response sends a completion item array or a completion list or null if completion item array is provided it's interpreted to be complete so it's the same as incomplete is incomplete faults and items okay so it looks like completion list is our most flexible response type so let's see what this is it has an is incomplete field is incomplete indicates whether the list is or isn't complete so if is incomplete is true we're telling the client that it should keep sending us requests as it accumulates more characters that the user's typing because we might be able to provide better results if is incomplete is false we're telling the client like hey what you've sent us so far that the user is typing is sufficient for us to give you exhaustive results and sending us more characters that they type isn't going to help you should just do all the filtering in memory on your end so we'll grab this item defaults is optional we'll skip it for now and then the last part is the completion items so let's copy those so we'll make a new folder and file for our method so in methods we'll make a new folder called text document and under text document we will make our completion. TS and in here we can paste in our interface so we'll do uh the part where we delete the things that we don't want and then delete a little more perfect so we need to Define completion item next completion item is is a lot of optional fields and only one required field which is label label is a string so this is the label that shows up while the user typing as the suggestion and then assuming you don't provide any of these other options it's also the text that gets inserted when they select that suggestion so we'll just paste that in here and now we can Define our function so we're going to export const completion it's going to take a message that is type request message and it's going to reply with a completion list oops completion list we'll hardcode a response here for now and it will be is incomplete faults and we'll have some items we'll do uh label of type script label of LS p and then lab label of Lua just so we have another l in there okay we don't know what request message is so let's import that and then back in our server we can wire this up so the same way we imported initialize we will import completion and then down here we can't just throw completion in here because it's not the full method name the full method name is text document completion so we'll copy that paste it over here and that's completion awesome uh let's give this a try so now we will reload and we'll type hello we don't see anything because none of our examples start with an H if we Type L we get the suggestion of LSP and Lua we can you know choose between those and then select one and then Ty gives us typescripts we also just get that for T this works awesome of course having a static list of items is not super exciting especially when it's only three things so let's make this a little more exciting we will get rid of all those examples and we're going to Define items so for items what I want to do is use the system dictionary so if you aren't aware of this on Mac OS and most Unix Z systems there is a dictionary file so I'll do command o and on Mac OS it actually is uh user share dicks words it's here in my history so I can pick that user share Dix words we'll hit enter and we see that it's an alias It's actually an alias to web two so when we open this up it'll say web two at the top but this is a list of a whole bunch of words and there's actually almost 20 almost 236,000 of them so this will give us us a lot of things to complete uh so over here we will import Star as FS from fs and then we'll say words is going to be fs. readfile sync and then we'll we'll give it that path and then we want to split this on new lines but uh this is actually a buffer so we should two string it first and then we can split it all right then we can say that items is going to be words. map word and return a completion item representation of the word which is label word all right that seems reasonable so let's close this and give that a try all right so now we're going to type a and after a moment we see a lot of suggestions if we scroll this down it goes a long way um but this is great so we can do do a a and get arvar uh we can do z y and get all of these words that you probably don't recognize I certainly don't so that's great but we do notice that when we press a there's a good bit of a delay before things show up why is that take a moment to guess if we look at our log we'll find out why this takes a moment to load this is loading and I'm waiting and then eventually it pops in and we see that the content length of our completion response is 5 milon 325,000 and if they had only typed a we wouldn't send every word in the dictionary we would just send you know some reasonable number of it but the question is if the user typed hello there and then types Ma how do we know that the thing that we're supposed to complete is just the ma part and this gets even more tricky if you imagine that you know there's a ton of lines in this file so there's stuff up here stuff up here and then on line two here uh they start typing Ma and we should give them the results for Ma so how do we get this context of where they are and what they're typing now it's time to look at our completion prams so we ignored these before but we'll we'll come back and take a look at them in force we'll scroll back up a little bit completion prams okay so completion prams extend text document position prams work done progress prams and partial result prams they also optionally have a context of completion context now since this is optional we already know we can't rely on it but let's just see what it is okay completion context is a trigger kind which is just invoked trigger character or trigger for incomplete completions that's not helpful for giving us context and then it could also have a trigger character but this says that it's a single character so that definitely could not help in our ma example so we can rule out the completion context we can look at partial results pram this is just about handling partial results when you're doing streaming and since they're about the results and not the request that doesn't really help us we have work done progress commands which the server can use to report progress that doesn't help either so all we really have to work with is text document position prams let's take a look at that text document position prams has a representation of the text document which is a text document identifier that is a URI which is just a string that represents the file path so that'll give us the files path on disk but if the user hasn't saved the file then we definitely can't read the content out of that file so that's not going to get us where we need to go by itself we also have the position this tells us where we are in the document it has a line and a character it sure looks like we don't have enough information to figure out what the user is typing currently uh we can know where they're typing what file they're in and what position they're at thanks to these text document position programs but we don't know what they're typing and that's a problem and this is where another part of the language server protocol comes into play which is document synchronization document synchronization is the client's ability to send notification to the server that a text document changed opened closed Etc and so we can use this information to keep an inmemory representation of the documents the user is working with and then combine that with our position to figure out where they're at in the document and from that we can extract the word that's in progress if that sounds confusing don't worry we'll break it down one part at a time uh there's lots of good notifications here we're going to work with text document did change um so did change text document notification comes across as text document did change and includes did change text document prams let's take a look at those so those include a versioned document identifier but it also has the actual content changes and the text document content change event we see is an event describing a change to the text document if only a text is provided is considered to be the full content of the document okay so it could be this in which case there's like a range and it sort of represents the change that happened or it could be this which is the whole text of the document we're going to go uh this route because that's just easier for us to handle in this example so let's back up a little bit uh in order for this to work we need to see Server capabilities for server capab abilities we have a text document sync property and then the property uh type needs to be specified as one of these two things so we have a text document sync kind which is none of zero full of one or incremental of two so we can just say full as one okay so we'll add this to our initialize we're going to copy that hop over into initialize and after our complete completion provider We'll add this and say that it's one we want the full changes and let's restart and we'll do h l o and then look at our log this is still going to lag for the moment um yep still processing okay now we can scroll to the bottom and we should see we're not responding to this or logging out the prams but we did get the notific which is great um so let's just temporarily change this to also log out the um pams so we'll say pams are message message. prams just rerun this real quick topic over log wait for it to refresh okay scroll back down all right cool so text document did change it has a URI which represents the text document this is the file and disk example.txt and it has the content changes which is the full content that's exactly what we wanted hello hello great and if we look in here we can see that did change happened a I'm sorry that's a setup did change happened a few times here we just have our initial h h here's where I typed H.L as an accident uh here I changed it to hell and then we're to hello so awesome so we can use this to construct an inmemory representation of the documents the user's editing and then coordinate that with our completion prams to find out what they're typing at the moment that we want to complete to do that we need to respond to this method so let's get rid of that log line make this all be back on one line again perfect so let's grab this and we know in our server we're going to want to add that in here and it will be did change doesn't exist yet we'll make it exist we'll make a new file from completion we'll save it as did change. TS that's in our methods text document folder and so this is our first notification so it's not going to have the same shape of response that our request messages did so we'll get to these types in a moment let's go ahead and work on implementing this we'll grab our interfaces our types from over here back up okay so we're in the did change we're going to get the did change document forams great so we'll grab this we'll do the delete and clean up dance all right what is version text document identifier let's figure that out version text document identifier is a version which is an integer but it also extends text document identifier uh we'll call this a number and then we'll say um let's make this an interface just so we can clearly extend oops grab that oh come on so uh interface that is going to be document URI and then we know that document URI is just a string okay so now we need to handle this guy so we'll back up a little bit to get to where he was specified and again we're going to be on this side of the Union so it's just going to be a text string excellent so uh now we can Implement our function so we'll export const did change this is going to take a message that is a notification message so over in our [Music] server let's define that um we'll say that export interface notification notification message extends message and has these things and then this will just extend notification message um perfect so now over in our did change we do notification message we get our import that's great and this is not going to return anything we'll explicitly mark it as a void awesome so in our server we can now import this now that we've imported it we can put it into practice uh this is happy but you'll actually notice that this type is wrong we don't have a request method here we have a notification method and the reason that this works is because when we were typing the request method we were lazy and we made it an unknown so we're going to change that to an object and that starts to fail and it fails for the notification which we expect because did change is a void it fails for completion also because completion could be completion list which is an object or it could be null we'll do return type type of initialize Union return type type of completion so now we have something valuable here uh this is is understandably upset because uh did change does not satisfy this this is going to be get long as we add more things but we can always refactor our types later so now what we're going to do is type a notification method and it's going to take a message which is a notification message and it's going to return void it never returns anything so now this can be a request method or a notification method and this looks happy but uh this is actually wrong too because respond is not going to have a result for a notification method and we shouldn't respond to notification methods so we'll see this could actually be an object or null uh because we could also intentionally write out a null we have a null response that's valid so void is not assignable to type object or null so what we can say then is const result is going to be method message and so we can say if result is not undefined then we will respond and we'll respond with our result so now result could either be an initialized result a completion list or null and that's great uh and then we'll only be responding then with an object or null because both of these are an object uh so let's let's kick this and actually we'll do one more thing which is um let's just log here uh log. write message kick it hello this takes a moment h okay so here's what we have so we need to use our prams to get the URI that's going to be the index for our inmemory uh documents and then we'll store the text so let's make a documents file we'll make it in the same level as server so we'll save this as documents. TS and we will say const documents is going to be a new map of type document URI I'll have to move that over in a second and document body which we'll Define Okay so we've got some stuff in did change that probably belongs in our main document area I think it's going to be all of these things we'll do that for now can always move things around later um and then we'll say type document body is also going to be a string so what we end up here with here is a map and this is going to be exported so we can use it throughout our language server uh that lets us represent the the pair of for this document URI this is the document body great over in did change we want both of these things so version text document identifier text document content change event so let's just export those real quick and then over here we will import [Music] documents import all of those things uh which is great and so when we get our did change event come in we want to represent our prams as message. prams as did change document programs so we know what they are we can narrow those down here with an ad as uh and then we'll do documents. set pr. text document. to pam. content changes zero it's a a single item array. text that looks good uh so now as changes come through we're going to store them in our document store over in completion we can Now consume this document store and what we'll do here is we want to get our completion prams so effectively what we want to end up saying is const content is documents. getet document URI which we haven't parsed out yet um so let's figure out how to get document URI from our completion programs we'll hop over here perfect so we'll hop back down in our sidebar to our language features then our completion proposals and now we can actually skip over this and get skip over that and get to our completion programs so again completion programs has a bunch of stuff we don't care about what we really care about is that it extends this so we're going to copy that and we'll paste it into our editor and then we need to Define text document position prams it's going to be this come on you may have guessed by now that I don't normally nor use uh VSS code as my primary driver and sometimes I get a little hung up on some of the UI uh so text document identifier is the document URI we already defined this one let's take a quick look over in documents we did okay so we can grab text document identifier from there except that it's not exported so let's do that real quick and then we need to Define position which is uh straightforward we'll grab that from back here position is simply a line and character change these to number and to number okay so now what we can do is say cons PRS is going to be message. prams as completion prams and instead of document URI here we can do prams dot document. URI [Music] and don't really know what to do with this yet so let's just do a log. write and we'll write out content like that just so we can see and we'll also actually we'll say completion content great we just need to import log and We're Off to the Races import log from log log great uh we'll kick it we'll type hello we'll hop back to our log we'll wait a moment we're almost done waiting folks uh we'll scroll back up actually let's look for completion great so completion H um then oh man it's also in here huh okay eventually we get to completion hello though which is great so everything is working we have all of our content being stored in our memory our in memory representation of the documents we have our complete method able to access that content so now what we need to do is to use the content and the position for our completion to extract what's currently being written all right so we'll say con line we'll name this current line is going to be content dosit and then we'll get the position dot oh sorry rams. position. line all right this tells us that we've done something wrong here so documents actually could not have this content in it uh this will only happen in an extreme case where like something went wrong but we should still handle it just to make our types happy so if not content we're just going to return null we're we're not going to do anything here um null is not assignable to type completion list that's true perfect so now we can reliably work with our content and we can get the current line which is great uh so then what we want to do is to ignore all the content uh before the current positions current word being type uh but to do that let's do cons line until cursor is going to be current line. sli0 and we will end at pam. position. character this will take us up into the character so now we can say that con current word is going to be line until cursor. replace and we're going to replace everything until a non-word character then we'll capture whatever's after that and we'll replace that with I think the dollar sign one is what we want here let's test this out so we'll do log. write and we'll do completion and let's just dump these all out current line line until cursor and current Word Perfect to make our log a little more readable we're going to temporarily uh disable writing to it from our respond this will kill all of the uh completion results being dumped to our log we'll put this back though once we're in a more manageable place so we'll do hello there [Music] puppy and then after there we'll do Ma okay let's check this out all right for so for the last completion we have a current line which is there uh with an M at the end so we don't have our a yet that's sort of interesting oh actually I know why that is it's because we're saying is in complete false so we want to say true so that we keep getting additional completion requests let's try that again hello their puppy this should hopefully be better awesome their Ma so the line until cursor is their Ma and the current word is ma let's do a little more testing so if we type stuff after this and then we come back and do a we should hopefully still see ma as our uh so line until cursor is ma a current word is ma a there's actually probably an issue here though which is if we're just completing the beginning of a word do na there know our current word is still in a great awesome wonderful love it I was a little worried that our Rex wasn't going to work there but uh it did so that's always a pleasant surprise isn't it excellent um this is really great so now what we have is the current word and we can do some filtering so in instead of items always being this we're going to bring it down here and we're going to say the items is words. filter uh word and we will return word. starts with current word current word a little misleading um let's rename this to be current prefix great so now we're only returning words that start with that prefix so now what we can do is restore this because we should always have a reasonable number of items except in the pathological case of when we have a single character so we'll make one more change actually so that even that's okay over in our completion we'll also do a slice here so we'll limit this to the first thousand results and we can retest so if I do a the results come back immediately and that's great but I can still uh dig down deep into there because we're matching the prefix so even though we're only returning the first thousand results we see the prefix is working ma MZ no nothing there my there we go and then if we look at our log we'll see that this is actually uh these are reasonably sized responses so even in the pathological case let's actually just uh clear this out real quick so even in the case of us typing the letter A the most that we're sending across is uh 22,000 bytes that's still maybe a lot but it's super fast compared to shipping across the whole five million plus bytes uh and so this is cool so we could use this to build things like uh HTML completion we could have a we could filter so that if we have the less than symbol and then B we know that we can complete it as body um those sorts of things there's a lot of cool stuff that you can do with completion but we're going to stop here I wanted to close out this video by giving a brief example of connecting to our language server from another client language servers are client agnostic so we should be able to connect from any editor uh that supports a language server protocol I'll connect from neovim so in neovim we're going to open our after ftplugin text. file this is a code that gets a Val ated and run whenever a.txt file opens so I'll paste in our config here to get our language server started so neovim will start this when you open a text file or it will check to see if one's already running uh when you open a new text file vm. lp. start and then we give it some config we pass in the name of LSP from scratch our Command is npx TS node and then the full path to our server. TS file we're using TS node just so we don't have to trans transpile things to JavaScript first we can run the typescript directly and then we tell it to use the normal uh newm language server capabilities we're going to save this and then we'll enm TMP example.txt and we can actually do LSP info here and see that we are attached to LSP from scratch awesome so now if we start typing we get our suggestions as we hoped excellent we did it take a deep breath we're done we built a fully functional language server and we have a stable core that we can build new functionality on I'd like to continue this it'd be cool to add hover support code actions those sorts of things maybe we could support CSV files it'd be cool to do like some uh have some code actions for math in there or maybe we support some kind of web framework I don't know there's a lot of directions to go in another thought I have is building out some tdd tools so that you can do blackbox testing on both this and other language servers anyway if you're interested in this sort of thing give me a like hit the Subscribe button and let me know in the comments what you'd like to see until next time

Channel: Jeffrey Chupp

Views: 30,998

Keywords:

Id: Xo5VXTRoL6Q

Length: 69min 7sec (4147 seconds)

Published: Mon Jan 22 2024