HTTP Client - Secret Weapon for Web Service Testing

testing is an important part of developing any web service we all agree there are no doubts about it but what about the testing process is it obvious simple not so much so would you prefer a simplest solution i guess yes what if there is a way to test web services without writing lots of code which also helps your team they've got you covered in today's session you will see http client in action a special tool that ships with intellij id ultimate and aims to simplify testing of web services the live demo will cover not just the basics but also advanced topics like streaming file downloading and many more hi everyone thanks for joining in i'm your host mala gupta and today i'm pleased to welcome yuri ahmadinov from jetbrains we'll demonstrate http oh sorry yeah hi mama i messed up with the pronunciation of the last name and i'm sorry about that so yuri today will demonstrate uh http client which i prefer to call a secret weapon for web service testing and yuri it's a pleasure to have you present today again on intelligent idea live stream yeah thank you mala it's pleasure to me to present this material to all our audience and i really like our functionality and we'll try to show its strong points to you guys and tell you more about our features so you will be able to use them effectively in your daily uh routine and maybe uh you'll find the new hidden gems today and try them uh later so mala how is it going actually it's really good it's going great and i'm based in new delhi and it's really cold and i don't like winters and i'm sure it's colder at the place that you live yeah yeah we also have this uh cloudy evening today so what else we can do uh when the weather is not uh fine uh i believe we should learn some new stuff in this case uh actually in winter so we we may try to learn new things about http client for instance for sure and before i i let uh beauty take the stage i want to share some more details uh judy works as a software developer in the intellij idea team and he has been developing libraries frameworks and tools for the past 10 years so yuri let me ask you what is the favorite feature what what's yours favorite feature in http client that you're going to talk about today well if i need to answer exactly one feature i would say uh it's simplicity because hp client actually uses the same syntax as your application use when they reach uh http services so it's basically the same http protocol and it's simplicity is actually a feature the best feature for me here absolutely and with the development of software getting more complex by the day i think having something simpler would be a welcome for all the developers i think and before i let yuri take the stage let me share some quick housekeeping details everyone who is watching please use the youtube chat to post your questions you will answer the questions towards the end of the presentation but you don't really have to wait till the end of the session because alexander from the idea development team and i will answer your questions as you post them and the session is being recorded and will be hosted on intellij ideas youtube channel so if you haven't already subscribed to our channel now is the time to do that and if you like today's session which i'm sure you will please like the video now yuri let me add your screen to the stream and let me take myself off the stream good luck to you yeah thank you uh uh welcome guys folks to our session uh thank you all for joining us here let me uh start our uh session right now so i'm working at jetbrains intellij idea team but also sometimes i do micro services and begin development that's why i really like uh our tool called http client and today i will try to show you more and more exciting features uh in this uh tooling available for you in our jetbrains ids such as intellij id phpstorm webstorm and other ultimate ids so first of all i should mention that there are so many tools available for web service testing nowadays from a simple command line tools such as squirrel or http h2pi maybe ui tooling something like postman or insomnia and countless ui clients available also we have 10 or or maybe 15 plugins for intellij idea available on our marketplace that our ui clients for http requests but today i will try to show you our approach to this task that involves a simple file format that enables you to send hp requests yeah and i believe you'll like it so hp client is a tool available in intellij idea and it is based on http protocol syntax you'll see in my screen that here is a plain dot http file that just looks like a real http interaction between two uh apps client and server and here we see like hp method url some headers and the body and that's it it's like when you try to send your request from your app it will look basically like this so that's why it is kind of simpler maybe for uh many people than trying to use uh ui or some common line tools so let's uh try to see how it works uh today we have so many uh topics to cover we'll try to see basic usages and integrations intelligent idea and after that we'll see all the new features that are available since the previous stable release of intellij idea let me start with the basics let's go to our id let me go on to here what is this this is a microservice app based on spring framework but it is not that important for us today let me just start this app here we have three uh services it is uh standard rest services uh users products payments some domain uh here about this entities and uh those services they expose http endpoints and when we uh go to our project let's say i'm a new guy on this project how can i find all the services exposed my uh by my uh application there is a dedicated tool window for this and i want to show you this view endpoints here we can go and find all the rest services available in this app and let me wait for for the end points so we see all the spring services available for us uh well we start the app by the way i am starting this all three services all together using services to window which is available for you too so you can use it for uh all the rank figurations you are used for development so here we have three apps up and running so let me go to endpoints so here if we open endpoints to window we'll find that it exposes some like end points and if we try to maybe choose something let's say slash users here we can navigate to this uh endpoint in in code but also right here we uh like see this http client tab and this compact uh i would say panel provides you with a simple action to call your app right here without changing anything without writing code we just submit request and see the response right here it is empty right now but you can trigger your end points without even opening any additional files without creating any additional files you just go to endpoints to window open http client tab and that's it we can trigger some uh business logic with this panel it's like our first introduction to http client but also if we wanna uh deal with more complex queries such as if we need to enter some path variables or parameters we can go and create a special file let's create it new i know it will be dot paths.http uh extension of the file can be uh either http or dot rest uh we choose http here uh i will even add this file to my vcs which is git in my case so you uh can just commit this file to geared and share it via i don't github gitlab uh which one you use in your daily work so here we have this paths.http file let's try to run some requests to our app let's start with uh some uh get queries uh let's say get i wanna trigger some http localhost uh services and here i have a code completion available for me uh which can uh find all the endpoints of the app all the endpoints from i don't know open api or swagger specification uh it can also uh find uh various endpoints declared in your framework such as spring framework or maybe another framework micronaut juxta something else so here in my case i have users and we can go and say okay i wanna find all the users i will enter my headers here let's say i wanna have uh json as a result that's it and now we can run this uh query or request yeah uh we can go and just click here or run the corresponding intention action we'll see the same result here because we haven't yet created any users uh let's try to enter uh the second request to create something it will be post in our case i will uh post data to the same sorry to the same endpoint which is users and it will be payload of the type json here yeah if i use this special content time or json or maybe xml in this case i can start typing my body of my http client and http client will understand that this is json actually so i can even edit this json fragment in a separate tool window uh if i want but i prefer inline representation here so let me enter some parameters let's go to users and define which parameters are required if we need to create a new user so here we have various usages of our user's endpoint let's go to this and here if we post something uh we need to fill uh some data and data is actually a name name of a user so in my case name will be i don't know peter for instance okay and we can now go and run these requests uh i will use alt enter shortcut this time and use this run localhost intention okay so we post oh no not ssl yeah i don't have uh ssl available for uh my local host so let me repost this okay so now we just send this post request to our web service with some uh headers which is a content type and the type is json and the body body here is just this json payload it can be xml it can be um even binary data if you wanna i don't know send file for instance it can be um you know form uh encoded here on body or something else so you choose uh here how to write the only uh requirement here is that you have this separator like empty line before this body and when we wanna create one more request we uh divide them or separate them with this uh special um block of uh uh sharps this signs yeah to separate different uh requests here well we now uh tried basics here and tried to send some requests to our web servers uh let's think of uh more possible like application for this we uh not only can send a single request um we can also send this uh all requesting file in patch so we can just run them all together and see all the statuses here so with time and responses and also can open response in separate file editor as a response file so we have this available view and i can go and commit this file to my vcs as i said before well great how about parameters if i want to pass some i don't know url parameters you say then we just use this your parameter as right here and this syntax will show or we will send the query parameter to our web servers so if you need your clear parameters just uh attend this question mark to your url and you will have this query parameter available well great what if um i need some variables in my queries you say for instance i want to use different hosts not only localhost but sometimes i have stage and environment or even maybe production environment then we can use special kind of files called environment file let me show you this action if we go and click on add environment file there's a there are two options available uh regular it is not private file which you can commit to your vcs and it should not contain any of sensitive data please do not post any passwords there it is kind of you know dictionary of data let's say dev and in this case we'll have host i don't know localhost 8080 development host but in case of staging for instance i don't know uh we'll have host maybe oh sorry uh host equal to staging uh 88 for instance or saying 80 for instance yeah well now let's try to use this host variable in our query i go here to my uh this yeah and in this case i will be able to send uh my query to the host variable yeah so we just use this car liberated syntax and now we see this is connected to our environment file this environment environment file is regular it doesn't contain any sensitive data so i can just share it through my vcs and that that's why it is marked as green now let's see how it is applied if i click on my run gutter icon then i will see this run with dev or run with staging environment so if i run with dev it goes to localhost 8080 but if i switch to staging environment it goes to staging 80 which is a known host in my case it is not available to in my network right now but anyway that's how we can use environment files and even commit them to vcs in case we need to use some uh parameters uh well and if we try to run with this host it is invalid area of course uh well what else we can do with all this functionality we can uh not only uh use it to receive or send requests but also we can navigate from here as you as you already see to the source code so i already tried to navigate to our user's endpoint and here i see all the usages of this url pass in my app not only spring controller but also a user's client which is a client-side of my microservice in in the code so using just simple uh go to declaration action or control plus p shortcut on windows and linux you will be able to go to declaration of this endpoint so it's kind of simple navigation to this also you can go to web browser if you use um special intention action alt enter open in web browser that's how we can go and just do not copy this url manually but open this url in a browser right from the code well looks great now how we can deal with running this query if we don't know i don't wanna open them uh like don't wanna go and find which file contains all the requests in this in the latest table release we announced uh special naming for our requests for instance i want to name this query as create a new you know new user well now this uh request will have this identifier and if i try to use uh run everything run anything shortcut uh control control in my case and type just create create a new user i will see all the requests with such a name available in my project so i have various files with this uh the same name in my case it's bets http and if i just hit enter right now it will run this requests open services tool window uh show me the result of the requests and that's how you can easily trigger some hp request without even opening this http file so it's kind of uh easy task for you right now you just go and use this run anything action in your stuff okay uh that uh this is basic usages of our http client you can also use additional syntax i would say uh how you can learn this first of all i wanna show you this examples link of course if we click it you'll find uh various topics get pause request with authorization with tests what's new and if we open some uh examples like post requests you will be able to find here various examples of how to deal with files how to send requests with dynamic variables and many many examples of such usages well that's how we explore this uh stuff and that's how you can get acquainted with http request syntax but basically this is based on existing http protocol and if you know um i know see your command line utility or if you use http pi uh you will be able to use this without any differences just go and type your method and headers and body and that's it and you'll see the result okay what else we can do we can also try to test our app with our requests not only trigger the predefined like static uh like requests but also try to react on the result of the request for instance if i wanna go and write a test like real test like a unit test or integration test here i can create such a block of code with this syntax and here you'll see you can write javascript let me write it from uh from scratch for you okay so we gonna write a new test for get we're gonna use a predefined variable called client we create a test and test will be users response is not empty for instance here we declare a test function like this and we provide some assertions first of all we want to check that the result the response code of the http response is okay it's uh 200 so we say uh assert response which is predefined variable do and we check status and it should be 200 and we provide some uh result is okay and let's run this request well it runs if we switch to tests tab you'll see our test is here a user's response is not empty it is the exact string that i entered in the editor that's how we can add more tests and let's add one more test which is which will be not success i don't know response is result is ever error yeah uh such a contradict uh requests user's response is i don't know this um and then if we go to test we see all the test functions uh one of them is okay and one is failed that's how we can test statuses well now let's try some tests that will check refund body because we actually do not test only statuses we need to check some data now our response body will be we can even navigate to api declaration of http client with a control plus b and body is actually a string or line stream response or object and here we see this is response content and it can be json object if response type is json so we provide json from our web service that's why we can easily use nested property access here and denote parts json manually so we just go and let's check uh length of this uh users we we test users endpoint users and point provide list of users actually and if we go and use a length property of array in javascript it should be [Music] not now or even more than no okay so length is not zero okay and let's start this so our test like runs okay and if i let's say change this to less than zero it should fail run this test failed one okay if we switch to test we see this a lens is not zero uh failure we should uh go and check our application uh something went wrong uh i don't know if something has happened in our application that's like basic usage of tests in our http client you can use them uh like to quickly test your app or maybe even share some test procedures i don't know like a acceptance test for your staging environment for your deaf environment so you can easily run all the tests in this file and make sure your app functioning like as expected okay and see no failures in communication and i don't use any like heavy uh technologies here only http file plus some uh javascript code which is uh simple synchronous uh like asserts on the result of the response well it's uh quite quite simple and easy to write so you don't have any like complex thing here well let's maybe talk about some uh advanced usages let's say your application requires some authentication which is our daily life we usually protect our web services with some authentication it can be i don't know um not only basic but maybe some uh session protection maybe something else like all to protocol or maybe more advanced techniques but anyway how do you deal with authentication the easiest way to deal with authentication is provide first of all provide authorization token in authorization header if you use authorization header in your app to authenticate requests from your clients you may provide token as a inline data right here like really token but please don't do this uh i would strongly recommend do not use this inline tokens because they expire they should not be shared through vcs and you should not commit to it to version control please instead use environment files so if i for instance go and use here available with name bearer token uh let's share it uh let's extract this variable to some private environment file so it will be my own private file which is not shared with anyone in my team i go to add environment menu and use private option right now so it will be private for dev it will be bearer token and favela will be i don't know uh some for instance uh and i should not add this file to vcs so please please don't so you can go and add this pattern like star dot private dot and json as files that should be excluded from version control so in this case it should not be committed to vcs well now if we go back to http file and hit and run with with dev environment you'll see we uh provide this authorization token to our app but this is the simplest example yeah then we already have token we should not uh like go and uh authenticate what if we need additional means for authentication such as uh oauth.2 uh 2.0 protocol let me go to the second example project here and close all this close all this runway up this app is uh using a separate authentication service uh maybe you know this it is called key clock key clock is a well-known known product for authentication it provides various means for authentication and role management and user management and many more now it is running inside my docker environment so let me show you my docker compose file so it is a key clock docker container i have predefined users here in json file and this service exposed on this port on my machine and my app requires authentication uh token uh to proceed with uh some queries let's say if i go and try to use my endpoint like api users me without any authentication headers without an authorization header and go and run this well it says unauthorized so you haven't provided any uh authorization tokens how we should deal with it first of all we go and obtain this or ours to token and i have a special hp request for that in this file so i just post um some payload which is uh form and it includes my client id it includes my password run type and just hit here it will responds with access token so i can go and copy this access token to my private file uh well i used that dot admin this time so i have different environments for different users so path dot admin have this access token and if now i go and use this path dot admin environment and pass my access token to my service well it responds with that.admin nice so i see the result of the request and if i need to go and run my request with different user i use paths dot user environment file and it responds with paths dot user so nice we we can easily store the success token thing in a private environment json file and do not share it through dcs and just use it as a variable do not change and do not in line your tokens to http file just use your environment as a provider of this token thing nice okay so what if now we want to use not simple request that just responds with text but what if our servers respond with some uh binary data i don't know maybe uh image or maybe something else maybe file or i don't know let me show you how to deal with it first of all let's find some endpoint that provides uh binary data in my case it will be uh the most hated thing of this year of course this is qr code uh you know this yeah and because everyone uh should need this uh should have this and we will generate a request here so you can either use this drop-down or hit on line marker here to generate hp request so i will go and [Music] get this qr png file with authorization let me go run and that's it and if this response can be interpreted somehow by hp client for instance it is uh an image i don't know maybe png or gpa or pnp image intellij idea will show you result as an image as an actual image in your services to window so you'll see the preview of this file but also if you don't need to see this preview you can scroll down and click on uh result file yeah it will open separate file viewer for you of your operating system so you can also go and just click on the result a hyperlink to see the resulting response of the server that's how we improved uh our response view recently in the latest intellij idea version okay sometimes our service will not respond but instead provide redirects let's talk about redirects yeah in my case i have some spring service uh which do not do any real work but instead it responds with found hsp response and redirects to some animated gif available on our website if i go and just try to request this resource well let me check this should be download shake uh-huh it's forbidden bits of mean yeah it it it actually silently redirects me to some uh resource which is uh research advanced.com i don't see why this sell error here but uh i see that my response is not shown response is actually a redirect how to see the actual rhetoric there are special options available for you if you prepared your request with this no redirect directive and you can check all the available options with a code completion here then our http client will show the original response from your service so now we see this response code here which which is found and not a cell error when we try to reach to resources advanced chrome but the actual response of our servers so that's how you can deal with uh various options uh for your interactions it is not only no redirect but also we have no cookie no log use ois credentials so you can use this uh special options in http client to assume behavior of um execution of http requests uh well uh what else we can do in the recent release of the idea we also supported streaming of results let's say we have some endpoint which try to fetch uh data from database and it takes so long that we cannot uh like wait for all the results but instead we want to show them right at the moment when they are available to user so we like streaming data from our database to our clients so they can show some progress instead of waiting for all their data available let's go and try to check this for instance if we have such a request that that uh uses stream plus json content type let's hit this with our http client uh-huh oh it's it's amps here with uh what should i do that's admin not let me let me check my credentials uh maybe it's that's the user no let me check the app what's going on here well we started the app we hit oh we do not provide uh authentication of course i forget that i forgot about the uh authentication let me provide some um authentication here so authorization uh well it should be there run with that's mean okay now what we see we see that server respond with this special kind of response application json stream plus json and we see results and they coming and coming to our response that's how we deal with streaming responses and that's how we can test them um like in http client and see all the results it simply provides uh time when uh it generated a new result so we just stopped this and we should not wait for uh infinite results because they are infinite we should not wait for the response data completed okay that's how we deal with uh streams and in http client you just go get this we detect content type so we read this and that's how we deal with it also we support extreme so not not only uh stream plus json but also if you use uh stream plus text it will also be handled by us uh and you will see this uh coming and coming and coming data to your output well what if at some point we want to collect all the data to some file because you know we want to have all these responses somewhere let's see how to deal with it then we just use this redirection syntax like data.json this let's run this and this uh request should collect all the data that is coming to this uh data.json uh file for us actually yeah that's how we redirect respond response here [Music] stop where is where is my file well let me let me check the maybe some finite not infinite but some end point where we have results uh right after they and we go and use bets with admin users and then it is empty well this but access token which is red stock user well let me go and try to that's admin no not this let me go in here here yeah i should use this because i have my environment file here and it is not available in the in this second uh file sorry well it will be page data json in my case and in this case we data will be saved not to this predefined file but to my file uh that i just mentioned here so response file saved and it will be http request data json that's how we can you can redirect the result of the request to any like file or even rewrite it were appended there are various options available for you please check them if you want to have for this like fixed name of file available for you well that's not everything we have in http client this release we also introduced a special support for grpc protocol so you can not only send plain http requests but also send some requests to grpc services let me start with this i have this jrpc service in my app called greetingservice like it's just greet someone um it is implemented as java program uses google grpc implementation let's start our server it should be running on board like a http server but it requires binary request response in a proto format and i want to test this jrpc service how can i deal with it i go to my proto file find my service and just click on this landmarker to generate this request here let's say i want to greet someone i don't know maybe i should provide some parameters i go and use autocompletion here to see all the parameters of my payload which i mentioned in protofile i will use name let's provide some i don't know bob and we run this okay and the server will respond with hello there bob so what is going on here we have a dedicated method here called grpc which is reserved for jrpc implementation of http client it hits grids in service and we can even navigate to protofile and its method greeting with this payload in in protoformat of course and we received this response so now you can not only test plain uh rest services web services but also grpc services uh using our integrated solution so you can you should not even open any additional uh tools download any additional command line utilities just go and use this this is not the only way we can start let's remove everything let's say i don't see this protofile i don't have it in in mind where it is available instead i go and just write grpc i have localhost i see all the services in auto completion what's a greeting service and it's greeting here is also here and i provide the same payload for name i don't know sully for instance that's how i can explore my jrpc services without even opening any specifications in my uh project and just go and test them so please enjoy use try in your daily routine maybe it should help you somehow to deal with your daily tasks fine well let's recap this so this release we have tons of new features please try them we provide you with binary data with you we supported tech streams and json streams we uh like provide you with a grpc client also supported by various output redirection options and we supported hp request id files so you will be able to identify them run them from around anything uh find them share them and uh many more so please check uh our what's new page of intellij idea to learn more about all those features and how to deal with them but also this time i want to say huge thank you to our community because our community is so awesome we uh have in this release previously our user linux china released a separate plugin on top of our http client which makes it possible to not only test http and jrpc built-in options but also brings our circuit support to http client so thank you uh please subscribe to their uh twitter twitter handle is the same linux china um huge thanks from uh intel jita team uh to you folks uh please um we all your extra plugins for intellij idea are very welcome uh thank you uh please try our socket request and http client plugin if you use our socket in your apps so we really enjoy having this extra plugins on top of our http client and we have more plans actually we will try to bring different protocols to hp clients soon we are thinking of graphql right now maybe we will uh talk about something else uh to if you have any feedbacks on w which feature let's say is missing in our hp client please reach out to our issue tracker we always uh read all the issues from you folks please don't hesitate to post them up vote them we have a couple of highly voted issues and we like checking how to deal with them right now again you can also check uh our documentation which is available in intellij idea documentation all the examples mentioned here today available there i will also share this presentation after the session so you'll be able to check this links again well i believe that's all folks from me today we should have many many many questions right now let's try to answer them all mala first of all thank you so much yuri it was a great session i was about to jump in to remind you about the time but you finished right in time so that was amazing we had a lot of questions but alex did an amazing job of answering a lot of questions a lot of participants were also answering each other queries so that's again amazing i'll quickly jump on to the first question so one question which was asked by multiple attendees was can i use a response from one request as a request parameter for another one for example auto update of access token yeah right now right now it is possible to only store some response to file and then supply this file from this like disk file to the second request we are thinking of some pre-processing for results let's say json path or xpath uh pre-processing for results so you will be able to query some data from from the result and use it as a variable later right now we are in like in the middle of designing the solution and thinking how we can bring this if you have some thoughts or maybe even desired solution for you please uh reach out to our buck tracker and don't hesitate to post your like even feature requests if you think of some feature it is really good to know about uh it uh from our side we really like all the feedbacks here guys providing on on http client right and um so yuri i was not able to concentrate completely on the presentation because i was trying to keep a track on the question so if there is any question which you already answered in your session you can just say so so the other question is is it possible to inject dynamic values for example authentication data from external files or shell service call yeah right now we are injecting external data from environment files which is kind of static i would say it is not that possible right now to hook into the dynamic part with javascript we should think of it it's not that it's not yet available sorry okay uh the other question is um how does one convert their existing curl requests how does one uh sorry could you please uh uh yes can http client be used to convert the existing girl request that they already have see you are a request yes uh let's say if you use the uh some url request or maybe you stumble upon some c url example on the internet and you want to use it in your http file there is a special option convert a c url to http request we just uh have this right here we click convert and then we have this syntax available for you so we just process all the options of your url convert them to http client options and you use http client instead of url right now okay and the other question is is it possible to run all the requests in patch yeah i already mentioned that if you want to write run all the requests it is right here on top of all the options so we just click run all requests with um we choose environment dev staging or no environment and they all run in batch you see all the results uh some they fail some they will succeed and you see all the statuses in services to window right how can a developer deal with the cookies yeah by default http client stores cookies provided by your web service automatically so let's say if you go and log in using password authentication with some i don't know post a request and your servers respond with a set cookie header so you receive some cookie hp client stores this cookie on disk and then sends it again to this service when you'll send subsequent queries yeah so it will manage them automatically but also it is always available here as header value you may provide your own cookie values as part of cookie header manually if needed thank you so much uh i think uh alex has answered all the remaining questions and we are already seeing a lot of thanks messages so there were a lot of good comments um and people are really appreciating the content and they really liked uh your live demo today so thank you uh yuri again again a pleasure to have you present yeah thank you guys thank you folks see you again maybe next time we have different topic and different conversation thank you and yes before we say bye i want to thank everyone who is involved uh eda from the marketing team sergey from the video team and alex of course from intellij idea team for making this happen and for helping the and another amazing session i would say and everyone who's watching and following the intellij idea live stream we have another session lined up next month which is on 16th of february so stay stay tuned for the details and until then bye bye you

Channel: IntelliJ IDEA by JetBrains

Views: 22,778

Keywords: HTTP Client, web service testing, intellij idea, HTTP protocol

Id: VMUaOZ6kvJ0

Length: 57min 22sec (3442 seconds)

Published: Wed Jan 19 2022