WAF Bypass Techniques Using HTTP Standard and Web Servers’ Behavior - Soroush Dalili

hello everyone good afternoon I am Suroosh steadily and I'm working for NCC group as a security consultant welcome to my past techniques so today I'm going to talk about a number of methods to bypass web application firewalls these methods are not new probably most of them have been discussed in the past but they have been overlooked so I'm going to discuss them today most nodes contain a lot of HTTP requests so if you're a HTTP logger you're in it for a tree so if you're a pen tester and you have a verification firewall in front of your web application when you're doing web application testing you're gonna be mad it's it's a nightmare because the testing is very slow and it so and it doesn't provide a satisfactory result because some of the automated testing requests are going to be killed silently so you're not sure if for example all your scans that means successful as and as a human being you can make mistakes you can't cover everything manually what you may ask you're the hacker you have been trained for this what can't you just boy pass the rough it's part of the assessment the answer is time so we don't have enough time during a bail application assessment to basically assess the valve as well we can only test the web application assessment during one week or two weeks that we have for the web app and then if there is a raft there then what would happen is it's going to block us and it's going to slow down the test so you don't get value for you for your money and also it does reduce the quality of the reports because it means that you are not going to have all the issues in your report at the same time math effectiveness test is completely a separate assessment so you basically if you you have to go for that assessment if you want to assess your valve or you have to go for a normal web application assessment so where can you find someone to bypass maybe first of all if you have permissions then you can do this you can just say and basic cross-site scripting or sequel injection to a website and to see if you can receive one of these error messages access denied or something and it doesn't matter if the parameter is valid or not you can just send it over and see what would happen if you get one of these access denied probably there is a path and you have to bypass it if you are doing tests on it so that's or based on whitelist or blacklist might lists are more secure you basically you have it it does only allow a good thing to go through and blacklist or less secure and they have a database of things that are bad and shouldn't go through that said whitelist vas or very hard to basically be trained to be configured you need a specialist for that for them and then you basically there are very high it meant high maintenance so if you change your web application what would happen is that you have to train you laugh again and you have to tell the valve that this is a new page these are the new parameters you have to allow this to go in so all that it's just hard work and it is very very expensive blacklists on the other hand or chip you can just basically pay a small minimum amount of money for a buff in the cloud put it in front of your web server and that's it and normally you don't need to change any configuration on them saying that sometimes you still need to do a little bit of configuration changes but as soon as you do that it's fine it everything is fine with the blacklist months so the blackness months are the most popular ones and the ones that I'm going to discuss here today before I start with methods and everything let's just talk about something that is out there and most people know about it you may know that it's very easy to put a rough in the cloud in front of your website by changing the DNS server to the rough company's DNS all the requests are not going through the valve and you're secure however if someone knows your IP address they can see they probably can still send the request directly to you without going through the bath thinking that the IP address is bay sickly a secret and you can keep it secret forever it's completely wrong there are a lot of methods to find our IP address and sooner or later someone will find the IP address and you vuf will be bypassed so if you're going to do that my recommendation is to set a very like a secret header or something in the in the vast panel and on the web application side you basically have to say if you can't see this header it's not coming from the bath so just kill it yes that can make a bottleneck and if there is a denial of service on the bath you may go down but at the same time it will it will make sure that you always have a bath in front of your web applications and these baths in the cloud are very useful they give you a lot of statistics you you don't need to maintain them they are very cheap that's why many people are using them these days so I have categorize the vast boy passes into a number of categories so the first category that you everyone knows about or the newer mist pilots so if a payload is not in that blacklist database and use that obviously can go through because it's not there also if you start changing the payload for example change the capitalization make it uppercase or apply encoding and do these kind of things on the payloads then it may go through as well at the same time some of the buffs have some exceptions set so developers think ok I we have we have this like certain user agent old Nokia phone but whenever they're browsing our website we're blocking them because they are sending something unusual so let's just say whenever this is the user agent you don't need to go through all these rules and all these last so if you find those kind of like exceptions you can bypass the buff easily well it's kind of sometimes it's very difficult to find those because there are secrets but also there has been something in the past that if you were sending a large request to above you could have bypassed it as well because graphs were like this is very large for me to handle it may cause then a lot of service it looks like a file upload I can do anything about it and without inspection it can go through so that's another method however today I'm going to talk about payload delivery and request mutation techniques so let us start with payload delivery there are a number of methods that you can send your HTTP requests to a server so depends on these methods if laughs don't know about them they can be bypassed so for example a few days ago I saw this a very good vulnerability blog posts about unsupported SSL TLS ciphers boy the bass so basically the guy who found this or identified this issue that there was a bath and when the bath couldn't understand this SL cipher and bought that cipher was being supported by the web server was basically passing everything through to the web server and was boy passing the bath and it was very good research and I was really amazed by that and I was thinking maybe you can just apply to HTTP 2 as well I don't know it might be possible but then now if you set that cipher in bear suit and put it in front of like input it setting on the browser then maybe you can just bypass the bath and just test as usual today however I'm just going to talk about HTTP version 0.9 and HTTP pipelining because those are interesting and also normally developers even testers don't know about them that much and they have me forgotten so I'm going to talk about those things so let's start with HTTP version 0.9 this is the this is basically the first HTTP that came out didn't have any version it was just one line of a get request get a slash that's it or get a slash page that's it that was the request no header was so being supported no HTTP version nothing it was just a request to an IP address and that was it so and so it's very old and because it's very old most of the baths are non-new they may not know about these these kind of like requests so but what can go wrong with HTTP version it doesn't support header it doesn't support a lot of things but and and the latest version of RFC for HTTP 1.1 says that it should not being supported at all but during my research I realized that all the web servers that I was testing we are supporting HTTP version 0.9 is still like today so it's being supported at the same time you can still send an absolute URL in a get request so it can be like this here so as you can see we have get and then you have an absolute URL and domain name here and some parameters and there is no HTTP version as I said so you can still target those applications that accept requests using get and there are just some parameters interestingly Apache Tomcat supports headers with HTTP version 0.9 it's just against a standard but they are doing it there it's just fun so now you can whenever you're testing something on apache tomcat you can just send her she'd be version 0.9 and that's it it's gonna work they don't care about it and okay and also this talk in DEFCON 20 for hiding wiki's in HTTP basically covers some interesting issues using HTTP version 0.9 I really recommend you to actually go and look at that and this like the short recommendation here is before I start anything with HTTP version 0.9 just disable it it's safer it's not being used by anything just disable just make sure it can't go to your web server so how can you send HTTP version 0.9 I tried fiddler I tried the boy tried web site I tried all these like web proxies none of them could send HTTP version 0.9 actually bear sweet could but it was not showing the response so I had to use Wireshark but you can always use net-net cat OpenSSL so on so forth but who wants to use these tools when when testing about application you really don't want to use those low-level tools so I'm going to show you a method to sent HTTP version 0.9 in birth suite using HTTP pipelining so let's start using HTTP pipelining so what is how should it be populating this is very quick introduction and I was stolen that images well from somewhere so in HTTP you can basically send your request and then there is a response send a request there is a response however they introduce pipelining feature to HTTP version 1.1 and 1.0 so now you can basically send multiple HTTP requests as part of one HTTP request and the server will respond to them at the same time however when I say at the same time if you process the first one then the next one then the one after and then we will send all the responses in one request and one in one response and you will see everything at the same time the only bad thing about HTTP pipelining for Ted for testing and for bypassing laughs is that it's hop by hop it means if it goes through the proxy it's up to the proxy to decide what to do with it some proxies basically will say okay I'm gonna send multiple requests I'm not going to continue sending HTTP pipelining some proxies just let it go and some other proxies just send the first request it all depends on how the actual proxy works but on the standard they should really send all the requests how it's up to them so here is an example of HTTP pipelining so as you can see here this is the first request imagine this is in pursuit for example and this is the first request that's a get request and then I have the post request immediately after because the first request is HTTP 1.0 it's just I chose to use that in here just as an example I had to include connection keepalive otherwise it would think that connection is closed because that's default on 1.0 and it wouldn't do HTTP pipelining for 1.1 I don't need to mention this because by default it's on keep alive so it will go through fine and you can send multiple messages you don't need to send two but I could only fit two in these slides so and I can even send them the other way around so I can send the post request first and then the get request however if I'm doing that you have to make sure the content length is actually showing the characters that are in the post body of your request otherwise the server won't see it as you can see I've just misplaced them so the server this time is going to see the post and the negative request you could have had three posts requires one get requires anything this is just an example so how can you do that using Pepsi so embarrassing if you want to do that first of all make sure that the content links update is disabled otherwise it will update the content lengths and it will ruin it for you also what I found was if there is accept encoding sometimes you can see the response in the response panel because virtually doesn't show the responses when it's compressed and it's pipelined in here I'm sending two requests at the same time with one HTTP request and as you can see I've received the first response here and the next response followed by HTTP slash 1.5 200 the status code and the response for the second request so you can use this to basically send HTTP pipelining to a web server so what has this to do with the HTTP version 0.9 because now I should if you send just this get without any kind of like header yep you can't see the response also if you send used other tools they will say it's invalid and they will not send it over but if you send it through pipelining you will see the response and that's cool because now I can use my favorite tool to basically perform my test so this really happened in one of the tests I had so it wasn't admin but it was something else so imagine there is a website that has a wife in front of it and that valve has blocked the word admin whenever it sees admin in the URL anywhere it blots the request and it cares about it doesn't care about the character cases so it's case insensitive so you couldn't boy pass it by opera casing admin or anything like that also it had a feature that was supporting URL encoding so you could understand if I was encoding it so I had no way to bypass it also directed traversal in here can't work because it can still see admin in the request so what I did basically i targeted a page that was a normal page indexed DOJ's me in this example and the content links here is 10 so as you can see I have 10 characters before the get request to the to the target page and so I sent that request an event through the valve the valve didn't know about HTTP version 0.9 and because of that it didn't even see this as the second request just let it go to the web server and in the response I had the response of index dot JSP as well as this receptor JSP page on the admin side the now imagine if that reset page was resetting something on the server and because this is JSP on apache tomcat i could have included headers and this is the this is an example here to confuse the vApps even further so apache tomcat also supports carriage return rather than the whole carriage return line field so basically after you have your first request you can start the second request in HTTP version 0.9 you don't need to mention the version so header starts from here because I have a carriage return here and then I'm sending a post request because why not Apache Tomcat supports it so that can go through and that can really hit that like add user dot JSP page so if you have an Apache Tomcat verbs or even more confused can be more confused in the example that I had so the solution we found was there was a tick box somewhere hidden inside the bath and it was saying disable HTTP version 0.9 or something like that as soon as we did that it worked one but by default it wasn't it so it should be version 0.9 by default was supported I think it should be daughter very round I have also created a poison like a module or just function that you can use if you want to if you want to basically send your requests they're not using burp suite just using Python because you need HTTP pipelining or something like that it basically fixed the pipelining for you it fixed the headers the it cares about content links and everything so it's very functional and you can use it if you want to okay that was about the delivery and now I'm going to talk about request mutation that's the actual part I like more because I can do more with this so what is request mutation and what is this category so basically web servers may act differently depends on how they received the requests something that is valid for one web server may not be valid for another web server because they have implemented things differently and when this happens when there are kind of like different features or different things being supported by the web servers you probably can't find boy passes because graphs can also implement them differently so we know about like HTTP may know about HTTP parameter pollutions like if you duplicate some parameters if apps don't know about it they may be able to find a boy pass the directory traversal and things like that might lead to a boy pass as well today I'm going to talk about miss shaped requests but before doing that I'm going to talk about RFC so you can basically it's like I have seen other talks also in app psyche you they were saying read RFC find the vulnerabilities that's true so if you really want to find some good vulnerabilities read the RFC find the vaguest statements like recommended suggests that may may not so any web server or web application firewalls or proxies in the middle may have implemented these things differently and because of these differences a request that can be that is understandable by the web server cannot even be parsed by the valve so that is where it becomes interesting however that said if you're just looking in normal features in RFC that's probably know about them because they're they're known they people have talked about them in the past of us like the verbs are not as stupid they will go and read or if sees as well themselves and if there is something like this they understand it so what what this one is it's just an example in RFC says basically it has been obsoleted in the latest version but it is a still being supported you can in the request that you're sending you can have the header value in multi lines as long as the next line a start with a space or tab long tab character then it will be part of the the header as you can see it here for example host header and then I have some space characters tab characters and then I have a domain name that what that had been filtered so basically I haven't boy passed any buffs using this technique using like line folding in headers but in the past I've both has some websites that they're filtered by the company I was working for using this because I really needed to access that Facebook page so that's it now I'm going to talk about custom implementation that these are the things that are basically very interesting these are the things that are not in the RFC but then developers of the web servers or proxy decided to basically support this or develop web technology support decided to support these kind things because they are cool because why not so the only way of finding these things are to fuzz it to basically go there and see what would happen if I add this character if I do this or do that and I really like the semicolon character it does different things in different web servers and web applications to just give you some examples you know like in the in the URL or in the body of a request ampersand is a delimiter between parameters however important Django you can replace the ampersand with semicolon and send it over and it will understand it as a delimiter so if there is a valve that cares about the parameters and it says it has a rule that for example menu C is admin equal to one or something just like to request if you add a parameter like this and add semicolon after that and so on you can basically bypass it and the other thing is like ASB Classic on OAS is always crazy about stuff like just to give you an example if you have a person character not being followed by a hex value it can't be seen from an HP classic application so percent s is equal to s that's quite good also it converts it converts the characters in UTF like for example utf-8 so when you have like - a or something like that if you convert it to a and in here like we have I here letter I this is in utf-8 it's probably I some sort of avi dots on top of it or something like that it will be converted to I and it will be a script again so you can bypass buffs using that even some of the like a conte cross-site scripting features on the browsers can be bypassed using this features and so Apache Tomcat uses Simic semicolon in the URL path differently as well so anything in the path when you're sending something to Apache Tomcat is a comment so unless it sees a slash so basically this path here is equal to this the red one in front of it so part one semicolon foo is equal to slash path one so you can still go there and see a page if it cares about different things in the URL however I'm going to talk about something completely different today and that's content encoding that is what I like and I think it hadn't been covered before we published some blog posts in NCC website so basically if you read the RFC it says yes we can have content encoding in responses actually it doesn't say responses it says has TTP messages and because of that you can use content encoding in the requests as well as the response response requests so what would happen is if you use care set in a request you will encode all the characters that are inside the request and you will send it to the server from server point of view it's completely fine and it's normal but it will bypass all the buffs in the middle so request encoding is also can also be challenging because any web server can implement it differently there I couldn't basically so when I tried HP classic on Ras did not support it also Apache is when I had PHP did not support it so that's it but for a sspx Apache Tomcat and for Python to entry on Django when I tried it worked however if you look at this table you will see the differences any single one of them is different than the others so if you really want to use this technique during testing to bypass maths you will have a hard time to create a tool and you need to customize your tool every time and so this is how you basically need to encode your parameters you this this simple Python code here can be used to basically encode the payload using IBM 0:37 and then it will give you this I have your L encoded it in here and that's it it's very easy however you can't use this easily during a test we need something to basically automate the whole thing and I have created a burp suite plug-in extension called HTTP smuggler and you can download it from github it's accessible it's still not in bad pasture because this is an early version of it and but there is a Java file you can just download it and add it your birth seed and you can select the policy you want and if for example you're testing a new web server that is not there then you need to use the custom one and just go there and see which one will work on the web server just give you some examples like this is CloudFlare and I'm sending a sequel injection payload for you find the pointer yeah here so this is the normal security injection payload that I'm sending if I send it normally I will get for a tree which is like I have been blocked if I use HTTP a smuggler and because this was JSP I was using Apache Tomcat policy and I'm sending this using HTTP smuggler then can go through CloudFlare without being blocked the same with other paths I have even tried in a mod security but as you can see I have cheated a little bit of added ampersand here I've also capitalized our letter here was needed and now if you send it without HTTPS Mugler you will be blocked but if you send it through HTTPS Mugler then it will be bypassed and there is no in like alerts from or warnings from mod security so this method can be quite useful however HTTP encoding is not always useful for testing if you have utf-8 characters in your request it may replace some of the characters with some question marks or something so if the request that you're sending to the server is changing something on the server side you have to be careful about using that using HTTP smuggler there is a checkbox that is disabled by default and so it's safe to use HTTP smuggler as soon as it sends that it is changing a character to something it doesn't have any idea about it will not work it will just return the actual HTTP request and it will not encode it and I found very interesting like I can't side-effect I should say because this wasn't a bath bypass this was just a side effect that again we published in NCC group blog and Microsoft basically said it is very minor and it doesn't really pass the security bar but it's very good and very interesting so if you use request value and so request validation in.net basically blocks any HTML codes that are being sent to a dotnet page and it will show you an error so it's an auntie CSS XSS protection so that's it however I found out if you have uh narrow resume next in vb.net or for example an empty catch like we have here in seashore and it is reading some query string or foreign parameters the first time that it reads query string or foreign parameters it is basically going to say okay this is an empty value the second time it reads the same parameters or another parameter it actually sees the value so this is useful to basically exploit some stored XSS and it it is also useful to bypass some validations because it has time of check time of use vulnerability because when it reads it for the first time it's empty and then it wants to read it again it's not empty so that's when you can basically bypass some some protections but it's rare to see it but whenever you see it it's vulnerable and the twist here is if your payload is in the URL you have to send a post request and if the payload is in the body you have to send a get request as long as you keep the content type from our s point of view it's a still a post request so it doesn't matter but it will bypass the request validation so that's the small choice there here is an example so in this aspx page we had that on error regime next and it was basically showing the post param one once twice and host param two only once let me see if I can show the pointer so as you can see here the post param one and post round two was being sent to the server encoded as you can see here and they have been encoded using IBM 500 and because they are in the of this request I'm sending it get where so that was needed but I'm because I'm still keeping the content time as is the survival state as a post request so if there wasn't any request validation if it was disabled I would have seen three alerts here to like because I was printing this parameter twice and this parameter once I would have seen a lot more but there is an empty string and then this and here is an example for a sequel injection challenged or basically put on my Twitter unfortunately nobody solved this on time some people solved it afterwards after the deadline but was quite past the deadline and I had they had help so for that reason I'm not announcing them as the winner but this this is the this is the code so honorarium next and then I am reading the UID here saying if it contains single code then it's unsafe and des know it if it doesn't then read it again and create a sequel query obviously this is unsafe and then I was doing something afterward in this example I'm just showing it out you shouldn't like showing it on the page so if you send the request like this UID some haich-d a HTML tag here to trigger request validation and then your sequel injection afterwards then you can exploit this as you can see because the parameter was in the URL I had to send the post message here and this is all being 500 encoded again and it goes to the server and exploit the application as you can see it will see it as like being injected which was very good so if how can you stop this request encoding is bad how can you stop it the way I found to stop this using mod security was using this rule so basically this rule says whenever you see care set in a request or the the cache is not utf-8 if it is in the request then just deny the that Qwest this is also a rule for encapsulate just an example to say like this we can basically implement at a similarly on other laughs so it will say again the same thing character set shouldn't be there but if it is there then it shouldn't be utf-8 it should be only utf-8 and nothing else maybe I brought I've written that wrong but not sure okay so now I have a test case walk through to just talk about things very fast so in here I'm just going to show you how to bypass some web application firewalls when I'm when I have OS 10 and and a speaks page which is very fast I'm going through some simple steps and will show you how you can bypass them just by using some simple steps so I have an a speaks page that I accept a post request and it's vulnerable so for in the first step I'm just replacing post we'd get so this is my request I'm just replacing this post we'd get as you can see I have a sequel injection payload here that is going to be blocked by all the buffs like in this example I haven't covered any other just because these were easier to be tested I know it quickly tested them during the slides just like a few days ago so as you can see I've replaced post we'd get nothing happened they still know about it haven't been bypassed and everything is good so in the second step I'm going to change the content type from URL encoded to multi-part standard everything should work fine and all the web servers should support it except Apache Tomcat and it should be fine so for this example that I have oh yes and a space so this is the content type URL encoded I have just changed this using multi-part form data and boundary so you can just embarr so you can just right-click change the content type it's very easy to do that still because this is quite a standard it's being blocked by now from this point it becomes interesting now I'm going to remove unnecessary parts that web servers don't need basically to understand a post request on our yes I should say oh yes doesn't need these some other web servers have other behaviors so in here what I'm going to do I'm going to remove these highlighted in red character so - - characters here here here and also after the last boundary it's not a standard but I'm doing it because I yes doesn't care and then I'm also removing this form data just by doing that CloudFlare an Akuma have been bypassed immediately yes in capsule or haven't been by us so they had some protections in place to know about this but this can't be parsed by those two so they have been bought us was very easy bought us so now I'm going to add some unused part or unnecessary parts to the same request to see what would happen if I confuse the vast even further to see if I can bypass them completely in here this was the previous request that I had I've already bypassed two of the buffs fair enough but I'm now going to add more useless parts to it as you can see of I have added an additional boundary that is not needed highest ignores that I've also added these green parts here as well as this filename here the space after the file name was actually important because if you don't do that aspx is this is a file upload so it's not going to be a request form a valid post request so for that reason oh yes still sees this as a valid post request and it ignores the file name however at this point all the VAS that I was testing have been bypassed which was quite cool but you may say two of those valves have already been bypassed so for that reason I'm going to jump from a step two to a step four so after I change the URL encoded to multi-part I'm not going to just add unnecessary parts without removing any any actually any parts that was not used so I'm just adding a stuff to the same request that I had previously just by doing that in capsule and akamai have been bypassed this time CloudFlare actually wasn't bypassing so that as it saw that as a as an as a dangerous input so still there and no just basically okay I only have five minutes I know so I'm gonna finish this early so if I just encode the same thing like if I applied request encoding on the request that I had it has already bypass all the valves I know the request encoding on its own can bypass all the vas as well but if I do that just that this was the first request that I had and this request is actually equal to the request that I had initially and it will go through all the baths and boy pass everything and nothing can understand it except OS and aspx page that I had there and it's a valid sequel injection and it's it's fine it does work so lesson learned here just quickly go through them so if you're using vats in the cloud just do something so using IP address directly can't bypass it also don't support HTTP version 0.9 at all do not accept that only accept known character sets also discard malformed HTTP requests don't accept invalid ones and also if you can afford using a voiceless buff then use that it's it's the best and remember to white lace pen Tessa's IP address during an assessment and remove remove that afterwards because if you don't do that you don't get the value for money that's it with that thank you very much [Applause] do we have any questions before we break for coffee thanks for a great talk Suresh obviously the fact that some of the cloud woth providers like CloudFlare leak capsular behave differently for different tests have you relayed the results of his research to them and - they actually have a way of fixing their rule set to make sure they catch those exploits in the future so basically I send the resulting cloud flare and in capsular in the first place and what I got was like this is very standard to bypass blacklist buffs and it is quite normal every now and then we will be bypassed so then I was like are you going to patch this and they said yes we are going to do something about it I reported this basically last year around the same time as now and our blog post was published in September so basically it's completely public they know about it they haven't done anything for it so any more questions and of text for the great presentation and the question is under which conditions all this test was run because there's a different configuration different rule sets yeah I don't know in which so so the walkthrough was Ras a specs but when I was doing my research I was I I had those web servers that in that table that you saw earlier on the request encoding of assessing against those I have created a github project called HTTP ninja if actually if you go to http the ninja website you will be redirected to that I've forced a lot of web servers and things like that but it sees still in progress so if you go there you will see a table of all different behaviors a lot more than you saw here that how they basically act differently and depends on what you have you need to basically decide what you want to do the question is about the configuration of the work itself okay and there is all the rules and your enable so there were all defaults so I did not add anything to them and I did not remove anything from them so that's right any more questions okay [Applause]

Channel: OWASP Foundation

Views: 7,430

Keywords: owasp, appsec

Id: tSf_IXfuzXk

Length: 42min 34sec (2554 seconds)

Published: Tue Oct 02 2018