Projecting Your Burp

If you’re a security researcher or penetration tester you’re probably already well aware of the extensive array of tools available to help you. OpenVAS, Qualys, Nessus, Arachni, Burp, Wapiti, Skipfish, w3af … the list goes on and on. Choosing which tool to use may not be a simple task and should wind up raising even more questions to be answered. What are you looking to accomplish? What features (and support) do you need? Do you need customization? And most importantly, as a hacker how dirty do you want to get your hands to attain that perfect solution? For a recent initiative we decided to pursue a route that balanced commercial support and reputation with just enough openness to fit it to specific needs. That route is Burp by Portswigger.

Burp is a tried and true, feature rich commercial application that offers crawling, scanning, and intrusion capabilities to audit the security of any web application you might throw its way. But just because you need to purchase a license (for the professional version), it doesn’t mean you’re only getting what the Portswigger developers are putting in the box. Burp comes with what is called the Extender API which opens up a world of opportunities for you and developers alike to create even more useful features to build up this proverbial Swiss Army knife of an application. What Portswigger gives you out-of-the-box:

Proxy: This module is the core of capturing requests and responses ultimately used for spidering and scanning analysis. When messages are captured you’re free to modify their contents at will via rules in the Proxy’s Options tab

Spider: This module is used to scope out the target and build a thorough map. The results of spidering are easily sent to the scanner to locate intrusion points

Scanner: The Scanner module is the attacking engine that identifies the vulnerabilities in any of the pages mapped out by the Spider. Its toolbox is a configurable arsenal of injection techniques (SQL, XSS, LDAP, HTTP, etc).

Intruder: If you’re already a web ninja looking for a tool to help send your own custom payloads, the Intruder module will be a good friend. The Intruder module provides a workbench for you to put your mischievous creativity to work with the aid of built-in processing automation

Repeater: When was the last time one of your pen tests worked on the very first shot? If an occasion even comes to mind, I salute you. If not, use Burp’s Repeater module. As the name implies, Repeater allows you to reissue HTTP requests as much as you need

Sequencer: This module is geared towards analyzing the randomness of sample data. Use this to try to identify patterns in sensitive data elements such as tokens

Decoder: The aptly named Decoder module helps you transform data into more useful forms such as hex, base64, ASCII, and more

Comparer: When performing a penetration test seeing the differences in responses based on different input is your most important data point. Comparer gives you a nice way to quickly see the differences between collected data. This is pretty much Burp’s version of ‘diff’/’vimdiff’ and can quickly shed light on the target application’s behavior

Extender: Last but certainly not least… the Extender module is the developer’s gateway into hooking into the previous modules with his/her own code. It is also the focus of the rest of this post

Portswigger’s foresight to anticipate the potential in opening up just enough accessibility to feed a development community has, in my opinion, paid off well. Since Burp is a Java application it was only a short matter of time until Java coders began to take advantage of the Extender API. This API is a library of interface classes that provide the coding “contracts” to interact with Burp’s internals. I won’t be going into extreme detail of building a Burp extender from scratch, but I’d like to provide some tips, thoughts, observations and insight that may be useful to anyone working on his/her own extender as well to the Burp developers themselves. The first thing I’d like to mention is that the usage of the Extender API is not very well documented beyond the Javadocs found in the interface classes. Depending on what you’re trying to implement you will find yourself learning a large amount from existing open-source extenders (listed below) and relying on your own craftiness to figure things out. The second thing to bring up here is that the API in its current state leaves a bit to be desired; don’t expect that all of your wildest Burp ambitions will be fulfilled with the current API. The third, and last thing I’ll point out before going more in depth is that the Portswigger developers are great at addressing issues and enhancing the API, so updates are always rolling in to make extending Burp even better. Let’s dive into these topics.

Burp Extender API Documentation:

I haven’t found official documentation for the Burp Extender API outside of the Javadocs that provides real world usage explanation. One of the biggest areas that needs documentation is the loadConfig()API method. As described, this method takes a Map<> interface object of Strings to set the various Burp-wide configurations. These configurations are the options found throughout the module tabs in the application. As you can imagine, having programmatic control over these configurations can be a very powerful utility. You may want to adjust the number of threads the Scanner will run on, tell the Spider to avoid forms, only scan for certain vulnerabilities, and create your own configuration permutations. The loadConfig() method is the method you would use to set these parameters, but how do you know what exactly should be put into this map? I began digging around to solve this issue. One useful piece of information I found was around the saving and restoring of Burp’s “state”. In the GUI you can save the state to a file, and this state includes configurations as per the web page. Cool, this file should shed some light on what we would use here. After saving a state and opening it up in vim I see:

PK^C^D^T^@^H^H^H^@¢MmE^@^@^@^@^@^@^@^@^@^@^@^@^D^@^@^@burpí] sÛF<96><8e>ï+:,E<8a>íØ<89>fâ©ÝulJ….

Fantastic, this is not a regular file. Well, now what? We know that Burp is a Java application, and that the common way Java writes to files is through a FileOutputStream, or some derivative thereof. If you’re feeling ambitious you can “unjar” the BurpSuite jar file and do some “grep”-ping around for various uses of the different kinds of OutputStream uses, eventually you’ll find ZipOutputStream. If you’re not up for the task you can just use the Linux “file” command on your saved state and uncover something similar to the following:

name_of_file: Zip archive data, at least v2.0 to extract

Either way, bingo! We know this is a zip file. Now go ahead and unzip it and take a look at the resulting file. You’ll find clear-text “name/value” tags that you can trace back to the GUI elements. All set now, right? Not so fast. There’s more to understand and notice in this file. Let’s go back the use-case of adjusting the number of threads for the Scanner. If we jump to a pattern match of “threads” we’ll noticed two matches for “numthreads”, and if we look around even more we’ll find that they’re within “spider” and “scanner” tags respectively. How do we know what the correct string to pass to differentiate each is now? This puts us in a bind, but there’s still one way to solve this puzzle. Take a look at the saveConfig() API callback method and read the Javadoc. This method returns a Map of the name/value pairs! Call this method in an extender, fire up an iterator and save your results. You’ll now find what you’ve been looking for. The configurations are represented in the “tool.configuration name” format (e.g. “scanner.numthreads”, “spider.numthreads”).

Something else you may have noticed by now are some funky looking values such as “1.1.4.last.6.Winter”. How is this read? A close look at this and similar values will show you that, in this example, the “4” represents the number of characters in the subsequent string “last” and the “6” is the same for “Winter”. To be honest, I’m still not even sure about the leading “1.1”. The last thing to keep in mind about these configurations are the values attributed to the dropdown selection configurations in the GUI. Adjusting the scan speed in the Scanner for example: Scan speed “Fast” maps to 10, “Normal” maps to 20, and “Thorough” maps to 30. This convention does not apply to all dropdown selection though. Others tend to map to values 0,1,2,3 etc. and even then there are inconsistencies. Sometimes values will be skipped, and sometimes the values do not map to any index within the list. The best way to find this information out for yourself is to save the states testing the different configurations, unzip the state files, and check out the values.

One last (and important) piece of information regarding the Burp configurations is the difference between behavior when running in “headless” and “GUI” mode. Sometimes when running in GUI mode you’ll notice pop-up alerts asking for “yes” or “no” user input. In “headless” mode these are not accessible, and Burp assumes responses on your behalf- which is actually OK because in most cases that will be the desired response in those circumstances. Another difference is behavior when loading saved “state” files. When restoring a state with the GUI you may find that scans in the Scanner tab will be put in “Paused/waiting” state, but when restoring a state via the restoreState() callback method this is not the case and the scans are free to run.

Wow. That was a lot of work just to figure out how to use Burp’s configurations.

What We’re Still Looking For:

At first glance over the Extender API and some of the existing extenders living out there on GitHub it may appear that you have all that you need to fully control Burp in this handful of interfaces. Sure you can read the outgoing requests and incoming responses, get scan results, and send messages to the tools of Burp, but if you want all of the control that you get in the GUI you’ll find the API falls short. Take the Scanner for example. In the GUI the user has the ability to pause, resume, and cancel active scans with a simple right-click context menu, but in the extender the pause and resume functions are not exposed at all. Another improvement I would like to see is a cleaner way to inject cookies into the Burp Cookie Jar. Adding cookies into the cookie jar to perform authenticated operations is available via the updateCookieJar() callback which takes an ICookie implementation. This makes perfect sense, but being stuck with the ICookie interface that only provides “getters” and no “setters” means we need to actually pass not a traditional object, but an implementation of the ICookie interface as a parameter into this call. Luckily none of the gaps I’ve come across would I classify as crippling to an extender being developed. Some tasks do have workarounds while those that don’t we’ll just have to wait for in a future release.

Portswigger Developers Pull Through:

Spending significant time with any third party API will at some point raise as many eyebrows as questions, and sometimes those situations may lead to legitimate bugs beyond your control. One such bug that I came across was within the previously discussed loadConfig() method. When writing an extender that implemented the IHttpListener interface and used the loadConfig() method I found that after the new configurations were reloaded, the extender was deregistered as an HttpListener and the processHttpMessage() callback completely ceased to be invoked. This behavior is definitely a showstopper as it ultimately renders your extender useless when implementing certain interfaces. I shot the Portswigger developers an email describing the issue and was very happy to receive a very prompt response acknowledging the issue and plan to release a fix. This and a few other questions about some Burp nuances continue to present themselves and are given the same expeditious attention from Portswigger. As a developer it’s always great to work with other teams. This is especially true with vendors, because based on their receptiveness to feedback you really do get a sense of contribution to the industry and coder community.

Wrapping Up:

By now I hope you feel more informed about things you may encounter writing your own code to leverage Burp. Maybe you’ve also gotten closer to answering the question “How much work do I want to put in to get the results I’m after?”. Using a semi-open tool lets you implement ideas, but ultimately you rely on maintainers of the application if you hit a wall. Going fully open-source gives you entire control over every line of code, but there’s a good chance you’ll be lacking features you want out of the box, and adding them yourself would be too time consuming. Purchasing a licensed, closed source product gives you no flexibility, but the product is likely to be very feature rich. There isn’t a right or wrong answer. If you have some time to get your hands dirty and make Burp’s horsepower work for you, I recommend reading the Javadoc API and the source code of the links below. Good luck!

Cheers,

Nick

References:

• Burp-CSJ provides a hook into the Crawljax web crawler that does an excellent job of interpreting and traversing JS in web pages. This is very important because as web pages become more dynamic they also become more difficult to correctly spider and map out.
• Resty-Burp gives the user a RESTful API into Burp. This is a great feature for programmatic control over Burp Suite
• Burp-C02 is a collection of tools with a wide range of uses. It provides a workspace to create complete copy/pasteable SQL Map commands, generate form inputs, create intruder payloads, and more.