Doesn't show extension changes in proxy

[irsdl]

If an extension changes a request within the proxy tool, Logger++ does not show the changes.

The extension I was testing this with was using processHttpMessage to manipulate the requests.

Moving Logger++ before or after the other extension did not resolve the issue. Therefore, order is not an issue here.

[CoreyD97]

Is it possible you can provide the extension which is not having the changed results displayed?
I have tested with Collaborator Everywhere and can see the changed request is being displayed successfully.

[irsdl]

So this issue is still there. Just use this sample (the changes won't be visible in Logger++):

from burp import IBurpExtender
from burp import IHttpListener

class BurpExtender(IBurpExtender, IHttpListener):

    #
    # implement IBurpExtender
    #
    
    # The only method of the IBurpExtender interface.
    # This method is invoked when the extension is loaded and registers
    # an instance of the IBurpExtenderCallbacks interface
    # A set of callback methods that can be used by extensions to perform various actions within Burp.
    # https://portswigger.net/burp/extender/api/burp/iburpextendercallbacks.html
    def registerExtenderCallbacks(self, callbacks):
        # Put the callbacks parameter into a class variable so we have class-level scope
        self._callbacks = callbacks
 		# save the helpers for later. This extension doesn't need them but most will.
        self._helpers = callbacks.getHelpers()
        # Set the name of our extension, which will appear in the Extender tool when loaded
        callbacks.setExtensionName("Add HTTP Header")

 	# register ourselves as an HTTP listener
 	# The listener will be notified of requests and responses made by any Burp tool. 
 	# Extensions can perform custom analysis or modification of these messages by registering this HTTP listener.
        callbacks.registerHttpListener(self)
        print "hello world"
        return

    def processHttpMessage(self, toolFlag, messageIsRequest, currentMessage):
    	# processHttpMessage is invoked when an HTTP request is about to be issued, and when an HTTP response has been received
    	# https://portswigger.net/burp/extender/api/burp/IHttpListener.html

        # only operate on Proxy requests, not TOOL_SCANNER, etc etc.
        if toolFlag == self._callbacks.TOOL_PROXY:
        # only operate on requests (not responses)
            if messageIsRequest:
                self.processRequest(currentMessage)
                
    def processRequest(self, currentMessage):
        # boilerplate for getting the HTTP request and splitting it into header and body.

        request = currentMessage.getRequest()
        parsedRequest = self._helpers.analyzeRequest(request)
        requestHeaders = parsedRequest.getHeaders()
        requestBody = self._helpers.bytesToString(request[parsedRequest.getBodyOffset():])

        # This is the actual injected header.
        requestHeaders.add('Client-IP: 111.222.222.111') 

        # build the HTTP request back out of the new request Headers and Body
        updatedRequest = self._helpers.buildHttpMessage(requestHeaders, requestBody)

        # set this newly built request as the current request, replacing the original. 
        currentMessage.setRequest(updatedRequest)

so if you use another burp in front of burp, you will see the new header. But no matter where you put the Logger++, it will not show it.

[federicodotta]

Hi! I have the same issue.

I add some details. It is not possible to see modifications executed on Proxy requests/responses by extensions that implements a IHttpListener listener (and not the IProxyListener one).

Maybe the requests/responses in the IProxyListener queue are all processed before the requests/responses in the IHttpListener queue and consequently modifications executed on proxy requests from IHttpListener extensions are invisible to Logger++, that handles those requests in the IProxyListener queue (but this is only a guess).

In this case, the bug could be fixed by handling all requests/reponses (including the proxy ones) with a IHttpListener listener instead of a IProxyListener one.

[CoreyD97]

Thank you for the additional information. I have been doing some testing of the Burp Extender API recently and the processProxyMessage does indeed get processed before the processHttpMessage, with extensions higher in the list being processed before those lower in the list.

While switching to an IHttpListener and having the extension loaded last would allow you to see the final request sent to the server, you lose the ability to see the original request sent before any extensions made changes. In addition, processProxyMessage has the benefit of being able to correlate requests with their responses, which is used to allow L++ to log the request being made before the response has come back to show a more real-time list of what requests are being sent. While I have utilised the comment field on IRequestResponse objects to achieve this for the processHttpMessage method it is not as reliable as the correlation in the processProxyMessage method.

One workaround I am looking to implement, is to have both the processProxyMessage and processHttpMessage methods handle requests made with the proxy, and to compare them to determine if changes have been made. However to capture all changes, processProxyMessage must be the first extension in the list for requests, and the last extension for responses. While the inverse is true for processHttpMessage. Though it isn't possible to change the priority of handlers in the Burp API at this time.

Sorry if I haven't quite explained that well, the diagram below may help to understand it better.

[federicodotta]

Hi Corey42!

Thank you for the quick reply! You explained it very well. I understand the limitations due to Burp Extender API. Maybe we can ask to Burp Suite to add a feature to the Extender GUI pane, allowing to move up and down extensions in the different queues instead of moving the extensions in all the queues together.

By the way, thank you for the clarification.

[CoreyD97]

That would definitely be useful, though I know similar items have been on their development backlog for a while. Maybe when they come to update the API it is something they will add, though I don't think it'll be any time soon :(

[CoreyD97]

Just had a 'why didn't I think of that earlier?' moment.

For capturing the modifications to the request, I can store the request that's received initially when the request is sent via either processProxyMessage or processHttpMessage as normally, then when the response comes back through processHttpMessage check to see if the request was modified after L++ captured it.

I don't have a solution for responses modified after L++ has grabbed them yet though.
Ideally, PortSwigger would process extensions implementing the methods in the inverse order for responses, but I don't think that'll happen anytime soon.

I'll add this to my todo list 😃

[CoreyD97]

Reopened the issue, as this fix could be improved quite a bit. Due to Burp's API limitations it can never be perfect, but using processHTTPMessage for requests, and processProxyMessage for responses should allow the actual message sent to the server and client to be obtained by L++.

[floyd-fuh]

Hey there, don't want to wake up old issues, but at least for Burp Suite professional v2022.2.4 the diagram above is not correct anymore, see https://www.pentagrid.ch/en/blog/teaching_burp_a_new_http_transport_encoding/ . If you disagree feel free to take the diagram from the github https://github.com/pentagridsec/PentagridBurpTransportEncoding repository and modify it at https://draw.io

[CoreyD97]

Thanks for sharing this @floyd-fuh, it's nice to see a better diagram of what I was trying to figure out. The diagram here is flipped from what I had originally drawn, but it looks to be essentially the same process, with the only difference being where the match+replace rules are executed. However, they've clearly spent much more time to test this, so I'm inclined to agree that they're correct here.

Ultimately, I believe that Logger++ hooks into things at the best possible locations currently. The use of two extensions at the first and last extension positions as they have demonstrated in the blog post is something I had considered for Logger++, but unfortunately this would make it much harder to ship, especially via the BApp store. Though if anything changes with Burp's upcoming API update, I'll be sure to look into it.