Client Script Based Auth (Ajax Spider)

[kingthorin]

Overview

Allow the Ajax Spider to use Client Side scripts to authenticate, when used in conjunction with the Auth Helper add-on.

• Update change logs.
• Update build files with necessary config/dependencies.
• Auth Helper:
  • Expand visibility of findSessionTokenSource in AuthUtils.
  • Add new authentication method type, ClientScriptBasedAuthenticationMethodType.
  • ExtensionAuthhelperClient updated to facilitate the new functionality.
  • Added intertnal.ZestAuthRunner which will also be used by the Client Spider in the future. This ensures that browsers are launched as expected and auth handled before other actions.
  • ClientScriptBasedAuthHandler handler for the new auth method type, responsible for enabling/disabling auth methods for Users, and providing the necessary BrowserHook.
  • ExtensionAuthhelperAjax updated to 'hook' and use the new auth method.
  • Help updates/additions.
  • Messages.properties added necessary KVPs to support the functionality.
• Automation Framework
  • Add handling to AuthenticationData.
  • Help updates/additions.
• Zest
  • Minor changes to ZestAuthenticationRunner to facilitate the new functionality.

Related Issues

n/a

Checklist

• Update help
• Update changelog
• Run ./gradlew spotlessApply for code formatting
• Write tests
• Check code coverage
• Sign-off commits
• Squash commits
• Use a descriptive title

[kingthorin]

I believe this is ready for review.

There is an outstanding issue where one of the browser windows may not close. I'm investigating.

[psiinon]

Just one minor comment. I'm sure theres a load of dup code from core here so at some point it would be really good to rationalise it. But maybe not today ;)

[kingthorin]

Thanks for reviewing, I figured things would need to move around and that dependencies might be wrong, some of that I blindly trusted and some of it I just wasn't confident about the changes myself.

[kingthorin]

At d02f036 it's failing to load the script when running via automation:

904848 [ZAP-Automation] ERROR org.zaproxy.zap.ZAP.UncaughtExceptionLogger - Exception in thread "ZAP-Automation"
java.lang.NoClassDefFoundError: org/zaproxy/zest/core/v1/ZestScript
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method) ~[?:?]
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3402) ~[?:?]
	at java.base/java.lang.Class.getMethodsRecursive(Class.java:3543) ~[?:?]
	at java.base/java.lang.Class.getMethod0(Class.java:3529) ~[?:?]
	at java.base/java.lang.Class.getMethod(Class.java:2225) ~[?:?]
	at org.apache.commons.lang3.reflect.MethodUtils.getMethodObject(MethodUtils.java:439) ~[commons-lang3-3.17.0.jar:3.17.0]
	at org.apache.commons.lang3.reflect.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:323) ~[commons-lang3-3.17.0.jar:3.17.0]
	at org.apache.commons.lang3.reflect.MethodUtils.invokeMethod(MethodUtils.java:843) ~[commons-lang3-3.17.0.jar:3.17.0]
	at org.apache.commons.lang3.reflect.MethodUtils.invokeMethod(MethodUtils.java:931) ~[commons-lang3-3.17.0.jar:3.17.0]
	at org.apache.commons.lang3.reflect.MethodUtils.invokeMethod(MethodUtils.java:905) ~[commons-lang3-3.17.0.jar:3.17.0]
	at org.zaproxy.addon.automation.AuthenticationData.initContextAuthentication(AuthenticationData.java:351) ~[?:?]
	at org.zaproxy.addon.automation.ContextWrapper.createContext(ContextWrapper.java:329) ~[?:?]
	at org.zaproxy.addon.automation.AutomationEnvironment.create(AutomationEnvironment.java:174) ~[?:?]
	at org.zaproxy.addon.automation.ExtensionAutomation.runPlan(ExtensionAutomation.java:369) ~[?:?]
	at org.zaproxy.addon.automation.ExtensionAutomation.lambda$runPlanAsync$4(ExtensionAutomation.java:437) ~[?:?]
	at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.lang.ClassNotFoundException
	at org.zaproxy.zap.control.AddOnClassLoader.findClass(AddOnClassLoader.java:330) ~[main/:?]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
	... 16 more

Edit: 13c993a should be the same.

[kingthorin]

Here's the plan and script to re-create, you'll have to edit the script path to be correct for your system.

Zest Test Script

{
  "about": "This is a Zest script. For more details about Zest visit https://github.com/zaproxy/zest/",
  "zestVersion": "0.8",
  "title": "RecordedTestAuth",
  "description": "",
  "prefix": "",
  "type": "StandAlone",
  "parameters": {
    "tokenStart": "{{",
    "tokenEnd": "}}",
    "tokens": {},
    "elementType": "ZestVariables"
  },
  "statements": [
    {
      "windowHandle": "windowHandle1",
      "browserType": "firefox",
      "url": "http://localhost:9091/auth/simple-json-cookie/",
      "capabilities": "",
      "headless": false,
      "profilePath": "",
      "index": 1,
      "enabled": true,
      "elementType": "ZestClientLaunch"
    },
    {
      "windowHandle": "windowHandle1",
      "type": "id",
      "element": "user",
      "index": 2,
      "enabled": true,
      "elementType": "ZestClientElementClick"
    },
    {
      "value": "[email protected]",
      "windowHandle": "windowHandle1",
      "type": "id",
      "element": "user",
      "index": 3,
      "enabled": true,
      "elementType": "ZestClientElementSendKeys"
    },
    {
      "value": "password123",
      "windowHandle": "windowHandle1",
      "type": "id",
      "element": "password",
      "index": 4,
      "enabled": true,
      "elementType": "ZestClientElementSendKeys"
    },
    {
      "windowHandle": "windowHandle1",
      "type": "id",
      "element": "login",
      "index": 5,
      "enabled": true,
      "elementType": "ZestClientElementClick"
    }
  ],
  "authentication": [],
  "index": 0,
  "enabled": true,
  "elementType": "ZestScript"
}

AF Test Plan

env:
  contexts:
  - name: AjaxSpiderAuthTest
    urls:
    - http://localhost:9091/auth/simple-json-cookie
    includePaths:
    - http://localhost:9091/auth/simple-json-cookie.*
    authentication:
      method: client
      parameters:
        script: /some/path/RecordedTestAuth.zst
        scriptEngine: Mozilla Zest
      verification:
        method: poll
        loggedInRegex: \Q 200 OK\E
        loggedOutRegex: \Q 403 Forbidden\E
        pollFrequency: 60
        pollUnits: seconds
        pollUrl: http://localhost:9091/auth/simple-json-cookie/user
        pollPostData: ""
    sessionManagement:
      method: headers
      parameters:
        Cookie: "sid={%cookie:sid%}; _random=blahblah"
    technology: {}
    structure: {}
    users:
    - name: test
      credentials:
        Username: [email protected]
        Password: password123
  parameters: {}
jobs:
- type: spiderAjax
  parameters:
    context: AjaxSpiderAuthTest
    user: test
    browserId: firefox
  tests:
  - name: At least 30 URLs found
    type: stats
    onFail: INFO
    statistic: spiderAjax.urls.added
    operator: '>='
    value: 30

[thc202]

#6112 (comment)

I didn't double check but that seems to be because of #6112 (comment) since the add-on itself doesn't have access to Zest.

Edit: further edits, I think the first version was/is correct as it's trying to instantiate the other class, which is the one that needs access to but we'll see.

[kingthorin]

Should be functional now, with the build file changes.

[psiinon]

Should also update the AF docs, e.g. method on https://www.zaproxy.org/docs/desktop/addons/automation-framework/environment/
Happy for that to be in another PR though - we'll need more docs once we "announce" client script auth support

[kingthorin]

Added help content.

[kingthorin]

Now with updated Auth Helper help too 😀

[kingthorin]

Yes these white spaces changes are unrelated but it looked horrible, could we just go with it 😀

[kingthorin]

Tweaked

[kingthorin]

Got those.

[kingthorin]

Got all those now too 🤞

[kingthorin]

Tweaked again

[thc202]

Thank you!

[kingthorin]

Yay 🥳