xmemdump completion status

[eddietorial]

How do I check the status of xmemdump. Is there a way to get a confirmation that it is in progress or has completed?

The output is sent to a network share as local drive output is not an option in our environment. However, xmemdump takes over an hour to complete, so knowing that it has successfully completed or has failed would be useful.

[bk-cs]

There's no progress output when creating the xmemdump output, so my guess is that you'll need a script that checks for the presence of the output file, and determines whether or not it's complete. Maybe run something like Invoke-FalconRtr to start the memdump generation, record the path of the dump file you create, check for the file to verify that it's no longer locked (using RTR), and once it's available, get it?

I'll try and put together an example in my spare time, but the next couple of weeks are going to be pretty busy. If you figure it out beforehand, please reply with your solution!

[bk-cs]

It seems to me that your proxy is doing something with the SSL connection, or the API is returning something that your proxy does not expect. If it doesn't happen when the proxy is removed, the module is not the problem.

When it automatically requests a new token at 30 minutes, it's running the exact same command as it is the first time -- Request-FalconToken.

[eddietorial]

So, when it automatically requests a new token at 30 minutes with Request-FalconToken, does it first test if the existing token exists? And if so, does it revoke it before the new token request?

I agree that there may be something happening on the wire that we're not seeing in the logs. However, I am able to reproduce the same error in the proxy logs without having to wait 30 minutes, just by running Request-FalconToken without revoking the existing token.

I revoke the existing token in my script before requesting a new token regardless of whether ((Test-FalconToken).Token -eq $true) else the new token request fails just at it would in the predefined auto-check at 30 minutes.

In my environment, xmemdump is the only RTR process that takes over 30 minutes to output to a network share, I run the new token request every 10 minutes while checking the file lock. Sure, this delays the confirmation of the xmemdump output completion but at least the script ends gracefully.

[bk-cs]

It does not revoke before a request. Here is the exact logic that happens before every request (if token is due to expire within 60 seconds, request a new token):

if ($Script:Falcon.Expiration -le (Get-Date).AddSeconds(60)) { Request-FalconToken }

Although you can already reproduce it manually, you could make PSFalcon think the expiration is much lower by modifying the "expiration time" in Public\oauth2.ps1. Change this line from:

$Script:Falcon.Expiration = (Get-Date).AddSeconds($Result.expires_in)

to:

$Script:Falcon.Expiration = (Get-Date).AddSeconds(120)

That would cause your tokens to expire every two minutes, causing you to wait about a minute to see the failure happen.

I haven't received any reports about this sort of failure beyond yours, and (last I checked) there were about 1,500 different customers using PSFalcon daily. If there's something to change, I'd like to narrow it down and get it fixed, but if eliminating the proxy solves the issue, I have to think that the proxy is the reason.

[eddietorial]

Is it feasible to implement multiple retries (at least 3) on the request for a new token? Or maybe even overall for the module commands that may fail due to temporary and unavoidable network latency?

My observation over weeks of testing is that the default retry fails at the first attempt (in my environment).
A second attempt goes through without any errors.

As this only seems to be affecting my environment, I have put in new token requests within my script as a workaround.

However, another issue that I've frequently obeserved is that even if a target workstation is online, sometimes the Invoke-FalconRtr fails with a "code":404,"message":"no successful hosts initialized on RTR".

So, I've put in a retry attempt similar to a solution described here
https://stackoverflow.com/questions/45470999/powershell-try-catch-and-retry

And, as expected, whenever I've encountered "code":404,"message":"no successful hosts initialized on RTR" again, it goes through successfully in the second attempt.

[bk-cs]

You can also use -QueueOffline $true with Start-FalconSession to queue for offline hosts and avoid the "code":404,"message":"no successful hosts initialized on RTR" error (most of the time). This is a different problem than the token request, though.

I'm not sure it's appropriate to add an authorization token retry request to Request-FalconToken. This is the only report of token request issues that I've heard that isn't due to IP-based allowlisting or invalid credentials, and it seems restricted to your environment. The token request API also redirects if provided with the wrong hostname, so it might be a little tricky to add in a retry that still properly respects that redirection (i.e. don't retry if redirected, only if due to legitimate failure). I think the try/catch you're using in your script is the easiest way to handle it... unless you have other ideas?