Skip to content
This repository has been archived by the owner on Dec 4, 2023. It is now read-only.

fix: remove total class to fix pagination bug #169

Merged
merged 4 commits into from
Sep 27, 2023
Merged

fix: remove total class to fix pagination bug #169

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bc7238b
fix: remove total class to fix pagination bug
zemberdotnet Sep 27, 2023
c9ab1a6
add failsafe
zemberdotnet Sep 27, 2023
ef8c8c3
Update src/crowdstrike/CrowdStrikeApiGateway.ts
zemberdotnet Sep 27, 2023
9d94adc
prettier
zemberdotnet Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 14 additions & 11 deletions src/crowdstrike/CrowdStrikeApiGateway.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import {
import fetch, { RequestInit } from 'node-fetch';
import { FalconAPIResourceIterationCallback } from './FalconAPIClient';
import { ICrowdStrikeApiClientQueryBuilder } from './CrowdStrikeApiClientQueryBuilder';
import { Total } from './Total';
import { httpErrorPolicy } from './HttpErrorPolicy';

function getUnixTimeNow() {
Expand Down Expand Up @@ -54,7 +53,6 @@ export class CrowdStrikeApiGateway {
private attemptOptions: AttemptOptions;
private logger: IntegrationLogger;
private readonly rateLimitConfig: RateLimitConfig = DEFAULT_RATE_LIMIT_CONFIG;
private total: Total;
private queryBuilder: ICrowdStrikeApiClientQueryBuilder;
private fetcher: typeof fetch;

Expand All @@ -66,7 +64,6 @@ export class CrowdStrikeApiGateway {
attemptOptions?: AttemptOptions,
) {
this.queryBuilder = queryBuilder;
this.total = new Total();
this.credentials = credentials;
this.logger = logger;
this.attemptOptions = attemptOptions ?? DEFAULT_ATTEMPT_OPTIONS;
Expand Down Expand Up @@ -375,17 +372,23 @@ export class CrowdStrikeApiGateway {
paginationParams.offset = seen;
}

const baseUrl = this.queryBuilder.buildResourcePathUrl(
availabilityZone,
resourcePath,
);

this.total.setValue(baseUrl, paginationParams?.total);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logic was confusing. I hope it's gone forever.


total = this.total.getValue(baseUrl);
if (paginationParams?.total) {
total = paginationParams.total;
}

finished = seen === 0 || seen >= total;

if (
(!response.resources || response.resources.length === 0) &&
!finished
) {
Comment on lines +381 to +384
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just noticed that response.resources.length is already accessed on line 364

Suggested change
if (
(!response.resources || response.resources.length === 0) &&
!finished
) {
if (response.resources.length === 0 && !finished) {

this.logger.info(
{ resourcesLength: response.resources?.length },
'response with zero resources, but finished is not true',
);
finished = true;
}

this.logger.info(
{ seen, total, finished },
'post-request pagination state',
Expand Down