Skip to content

updates to paginator to fix issue with excatly 250 reults #62

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
darrell-thobe-sp wants to merge 2 commits into
base: main
Choose a base branch
from
Open

updates to paginator to fix issue with excatly 250 reults #62

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ff08f38
updates to paginator to fix issue with excatly 250 reults
darrell-thobe-sp Jul 23, 2025
c130c6b
merge main fix conflicts
darrell-thobe-sp Jul 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 45 additions & 20 deletions sdk-output/paginator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,64 +103,89 @@ export class Paginator {

let modified: TResult[] = [];
while (true) {
let results = await callbackFn.call(thisArg, params);
modified.push.apply(modified, results.data);
if (
results.data.length < increment ||
(modified.length >= maxLimit && maxLimit > 0)
) {
results.data = modified;
return results;
try {
let results = await callbackFn.call(thisArg, params);
modified.push.apply(modified, results.data);
if (
results.data.length < increment ||
results.data.length === 0 ||
(modified.length >= maxLimit && maxLimit > 0)
) {
results.data = modified;
return results;
}
params.offset += increment;
} catch (error: any) {
if (
error.response &&
error.response.status === 400 &&
error.response.data &&
error.response.data.detailMessage &&
error.response.data.detailMessage.includes("Specified offset is invalid")
) {
const finalResults = await callbackFn.call(thisArg, {
...params,
offest: 0,
limit: 1,
});
finalResults.data = modified;
return finalResults;
}
}
params.offset += increment;
}
}

public static async paginateSearchApi<T extends ApiType>(
searchAPI: ApiInstanceMap[T],
search: SearchApiTypeMap[T]['search'],
search: SearchApiTypeMap[T]["search"],
increment?: number,
limit?: number
): Promise<AxiosResponse<SearchApiTypeMap[T]['document'][], any>> {
): Promise<AxiosResponse<SearchApiTypeMap[T]["document"][], any>> {
increment = increment ? increment : 250;

let offset = 0;
const maxLimit = limit ? limit : 0;
let modified: SearchApiTypeMap[T]['document'][] = [];
let modified: SearchApiTypeMap[T]["document"][] = [];

if (!search.sort || search.sort.length != 1) {
throw "search must include exactly one sort parameter to paginate properly";
}

while (true) {
console.log(`Paginating call, offset = ${offset}`);
let results: AxiosResponse<SearchApiTypeMap[T]['document'][], any>;
let results: AxiosResponse<SearchApiTypeMap[T]["document"][], any>;

// Handle each API type separately to avoid type conflicts
if (searchAPI instanceof SearchApi) {
const searchParams: SearchApiSearchPostRequest = {
search: search as Search,
limit: increment,
};
results = await (searchAPI as SearchApi).searchPost(searchParams) as AxiosResponse<SearchApiTypeMap[T]['document'][], any>;
results = (await (searchAPI as SearchApi).searchPost(
searchParams
)) as AxiosResponse<SearchApiTypeMap[T]["document"][], any>;
} else if (searchAPI instanceof SearchV2024Api) {
const searchParams: SearchV2024ApiSearchPostRequest = {
searchV2024: search as SearchV2024,
limit: increment,
};
results = await (searchAPI as SearchV2024Api).searchPost(searchParams) as AxiosResponse<SearchApiTypeMap[T]['document'][], any>;
results = (await (searchAPI as SearchV2024Api).searchPost(
searchParams
)) as AxiosResponse<SearchApiTypeMap[T]["document"][], any>;
} else if (searchAPI instanceof SearchV2025Api) {
const searchParams: SearchV2025ApiSearchPostRequest = {
searchV2025: search as SearchV2025,
limit: increment,
};
results = await (searchAPI as SearchV2025Api).searchPost(searchParams) as AxiosResponse<SearchApiTypeMap[T]['document'][], any>;
results = (await (searchAPI as SearchV2025Api).searchPost(
searchParams
)) as AxiosResponse<SearchApiTypeMap[T]["document"][], any>;
} else {
throw new Error("Unsupported API type");
}

modified.push.apply(modified, results.data);

if (
results.data.length < increment ||
(modified.length >= maxLimit && maxLimit > 0)
Expand Down