[Bug] listPaginated not returning all results

[harrisonlinowes]

Is this a new bug?
In other words: Is this an error, flaw, failure or fault? Please search Github issues and check our Community Forum to see if someone has already reported the bug you encountered.

If this is a request for help or troubleshooting code in your own Pinecone project, please join the Pinecone Community Forum.

• I believe this is a new bug
• I have searched the existing Github issues and Community Forum, and I could not find an existing post for this bug

Describe the bug
When calling the listPaginated() function with a prefix id, I am not able to receive all the expected results. I was able to validate this by calling the analogous function in python which returned different results.

Typescript code:

const recordsWithPrefix = [];
const response: ListResponse = await index.listPaginated({
  prefix: prefix,
});
recordsWithPrefix.push(...extractIdFromResponse(response));
let nextPage = response.pagination?.next;

while (nextPage !== undefined) {
  const nextResponse: ListResponse = await index.listPaginated({
    prefix: prefix,
    paginationToken: nextPage,
  });
  recordsWithPrefix.push(...extractIdFromResponse(response));
  nextPage = nextResponse.pagination?.next;
}

Python Code:

index = pc.Index("my-index")
allRecordIds = []
for ids in index.list(prefix="my_prefix"):
  allRecordIds += ids

Error information
The results from the two snippets of code above should be equivalent however they are not. The python code returns all expected record ids while the typescript code only returns a subset of ids.

Steps to reproduce the issue locally
See code above.

Environment

• OS and version: Mac
• Node version: v20.15.0
• Typescript SDK version: 3.0.1

Additional context
Add any other context about the problem here.

[anawishnoff]

Thanks for opening - the team is investigating this issue and will follow up

[max-tano]

+1 our team would really like this fixed too