Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/data-history-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@polymarket/bindings": patch
"@polymarket/client": patch
---

Preserve exact combo entry-basis fields and reject Data history pagination past documented offset ceilings.
9 changes: 9 additions & 0 deletions packages/bindings/src/data/portfolio.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import {
ComboConditionIdSchema,
ConditionIdSchema,
DecimalishSchema,
DecimalStringSchema,
EpochSecondsToMillisecondsSchema,
emptyStringToNull,
IsoCalendarDateStringSchema,
Expand Down Expand Up @@ -211,6 +212,10 @@ export const ComboPositionSchema = z
shares_balance: DecimalishSchema,
entry_avg_price_usdc: DecimalishSchema.nullish(),
entry_cost_usdc: DecimalishSchema.nullish(),
// These fields exist to preserve the exact entry basis and must never
// accept numeric JSON that has already lost decimal precision.
gross_entry_cost_usdc: DecimalStringSchema.nullish(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ClickHouse fallback serializes these unavailable decimal fields as "". We already have OptionalDecimalStringSchema specifically for this wire behavior, normalizing "" to null while preserving populated decimal strings. Can we use it for gross_entry_cost_usdc and entry_fees_usdc instead of DecimalStringSchema.nullish(), and cover the empty fallback shape? Otherwise "" is branded as a valid DecimalString.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trial period: automated Brunson reviews on this repo are being evaluated. Suggestions are a work in progress and not necessarily accurate or vetted by the team yet — please verify before acting on them.

Confirmed — [bug]. Verified at the source in Polymarket/data-api:

  • packages/api/pkg/repository/clickhouse/v1_combos.go:122 builds CombinatorialPositionPayload without GrossEntryCostUsdc / EntryFeesUsdc — the CH arm never scans them. Only pg_repository.go populates them.
  • packages/api/pkg/payload/combos_payloads.go:16-17 declares both as plain string with no omitempty, so a CH-served page emits "gross_entry_cost_usdc": "" — present, non-null, empty.

Two consequences beyond what you flagged:

  • .nullish() is dead code on this endpoint. Because those struct fields are non-pointer and lack omitempty, null/absent is unreachable; "" is the only missing encoding. DecimalStringSchema is z.string().transform(toDecimalString) with no non-empty check, so "" is branded DecimalString and flows straight into grossEntryCostUsdc − entryFeesUsdc.
  • packages/client/tests/integration/portfolio.test.ts:110-111 asserts expect.any(String), which passes on "". Even a live CH-served page wouldn't fail the suite — so the "cover the empty fallback shape" ask is a real coverage gap, not just belt-and-braces.

OptionalDecimalStringSchema is the right primitive here.

Adjacent, pre-existing, not this PR: first_entry_at has the same hole — formatUTC (v1_combos.go:263) returns "" for a NULL first_entry_at, and IsoDateTimeStringSchema brands without validating, so "" becomes an IsoDateTimeString.

Slack discussion

entry_fees_usdc: DecimalStringSchema.nullish(),
realized_payout_usdc: DecimalishSchema.nullish(),
total_cost_usdc: DecimalishSchema.nullish(),
status: ComboPositionStatusSchema,
Expand All @@ -233,6 +238,8 @@ export const ComboPositionSchema = z
shares_balance,
entry_avg_price_usdc,
entry_cost_usdc,
gross_entry_cost_usdc,
entry_fees_usdc,
realized_payout_usdc,
total_cost_usdc,
first_entry_at,
Expand All @@ -252,6 +259,8 @@ export const ComboPositionSchema = z
shares: shares_balance,
entryAvgPriceUsdc: entry_avg_price_usdc,
entryCostUsdc: entry_cost_usdc,
grossEntryCostUsdc: gross_entry_cost_usdc,
entryFeesUsdc: entry_fees_usdc,
realizedPayoutUsdc: realized_payout_usdc,
totalCostUsdc: total_cost_usdc,
firstEntryAt: first_entry_at,
Expand Down
18 changes: 16 additions & 2 deletions packages/client/src/actions/activity.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ import { snakeCase, toDataSearchParams, toSearchParams } from './params';

export { ComboActivityType } from '@polymarket/bindings/data';

const MAX_TRADES_OFFSET = 10_000;
const MAX_ACTIVITY_OFFSET = 5_000;

const TradeFilterTypeSchema = z.enum(['CASH', 'TOKENS']);

const ListTradesRequestSchema = z
Expand Down Expand Up @@ -86,6 +89,12 @@ export const ListTradesError = makeErrorGuard(
* @remarks
* This is a low-level function. Most SDK consumers should prefer the client instance API.
*
* Pagination rejects continuation past the documented 10,000 offset ceiling.
* Use `start` and `end` to query bounded time windows when deeper history is
* required. Without `start`, queries default to roughly three years of
* history. A positive `start` can extend user-scoped queries further back,
* while market- and event-scoped queries retain the three-year floor.
*
* @throws {@link ListTradesError}
* Thrown on failure.
*
Expand Down Expand Up @@ -128,7 +137,7 @@ export function listTrades(
);

return paginate((cursor) => {
const decoded = decodeOffsetCursor(cursor, pageSize);
const decoded = decodeOffsetCursor(cursor, pageSize, MAX_TRADES_OFFSET);

return client.data
.get('/trades', {
Expand Down Expand Up @@ -203,6 +212,11 @@ export const ListActivityError = makeErrorGuard(
* @remarks
* This is a low-level function. Most SDK consumers should prefer the client instance API.
*
* Pagination rejects continuation past the documented 5,000 offset ceiling.
* Use `start` and `end` to query bounded time windows when deeper history is
* required. Without `start`, descending queries default to roughly three
* years of history; ascending queries read from the beginning.
*
* @throws {@link ListActivityError}
* Thrown on failure.
*
Expand Down Expand Up @@ -245,7 +259,7 @@ export function listActivity(
);

return paginate((cursor) => {
const decoded = decodeOffsetCursor(cursor, pageSize);
const decoded = decodeOffsetCursor(cursor, pageSize, MAX_ACTIVITY_OFFSET);

return client.data
.get('/activity', {
Expand Down
8 changes: 8 additions & 0 deletions packages/client/src/decorators/account.ts
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,10 @@ export type PublicAccountActions = Prettify<
* Lists wallet activity.
*
* All activity types are returned by default, including deposits and withdrawals; use the `type` filter to narrow results.
* Pagination rejects continuation past offset 5,000. Without `start`,
* descending queries default to roughly three years of history; ascending
* queries read from the beginning. Use `start` and `end` to split deeper
* history into bounded windows.
*
* @throws {@link ListActivityError}
* Thrown on failure.
Expand Down Expand Up @@ -400,6 +404,10 @@ export type SecureAccountActions = Prettify<
* Defaults to the authenticated account's wallet when `user` is omitted.
*
* All activity types are returned by default, including deposits and withdrawals; use the `type` filter to narrow results.
* Pagination rejects continuation past offset 5,000. Without `start`,
* descending queries default to roughly three years of history; ascending
* queries read from the beginning. Use `start` and `end` to split deeper
* history into bounded windows.
*
* @throws {@link ListActivityError}
* Thrown on failure.
Expand Down
6 changes: 6 additions & 0 deletions packages/client/src/decorators/data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,12 @@ export type DataActions = {
/**
* Lists trades for a wallet, market, or event.
*
* Pagination rejects continuation past offset 10,000. Without `start`,
* queries default to roughly three years of history. A positive `start` can
* extend user-scoped queries further back, while market- and event-scoped
* queries retain the three-year floor. Use `start` and `end` to split deeper
* history into bounded windows.
*
* @throws {@link ListTradesError}
* Thrown on failure.
*
Expand Down
35 changes: 35 additions & 0 deletions packages/client/src/pagination.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import { describe, expect, it } from 'vitest';
import { UserInputError } from './errors';
import { decodeOffsetCursor, encodeOffsetCursor, paginate } from './pagination';

describe('capped offset pagination', () => {
it('allows the last documented offset', () => {
const cursor = encodeOffsetCursor({ offset: 10_000, pageSize: 500 });

expect(decodeOffsetCursor(cursor, 1, 10_000)).toEqual({
offset: 10_000,
pageSize: 500,
});
});

it('rejects a continuation computed from the last legal offset', () => {
const lastPage = { offset: 10_000, pageSize: 500 };
const cursor = encodeOffsetCursor({
offset: lastPage.offset + lastPage.pageSize,
pageSize: lastPage.pageSize,
});

expect(() => decodeOffsetCursor(cursor, 500, 10_000)).toThrow(
UserInputError,
);
});

it('rejects synchronous page setup failures through the promise contract', async () => {
const failure = new UserInputError('Invalid page');
const paginator = paginate(() => {
throw failure;
});

await expect(paginator.firstPage()).rejects.toBe(failure);
});
});
18 changes: 15 additions & 3 deletions packages/client/src/pagination.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ export type Page<T> = {
*
* On methods without a server-provided continuation signal, a full page
* reports `true` and the follow-up request returns an empty final page when
* the collection ended exactly on a page boundary.
* the collection ended exactly on a page boundary. On offset-capped history
* methods, following a full page past the endpoint maximum rejects instead,
* so capped results are not reported as complete history.
*/
hasMore: boolean;
nextCursor?: PaginationCursor;
Expand Down Expand Up @@ -63,7 +65,7 @@ export function paginate<T, TError>(

function createPaginator(cursor = initialCursor): Paginated<T> {
return {
firstPage() {
async firstPage() {
return unwrap(fetchPage(cursor));
},
from(nextCursor) {
Expand Down Expand Up @@ -105,6 +107,7 @@ export function encodeOffsetCursor(state: OffsetCursorState): PaginationCursor {
export function decodeOffsetCursor(
cursor: PaginationCursor | undefined,
pageSize: number,
maxOffset?: number,
): OffsetCursorState {
if (cursor === undefined) {
return {
Expand All @@ -113,9 +116,18 @@ export function decodeOffsetCursor(
};
}

let state: OffsetCursorState;
try {
return OffsetCursorStateSchema.parse(JSON.parse(atob(cursor)));
state = OffsetCursorStateSchema.parse(JSON.parse(atob(cursor)));
} catch (error) {
throw new UserInputError('Invalid pagination cursor', { cause: error });
}

if (maxOffset !== undefined && state.offset > maxOffset) {
throw new UserInputError(
`Pagination cannot continue past the endpoint maximum offset of ${maxOffset}; narrow the query before continuing`,
);
}

return state;
}
15 changes: 5 additions & 10 deletions packages/client/src/websockets/perps/session.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1497,21 +1497,16 @@ describe('PerpsSession', () => {
});
});

it('throws user input errors for invalid account pagination cursors', () => {
it('rejects user input errors for invalid account pagination cursors', async () => {
const cursor = toPaginationCursor(
btoa(JSON.stringify({ kind: 'perpsTrades' })),
);
const session = createSession();
const result = session.listFundingPayments({ cursor }).firstPage();

let thrown: unknown;
try {
session.listFundingPayments({ cursor }).firstPage();
} catch (error) {
thrown = error;
}

expect(thrown).toBeInstanceOf(UserInputError);
expect(thrown).toMatchObject({
await expect(result).rejects.toBeInstanceOf(UserInputError);
await expect(result).rejects.toMatchObject({
name: UserInputError.name,
message: 'Invalid Perps account pagination cursor',
cause: expect.any(Error),
});
Expand Down
41 changes: 40 additions & 1 deletion packages/client/tests/integration/activity.test.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,28 @@
import { ActivityType, ComboActivityType } from '@polymarket/client';
import {
ActivityType,
ComboActivityType,
UserInputError,
} from '@polymarket/client';
import { expectPresent, isSameEvmAddress } from '@polymarket/types';
import { afterEach, vi } from 'vitest';
import { describe, expect, it } from './fixtures';
import { expectNonEmptyPage, expectPageWindow } from './helpers';

const TEST_USER = '0x7c3db723f1d4d8cb9c550095203b686cb11e5c6b';

describe('Activity', () => {
afterEach(() => {
vi.restoreAllMocks();
});

describe('listTrades', () => {
it('lists trades for a wallet', async ({ publicClient }) => {
const result = await publicClient
.listTrades({
user: TEST_USER,
pageSize: 1,
start: 1_700_000_000,
end: 2_000_000_000,
})
.firstPage();

Expand All @@ -34,6 +45,18 @@ describe('Activity', () => {

expect(firstPage.items.length).toBeGreaterThan(0);
});

it('rejects a cursor above the documented ceiling before transport', async ({
publicClient,
}) => {
const fetchSpy = vi.spyOn(globalThis, 'fetch');
const cursor = btoa(JSON.stringify({ offset: 10_001, pageSize: 20 }));

await expect(
publicClient.listTrades({ cursor: cursor as never }).firstPage(),
).rejects.toThrow(UserInputError);
expect(fetchSpy).not.toHaveBeenCalled();
});
});

describe('listActivity', () => {
Expand All @@ -43,6 +66,8 @@ describe('Activity', () => {
user: TEST_USER,
pageSize: 100,
type: [ActivityType.TRADE],
start: 1_700_000_000,
end: 2_000_000_000,
})
.firstPage()
.then(expectNonEmptyPage);
Expand Down Expand Up @@ -70,6 +95,20 @@ describe('Activity', () => {
isSameEvmAddress(wallet, depositWalletAddress),
);
});

it('rejects a cursor above the documented ceiling before transport', async ({
publicClient,
}) => {
const fetchSpy = vi.spyOn(globalThis, 'fetch');
const cursor = btoa(JSON.stringify({ offset: 5_001, pageSize: 20 }));

await expect(
publicClient
.listActivity({ cursor: cursor as never, user: TEST_USER })
.firstPage(),
).rejects.toThrow(UserInputError);
expect(fetchSpy).not.toHaveBeenCalled();
});
});

describe('listComboActivity', () => {
Expand Down
2 changes: 2 additions & 0 deletions packages/client/tests/integration/portfolio.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ describe('Portfolio', () => {
positionId: expect.any(String),
redeemable: expect.any(Boolean),
wallet: TEST_USER,
grossEntryCostUsdc: expect.any(String),
entryFeesUsdc: expect.any(String),
realizedPayoutUsdc: expect.any(String),
totalCostUsdc: expect.any(String),
}),
Expand Down