Skip to content

suix_getBalance intermittently returns a huge u128 value for a small-balance address #26776

@NoahGao96

Description

@NoahGao96

Title

suix_getBalance intermittently returns a huge u128 value for a small-balance address

Body

Steps to Reproduce Issue

The issue is intermittent and appears to be node-local. The same suix_getBalance request can return a normal value on one fullnode and a huge value on another fullnode at roughly the same time. Repeating the same request later against the same node may return the normal value again.

  1. Call suix_getBalance for this mainnet address and the SUI coin type:
curl --location 'https://rpc.mainnet.sui.io' \
  --header 'content-type: application/json' \
  --data '{
    "method": "suix_getBalance",
    "jsonrpc": "2.0",
    "params": [
      "0x8f1ab753c0dffbb296541b2ad22ada6437f17ed9e102dece186e4a6575ad3e68",
      "0x2::sui::SUI"
    ],
    "id": "1"
  }'
  1. Repeat the same request against multiple fullnodes at approximately the same time.

  2. If one node returns a huge value, compare it with the sum of balances returned by suix_getAllCoins for the same owner and coin type:

suix_getAllCoins(owner, "0x2::sui::SUI")
  1. Repeat the same suix_getBalance request later against the node that returned the huge value.

Optional diagnostic step:

Start a fullnode with the JSON-RPC index cache disabled and retry the same balance query:

DISABLE_INDEX_CACHE=1 sui-node --config-path /path/to/fullnode.yaml

The current source enables this variable when its parsed value is greater than zero:

read_size_from_env("DISABLE_INDEX_CACHE").unwrap_or(0) > 0

Expected Result

suix_getBalance should return the actual SUI balance for the address.

If the node detects an internally inconsistent balance state, it should return a JSON-RPC error instead of returning a valid-looking huge balance.

The API should not return a value close to u128::MAX for an address whose actual balance is small.

Actual Result

Occasionally, suix_getBalance returns a huge value close to u128::MAX.

One observed value was:

340282366920938463463374607356230107723

This value looks like a signed-to-unsigned conversion of a negative i128 balance:

2^128 = 340282366920938463463374607431768211456

2^128 - 340282366920938463463374607356230107723
= 75,538,103,733 MIST
= 75.538103733 SUI

So the observed JSON-RPC value is equivalent to:

(-75_538_103_733_i128) as u128

The behavior observed from RPC responses:

  • Direct curl requests to a fullnode can return the huge value.
  • Different fullnodes can return different values for the same request at approximately the same time.
  • The same node can return to the normal balance later.

Analysis

The suspicious path is the JSON-RPC balance index/cache path.

TotalBalance is stored internally with signed fields:

struct TotalBalance {
    balance: i128,
    num_coins: i64,
    address_balance: u64,
}

The JSON-RPC response path converts the signed balance into the unsigned API response. In the affected path, unchecked casts can expose a negative internal balance as a huge unsigned value:

coin_object_count: balance.num_coins as usize,
total_balance: balance.balance as u128,

If balance.balance becomes negative due to a transient index/cache inconsistency, balance.balance as u128 wraps the negative value and returns a number close to u128::MAX.

A likely race is:

  1. The balance index computes a delta for a transaction.
  2. commit_index_batch writes the new DB state.
  3. Before the cache delta is merged, an RPC balance request has a cache miss.
  4. That RPC request reads the already-updated DB state and fills the cache with the new balance.
  5. The commit path then merges the old delta into this already-new cache value.
  6. For a spend/delete path, this can double-apply the negative delta and make the cached TotalBalance.balance negative.
  7. The RPC layer casts that negative i128 to u128, returning a huge value.

This would explain why the issue is intermittent, node-local, and can disappear after the cache entry is rebuilt or invalidated.

Suggested Fix

Since a PR submission was not available from my side, here are the suggested code changes directly.

1. Prevent RPC from exposing negative internal balances as huge u128

File:

crates/sui-json-rpc/src/coin_api.rs

Add a helper that uses checked conversions:

fn build_balance_response(coin_type: String, balance: TotalBalance) -> RpcInterimResult<Balance> {
    let coin_object_count = usize::try_from(balance.num_coins).map_err(|_| {
        Error::from(SuiErrorKind::ExecutionError(format!(
            "negative coin object count in JSON-RPC balance cache for {coin_type}: {}",
            balance.num_coins
        )))
    })?;
    let total_balance = u128::try_from(balance.balance).map_err(|_| {
        Error::from(SuiErrorKind::ExecutionError(format!(
            "negative total balance in JSON-RPC balance cache for {coin_type}: {}",
            balance.balance
        )))
    })?;

    Ok(Balance {
        coin_type,
        coin_object_count,
        total_balance,
        locked_balance: Default::default(),
        funds_in_address_balance: balance.address_balance as u128,
    })
}

Then use this helper in get_balance:

let balance = self
    .internal
    .get_balance(owner, coin_type_tag.clone())
    .await
    .tap_err(|e| {
        debug!(?owner, "Failed to get balance with error: {:?}", e);
    })?;

build_balance_response(coin_type_tag.to_string(), balance)

And in get_all_balances:

all_balance
    .iter()
    .map(|(coin_type, balance)| build_balance_response(coin_type.to_string(), *balance))
    .collect::<RpcInterimResult<Vec<_>>>()

This prevents a negative internal balance from being serialized as a valid huge u128 balance.

2. Invalidate negative balance cache entries and recompute from DB

File:

crates/sui-core/src/jsonrpc_index.rs

Add helpers to detect negative cached balance values:

fn contains_negative_total_balance(balance: &SuiResult<TotalBalance>) -> bool {
    matches!(balance, Ok(balance) if balance.balance < 0 || balance.num_coins < 0)
}

fn contains_negative_all_balance(balances: &SuiResult<Arc<AllBalance>>) -> bool {
    matches!(balances, Ok(balances) if balances
        .values()
        .any(|balance| balance.balance < 0 || balance.num_coins < 0))
}

In get_coin_object_balance, invalidate a negative per-coin cache entry instead of returning it:

let cache_key = (owner, coin_type.clone());
let balance = self.caches.per_coin_type_balance.get(&cache_key);
if let Some(balance) = balance {
    if Self::contains_negative_total_balance(&balance) {
        debug!(
            ?owner,
            ?coin_type,
            "invalidating negative balance cache entry"
        );
        self.caches.per_coin_type_balance.invalidate(&cache_key);
    } else {
        return balance;
    }
}

Also reject a negative balance found through the all-balances cache fallback:

let all_balance = self.caches.all_balances.get(&owner);
if let Some(Ok(all_balance)) = all_balance
    && let Some(balance) = all_balance.get(&coin_type)
{
    if balance.balance < 0 || balance.num_coins < 0 {
        debug!(
            ?owner,
            ?coin_type,
            "invalidating all-balance cache entry containing negative balance"
        );
        self.caches.all_balances.invalidate(&owner);
    } else {
        return Ok(*balance);
    }
}

In get_all_coin_object_balances, invalidate an all-balances cache entry if any cached coin balance is negative:

if let Some(all_balance) = self.caches.all_balances.get(&owner) {
    if Self::contains_negative_all_balance(&all_balance) {
        debug!(
            ?owner,
            "invalidating all-balance cache entry containing negative balance"
        );
        self.caches.all_balances.invalidate(&owner);
    } else {
        return all_balance;
    }
}

The DB recomputation path sums persisted coin object balances, so it should rebuild a non-negative value under normal conditions.

Tests Added

Suggested tests for crates/sui-core/src/jsonrpc_index.rs:

test_negative_per_coin_balance_cache_is_recomputed_from_db
test_negative_all_balance_cache_is_recomputed_from_db

Suggested tests for crates/sui-json-rpc/src/coin_api.rs:

test_get_balance_negative_total_balance_returns_error
test_negative_total_balance_returns_error

Suggested validation commands:

cargo fmt --all -- --check
SUI_SKIP_SIMTESTS=1 cargo nextest run -p sui-core test_negative --no-fail-fast
SUI_SKIP_SIMTESTS=1 cargo nextest run -p sui-json-rpc negative_total_balance --no-fail-fast

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions