Title
suix_getBalance intermittently returns a huge u128 value for a small-balance address
Body
Steps to Reproduce Issue
The issue is intermittent and appears to be node-local. The same suix_getBalance request can return a normal value on one fullnode and a huge value on another fullnode at roughly the same time. Repeating the same request later against the same node may return the normal value again.
- Call
suix_getBalance for this mainnet address and the SUI coin type:
curl --location 'https://rpc.mainnet.sui.io' \
--header 'content-type: application/json' \
--data '{
"method": "suix_getBalance",
"jsonrpc": "2.0",
"params": [
"0x8f1ab753c0dffbb296541b2ad22ada6437f17ed9e102dece186e4a6575ad3e68",
"0x2::sui::SUI"
],
"id": "1"
}'
-
Repeat the same request against multiple fullnodes at approximately the same time.
-
If one node returns a huge value, compare it with the sum of balances returned by suix_getAllCoins for the same owner and coin type:
suix_getAllCoins(owner, "0x2::sui::SUI")
- Repeat the same
suix_getBalance request later against the node that returned the huge value.
Optional diagnostic step:
Start a fullnode with the JSON-RPC index cache disabled and retry the same balance query:
DISABLE_INDEX_CACHE=1 sui-node --config-path /path/to/fullnode.yaml
The current source enables this variable when its parsed value is greater than zero:
read_size_from_env("DISABLE_INDEX_CACHE").unwrap_or(0) > 0
Expected Result
suix_getBalance should return the actual SUI balance for the address.
If the node detects an internally inconsistent balance state, it should return a JSON-RPC error instead of returning a valid-looking huge balance.
The API should not return a value close to u128::MAX for an address whose actual balance is small.
Actual Result
Occasionally, suix_getBalance returns a huge value close to u128::MAX.
One observed value was:
340282366920938463463374607356230107723
This value looks like a signed-to-unsigned conversion of a negative i128 balance:
2^128 = 340282366920938463463374607431768211456
2^128 - 340282366920938463463374607356230107723
= 75,538,103,733 MIST
= 75.538103733 SUI
So the observed JSON-RPC value is equivalent to:
(-75_538_103_733_i128) as u128
The behavior observed from RPC responses:
- Direct
curl requests to a fullnode can return the huge value.
- Different fullnodes can return different values for the same request at approximately the same time.
- The same node can return to the normal balance later.
Analysis
The suspicious path is the JSON-RPC balance index/cache path.
TotalBalance is stored internally with signed fields:
struct TotalBalance {
balance: i128,
num_coins: i64,
address_balance: u64,
}
The JSON-RPC response path converts the signed balance into the unsigned API response. In the affected path, unchecked casts can expose a negative internal balance as a huge unsigned value:
coin_object_count: balance.num_coins as usize,
total_balance: balance.balance as u128,
If balance.balance becomes negative due to a transient index/cache inconsistency, balance.balance as u128 wraps the negative value and returns a number close to u128::MAX.
A likely race is:
- The balance index computes a delta for a transaction.
commit_index_batch writes the new DB state.
- Before the cache delta is merged, an RPC balance request has a cache miss.
- That RPC request reads the already-updated DB state and fills the cache with the new balance.
- The commit path then merges the old delta into this already-new cache value.
- For a spend/delete path, this can double-apply the negative delta and make the cached
TotalBalance.balance negative.
- The RPC layer casts that negative
i128 to u128, returning a huge value.
This would explain why the issue is intermittent, node-local, and can disappear after the cache entry is rebuilt or invalidated.
Suggested Fix
Since a PR submission was not available from my side, here are the suggested code changes directly.
1. Prevent RPC from exposing negative internal balances as huge u128
File:
crates/sui-json-rpc/src/coin_api.rs
Add a helper that uses checked conversions:
fn build_balance_response(coin_type: String, balance: TotalBalance) -> RpcInterimResult<Balance> {
let coin_object_count = usize::try_from(balance.num_coins).map_err(|_| {
Error::from(SuiErrorKind::ExecutionError(format!(
"negative coin object count in JSON-RPC balance cache for {coin_type}: {}",
balance.num_coins
)))
})?;
let total_balance = u128::try_from(balance.balance).map_err(|_| {
Error::from(SuiErrorKind::ExecutionError(format!(
"negative total balance in JSON-RPC balance cache for {coin_type}: {}",
balance.balance
)))
})?;
Ok(Balance {
coin_type,
coin_object_count,
total_balance,
locked_balance: Default::default(),
funds_in_address_balance: balance.address_balance as u128,
})
}
Then use this helper in get_balance:
let balance = self
.internal
.get_balance(owner, coin_type_tag.clone())
.await
.tap_err(|e| {
debug!(?owner, "Failed to get balance with error: {:?}", e);
})?;
build_balance_response(coin_type_tag.to_string(), balance)
And in get_all_balances:
all_balance
.iter()
.map(|(coin_type, balance)| build_balance_response(coin_type.to_string(), *balance))
.collect::<RpcInterimResult<Vec<_>>>()
This prevents a negative internal balance from being serialized as a valid huge u128 balance.
2. Invalidate negative balance cache entries and recompute from DB
File:
crates/sui-core/src/jsonrpc_index.rs
Add helpers to detect negative cached balance values:
fn contains_negative_total_balance(balance: &SuiResult<TotalBalance>) -> bool {
matches!(balance, Ok(balance) if balance.balance < 0 || balance.num_coins < 0)
}
fn contains_negative_all_balance(balances: &SuiResult<Arc<AllBalance>>) -> bool {
matches!(balances, Ok(balances) if balances
.values()
.any(|balance| balance.balance < 0 || balance.num_coins < 0))
}
In get_coin_object_balance, invalidate a negative per-coin cache entry instead of returning it:
let cache_key = (owner, coin_type.clone());
let balance = self.caches.per_coin_type_balance.get(&cache_key);
if let Some(balance) = balance {
if Self::contains_negative_total_balance(&balance) {
debug!(
?owner,
?coin_type,
"invalidating negative balance cache entry"
);
self.caches.per_coin_type_balance.invalidate(&cache_key);
} else {
return balance;
}
}
Also reject a negative balance found through the all-balances cache fallback:
let all_balance = self.caches.all_balances.get(&owner);
if let Some(Ok(all_balance)) = all_balance
&& let Some(balance) = all_balance.get(&coin_type)
{
if balance.balance < 0 || balance.num_coins < 0 {
debug!(
?owner,
?coin_type,
"invalidating all-balance cache entry containing negative balance"
);
self.caches.all_balances.invalidate(&owner);
} else {
return Ok(*balance);
}
}
In get_all_coin_object_balances, invalidate an all-balances cache entry if any cached coin balance is negative:
if let Some(all_balance) = self.caches.all_balances.get(&owner) {
if Self::contains_negative_all_balance(&all_balance) {
debug!(
?owner,
"invalidating all-balance cache entry containing negative balance"
);
self.caches.all_balances.invalidate(&owner);
} else {
return all_balance;
}
}
The DB recomputation path sums persisted coin object balances, so it should rebuild a non-negative value under normal conditions.
Tests Added
Suggested tests for crates/sui-core/src/jsonrpc_index.rs:
test_negative_per_coin_balance_cache_is_recomputed_from_db
test_negative_all_balance_cache_is_recomputed_from_db
Suggested tests for crates/sui-json-rpc/src/coin_api.rs:
test_get_balance_negative_total_balance_returns_error
test_negative_total_balance_returns_error
Suggested validation commands:
cargo fmt --all -- --check
SUI_SKIP_SIMTESTS=1 cargo nextest run -p sui-core test_negative --no-fail-fast
SUI_SKIP_SIMTESTS=1 cargo nextest run -p sui-json-rpc negative_total_balance --no-fail-fast
Title
suix_getBalanceintermittently returns a hugeu128value for a small-balance addressBody
Steps to Reproduce Issue
The issue is intermittent and appears to be node-local. The same
suix_getBalancerequest can return a normal value on one fullnode and a huge value on another fullnode at roughly the same time. Repeating the same request later against the same node may return the normal value again.suix_getBalancefor this mainnet address and the SUI coin type:Repeat the same request against multiple fullnodes at approximately the same time.
If one node returns a huge value, compare it with the sum of balances returned by
suix_getAllCoinsfor the same owner and coin type:suix_getBalancerequest later against the node that returned the huge value.Optional diagnostic step:
Start a fullnode with the JSON-RPC index cache disabled and retry the same balance query:
The current source enables this variable when its parsed value is greater than zero:
Expected Result
suix_getBalanceshould return the actual SUI balance for the address.If the node detects an internally inconsistent balance state, it should return a JSON-RPC error instead of returning a valid-looking huge balance.
The API should not return a value close to
u128::MAXfor an address whose actual balance is small.Actual Result
Occasionally,
suix_getBalancereturns a huge value close tou128::MAX.One observed value was:
This value looks like a signed-to-unsigned conversion of a negative
i128balance:So the observed JSON-RPC value is equivalent to:
The behavior observed from RPC responses:
curlrequests to a fullnode can return the huge value.Analysis
The suspicious path is the JSON-RPC balance index/cache path.
TotalBalanceis stored internally with signed fields:The JSON-RPC response path converts the signed balance into the unsigned API response. In the affected path, unchecked casts can expose a negative internal balance as a huge unsigned value:
If
balance.balancebecomes negative due to a transient index/cache inconsistency,balance.balance as u128wraps the negative value and returns a number close tou128::MAX.A likely race is:
commit_index_batchwrites the new DB state.TotalBalance.balancenegative.i128tou128, returning a huge value.This would explain why the issue is intermittent, node-local, and can disappear after the cache entry is rebuilt or invalidated.
Suggested Fix
Since a PR submission was not available from my side, here are the suggested code changes directly.
1. Prevent RPC from exposing negative internal balances as huge
u128File:
Add a helper that uses checked conversions:
Then use this helper in
get_balance:And in
get_all_balances:all_balance .iter() .map(|(coin_type, balance)| build_balance_response(coin_type.to_string(), *balance)) .collect::<RpcInterimResult<Vec<_>>>()This prevents a negative internal balance from being serialized as a valid huge
u128balance.2. Invalidate negative balance cache entries and recompute from DB
File:
Add helpers to detect negative cached balance values:
In
get_coin_object_balance, invalidate a negative per-coin cache entry instead of returning it:Also reject a negative balance found through the all-balances cache fallback:
In
get_all_coin_object_balances, invalidate an all-balances cache entry if any cached coin balance is negative:The DB recomputation path sums persisted coin object balances, so it should rebuild a non-negative value under normal conditions.
Tests Added
Suggested tests for
crates/sui-core/src/jsonrpc_index.rs:Suggested tests for
crates/sui-json-rpc/src/coin_api.rs:Suggested validation commands: