Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 25 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,31 @@ Top-level modules split into **public** (part of the API surface, re-exported fr

### Feature flags

- `default = []` — minimal core
- `ws` — enables WebSocket transport (`alloy-provider/pubsub` + `ws`) and `create_ws_provider`
- `odos-example` — pulls in `odos-sdk` and enables `OdosPriceSource`, `PriceCalculator`, and the `router_token_discovery` example

Any new feature-gated public export needs the matching `#[cfg(feature = "...")]` on the `pub use` line in `lib.rs`.
Each domain is a feature; `default` enables them all, so default builds are
source-compatible. Inter-feature dependencies mirror real module dependencies:

- `blocks` — block-window calculations (no domain deps)
- `events` — log scanning and event decoding (no domain deps)
- `transport` — Tower rate-limit/retry layers (no domain deps)
- `gas = ["events"]` — decodes `Transfer`/`Approval`, so pulls in `events`
- `price = ["dep:alloy-erc20"]` — DEX price extraction; on-chain decimals via `alloy-erc20`
- `provider = ["transport"]` — provider construction layered with `transport`
- `retrieval = ["events", "gas", "dep:alloy-erc20"]` — combined gas/price/balance orchestration
- `ws = ["provider", ...]` — WebSocket transport and `create_ws_provider`; composes with `provider`

`alloy-erc20` is the one optional dependency (pulled only by `price`/`retrieval`);
every other dependency is shared across enough domains to stay always-on.

The core modules `config`, `errors`, `types`, `cache`, `scan`, and `tracing` are
always compiled, but domain-specific items inside them (per-domain error variants,
gas/blocks/retrieval span helpers) are `#[cfg]`-gated to their owning feature so a
minimal build stays warning-clean under `-D warnings`.

Any new feature-gated public export needs the matching `#[cfg(feature = "...")]` on
the `pub use` line in `lib.rs`. Feature-specific integration tests carry a
`#![cfg(feature = "...")]` crate attribute; feature-specific examples declare
`required-features` in `Cargo.toml`. CI exercises `default`, `--all-features`, and
`--no-default-features` for test, clippy, and doc.

### Testing strategy

Expand Down
37 changes: 34 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,23 @@ keywords = ["ethereum", "blockchain", "analytics", "defi", "evm"]
categories = ["cryptography::cryptocurrencies", "development-tools"]

[features]
default = []
ws = ["alloy-provider/pubsub", "alloy-provider/ws"]
default = ["blocks", "events", "gas", "price", "provider", "retrieval", "transport"]
# Block-window calculations: map UTC dates to block ranges.
blocks = []
# Log scanning and event decoding (Transfer/Approval, EventScanner).
events = []
# Transport-layer Tower middleware (rate limiting, retries).
transport = []
# Gas cost calculation (L1 + L2). Decodes Transfer/Approval, so pulls in `events`.
gas = ["events"]
# DEX price extraction. Uses on-chain token decimals via `alloy-erc20`.
price = ["dep:alloy-erc20"]
# Runtime provider construction and pooling; layers rate limiting from `transport`.
provider = ["transport"]
# High-level orchestration combining gas, events, and balance lookups.
retrieval = ["events", "gas", "dep:alloy-erc20"]
# WebSocket transport; composes with `provider`.
ws = ["provider", "alloy-provider/pubsub", "alloy-provider/ws"]

[dependencies]
# Core blockchain dependencies (always required)
Expand All @@ -24,7 +39,7 @@ alloy-contract = { version = "2.0", default-features = false }
alloy-dyn-abi = { version = "1.6", default-features = false }
alloy-json-rpc = "2.0"
alloy-eips = { version = "2.0", default-features = false }
alloy-erc20 = { version = "2.0", default-features = false }
alloy-erc20 = { version = "2.0", default-features = false, optional = true }
alloy-network = { version = "2.0", default-features = false }
alloy-primitives = { version = "1.6", default-features = false, features = [
"std",
Expand Down Expand Up @@ -75,3 +90,19 @@ tracing-subscriber = { version = "0.3", default-features = false, features = [
"fmt",
"json",
] }

[[example]]
name = "custom_dex_integration"
required-features = ["price"]

[[example]]
name = "daily_block_window"
required-features = ["blocks"]

[[example]]
name = "eip4844_blob_gas"
required-features = ["gas"]

[[example]]
name = "zksync_combined_probe"
required-features = ["retrieval", "provider"]
24 changes: 23 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,29 @@ semioscan = "0.12"

### Feature Flags

- **`ws`**: Enables WebSocket transport (`alloy-provider/pubsub` + `ws`) and `create_ws_provider` for streaming event subscriptions
Each domain is a Cargo feature. The `default` feature enables all of them, so a
plain `semioscan = "0.14"` dependency gives you the full library. To slim the
build down to just the domains you use, disable defaults and opt in:

```toml
[dependencies]
# Just block-window calculations:
semioscan = { version = "0.14", default-features = false, features = ["blocks"] }
```

| Feature | Provides | Enables |
| --- | --- | --- |
| `blocks` | block-window (date → block range) calculations | — |
| `events` | log scanning and event decoding (`Transfer`/`Approval`, `EventScanner`) | — |
| `transport` | Tower rate-limit and retry layers | — |
| `gas` | L1/L2 gas cost calculation | `events` |
| `price` | DEX price extraction (reads on-chain token decimals) | — |
| `provider` | runtime provider construction and pooling | `transport` |
| `retrieval` | combined gas/price/balance orchestration | `events`, `gas` |
| `ws` | WebSocket transport and `create_ws_provider` for streaming subscriptions | `provider` |

A `--no-default-features` build with no domains selected compiles only the core
configuration, error, and strong-type machinery.

## Quick Start

Expand Down
3 changes: 3 additions & 0 deletions src/cache/block_range.rs
Original file line number Diff line number Diff line change
Expand Up @@ -252,16 +252,19 @@ where
}

/// Get the total number of cached entries
#[cfg_attr(not(feature = "gas"), allow(dead_code))]
pub fn len(&self) -> usize {
self.cache.len()
}

/// Check if the cache contains no entries
#[cfg_attr(not(feature = "gas"), allow(dead_code))]
pub fn is_empty(&self) -> bool {
self.cache.is_empty()
}

/// Clear all entries matching a predicate on the key
#[cfg_attr(not(feature = "gas"), allow(dead_code))]
pub fn retain<F>(&mut self, mut predicate: F)
where
F: FnMut(&K, BlockNumber, BlockNumber) -> bool,
Expand Down
15 changes: 15 additions & 0 deletions src/errors/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,17 +66,27 @@
//! }
//! ```

#[cfg(feature = "blocks")]
mod blocks;
#[cfg(feature = "events")]
mod events;
#[cfg(feature = "gas")]
mod gas;
#[cfg(feature = "price")]
mod price;
#[cfg(feature = "retrieval")]
mod retrieval;
mod rpc;

#[cfg(feature = "blocks")]
pub use blocks::BlockWindowError;
#[cfg(feature = "events")]
pub use events::EventProcessingError;
#[cfg(feature = "gas")]
pub use gas::GasCalculationError;
#[cfg(feature = "price")]
pub use price::PriceCalculationError;
#[cfg(feature = "retrieval")]
pub use retrieval::RetrievalError;
pub use rpc::RpcError;

Expand Down Expand Up @@ -112,22 +122,27 @@ pub use rpc::RpcError;
#[derive(Debug, thiserror::Error)]
pub enum SemioscanError {
/// Error from block window calculations.
#[cfg(feature = "blocks")]
#[error("Block window error: {0}")]
BlockWindow(#[from] BlockWindowError),

/// Error from gas cost calculations.
#[cfg(feature = "gas")]
#[error("Gas calculation error: {0}")]
Gas(#[from] GasCalculationError),

/// Error from price calculations.
#[cfg(feature = "price")]
#[error("Price calculation error: {0}")]
Price(#[from] PriceCalculationError),

/// Error from event processing operations.
#[cfg(feature = "events")]
#[error("Event processing error: {0}")]
Events(#[from] EventProcessingError),

/// Error from combined data retrieval operations.
#[cfg(feature = "retrieval")]
#[error("Data retrieval error: {0}")]
Retrieval(#[from] RetrievalError),
}
2 changes: 1 addition & 1 deletion src/gas/calculator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ use tokio::sync::Mutex;

use crate::config::SemioscanConfig;
use crate::gas::cache::GasCache;
use crate::retrieval::DecimalPrecision;
use crate::types::config::TransactionCount;
use crate::types::decimal_precision::DecimalPrecision;
use crate::types::fees::L1DataFee;
use crate::types::gas::{BlobCount, BlobGasPrice, GasAmount, GasBreakdown, GasPrice};
use crate::types::wei::WeiAmount;
Expand Down
71 changes: 65 additions & 6 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,24 +32,63 @@
//! live under their public modules — for example a custom price source pulls
//! `PriceSource` and `PriceSourceError` from `semioscan::price`, and transport
//! layers come from `semioscan::transport`.
//!
//! # Feature flags
//!
//! Every domain is a Cargo feature, and the `default` feature enables all of
//! them — so a default `semioscan` dependency behaves exactly as before.
//! Downstream crates that only need part of the library can opt out and pull
//! in just the domains they use:
//!
//! ```toml
//! # Just block-window calculations, nothing else:
//! semioscan = { version = "0.14", default-features = false, features = ["blocks"] }
//! ```
//!
//! Feature dependencies follow the real module dependencies:
//!
//! - `blocks`, `events`, `transport` — standalone domains
//! - `gas` enables `events` (it decodes `Transfer`/`Approval` logs)
//! - `price` — DEX price extraction (reads on-chain token decimals)
//! - `provider` enables `transport`
//! - `retrieval` enables `events` and `gas` (combined gas/price/balance lookups)
//! - `ws` enables `provider` and adds WebSocket transport (`create_ws_provider`)
//!
//! A `--no-default-features` build with no domains selected compiles the core
//! configuration, error, and type machinery only.

// === Module Declarations ===
#[cfg(feature = "blocks")]
mod blocks;
#[cfg(any(feature = "gas", feature = "price"))]
mod cache;
pub mod config;
pub mod errors;
#[cfg(feature = "events")]
mod events;
#[cfg(feature = "gas")]
mod gas;
#[cfg(feature = "price")]
pub mod price;
#[cfg(feature = "provider")]
pub mod provider;
#[cfg(feature = "retrieval")]
mod retrieval;
#[cfg(any(
feature = "events",
feature = "gas",
feature = "price",
feature = "retrieval"
))]
mod scan;
mod tracing;
#[cfg(feature = "transport")]
pub mod transport;
mod types;

// === Core Types (from types/) ===
pub use types::config::{BlockCount, MaxBlockRange, TransactionCount};
pub use types::decimal_precision::DecimalPrecision;
pub use types::fees::{L1DataFee, Percentage};
pub use types::gas::{
BlobCount, BlobGasAmount, BlobGasPrice, GasAmount, GasBreakdown, GasBreakdownBuilder, GasPrice,
Expand All @@ -67,23 +106,36 @@ pub use config::policy::{
pub use config::{ChainConfig, SemioscanConfig, SemioscanConfigBuilder};

// === Error Types (from errors/) ===
pub use errors::{
BlockWindowError, EventProcessingError, GasCalculationError, PriceCalculationError,
RetrievalError, RpcError, SemioscanError,
};
#[cfg(feature = "blocks")]
pub use errors::BlockWindowError;
#[cfg(feature = "events")]
pub use errors::EventProcessingError;
#[cfg(feature = "gas")]
pub use errors::GasCalculationError;
#[cfg(feature = "price")]
pub use errors::PriceCalculationError;
#[cfg(feature = "retrieval")]
pub use errors::RetrievalError;
pub use errors::{RpcError, SemioscanError};

// === Gas Calculation (from gas/) ===
#[cfg(feature = "gas")]
pub use gas::adapter::{EthereumReceiptAdapter, OptimismReceiptAdapter, ReceiptAdapter};
#[cfg(feature = "gas")]
pub use gas::blob;
#[cfg(feature = "gas")]
pub use gas::cache::GasCache;
#[cfg(feature = "gas")]
pub use gas::{EventType, GasCostCalculator, GasCostResult, GasForTx};

// === Price Extraction (from price/) ===
#[cfg(feature = "price")]
pub use price::{
PriceCalculator, PriceSource, PriceSourceError, RawSwapResult, SwapData, TokenPriceResult,
};

// === Block Windows (from blocks/) ===
#[cfg(feature = "blocks")]
pub use blocks::{
BlockWindowCache, BlockWindowCalculator, CacheKey, CacheStats, DailyBlockWindow, DiskCache,
MemoryCache, NoOpCache, UnixTimestamp, DEFAULT_HEAD_TTL,
Expand All @@ -93,29 +145,36 @@ pub use blocks::{
pub use types::cache::{AccessSequence, TimestampMillis};

// === Events (from events/) ===
#[cfg(feature = "events")]
pub use events::fetch_logs_chunked;
#[cfg(feature = "events")]
pub use events::EventScanner;
#[cfg(feature = "events")]
pub use events::{extract_transferred_to_tokens, extract_transferred_to_tokens_with_config};
#[cfg(feature = "events")]
pub use events::{AmountCalculator, AmountResult};
#[cfg(feature = "events")]
pub use events::{Approval, Transfer};

// === Retrieval (Data Orchestration) ===
#[cfg(feature = "retrieval")]
pub use retrieval::{
batch_fetch_balances, batch_fetch_eth_balances, get_token_decimal_precision,
u256_to_bigdecimal, BalanceError, BalanceQuery, BalanceResult, CombinedCalculator,
CombinedDataLookupAttempt, CombinedDataLookupFailure, CombinedDataLookupPass,
CombinedDataLookupStage, CombinedDataResult, CombinedDataRetrievalMetadata, DecimalPrecision,
GasAndAmountForTx,
CombinedDataLookupStage, CombinedDataResult, CombinedDataRetrievalMetadata, GasAndAmountForTx,
};

// === Transport Layers ===
#[cfg(feature = "transport")]
pub use transport::{
RateLimitLayer, RateLimitService, RetryConfig, RetryLayer, RetryLayerBuilder, RetryService,
};

// === Provider Utilities ===
#[cfg(feature = "ws")]
pub use provider::create_ws_provider;
#[cfg(feature = "provider")]
pub use provider::{
create_http_provider, create_typed_http_provider, network_type_for_chain,
rate_limited_http_provider, simple_http_provider, AnyHttpProvider, ChainAwareProvider,
Expand Down
2 changes: 0 additions & 2 deletions src/retrieval/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
// Combined retrieval sub-modules
pub mod balance;
mod calculator;
mod decimal_precision;
mod failure;
mod gas_calculation;
mod gas_extractor;
Expand All @@ -27,7 +26,6 @@ pub use balance::{
batch_fetch_balances, batch_fetch_eth_balances, BalanceError, BalanceQuery, BalanceResult,
};
pub use calculator::CombinedCalculator;
pub use decimal_precision::DecimalPrecision;
pub use types::{
CombinedDataLookupAttempt, CombinedDataLookupFailure, CombinedDataLookupPass,
CombinedDataLookupStage, CombinedDataResult, CombinedDataRetrievalMetadata, GasAndAmountForTx,
Expand Down
3 changes: 1 addition & 2 deletions src/retrieval/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ use std::str::FromStr;

use crate::config::constants::stablecoins::BSC_BINANCE_PEG_USDC;
use crate::errors::RetrievalError;

use super::decimal_precision::DecimalPrecision;
use crate::types::decimal_precision::DecimalPrecision;

/// Get the decimal precision for a specific token on a specific chain.
/// Native tokens (Address::ZERO) use 18 decimals.
Expand Down
1 change: 1 addition & 0 deletions src/scan/logs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -705,6 +705,7 @@ mod tests {
);
}

#[cfg(feature = "events")]
#[tokio::test]
async fn event_scanner_wrapper_skips_failed_chunks() {
// Integration test: confirm the public EventScanner wrapper actually
Expand Down
1 change: 1 addition & 0 deletions src/tracing/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
//!
//! This module provides structured tracing support for semioscan operations.

#[cfg(any(feature = "blocks", feature = "gas", feature = "retrieval"))]
pub(crate) mod spans;

// Note: All span functions are internal (pub(crate)) and not re-exported
Loading
Loading