Publish rskim-tokens to crates.io for external consumers (follow-up from #300)

[dean0x]

Context

Tracking issue filed as required by Decision 5 / ADR-004 in the L3 Wave-1 plan.
Per that decision: "The crates.io publish follow-up ticket is filed and linked before #300 merges to main."

This is a merge gate for PR #329 (#300 implementation).

Background

rskim-tokens was created with publish = false (matching the rskim-search precedent) because:

• The library is currently tightly coupled to workspace-internal tiktoken-rs version pinning
• External publish needs API stability review and SemVer commitment
• No external consumers exist yet — the initial consumer is the skim binary itself

Work Required

When ready to publish externally:

1. Version audit — ensure rskim-tokens version is SemVer-committed (start at 0.1.0)
2. Remove publish = false from crates/rskim-tokens/Cargo.toml
3. API stability review — Counter, Encoding, encoding_for_model, TokenError, counter_for_model are the public API surface
4. Add rskim-tokens to the release pipeline in .github/workflows/release.yml — publish after rskim-core, before rskim
5. Update README.md with crates.io badge and usage example for external consumers
6. net-anthropic feature decision — document whether the optional feature is part of the published API or kept unstable

Out of Scope for This Ticket

• Changing the current behavior (offline-first, net-anthropic feature-gated)
• Breaking API changes

Notes

• rskim-core (the parsing library) was already published on crates.io — rskim-tokens would follow the same pattern
• The tiktoken-rs workspace version pin (0.7.0) must be evaluated for stability before publish

Linked from: #329 (PR implementing #300)
Decisions: Decision 5, ADR-004