Skip to content

Pull requests: openai/tiktoken

New pull request New
61 Open 117 Closed
61 Open 117 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Avoid panic when byte_pair_encode is given an empty piece
#559 opened May 25, 2026 by adityasingh2400 Loading…
Fix CoreBPE::new type inference failure
#558 opened May 24, 2026 by fallintoplace Loading…
1
fix(py): make encode_with_unstable handle surrogates like encode (#541)
#556 opened May 24, 2026 by LeSingh1 Loading…
1
Recognise gpt-5.1 model identifier (Fixes #464)
#554 opened May 23, 2026 by jbbqqf Loading…
1
Repair surrogates in encode_with_unstable (Fixes #541)
#553 opened May 23, 2026 by jbbqqf Loading…
1
Add lexer fast path for o200k_base, cl100k_base, and gpt-2 pretokenization patterns; custom regexes fall back to fancy_regex
#552 opened May 22, 2026 by augustasio Loading…
Add 2-byte pair lookup table to skip hashmap for initial BPE merge scan
#551 opened May 22, 2026 by augustasio Loading…
bench: add tail-latency benchmark for encode_ordinary_batch
#548 opened May 20, 2026 by alobroke Loading…
1
build: enable line tables in release builds so backtraces show file:line (#103)
#546 opened May 18, 2026 by LeSingh1 Loading…
1
fix(py): report element count in buffer.shape, not byte length (#405)
#545 opened May 18, 2026 by LeSingh1 Loading…
1
1
Use user-specific cache directory by default
#544 opened May 15, 2026 by MukundaKatta Contributor Loading…
1
Handle surrogate pairs in unstable encoding
#542 opened May 13, 2026 by shreejaykurhade Loading…
1
Add HTTP timeout to read_file network fetch
#539 opened May 9, 2026 by Spectual Loading…
1 of 3 tasks
perf: 3.3x faster encode_ordinary_batch for single-threaded workloads
#531 opened Apr 22, 2026 by homanp Loading…
build: add PyInstaller hooks for dynamic encodings
#529 opened Apr 19, 2026 by MukundaKatta Contributor Loading…
1
fix(cache): respect TIKTOKEN_OFFLINE env var to prevent network downloads
#527 opened Apr 18, 2026 by xodn348 Loading…
1
Honor offline cache for encoding downloads
#525 opened Apr 18, 2026 by MukundaKatta Contributor Loading…
1
Respect TIKTOKEN_OFFLINE env var to prevent network access in offline mode
#514 opened Mar 27, 2026 by Spectual Loading…
3 tasks
1
1
Add command-line interface for tiktoken
#508 opened Mar 14, 2026 by alpnix Loading…
1
bug fixes
#507 opened Mar 14, 2026 by eeea2222 Loading…
7
ci: add riscv64 wheel builds
#506 opened Mar 12, 2026 by gounthar Loading…
1
9
Add support for armv7l wheel building
#505 opened Mar 12, 2026 by ekarad1um Loading…
3
Upgrade GitHub Actions to latest versions
#504 opened Mar 11, 2026 by salmanmkc Contributor Loading…
Upgrade GitHub Actions for Node 24 compatibility
#503 opened Mar 11, 2026 by salmanmkc Contributor Loading…
Add support for building and testing tiktoken on Windows ARM64
#501 opened Mar 9, 2026 by MugundanMCW Loading…
1
ProTip! Add no:assignee to see everything that’s not assigned.