Release v0.2.17

Lightning AI ⚡ is excited to announce the release of LitServe v0.2.17

Highlights

Automatic Worker Restart

LitServe now supports automatic restarting of inference workers when they die, ensuring high availability and resilience in production environments. This prevents server shutdown due to isolated worker failures and maintains service continuity.

import litserve as ls

server = ls.LitServer(
    MyAPI(),
    restart_workers=True,  # Automatically restart failed workers
    workers_per_device=4
)
server.run()

When a worker terminates unexpectedly, the server automatically spawns a replacement, keeping requests flowing without interruption.

Changes

Added

• Add support for restarting the inference worker when they die by @tchaton in #624

Changed

• Update README to reflect inference engines terminology by @williamFalcon in #625
• chore: drop support for Python 3.9 by @bhimrazy in #641

Fixed

• Add warning for dict/set outputs in batched predict to catch edge cases by @Copilot in #612
• fix(sdk): Reduce the quantity of warning emitted by @tchaton in #631
• fix(litServe): Use asyncio.sleep instead of time.sleep by @tchaton in #633
• fix(cli): lightning-sdk installation process with uv by @bhimrazy in #640
• Fix on_request callback not triggering for API specs by @bhimrazy in #642

Chores

• Bump the gha-updates group with 2 updates by @dependabot[bot] in #626
• [pre-commit.ci] pre-commit suggestions by @pre-commit-ci[bot] in #629
• feat(litServe): Bump version 0.2.17 by @tchaton in #632
• Bump actions/checkout from 5 to 6 in the gha-updates group by @dependabot[bot] in #636
• [pre-commit.ci] pre-commit suggestions by @pre-commit-ci[bot] in #637
• Bump mypy from 1.18.2 to 1.19.0 by @dependabot[bot] in #639

🧑‍💻 Contributors

Thank you ❤️ to all contributors for making LitServe better!

Full Changelog: v0.2.16...v0.2.17