Skip to content

Draft: Async refactor with aiohttp core and sync wrapper#583

Draft
kaya70875 wants to merge 7 commits intojdepoix:masterfrom
kaya70875:feature/async-api
Draft

Draft: Async refactor with aiohttp core and sync wrapper#583
kaya70875 wants to merge 7 commits intojdepoix:masterfrom
kaya70875:feature/async-api

Conversation

@kaya70875
Copy link
Copy Markdown
Contributor

@kaya70875 kaya70875 commented Mar 14, 2026

Overview

This draft PR proposes a major refactor of the YouTube Transcript API library to:

  • Replace requests with aiohttp for asynchronous HTTP handling.
  • Introduce a first-class async API: YouTubeTranscriptAsyncApi.
  • Provide a synchronous wrapper: YouTubeTranscriptApi using asyncio.run() internally for backward compatibility.

The goal is to modernize the library and make it fully async-capable while still supporting users who want synchronous usage.

Key Changes

1. Async core API (YouTubeTranscriptAsyncApi)

  • Uses aiohttp.ClientSession internally.
  • Methods are async: fetch, list, etc.
  • Resource management via async with ensures sessions are always closed properly.
  • Supports passing a custom ClientSession or proxy configuration.

2. Sync wrapper (YouTubeTranscriptApi)

  • Wraps async API via asyncio.run() for fetch and list.
  • No longer depends on requests.
  • Maintains the old sync method signatures.

Note: sync wrapper should not be called from an existing async event loop; use async API in that case.

3. Private async helpers

  • _fetch_async / _list_async for internal use.
  • Encapsulates async with usage cleanly and avoids nested coroutines.

4. Breaking changes

  • requests is removed; any code relying on the old sync-only HTTP client will break.
  • Users running in already running async loops must switch to YouTubeTranscriptAsyncApi.

5. Tests

  • Tests are not fully migrated yet.
  • This draft focuses on architecture and design; test migration to aiohttp / aioresponses will follow after this PR is approved.

Additional Notes on Proxies and Retry Logic

  • The old sync implementation used requests with manual retry logic and mounted HTTPAdapter for handling 429 Too Many Requests responses.

  • In the new async core (aiohttp):

    1. Retries are no longer handled globally. We can implement custom retry logic in the async API if needed, but aiohttp does not require the same adapter pattern.
    2. Proxy configuration is now applied per request using ClientSession.get/post(..., proxy=...) instead of session.proxies.
    3. prevent_keeping_connections_alive is still respected by passing Connection: close headers.

This simplifies the API and reduces internal complexity, but it is a breaking change for any users relying on the old requests retry/mount behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant