Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
feat: add async seeders (AsyncSeeder, AsyncHybridSeeder)
Bridge the existing sync seeders onto AsyncSession via run_sync, so
projects using an async engine can seed without a separate sync engine.
The hybrid seeder's mid-traversal filter queries run through the async
driver inside the greenlet, needing no async reimplementation.

Exports are guarded in __init__ so sync-only installs without greenlet
still import cleanly. Adds an `async` install extra, aiosqlite/greenlet
dev deps, async tests, and README/docs coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
  • Loading branch information
jedymatt and claude committed Jul 4, 2026
commit 9b09f0e651ea5e8fc747568941bc6c20b41188ba
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,35 @@ tests never see each other's rows.
> if you already use those names for something else, your definitions take
> precedence (pytest resolves conftest fixtures over plugin fixtures).

## Async usage

If your application only has an `AsyncSession`, use `AsyncSeeder` and
`AsyncHybridSeeder`. They accept the same entities as their sync counterparts
and run the seeding through `AsyncSession.run_sync`, so `filter`-key queries
execute against your async driver.

Install with the `async` extra (pulls in greenlet); you also need an async
driver such as `aiosqlite` or `asyncpg`:

```shell
pip install "sqlalchemyseed[async]" aiosqlite
```

```python
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemyseed import AsyncSeeder

engine = create_async_engine("sqlite+aiosqlite:///app.db")

async with AsyncSession(engine) as session:
seeder = AsyncSeeder(session)
await seeder.seed(entities)
await session.commit()
```

Use `AsyncHybridSeeder` when the entities contain a `filter` key, exactly as
you would reach for `HybridSeeder` in synchronous code.

## Documentation

<https://sqlalchemyseed.readthedocs.io/>
Expand Down
82 changes: 82 additions & 0 deletions docs/async.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
Async usage
===========

If your application uses an :class:`~sqlalchemy.ext.asyncio.AsyncSession`
instead of a synchronous ``Session``, use the async seeders. ``AsyncSeeder``
and ``AsyncHybridSeeder`` mirror :class:`~sqlalchemyseed.Seeder` and
:class:`~sqlalchemyseed.HybridSeeder`: they accept the same entities and expose
the same ``instances`` property, but their ``seed`` method is awaitable.

Under the hood they run the existing synchronous seeding logic through
:meth:`AsyncSession.run_sync`, so a ``filter`` key still issues a real query --
executed against your async driver -- during the seed traversal.

Installation
------------

Install the ``async`` extra (which pulls in greenlet, required by SQLAlchemy's
asyncio support) together with an async driver such as ``aiosqlite`` or
``asyncpg``:

.. code-block:: shell

pip install "sqlalchemyseed[async]" aiosqlite

AsyncSeeder
-----------

.. code-block:: python

from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemyseed import AsyncSeeder

engine = create_async_engine("sqlite+aiosqlite:///app.db")

data = {
"model": "models.Company",
"data": {
"name": "MyCompany",
"!employees": [
{"data": {"name": "Alice"}},
{"data": {"name": "Bob"}},
],
},
}

async with AsyncSession(engine) as session:
seeder = AsyncSeeder(session, ref_prefix="!")
await seeder.seed(data)
await session.commit()

AsyncHybridSeeder
-----------------

Use ``AsyncHybridSeeder`` when the entities contain a ``filter`` key, just as
you would reach for :class:`~sqlalchemyseed.HybridSeeder` in synchronous code.

.. code-block:: python

from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemyseed import AsyncHybridSeeder

data = {
"model": "models.Employee",
"data": {
"name": "Carol",
"!company": { # references an existing row
"model": "models.Company",
"filter": {"name": "Acme"},
},
},
}

async with AsyncSession(engine) as session:
seeder = AsyncHybridSeeder(session)
await seeder.seed(data)
await session.commit()

.. note::
The async seeders are only importable when SQLAlchemy's asyncio support
(greenlet) is installed. Without it, the rest of ``sqlalchemyseed`` still
imports normally -- only ``AsyncSeeder`` and ``AsyncHybridSeeder`` are
unavailable.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Project Links: `Github`_ | `PyPI`_
intro
seeding
relationships
async
examples
cli
pytest
Expand Down
9 changes: 9 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ dynamic = ["version"]
yaml = [
"PyYAML>=6.0",
]
async = [
# Pulls greenlet, which backs AsyncSession.run_sync. You still need an
# async driver of your own (e.g. aiosqlite, asyncpg).
"SQLAlchemy[asyncio]>=2.0",
]

[project.scripts]
sqlalchemyseed = "sqlalchemyseed.cli:main"
Expand All @@ -45,6 +50,10 @@ dev = [
"pytest>=9.0.3",
"coverage>=6.2",
"PyYAML>=6.0",
# Async seeder tests: aiosqlite provides an async SQLite driver, greenlet
# backs SQLAlchemy's AsyncSession.run_sync bridge.
"aiosqlite>=0.19",
"greenlet>=3.0",
]

[build-system]
Expand Down
7 changes: 7 additions & 0 deletions src/sqlalchemyseed/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@
from . import util
from . import attribute

try:
# Requires SQLAlchemy's async extra (greenlet); optional for sync-only users.
from .aio import AsyncHybridSeeder
from .aio import AsyncSeeder
except ImportError:
pass


__version__ = "2.4.0"

Expand Down
61 changes: 61 additions & 0 deletions src/sqlalchemyseed/aio.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
"""
Async wrappers around the synchronous seeders.

These bridge the existing sync-only seeder logic onto an ``AsyncSession``
using :meth:`AsyncSession.run_sync`, which runs the sync code inside a
greenlet where the driver's blocking I/O is translated into ``await`` calls.
No parallel async reimplementation of the traversal is needed.
"""

from typing import Union

from sqlalchemy.ext.asyncio import AsyncSession

from .seeder import HybridSeeder, Seeder


class AsyncSeeder:
"""Async counterpart of :class:`~sqlalchemyseed.seeder.Seeder`."""

def __init__(self, session: AsyncSession, ref_prefix: str = "!"):
self.session = session
self.ref_prefix = ref_prefix
self._seeder: Seeder = None

async def seed(self, entities: Union[list, dict], add_to_session: bool = True):
def _run(sync_session):
seeder = Seeder(sync_session, ref_prefix=self.ref_prefix)
seeder.seed(entities, add_to_session=add_to_session)
return seeder

self._seeder = await self.session.run_sync(_run)

@property
def instances(self) -> tuple:
return self._seeder.instances if self._seeder is not None else ()


class AsyncHybridSeeder:
"""Async counterpart of :class:`~sqlalchemyseed.seeder.HybridSeeder`.

The hybrid seeder issues queries (``filter`` keys) *during* the seed
traversal, so it genuinely needs a live connection; ``run_sync`` supplies
a real sync ``Session`` whose queries are proxied to the async driver.
"""

def __init__(self, session: AsyncSession, ref_prefix: str = "!"):
self.session = session
self.ref_prefix = ref_prefix
self._seeder: HybridSeeder = None

async def seed(self, entities: Union[list, dict]):
def _run(sync_session):
seeder = HybridSeeder(sync_session, ref_prefix=self.ref_prefix)
seeder.seed(entities)
return seeder

self._seeder = await self.session.run_sync(_run)

@property
def instances(self) -> tuple:
return self._seeder.instances if self._seeder is not None else ()
91 changes: 91 additions & 0 deletions tests/test_async_seeder.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
import unittest

from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine

from sqlalchemyseed.aio import AsyncHybridSeeder, AsyncSeeder
from tests.models import Base, Company, Employee


class AsyncSeederTestCase(unittest.IsolatedAsyncioTestCase):
"""Tests the async wrappers against an in-memory async SQLite engine."""

async def asyncSetUp(self) -> None:
self.engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
self.session = AsyncSession(self.engine)

async def asyncTearDown(self) -> None:
await self.session.close()
await self.engine.dispose()

async def test_seed_nested_relationship(self):
entities = {
"model": "tests.models.Company",
"data": {
"name": "MyCompany",
"!employees": [
{"data": {"name": "Alice"}},
{"data": {"name": "Bob"}},
],
},
}

seeder = AsyncSeeder(self.session)
await seeder.seed(entities)
await self.session.commit()

companies = (await self.session.execute(select(Company))).scalars().all()
employees = (await self.session.execute(select(Employee))).scalars().all()
self.assertEqual(len(companies), 1)
self.assertEqual(companies[0].name, "MyCompany")
self.assertEqual({e.name for e in employees}, {"Alice", "Bob"})
self.assertEqual(len(seeder.instances), 1)

async def test_seed_without_add_to_session(self):
entities = {"model": "tests.models.Company", "data": {"name": "Ghost"}}

seeder = AsyncSeeder(self.session)
await seeder.seed(entities, add_to_session=False)
await self.session.commit()

companies = (await self.session.execute(select(Company))).scalars().all()
self.assertEqual(companies, [])
self.assertEqual(len(seeder.instances), 1)

async def test_hybrid_seed_filter_references_existing_row(self):
"""The 'filter' key runs a query mid-seed, exercising real async
driver I/O through run_sync."""
await AsyncHybridSeeder(self.session).seed(
{"model": "tests.models.Company", "data": {"name": "Acme"}}
)
await self.session.commit()

hybrid = AsyncHybridSeeder(self.session)
await hybrid.seed(
{
"model": "tests.models.Employee",
"data": {
"name": "Carol",
"!company": {"filter": {"name": "Acme"}},
},
}
)
await self.session.commit()

carol = (
await self.session.execute(
select(Employee).where(Employee.name == "Carol")
)
).scalar_one()
acme = (
await self.session.execute(
select(Company).where(Company.name == "Acme")
)
).scalar_one()
self.assertEqual(carol.company_id, acme.id)


if __name__ == "__main__":
unittest.main()
Loading
Loading