Skip to content

Python overview says cursors "cannot run queries at the same time" which contradicts behavior and multithreading guide #7228

Description

@vimota

https://duckdb.org/docs/current/clients/python/overview#about-cursor (section on thread safety)\ says:

A DuckDBPyConnection.cursor() method creates another handle on the same connection. It does not open a new connection. Therefore, all cursors created from one connection cannot run queries at the same time.

This contradicts two other places in the docs:

In practice and in simulation of workloads I've also observed that cursors execute in parallel and have independent session and transaction state.

I was able to verify that 4 cursors complete 4 queries in the time of one, but session variables and registered Python objects are all per cursor which matches cursor() constructing a new Connection (its own ClientContext) on the parent's DatabaseInstance.

Perhaps it should say something like the below?

A DuckDBPyConnection object is not safe for concurrent use: calls on the same handle from multiple threads are serialized. DuckDBPyConnection.cursor() creates a new connection to the same database instance with its own session settings and transaction context. Cursors share the instance and can execute queries in parallel. To run queries concurrently, create one cursor per thread.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions