Merge branch 'release-1.12.0' into fix/background-job-timeout-ignored

Author: thesaadmirza

.secrets.baseline

@@ -191,6 +191,7 @@ Use this mode to previews the changes that would be made to the database schema
     :::warning
     `langflow migration --fix` is a destructive operation that can delete data.
     Always run `langflow migration` first to preview the changes.
+    For more information, see [Database migrations](./database-migrations).
     :::
 
 <Tabs groupId="Invocation">

docs/docs/Develop/configuration-cli.mdx

@@ -202,6 +202,7 @@ Since each Langflow instance runs in its own container on the Docker network, us
 
 ## See also
 
+* [Database migrations](./database-migrations)
 * [Langflow database guide for enterprise DBAs](/enterprise-database-guide)
 * [Memory management options](/memory)
 * [Logs](/logging)

docs/docs/Develop/configuration-custom-database.mdx

@@ -0,0 +1,95 @@
+---
+title: Database migrations
+slug: /database-migrations
+---
+
+Langflow stores application data such as flows, users, and API keys in a SQL database.
+
+When you upgrade Langflow or connect to a new empty database, Langflow must keep the database schema aligned with the application models.
+
+By default, the Langflow database uses [SQLite](https://www.sqlite.org/docs.html), and uses [Alembic](https://alembic.sqlalchemy.org/) for SQL database migrations.
+
+This page explains when migrations run and how to run and repair migrations safely.
+
+## Default migrations behavior and triggers
+
+Most Langflow users never need to run a database migration.
+When you start Langflow, or restart a container after upgrading the Langflow image, pending Alembic migrations run as part of database initialization.
+
+Langflow includes [Alembic revision scripts](https://github.com/langflow-ai/langflow/tree/main/src/backend/base/langflow/alembic) with the package.
+On startup, Langflow connects to the database at [`LANGFLOW_DATABASE_URL`](/memory#configure-external-memory) or the default `langflow.db` file, ensures the required tables exist, checks the database schema against the current models, and then applies any pending Alembic upgrades to bring the schema to its `head` revision.
+
+Alembic records the applied revision in the `alembic_version` table.
+
+Other events that trigger Alembic checks are a first Langflow startup against a new SQLite file or empty PostgreSQL database, upgrading to a Langflow version that includes new Alembic revisions, or pointing [`LANGFLOW_DATABASE_URL`](/memory#configure-external-memory) at an empty database.
+
+## Upgrade Langflow safely
+
+To upgrade Langflow or change the database URL in a deployment that already has data, you may need a database migration.
+To migrate your database, do the following:
+
+1. Stop Langflow.
+2. Back up the database.
+
+    If you're using SQLite, copy the `langflow.db` file.
+    The default path depends on your install method and whether `LANGFLOW_SAVE_DB_IN_CONFIG_DIR` is set.
+    For more information, see [Memory management options](/memory#storage-options-and-paths).
+
+    If you're using PostgreSQL, make a logical backup with `pg_dump`, or use your database provider's snapshot.
+    For more information, see [enterprise backup guidance](/enterprise-database-guide).
+
+3. Export any flows you want to save as JSON. For more information, see [Import and export flows](/concepts-flows-import).
+4. Upgrade the Langflow package or container image to the latest version, keeping the same [`LANGFLOW_DATABASE_URL`](/memory#configure-external-memory) or the same SQLite path.
+
+    ```bash
+    uv pip install langflow -U
+    ```
+
+5. Start a **single** Langflow instance, so database migrations run without concurrent database writes.
+6. Confirm that startup succeeds.
+7. If startup reports a schema mismatch, run `langflow migration` to inspect the problem and restore from backup if needed.
+   For more information, see [`langflow migration` CLI](/configuration-cli#langflow-migration).
+
+## Switch from SQLite to PostgreSQL
+
+Changing `LANGFLOW_DATABASE_URL` from SQLite to PostgreSQL does **not** copy existing SQLite data into PostgreSQL.
+Langflow initializes and migrates the new database schema, and your previous SQLite file remains where it was.
+
+There is no built-in SQLite-to-PostgreSQL data migrator.
+Langflow creates a new PostgreSQL schema, and then you can restore application data you saved.
+
+To migrate your flows while switching from SQLite to PostgreSQL, do the following:
+
+1. [Export flows](/concepts-flows-import), and note any [global variables](/configuration-global-variables) or credentials.
+2. Provision PostgreSQL 15+ and set `LANGFLOW_DATABASE_URL` as described in [Configure an external PostgreSQL database](/configuration-custom-database).
+3. If required, install PostgreSQL driver extras, for example `uv pip install "langflow[postgresql]"`.
+4. Start Langflow. The new PostgreSQL schema is created.
+5. Import the exported flows and recreate secrets or variables as needed.
+
+## Migration logging
+
+Alembic output is written to a log file by default at `alembic/alembic.log` under the Langflow config directory (`LANGFLOW_CONFIG_DIR`).
+Relative `LANGFLOW_ALEMBIC_LOG_FILE` values are also resolved against that config directory.
+In read-only or hardened containers, that path may not be writable, and Langflow will fall back to `stdout` and continue the migration.
+
+## Migration environment variables
+
+| Variable | Purpose |
+|----------|---------|
+| `LANGFLOW_ALEMBIC_LOG_TO_STDOUT` | Set to `true` to send Alembic migration output to `stdout`. |
+| `LANGFLOW_ALEMBIC_LOG_FILE` | Absolute or relative path for the Alembic log file when not logging to `stdout`. Relative paths resolve under the Langflow config directory. |
+| `LANGFLOW_MIGRATION_LOCK_TIMEOUT_S` | Seconds to wait for the PostgreSQL migration advisory lock. Default: `300`. |
+| `LANGFLOW_MIGRATION_LOCK_NAMESPACE` | Optional namespace for the Alembic PostgreSQL advisory lock. The default lock key coordinates instances that share a database. Set this variable to isolate different Langflow deployments that share one PostgreSQL server. Use the same value on every instance in that deployment. |
+
+## Troubleshooting
+
+For troubleshooting database migrations, see [Langflow upgrade issues](/troubleshoot#langflow-upgrade-issues).
+
+## See also
+
+* [Configure an external PostgreSQL database](/configuration-custom-database)
+* [Langflow database guide for enterprise DBAs](/enterprise-database-guide)
+* [Memory management options](/memory)
+* [`langflow migration` CLI](/configuration-cli#langflow-migration)
+* [Troubleshoot Langflow](/troubleshoot)
+* [Deploy Langflow on Docker](/deployment-docker)

docs/docs/Develop/database-migrations.mdx

@@ -119,7 +119,7 @@ For more information, see [Configure an external PostgreSQL database](/configura
     3. Run `langflow migration --fix` to run the migration and permanently apply the changes.
 
         This is a destructive operation that can delete data.
-        For more information, see [`langflow migration`](/configuration-cli#langflow-migration).
+        For more information, see [Database migrations](./database-migrations).
 
 6. To verify the configuration, create any flow using the Langflow visual editor or API, and then query your database to confirm the tables and activity are recorded there. The content of the flow doesn't matter; you only need to confirm that the flow is stored in your PostgreSQL database.
 You can query the database in two ways:
@@ -287,5 +287,6 @@ For example, use `pg_stat_activity` to monitor connection counts and contention.
 ## See also
 
 * [Configure an external PostgreSQL database](/configuration-custom-database)
+* [Database migrations](./database-migrations)
 * [Langflow architecture on Kubernetes](/deployment-architecture)
 * [Deploy the Langflow production environment on Kubernetes](/deployment-kubernetes-prod)

docs/docs/Develop/enterprise-database-guide.mdx

@@ -206,6 +206,8 @@ For more information and examples, see [**Message History** component](/message-
 
 ## See also
 
+* [Database migrations](./database-migrations)
+* [Configure an external PostgreSQL database](/configuration-custom-database)
 * [Langflow file management](/concepts-file-management)
 * [Langflow logs](/logging)
 * [Langflow environment variables](/environment-variables)

docs/docs/Develop/memory.mdx

@@ -498,12 +498,19 @@ The following error can occur during Langflow upgrades when the database schema
 There's a mismatch between the models and the database.
 ```
 
-To resolve this error, run the migration fix command:
+Back up your database first, and then run the migration fix command:
 
 ```bash
 uv run langflow migration --fix
 ```
 
+:::warning
+`langflow migration --fix` is destructive.
+Use a dedicated database for Langflow application data.
+Repair mode repeatedly downgrades and re-applies Langflow's schema revisions, which can delete Langflow data, and it might fail to converge when non-Langflow tables such as `langchain_pg_*` vector store tables share the same database.
+For more information, see [Database migrations](./database-migrations).
+:::
+
 If the migration fix doesn't resolve the issue, clear the cache by deleting the contents of your Langflow cache folder.
 The filepath depends on your operating system, installation type, and configuration options.
 For more information and default filepaths, see [Memory management options](/memory).

docs/docs/Support/troubleshooting.mdx

@@ -154,6 +154,11 @@ module.exports = {
               label: "Use Session IDs"
             },
             "Develop/configuration-custom-database",
+            {
+              type: "doc",
+              id: "Develop/database-migrations",
+              label: "Database migrations"
+            },
             {
               type: "doc",
               id: "Develop/enterprise-database-guide",

docs/sidebars.js

@@ -37,7 +37,7 @@
     get_enabled_providers_for_user,
     list_installed_tool_calling_models,
 )
-from langflow.api.utils.core import CurrentActiveUser, DbSession
+from langflow.api.utils.core import CurrentActiveUser, DbSession, release_db_transaction
 
 router = APIRouter(prefix="/agentic", tags=["Agentic"])
 
@@ -175,6 +175,10 @@ async def execute_named_flow(
     """
     ctx = await _resolve_assistant_context(request, current_user.id, session)
 
+    # The flow run below can wait on a model for minutes; don't hold the
+    # request transaction (and its pooled connection) open across it (#14445).
+    await release_db_transaction(session)
+
     global_vars = dict(ctx.global_vars)
     if request.component_id:
         global_vars["COMPONENT_ID"] = request.component_id
@@ -294,6 +298,10 @@ async def assist(
 
     logger.info(f"Executing {LANGFLOW_ASSISTANT_FLOW} with {ctx.provider}/{ctx.model_name}")
 
+    # The assistant run below can wait on a model for minutes; don't hold the
+    # request transaction (and its pooled connection) open across it (#14445).
+    await release_db_transaction(session)
+
     return await execute_flow_with_validation(
         flow_filename=LANGFLOW_ASSISTANT_FLOW,
         input_value=request.input_value or "",
@@ -318,6 +326,11 @@ async def assist_stream(
     await _validate_flow_access(request.flow_id, current_user.id, session)
     ctx = await _resolve_assistant_context(request, current_user.id, session)
 
+    # Dependency teardown only runs after the SSE stream finishes, so without
+    # this commit the request transaction (and its pooled connection) would
+    # stay open for the assistant's whole streaming run (#14445).
+    await release_db_transaction(session)
+
     return StreamingResponse(
         execute_flow_with_validation_streaming(
             flow_filename=LANGFLOW_ASSISTANT_FLOW,

src/backend/base/langflow/agentic/api/router.py

@@ -21,11 +21,12 @@
 from langflow.agentic.api.schemas import AssistantRequest
 from langflow.agentic.services.assistant_service import execute_flow_with_validation_streaming
 from langflow.agentic.services.flow_types import LANGFLOW_ASSISTANT_FLOW
-from langflow.api.v1.flows import _new_flow, _save_flow_to_fs
+from langflow.api.utils.core import release_db_transaction
+from langflow.api.v1.flows import _new_flow, _save_flow_to_fs, _validate_catalog_policy_for_write
 from langflow.initial_setup.setup import get_or_create_default_folder
 from langflow.services.database.models.flow.guards import ensure_flow_unlocked, lock_flow_for_update
 from langflow.services.database.models.flow.model import Flow, FlowCreate
-from langflow.services.deps import get_storage_service
+from langflow.services.deps import get_catalog_policy_service, get_storage_service
 
 if TYPE_CHECKING:
     from sqlmodel.ext.asyncio.session import AsyncSession
@@ -156,7 +157,8 @@ async def _consume_stream(
                 data = event.get("data") or {}
                 result_text = data.get("result")
                 working = get_working_flow()
-                if isinstance(working, dict) and working.get("data", {}).get("nodes"):
+                working_data = working.get("data") if isinstance(working, dict) else None
+                if isinstance(working_data, dict) and isinstance(working_data.get("nodes"), list):
                     working_snapshot = copy.deepcopy(working)
             elif event_type == "error":
                 error_text = event.get("message")
@@ -195,6 +197,12 @@ async def run_assistant_and_persist(
     )
     ctx = await _resolve_assistant_context(request, user_id, session)
 
+    # The agent loop below can run for minutes; end the read transaction now
+    # so it doesn't pin a pooled connection (Postgres: idle-in-transaction)
+    # for the whole run (#14445). Persistence re-reads under a fresh short
+    # transaction (the FOR UPDATE below), so the lock ordering is preserved.
+    await release_db_transaction(session)
+
     stream = execute_flow_with_validation_streaming(
         flow_filename=LANGFLOW_ASSISTANT_FLOW,
         input_value=instruction,
@@ -220,6 +228,19 @@ async def run_assistant_and_persist(
         # each to the working flow here or the text edit is dropped (Bug #13641).
         for edit in field_edits:
             _apply_field_edit(flow_data, edit)
+        try:
+            _validate_catalog_policy_for_write(
+                flow_data,
+                snapshot=get_catalog_policy_service().snapshot,
+            )
+        except HTTPException:
+            if created_new:
+                # The assistant needs a committed flow id while it runs. Do not
+                # leave that provisional row behind when its generated graph is
+                # rejected before the caller ever receives the id or link.
+                await session.delete(flow)
+                await session.commit()
+            raise
         flow.data = flow_data
         if created_new and canvas.name:
             flow.name = canvas.name