Problem
When trying to store text embeddings using OpenAI's text-embedding-3-large into CrateDB, LangChain trips.
OpenAIEmbeddings(model="text-embedding-3-large")
sqlalchemy.exc.ProgrammingError: (crate.client.exceptions.ProgrammingError)
SQLParseException[Field [6] vector's dimensions must be <= [2048]; got 3072]
[SQL: INSERT INTO langchain_embedding (id, collection_id, embedding, document, cmetadata) VALUES (?, ?, ?, ?, ?) ON CONFLICT (id) DO UPDATE SET embedding = excluded.embedding, document = excluded.document, cmetadata = excluded.cmetadata]
Details
Processing the SQL DDL statement using FLOAT_VECTOR(3072) succeeds.
CREATE TABLE langchain_embedding (
id STRING NOT NULL,
collection_id STRING,
embedding FLOAT_VECTOR(3072),
document STRING,
cmetadata OBJECT,
PRIMARY KEY (id)
)
The error happens when processing the INSERT statement.
INSERT INTO langchain_embedding (id, collection_id, embedding, document, cmetadata)
VALUES (?, ?, ?, ?, ?) ON CONFLICT (id) DO
UPDATE
SET embedding = excluded.embedding,
document = excluded.document,
cmetadata = excluded.cmetadata
This is the engine's FLOAT_VECTOR dimension cap surfacing at insert time — the DDL accepts 3072, the write path enforces 2048 — and it hits the defaults of current models: OpenAI's text-embedding-3-large and Gemini embeddings both produce 3072 dimensions out of the box, so this is many users' first-run experience. On the integration side: fail at construction with a clear message instead of deep inside an INSERT, and pass through the dimensions parameter both providers support, so users can fit under the cap deliberately. The cap itself needs a core-side conversation.
Problem
When trying to store text embeddings using OpenAI's
text-embedding-3-largeinto CrateDB, LangChain trips.Details
Processing the SQL DDL statement using
FLOAT_VECTOR(3072)succeeds.The error happens when processing the
INSERTstatement.This is the engine's
FLOAT_VECTORdimension cap surfacing at insert time — the DDL accepts 3072, the write path enforces 2048 — and it hits the defaults of current models: OpenAI'stext-embedding-3-largeand Gemini embeddings both produce 3072 dimensions out of the box, so this is many users' first-run experience. On the integration side: fail at construction with a clear message instead of deep inside an INSERT, and pass through thedimensionsparameter both providers support, so users can fit under the cap deliberately. The cap itself needs a core-side conversation.