feat(ai): add Valkey vector store backend

ai/README.md

@@ -39,6 +39,7 @@ The Conductor AI module provides built-in integration with 13 popular LLM provid
 | **PostgreSQL (pgvector)** | ✅ | ✅ | Postgres with vector extension |
 | **Pinecone** | ✅ | ✅ | Managed vector database |
 | **MongoDB Atlas** | ✅ | ✅ | MongoDB vector search |
+| **Valkey** | ✅ | ✅ | In-memory vector search via the valkey-search module |
 
 > **Note**: Multiple named instances of these providers can be configured. See [Vector Database Configuration](VECTORDB_CONFIGURATION.md) for details.
 

ai/VECTORDB_CONFIGURATION.md

@@ -15,6 +15,7 @@ Conductor supports multiple vector database providers with the ability to config
 - **PostgreSQL** (with pgvector extension)
 - **MongoDB** (with Atlas Vector Search)
 - **Pinecone**
+- **Valkey** (with the valkey-search module)
 
 ## Configuration Format
 
@@ -25,7 +26,7 @@ conductor:
   vectordb:
     instances:
       - name: "instance-name"        # Unique identifier for this instance
-        type: "database-type"        # Type: postgres, mongodb, or pinecone
+        type: "database-type"        # Type: postgres, mongodb, pinecone, or valkey
         <type-specific-config>:      # Configuration block for the database type
           # ... type-specific properties
 ```
@@ -101,6 +102,53 @@ conductor:
           apiKey: "your-pinecone-api-key"
 ```
 
+### Valkey (valkey-search)
+
+Requires a Valkey server with the [valkey-search](https://github.com/valkey-io/valkey-search)
+module loaded, which provides the `FT.CREATE` and `FT.SEARCH` commands. The
+`valkey/valkey-bundle` image ships the module; the plain `valkey` image does not.
+
+```yaml
+conductor:
+  vectordb:
+    instances:
+      - name: "valkey-embeddings"
+        type: "valkey"
+        valkey:
+          host: "localhost"
+          port: 6379
+          password: "${VALKEY_PASSWORD}"   # Resolve from the environment, never inline
+          database: 0
+          useTls: false
+          dimensions: 1536
+          distanceMetric: "cosine"         # Options: cosine, l2, ip
+          indexingMethod: "hnsw"           # Options: hnsw, flat
+          keyPrefix: "conductor"
+          requestTimeoutMs: 2000       # Also bounds FT.SEARCH/KNN latency
+```
+
+**Limitations of this release:**
+
+- Standalone mode only. Valkey Cluster is not supported yet, because a clustered
+  deployment requires additional key-slot design.
+- Not compatible with managed services that do not ship the search module. Verify
+  module availability with your provider before choosing this backend.
+
+**Score semantics:** `score` on returned documents is a Valkey Search distance, so
+**lower is closer** for `cosine`, `l2`, and `ip`. For cosine, an exact match scores `0.0`
+and an orthogonal vector scores `1.0`. For inner product, Valkey Search uses `1 - dot(X,Y)`.
+This matches the PostgreSQL backend. MongoDB and Pinecone instead return a similarity
+where higher is closer, so do not compare score values across backends.
+
+**Search timeout:** `requestTimeoutMs` is the timeout for every Valkey command, including
+`FT.SEARCH` KNN queries. Increase it for large indexes or higher `EF_RUNTIME` settings if
+searches exceed the configured deadline.
+
+**Keys and indexes:** documents are stored as hashes under
+`<keyPrefix>:<indexName>:<namespace>:<docId>`, and each `(indexName, namespace)` pair
+gets its own search index named `<keyPrefix>:<indexName>:<namespace>`. Two namespaces
+sharing an index name therefore stay isolated.
+
 ### Mixed Configuration (Multiple Types)
 
 ```yaml
@@ -177,6 +225,22 @@ When using vector database tasks in your workflows, reference the instance by it
 |----------|------|---------|-------------|
 | `apiKey` | String | Required | Pinecone API key |
 
+## Valkey Configuration Options
+
+| Property | Type | Default | Description |
+|----------|------|---------|-------------|
+| `host` | String | "localhost" | Valkey server hostname. Must not be blank |
+| `port` | Integer | 6379 | Valkey server port. Must be 1-65535 |
+| `username` | String | null | Username for ACL authentication |
+| `password` | String | null | Password. Source from the environment, not the config file |
+| `database` | Integer | 0 | Logical database index. Must be >= 0 |
+| `useTls` | Boolean | false | Enable TLS with full certificate and hostname verification |
+| `dimensions` | Integer | 256 | Vector dimensions. Must be positive and match your embedding model |
+| `distanceMetric` | String | "cosine" | Distance metric (cosine, l2, ip). Unknown values are rejected at startup |
+| `indexingMethod` | String | "hnsw" | Index algorithm (hnsw or flat). Unknown values are rejected at startup |
+| `keyPrefix` | String | "conductor" | Root segment for keys and index names. Trailing colons are stripped; must match `[a-zA-Z0-9_.-]+` |
+| `requestTimeoutMs` | Integer | 2000 | Per-command timeout in milliseconds, including FT.SEARCH/KNN. Must be positive |
+
 ## Migration from Old Configuration
 
 ### Old Format (Single Instance Per Type)
@@ -257,3 +321,18 @@ If you see an error like "Vector DB instance not found: xyz", check:
 - Verify API key is valid and has necessary permissions
 - Ensure index exists in your Pinecone account before using it
 
+### Valkey
+
+- `unknown command 'FT.CREATE'` means the server is running without the
+  `valkey-search` module. Load the module, or use the `valkey/valkey-bundle` image
+  instead of the plain `valkey` image.
+- Searches returning no results after successful writes usually means `dimensions`
+  does not match the vector length your embedding model produces. Compare the
+  configured value against `FT.INFO <keyPrefix>:<indexName>:<namespace>` and check
+  whether its `hash_indexing_failures` counter is climbing.
+- `dimensions` is fixed when the index is created and there is no ALTER equivalent.
+  Changing it requires `FT.DROPINDEX` on the affected index followed by re-indexing.
+- When reusing an existing index, its vector dimensions are validated with `FT.INFO` and
+  must match the configured `dimensions` value.
+- Remember that `score` is a distance for all supported metrics, so ascending order is
+  closest-first. Sorting descending will return the least relevant documents.

ai/build.gradle

@@ -43,6 +43,11 @@ dependencies {
 
     //Vector Databases
 
+    // Valkey GLIDE client for the Valkey vector store provider.
+    // The uber jar bundles Rust natives for 7 platforms (~40 MB). Upgrades require re-auditing
+    // native code because Java-layer patches cannot reach the bundled binaries.
+    api "io.valkey:valkey-glide:${revValkeyGlide}"
+
     api "org.mongodb:mongodb-driver-sync:${mongodb}"
     api "org.mongodb:mongodb-driver-core:${mongodb}"
     api "org.mongodb:bson:${mongodb}"
@@ -56,6 +61,9 @@ dependencies {
     testImplementation "com.squareup.okhttp3:mockwebserver:4.12.0"
     testImplementation "org.testcontainers:mongodb:${revTestContainer}"
     testImplementation "org.testcontainers:postgresql:${revTestContainer}"
+    // Core Testcontainers is used directly by ValkeyVectorDBRoundTripTest via GenericContainer.
+    // Declared explicitly rather than relied upon transitively through the modules above.
+    testImplementation "org.testcontainers:testcontainers:${revTestContainer}"
     testImplementation "org.postgresql:postgresql:${revPostgres}"
     testImplementation "org.apache.commons:commons-compress:${revCommonsCompress}"
     testImplementation "com.h2database:h2:2.4.240"

ai/src/main/java/org/conductoross/conductor/ai/vectordb/VectorDBInstanceConfig.java

@@ -12,13 +12,18 @@
  */
 package org.conductoross.conductor.ai.vectordb;
 
+import java.io.Closeable;
+import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
+import java.util.Locale;
 import java.util.Map;
 
 import org.conductoross.conductor.ai.vectordb.mongodb.MongoDBConfig;
 import org.conductoross.conductor.ai.vectordb.pinecone.PineconeConfig;
 import org.conductoross.conductor.ai.vectordb.postgres.PostgresConfig;
+import org.conductoross.conductor.ai.vectordb.valkey.ValkeyConfig;
+import org.conductoross.conductor.ai.vectordb.valkey.ValkeyConnectionException;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.stereotype.Component;
 
@@ -77,6 +82,8 @@ public Map<String, VectorDB> getVectorDBInstances() {
             return vectorDBMap;
         }
 
+        List<String> failedNames = new ArrayList<>();
+        List<Exception> failures = new ArrayList<>();
         for (VectorDBInstance instance : instances) {
             try {
                 VectorDB vectorDB = createVectorDB(instance);
@@ -87,7 +94,19 @@ public Map<String, VectorDB> getVectorDBInstances() {
                             instance.getName(),
                             instance.getType());
                 }
-            } catch (Exception e) {
+            } catch (IllegalArgumentException | IllegalStateException e) {
+                // The configuration itself is invalid (bad enum value, blank name, etc.) and needs
+                // an operator fix, not a retry, so fail startup loudly.
+                failedNames.add(instance.getName() + " (type: " + instance.getType() + ")");
+                failures.add(e);
+            } catch (ValkeyConnectionException e) {
+                // A connectivity failure (e.g. Valkey unreachable at boot), not a configuration
+                // error. Unlike the other vector DB backends, ValkeyVectorDB connects eagerly at
+                // construction, so without this distinction a transient network blip would abort
+                // the entire server instead of just this instance. Log and skip, matching how the
+                // other backends already tolerate this. Deliberately narrower than catching
+                // RuntimeException: a genuine bug (NPE, ClassCastException, etc.) from any backend
+                // must still propagate and fail loudly rather than being silently dropped here.
                 log.error(
                         "Failed to initialize vector DB instance: {} (type: {}), reason: {}",
                         instance.getName(),
@@ -96,9 +115,40 @@ public Map<String, VectorDB> getVectorDBInstances() {
             }
         }
 
+        if (!failures.isEmpty()) {
+            closeAlreadyCreated(vectorDBMap);
+            IllegalStateException aggregate =
+                    new IllegalStateException(
+                            "Failed to initialize vector DB instance(s): "
+                                    + String.join(", ", failedNames));
+            failures.forEach(aggregate::addSuppressed);
+            throw aggregate;
+        }
+
         return vectorDBMap;
     }
 
+    /**
+     * Closes any instances that were successfully created before a later instance failed. Without
+     * this, a successfully-connected instance (e.g. an open GLIDE client) would be discarded along
+     * with the map when the aggregate exception below is thrown, since {@link VectorDBProvider} is
+     * never constructed to close it later.
+     */
+    private void closeAlreadyCreated(Map<String, VectorDB> vectorDBMap) {
+        for (Map.Entry<String, VectorDB> entry : vectorDBMap.entrySet()) {
+            if (entry.getValue() instanceof Closeable) {
+                try {
+                    ((Closeable) entry.getValue()).close();
+                } catch (Exception e) {
+                    log.warn(
+                            "Failed to close vector DB instance '{}' during startup failure cleanup",
+                            entry.getKey(),
+                            e);
+                }
+            }
+        }
+    }
+
     /** Creates a VectorDB instance based on the configuration type. */
     private VectorDB createVectorDB(VectorDBInstance instance) {
         String type = instance.getType();
@@ -107,7 +157,7 @@ private VectorDB createVectorDB(VectorDBInstance instance) {
             return null;
         }
 
-        switch (type.toLowerCase()) {
+        switch (type.toLowerCase(Locale.ROOT)) {
             case "postgres":
             case "pgvectordb":
                 return createPostgresVectorDB(instance);
@@ -117,6 +167,9 @@ private VectorDB createVectorDB(VectorDBInstance instance) {
             case "pinecone":
             case "pineconedb":
                 return createPineconeVectorDB(instance);
+            case "valkey":
+            case "valkeyvectordb":
+                return createValkeyVectorDB(instance);
             default:
                 log.error("Unknown vector DB type: {} for instance: {}", type, instance.getName());
                 return null;
@@ -150,13 +203,23 @@ private VectorDB createPineconeVectorDB(VectorDBInstance instance) {
         return config.get(instance.getName());
     }
 
+    private VectorDB createValkeyVectorDB(VectorDBInstance instance) {
+        ValkeyConfig config = instance.getValkey();
+        if (config == null) {
+            log.error("Valkey configuration missing for instance: {}", instance.getName());
+            return null;
+        }
+        return config.get(instance.getName());
+    }
+
     /** Represents a single vector DB instance configuration. */
     public static class VectorDBInstance {
         private String name;
         private String type;
         private PostgresConfig postgres;
         private MongoDBConfig mongodb;
         private PineconeConfig pinecone;
+        private ValkeyConfig valkey;
 
         public String getName() {
             return name;
@@ -197,5 +260,13 @@ public PineconeConfig getPinecone() {
         public void setPinecone(PineconeConfig pinecone) {
             this.pinecone = pinecone;
         }
+
+        public ValkeyConfig getValkey() {
+            return valkey;
+        }
+
+        public void setValkey(ValkeyConfig valkey) {
+            this.valkey = valkey;
+        }
     }
 }