Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

### Added

- **`project search` gains an `asset_class` filter.** Takes a bare string (`"Blueprint"`) or an array (`["Blueprint","WidgetBlueprint"]`), matches case-insensitively, de-duplicates, and is capped at 32 entries. Searching a common token in a project with a large third-party content folder previously buried the target: a real `E_` search over this project returned 16 `UserDefinedEnum` rows under 21 `Texture2D`, 10 `NiagaraEmitter` and 3 `StaticMesh` — the data needed to filter was already on every result row, but there was no way to ask for it.

The predicate is applied **inside the SQL**, not as a pass over the returned array. `LIMIT` is applied by the query, so a post-hoc filter would return fewer rows than the caller asked for — frequently zero — while matching assets sat below the cut; `limit` now counts *matching* rows. Both FTS statements already `JOIN assets`, so no extra join was needed and a graph-node text hit inside a Blueprint still survives a `Blueprint` filter. Only the placeholder count is interpolated into the SQL; every class name stays bound.

The parameter is type-checked against `EJson` rather than read through `TryGetStringField`, which coerces — a numeric `7` would otherwise arrive as the string `"7"` and become a filter for a class of that name, i.e. zero results presented as a valid search instead of a `-32602`. Whitespace-only input is likewise a caller error rather than a silent widening to unfiltered. A class no asset uses is a successful empty result.

Mirrored into `monolith_query.exe` and `monolith_offline.py`, whose search path `verify_offline_parity.py` does not gate (`SPEC_MonolithIndex` asks for these three to be kept in step by hand). `Args::options` in the C++ tool is single-valued, so both CLIs take the list comma-separated; the Python tool additionally accepts a repeated `--asset-class`.

Two automation tests: `Monolith.ProjectSearch.AssetClassFilter` locks the in-SQL behaviour with a deliberately lopsided fixture (10 noise assets, 3 wanted) that a post-hoc implementation cannot pass by luck, and covers case-insensitivity, multi-class union and unknown classes; `Monolith.ProjectSearch.AssetClassValidation` covers the `-32602` paths.

## [0.22.0] - 2026-08-01

### Internal
Expand Down
31 changes: 30 additions & 1 deletion Docs/specs/SPEC_MonolithIndex.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@

| Action | Params | Description |
|--------|--------|-------------|
| `search` | `query` (required, max 4096 chars), `limit` (50, clamped 1-1000) | FTS5 full-text search over the `fts_assets` and `fts_nodes` columns. **Not** variables or parameters — those are structured rows, reachable via `get_asset_details`. See **Search Error Contract** below |
| `search` | `query` (required, max 4096 chars), `limit` (50, clamped 1-1000), `asset_class` (string or array, max 32, case-insensitive) | FTS5 full-text search over the `fts_assets` and `fts_nodes` columns. **Not** variables or parameters — those are structured rows, reachable via `get_asset_details`. See **Search Error Contract** and **Asset-Class Filtering** below |
| `find_references` | `asset_path` (required) | Bidirectional dependency lookup |
| `find_by_type` | `asset_type` (required), `limit` (100), `offset` (0) | Filter assets by class with pagination |
| `get_stats` | none | Row counts for all 13 tables + asset class breakdown (top 20) + `skipped_assets` / `skipped_asset_paths` (assets the poison-pill rule dropped from deep indexing — see **Full-Index Resume**) |
Expand Down Expand Up @@ -131,6 +131,35 @@ classification and these limits. Note `verify_offline_parity.py` has **no
`project.*` cases**, so nothing gates that mirroring — keep the three in step by
hand when editing any of them.

### Asset-Class Filtering

`asset_class` restricts results to one or more asset classes. It takes a bare
string (`"Blueprint"`) or an array (`["Blueprint","WidgetBlueprint"]`), matches
**case-insensitively** (`COLLATE NOCASE`, so `"blueprint"` works), de-duplicates,
and is capped at **32 entries** — each becomes one bound placeholder in an `IN`
clause, bounding the statement a single call can ask SQLite to prepare.

Three properties are load-bearing:

- **The predicate lives in the SQL, not in a post-pass over the results.** `LIMIT`
is applied by the query, so filtering the returned array would yield fewer rows
than the caller asked for — frequently zero — while matching assets sat below
the cut. `limit` counts *matching* rows. `Monolith.ProjectSearch.AssetClassFilter`
locks this with a deliberately lopsided fixture (10 noise assets, 3 wanted) that
a post-hoc implementation cannot pass by luck.
- **Node hits are filtered by their owning asset's class.** Both statements already
`JOIN assets`, so a graph-node text match inside a Blueprint still survives a
`Blueprint` filter.
- **The parameter is type-checked against `EJson`, not read through
`TryGetStringField`.** That accessor coerces, so a numeric `7` would arrive as
the string `"7"` and become a filter for a class of that name — zero results
presented as a valid search rather than a `-32602`. Whitespace-only input is
likewise a caller error, not a request to widen the search.

Absent `asset_class` is unfiltered, which is the pre-0.22 behaviour. A class no
asset uses is a successful empty result, not an error — it is a legitimate
question with a legitimate answer.

### Incremental Indexing

The project indexer uses a 3-layer architecture to keep `ProjectIndex.db` in sync without costly full rebuilds:
Expand Down
39 changes: 34 additions & 5 deletions Scripts/monolith_offline.py
Original file line number Diff line number Diff line change
Expand Up @@ -1513,6 +1513,8 @@ def generate_class_stub(self, args):
# Upper bound on a project search query, mirroring
# MonolithProjectSearchActionDetail::MaxQueryLength.
PROJECT_SEARCH_MAX_QUERY_LENGTH = 4096
# Each entry becomes one bound placeholder in an IN clause; mirrors the live cap.
PROJECT_SEARCH_MAX_CLASS_FILTERS = 32

# Diagnostics emitted by the FTS5 MATCH expression parser rather than by
# storage/schema access. Mirrors MonolithProjectSearchDetail::IsFts5QuerySyntaxError.
Expand Down Expand Up @@ -1581,14 +1583,39 @@ def search(self, args):
limit = max(1, min(1000, args.limit))
sqlite3 = self._sqlite3

# Mirrors the live `asset_class` filter (SPEC_MonolithIndex "Asset-Class
# Filtering"). Nothing gates this parity, so it is kept in step by hand.
class_filter = []
# Comma-separated within each occurrence, so `--asset-class A,B` and a
# repeated `--asset-class` both work. The C++ tool's option map is
# single-valued and only accepts the comma form, so this keeps the two
# tools accepting the same syntax.
for occurrence in (getattr(args, "asset_class", None) or []):
for entry in (occurrence or "").split(","):
entry = entry.strip()
if entry and entry not in class_filter:
class_filter.append(entry)
if len(class_filter) > PROJECT_SEARCH_MAX_CLASS_FILTERS:
emit_error(
f"'asset_class' must list {PROJECT_SEARCH_MAX_CLASS_FILTERS} classes or fewer "
f"(got {len(class_filter)})"
)
return

# Only the placeholder count is interpolated; the names stay bound.
class_predicate = ""
if class_filter:
placeholders = ",".join("?" for _ in class_filter)
class_predicate = f" AND a.asset_class COLLATE NOCASE IN ({placeholders})"

results = []
# Per-table verdicts: None = completed, or the unknown-column message.
not_applicable = []

def run_search(sql, table_name):
"""Returns True to continue, False once a failure has been emitted."""
try:
rows = self.db.execute(sql, (query, limit)).fetchall()
rows = self.db.execute(sql, (query, *class_filter, limit)).fetchall()
except sqlite3.DatabaseError as exc:
# DatabaseError (not just OperationalError) so DatabaseCorruptError
# is reported as a structured failure instead of a traceback.
Expand Down Expand Up @@ -1616,20 +1643,20 @@ def run_search(sql, table_name):
return True

if not run_search(
"""SELECT a.package_path, a.asset_name, a.asset_class, a.module_name,
f"""SELECT a.package_path, a.asset_name, a.asset_class, a.module_name,
snippet(fts_assets, 2, '>>>', '<<<', '...', 32) as ctx, rank
FROM fts_assets f JOIN assets a ON a.id = f.rowid
WHERE fts_assets MATCH ? ORDER BY rank LIMIT ?""",
WHERE fts_assets MATCH ?{class_predicate} ORDER BY rank LIMIT ?""",
"assets",
):
return

if not run_search(
"""SELECT a.package_path, a.asset_name, a.asset_class, a.module_name,
f"""SELECT a.package_path, a.asset_name, a.asset_class, a.module_name,
snippet(fts_nodes, 0, '>>>', '<<<', '...', 32) as ctx, f.rank
FROM fts_nodes f JOIN nodes n ON n.id = f.rowid
JOIN assets a ON a.id = n.asset_id
WHERE fts_nodes MATCH ? ORDER BY f.rank LIMIT ?""",
WHERE fts_nodes MATCH ?{class_predicate} ORDER BY f.rank LIMIT ?""",
"nodes",
):
return
Expand Down Expand Up @@ -2661,6 +2688,8 @@ def build_parser():
p = prj_sub.add_parser("search")
p.add_argument("query")
p.add_argument("--limit", type=int, default=50)
p.add_argument("--asset-class", dest="asset_class", action="append",
help="Restrict to this asset class (repeatable, case-insensitive)")

p = prj_sub.add_parser("find_by_type")
p.add_argument("asset_class")
Expand Down
22 changes: 21 additions & 1 deletion Skills/unreal-project-search/unreal-project-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ All asset paths follow UE content browser format (no .uasset extension):

| Action | Params | Purpose |
|--------|--------|---------|
| `search` | `query` (string), `limit`? (integer) | Full-text search across indexed asset metadata and graph-node metadata |
| `search` | `query` (string), `limit`? (integer), `asset_class`? (string or array) | Full-text search across indexed asset metadata and graph-node metadata, optionally narrowed to given asset classes |
| `find_references` | `asset_path` (string) | Find all assets that reference a given asset |
| `find_by_type` | `asset_type` (string), `module`? (string) | List all assets of a specific type, optionally filtered by plugin/module |
| `get_asset_details` | `asset_path` (string) | Detailed metadata for a specific asset |
Expand All @@ -47,6 +47,26 @@ All asset paths follow UE content browser format (no .uasset extension):
Variables and parameters are **not** in the full-text index — they are indexed as
structured rows, so reach them via `get_asset_details`, not `search`.

## Narrowing by Asset Class

A bare token in a project with a large third-party content folder buries what you
were after. `asset_class` restricts the result set:

```
project search query="E_" asset_class="UserDefinedEnum"
project search query="Ship" asset_class=["Blueprint","WidgetBlueprint"]
```

Case-insensitive, de-duplicated, max 32 entries. The filter runs **inside the
query**, so `limit` counts matching rows — you get the top N of what you asked
for, not the top N of everything followed by a cull. A node-text hit is filtered
by the class of the asset that **owns** the node, so searching graph text with
`asset_class="Blueprint"` behaves as expected.

A class no asset uses returns `count: 0` with `success: true` — it is a real
question with a real answer, not an error. Omit `asset_class` for the unfiltered
behaviour.

## FTS5 Search Syntax

The `search` action uses SQLite FTS5 under the hood. Key syntax:
Expand Down
85 changes: 84 additions & 1 deletion Source/MonolithIndex/Private/Actions/ProjectSearchAction.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
#include "MonolithIndexDatabase.h"
#include "MonolithIndexSubsystem.h"
#include "MonolithParamSchema.h"
#include "Dom/JsonObject.h"
#include "Dom/JsonValue.h"
#include "Editor.h"

// Named (never anonymous) so the forced-full-unity release pass cannot collide
Expand All @@ -15,6 +17,14 @@ namespace MonolithProjectSearchActionDetail
* mitigation if any query-analysing layer is ever added above the bound MATCH.
*/
static constexpr int32 MaxQueryLength = 4096;

/**
* Upper bound on the asset_class filter list. Each entry becomes one bound
* placeholder in an IN clause, so this caps how large a statement a single MCP
* call can ask SQLite to prepare. Well above any real filter -- the project has
* a few dozen asset classes in total.
*/
static constexpr int32 MaxClassFilters = 32;
}

FMonolithActionResult FProjectSearchAction::Execute(const TSharedPtr<FJsonObject>& Params)
Expand Down Expand Up @@ -53,6 +63,65 @@ FMonolithActionResult FProjectSearchAction::Execute(const TSharedPtr<FJsonObject
Limit = static_cast<int32>(FMath::Clamp(RawLimit, 1.0, 1000.0));
}

// asset_class accepts a bare string or an array of strings, matching how other
// actions take single-or-many. Absent means no filtering, which is the old behaviour.
//
// Type is checked against EJson rather than via TryGetStringField, because that
// coerces: a numeric 7 would come back as the string "7" and become a filter for a
// class named "7" -- zero results dressed up as a valid search instead of a rejection.
TArray<FString> AssetClassFilter;
const TSharedPtr<FJsonValue> ClassField = Params->TryGetField(TEXT("asset_class"));
if (ClassField.IsValid() && ClassField->Type != EJson::Null)
{
auto AddClassName = [&AssetClassFilter](FString ClassName)
{
ClassName.TrimStartAndEndInline();
if (!ClassName.IsEmpty())
{
AssetClassFilter.AddUnique(ClassName);
}
};

if (ClassField->Type == EJson::Array)
{
for (const TSharedPtr<FJsonValue>& Entry : ClassField->AsArray())
{
if (!Entry.IsValid() || Entry->Type != EJson::String)
{
return FMonolithActionResult::Error(
TEXT("'asset_class' array entries must be strings"), -32602);
}
AddClassName(Entry->AsString());
}
}
else if (ClassField->Type == EJson::String)
{
AddClassName(ClassField->AsString());
}
else
{
return FMonolithActionResult::Error(
TEXT("'asset_class' must be a string or an array of strings"), -32602);
}

// A filter that survives trimming to nothing is a caller mistake, not a request
// for unfiltered results -- silently widening the search would be the wrong guess.
if (AssetClassFilter.Num() == 0)
{
return FMonolithActionResult::Error(
TEXT("'asset_class' must name at least one non-empty class"), -32602);
}
if (AssetClassFilter.Num() > MonolithProjectSearchActionDetail::MaxClassFilters)
{
return FMonolithActionResult::Error(
FString::Printf(
TEXT("'asset_class' must list %d classes or fewer (got %d)"),
MonolithProjectSearchActionDetail::MaxClassFilters,
AssetClassFilter.Num()),
-32602);
}
}

UMonolithIndexSubsystem* Subsystem = GEditor
? GEditor->GetEditorSubsystem<UMonolithIndexSubsystem>()
: nullptr;
Expand Down Expand Up @@ -82,7 +151,7 @@ FMonolithActionResult FProjectSearchAction::Execute(const TSharedPtr<FJsonObject
TArray<FSearchResult> SearchResults;
FString SearchError;
const EMonolithProjectSearchStatus SearchStatus =
Database->FullTextSearch(Query, Limit, SearchResults, SearchError);
Database->FullTextSearch(Query, Limit, SearchResults, SearchError, AssetClassFilter);
if (SearchStatus != EMonolithProjectSearchStatus::Succeeded)
{
const bool bInvalidQuery = SearchStatus == EMonolithProjectSearchStatus::InvalidQuery;
Expand Down Expand Up @@ -111,6 +180,19 @@ FMonolithActionResult FProjectSearchAction::Execute(const TSharedPtr<FJsonObject
Result->SetBoolField(TEXT("success"), true);
Result->SetArrayField(TEXT("results"), ResultsArr);
Result->SetNumberField(TEXT("count"), SearchResults.Num());

// Echo the applied filter so a zero-result response is self-explaining: the caller
// can see whether the filter was understood without re-running the query.
if (AssetClassFilter.Num() > 0)
{
TArray<TSharedPtr<FJsonValue>> FilterArr;
for (const FString& ClassName : AssetClassFilter)
{
FilterArr.Add(MakeShared<FJsonValueString>(ClassName));
}
Result->SetArrayField(TEXT("asset_class_filter"), FilterArr);
}

return FMonolithActionResult::Success(Result);
}

Expand All @@ -119,5 +201,6 @@ TSharedPtr<FJsonObject> FProjectSearchAction::GetSchema()
return FParamSchemaBuilder()
.Required(TEXT("query"), TEXT("string"), TEXT("FTS5 search query (supports AND, OR, NOT, quoted phrases, prefix*, NEAR(a b, N)); max 4096 characters"))
.Optional(TEXT("limit"), TEXT("integer"), TEXT("Maximum results to return (clamped to 1-1000)"), TEXT("50"))
.Optional(TEXT("asset_class"), TEXT("string|array"), TEXT("Restrict results to these asset classes, e.g. \"Blueprint\" or [\"Blueprint\",\"WidgetBlueprint\"]. Case-insensitive; max 32 entries. Applied inside the query, so 'limit' counts matching rows only. Node-text hits are filtered by their OWNING asset's class"), TEXT("(unfiltered)"))
.Build();
}
2 changes: 1 addition & 1 deletion Source/MonolithIndex/Private/Actions/ProjectSearchAction.h
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ class FProjectSearchAction
public:
static FMonolithActionResult Execute(const TSharedPtr<FJsonObject>& Params);
static FString GetName() { return TEXT("search"); }
static FString GetDescription() { return TEXT("Full-text search across indexed project assets (name, class, description, path, module) and graph nodes (name, class, type)"); }
static FString GetDescription() { return TEXT("Full-text search across indexed project assets (name, class, description, path, module) and graph nodes (name, class, type), optionally filtered by asset_class"); }
static TSharedPtr<FJsonObject> GetSchema();
};
Loading