feat(project): add asset_class filter to project search - #137
Open
Alexbeav wants to merge 2 commits into
Open
Conversation
Searching a common token in a project with a large third-party content folder buried the target: an E_ search over a real project returned 16 UserDefinedEnum rows under 21 Texture2D, 10 NiagaraEmitter and 3 StaticMesh. Every result row already carried asset_class; there was no way to ask for it. The predicate is applied inside the SQL rather than to the returned array. LIMIT is applied by the query, so a post-hoc filter would return fewer rows than requested -- frequently zero -- while matching assets sat below the cut. limit now counts matching rows. Both FTS statements already JOIN assets, so no extra join was needed, and a graph-node hit inside a Blueprint still survives a Blueprint filter. Only the placeholder count is interpolated; class names stay bound. The parameter is type-checked against EJson rather than read through TryGetStringField, which coerces: a numeric 7 would arrive as "7" and become a filter for a class of that name -- zero results dressed up as a valid search instead of -32602. Whitespace-only input is a caller error, not a silent widening. Mirrored into monolith_query.exe and monolith_offline.py, whose search path verify_offline_parity.py does not gate and which SPEC_MonolithIndex asks to be kept in step by hand. Tests: Monolith.ProjectSearch.AssetClassFilter locks the in-SQL behaviour with a lopsided fixture (10 noise, 3 wanted) that a post-hoc implementation cannot pass by luck; AssetClassValidation covers -32602.
The first cut looked up args.opt("asset-class"), but parse_args
normalises hyphens to underscores, so the key is asset_class -- the
lookup never matched and the filter silently did nothing. It also was
not in value_options, so the space-separated form would not have
consumed its value even with the right key.
It compiled clean and did nothing at all; caught only by running the exe
against a real index and diffing against the Python tool.
Both --asset-class=A,B and --asset-class A,B now work, and the usage text
advertises the option.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Searching a common token in a project with a large third-party content folder buries the target. Real numbers from a live project index,
project search "E_" --limit 50:UserDefinedEnum(the target)Texture2DNiagaraEmitterStaticMeshEvery result row already carries
asset_class. There was just no way to ask for it.The change
asset_classonproject search: a bare string ("Blueprint") or an array (["Blueprint","WidgetBlueprint"]), case-insensitive, de-duplicated, capped at 32 entries. Absent means unfiltered, i.e. existing behaviour.The predicate is applied inside the SQL, not to the returned array.
LIMITis applied by the query, so a post-hoc filter would return fewer rows than asked for — often zero — while matching assets sat below the cut.limitnow counts matching rows. Both FTS statements alreadyJOIN assets, so no extra join was needed, and a graph-node text hit inside a Blueprint still survives aBlueprintfilter (it is filtered by the class of the asset that owns the node).Only the placeholder count is interpolated into the SQL; every class name stays bound.
The parameter is type-checked against
EJsonrather than read throughTryGetStringField, which coerces: a numeric7would otherwise arrive as the string"7"and become a filter for a class of that name — zero results dressed up as a valid search instead of a-32602. Whitespace-only input is likewise a caller error rather than a silent widening.A class no asset uses is a successful empty result, not an error.
Offline parity
Mirrored into
monolith_query.exeandmonolith_offline.py.verify_offline_parity.pyhas noproject.*cases, andSPEC_MonolithIndexasks for the three to be kept in step by hand — so this follows that rather than leaving the CLIs behind.Args::optionsin the C++ tool is single-valued, so both CLIs accept a comma-separated list (--asset-class A,B); the Python tool additionally accepts a repeated flag.The second commit fixes a bug in the first:
parse_argsnormalises hyphens to underscores, so the initialargs.opt("asset-class")never matched and the filter silently did nothing. It compiled clean and had no effect — caught only by running the exe against a real index and diffing against the Python tool.Tests
Monolith.ProjectSearch.AssetClassFilter— locks the in-SQL behaviour with a deliberately lopsided fixture (10 noise assets, 3 wanted) that a post-hoc implementation cannot pass by luck. Also covers case-insensitivity, multi-class union and unknown classes.Monolith.ProjectSearch.AssetClassValidation— the-32602paths.Verification
BuildPlugin, BUILD SUCCESSFUL, 0 errors, 0 warningsMonolith.ProjectSearch.*andMonolith.StructFields.*ProjectIndex.db, exe and Python agreeing exactlyNote on how it was verified
BuildPluginon currentmasterfails before reachingMonolithIndex:#136 touches exactly those two files. This branch was therefore built and tested on a throwaway branch with #136 merged in. It does not depend on #136 functionally and does not include it. Worth noting the release path uses host projects with different unity settings, which is presumably why this configuration is not covered by the existing gate.
Docs
CHANGELOG.md(Unreleased),SPEC_MonolithIndex.md(new Asset-Class Filtering section),Skills/unreal-project-search. README untouched — no rounded threshold crossed.