Skip to content
This repository was archived by the owner on Jan 24, 2026. It is now read-only.

Filter search - #205

Open
mattn wants to merge 2 commits into
nbd-wtf:masterfrom
mattn:filter-search
Open

Filter search#205
mattn wants to merge 2 commits into
nbd-wtf:masterfrom
mattn:filter-search

Conversation

@mattn

@mattn mattn commented Dec 26, 2025

Copy link
Copy Markdown
Contributor

Current implementation, filter.Match does not take care Search.

@fiatjaf

fiatjaf commented Dec 27, 2025

Copy link
Copy Markdown
Collaborator

I don't know about this one. Search is not absolute, different search algorithms can have different results.

And this can be an expensive operation that people will start doing without realizing.

I think it's better to have another function, like MatchesConsideringSearch().

@mattn

mattn commented Dec 28, 2025

Copy link
Copy Markdown
Contributor Author

Okay, I've found a solution. I need to provide addtional spec for NIP-50 and implementation.

NIP-50
======

Search Capability
-----------------

`draft` `optional` `relay`

## Abstract

Many Nostr use cases require some form of general search feature, in addition to structured queries by tags or ids. 
Specifics of the search algorithms will differ between event kinds, this NIP only describes a general 
extensible framework for performing such queries.

## `search` filter field 

A new `search` field is introduced for `REQ` messages from clients:
```jsonc
{
  // other fields on filter object...
  "search": <string>
}
```

`search` field is a string describing a query in a human-readable form  i.e. "best nostr apps". 

### Query Syntax

The query string is parsed into **terms** as follows:

- Terms are split by whitespace.
- Terms may be enclosed in double quotes (`"`) to include spaces (e.g., `"hello world"` is a single term containing a space).
- Quoted terms may contain escaped quotes if necessary (e.g., `"\"hello\" world"`), though most implementations can omit advanced escaping for simplicity.

Examples:
- `hello world` → terms: `hello`, `world`
- `"hello world" fun` → terms: `"hello world"`, `fun`
- `"hello" "world"` → terms: `hello`, `world`

#### Minimum Required Behavior (MUST)

Relays **MUST** support at least the following:

- Treat multiple terms as an implicit **AND** search: an event matches only if it contains **all** terms.
- Perform case-insensitive substring or full-text matching primarily against the `.content` field.
- MAY also match against other fields if appropriate for the event kind.

This minimal behavior ensures basic interoperability and is easy to implement even with simple string operations or basic full-text indexes.

#### Recommended Boolean Extensions (SHOULD)

Relays **SHOULD** support explicit boolean operators and grouping:

- Uppercase `AND` and `OR` operators (case-sensitive).
- `AND` has higher precedence than `OR`.
- Parentheses `(` and `)` for explicit grouping.

Examples:
- `hello AND world` → must contain both "hello" and "world"
- `hello OR world` → must contain either "hello" or "world"
- `cat AND dog OR bird AND fish``(cat AND dog) OR (bird AND fish)`

Quoted terms work with operators:
- `"hello world" AND fun` → must contain the exact phrase "hello world" and the term "fun"

Relays implementing these extensions **SHOULD** advertise their support in the NIP-11 relay information document, for example:
```json
{
  "supported_nips": [50],
  "nip50_search": {
    "boolean_operators": true,
    "parentheses": true,
    "phrase_search": true
  }
}
```

#### Optional Enhancements (MAY)

Relays **MAY** additionally support:
- Exact phrase matching for quoted terms
- Stemming, accent folding, or language-specific normalization
- Prefix/suffix matching or wildcards
- Searching additional fields beyond `.content`
- Fuzzy matching

### General Behavior

- Results **SHOULD** be returned in descending order by quality/relevance score (as defined by the implementation), **not** by `.created_at`.
- The `limit` filter **SHOULD** be applied after sorting by relevance.
- Matching is case-insensitive by default.
- Relays **SHOULD** exclude spam from search results by default if they support spam filtering.

A query string may also contain `key:value` pairs (two words separated by colon); these are extensions. Relays **SHOULD** ignore extensions they do not support.

Clients may specify multiple filters in a single subscription, e.g.:
```json
["REQ", "sub1", { "search": "orange" }, { "kinds": [1], "search": "purple" }]
```
Clients may combine `search` with `kinds`, `ids`, and other standard filters to narrow results.

Clients **SHOULD** check the `supported_nips` field (or NIP-11 metadata) to learn if a relay supports the `search` filter. Clients **MAY** send `search` queries to any relay and filter out extraneous responses from unsupported relays.

Clients **SHOULD** query multiple relays supporting this NIP to compensate for implementation differences.

Clients **MAY** verify returned events against the query on their side and **MAY** stop querying relays with consistently low precision or recall.

## Extensions

Relays **MAY** support these extensions (specified as `key:value` pairs in the query string):

- `include:spam` – disable default spam filtering
- `domain:<domain>` – restrict to events from users whose verified NIP-05 domain matches `<domain>`
- `language:<two-letter ISO 639-1 code>` – restrict to events in the specified language
- `sentiment:<negative|neutral|positive>` – restrict to events with the specified sentiment
- `nsfw:<true|false>` – include (true) or exclude (false) NSFW events (default: true)

Relays **SHOULD** document any additional extensions they support.

And https://github.com/mattn/go-searchquery

@mattn mattn closed this Dec 28, 2025
@mattn mattn reopened this Dec 28, 2025
@mattn

mattn commented Dec 28, 2025

Copy link
Copy Markdown
Contributor Author

Ah, I will need to add new func MatchesFunc(f func(e *nostr.Event) bool) bool if my proposal is accepted.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants