Regex literal # marker font-lock override not taking effect

[bbatsov]

The regex font-lock feature defines a rule to highlight the # marker in regex literals (#"pattern") with font-lock-property-face, using :override t to override the string feature's font-lock-regexp-face that covers the entire regex_lit node. However, the override doesn't take effect -- the # gets font-lock-regexp-face like the rest of the literal.

The query itself is correct (manually running it against the tree returns the right node), so this appears to be a tree-sitter font-lock engine issue with overriding a parent-node capture from a child-node capture in a later rule.

Options:

1. Remove the regex marker rule and let the whole #"pattern" be font-lock-regexp-face. The # is part of the regex syntax, so this is reasonable.
2. Change the string rule to capture regex parts individually (marker, open, content, close) instead of the whole regex_lit node, avoiding the override issue entirely.
3. Use a different face if we fix the override. font-lock-property-face is an odd choice for a reader dispatch character -- font-lock-delimiter-face would be more consistent with how #' and quote markers are handled.

I'm leaning towards option 1. Came across this while backfilling font-lock tests -- opening this so I don't forget to fix it eventually.

[rrudakov]

The reason it's not working is that we are using an undefined face. font-lock-property-face doesn't exist. I tried to change the query to:

'((regex_lit marker: _ @font-lock-property-name-face))

and it fixes the problem.

[bbatsov]

Yeah, just noticed this myself. Still, the question is whether it makes sense use a property face for this, as that's not what the property face semantics normally are.

[bbatsov]

I've opted to use font-lock-punctuation-face for this, but I'll keep this issue open in case someone comes up with a better idea.

[rrudakov]

IMO, it should be the same face as for # in sets and function literals. As far as I can see, there we don't have any special face, so default is used. For # in regex we could set the default face explicitly in the override query part:

:feature 'regex
   :language 'clojure
   :override t
   '((regex_lit marker: _ @default))

or we could remove the override part and modify the string feature:

:feature 'string
   :language 'clojure
   '((str_lit open: _ @font-lock-string-face
              (str_content) @clojure-ts--fontify-string
              close: _ @font-lock-string-face)
     (regex_lit open: _ @font-lock-regexp-face
                (regex_content) @font-lock-regexp-face
                close: _ @font-lock-regexp-face))

[rrudakov]

We can also do it like this to avoid override:

   :feature 'string
   :language 'clojure
   '((str_lit open: _ @font-lock-string-face
              (str_content) @clojure-ts--fontify-string
              close: _ @font-lock-string-face)
     (regex_lit open: _ @font-lock-regexp-face
                (regex_content) @font-lock-regexp-face
                close: _ @font-lock-regexp-face))

   :feature 'regex
   :language 'clojure
   '((regex_lit marker: _ @font-lock-punctuation-face))