Tx sub states/v2#15827
Open
victorjulien wants to merge 43 commits into
Open
Conversation
Allow registration of per substate progress name mappings. Implement logic for getting sub-state names, id's. Add transaction type and end of progress values to AppLayerTxData. Introduce helpers to keep logic clean.
Support for per transaction sub states: different state machines per transaction type. Skip engines not belonging to our substate. Store tx_type in DetectTransaction. Each protocol supporting sub states will register states from 1 and up. To support: Ticket: OISF#8386.
Split into 2 sub-states: - stream, which has the "HTTP" requests and responses, including DOH2 - global, which has the settings and other global or control handling Introduce a simpler progress tracking for the global sub state: - HTTP2ProgGlobalStart and HTTP2ProgGlobalComplete. The stream sub state uses the same state machine as before. Ticket: OISF#8386.
Hard coded for now.
Each keyword supporting DOH2 must register explicitly
Now that DNS keywords are no longer registered for DOH2, the keywords need to manually registered for DOH2.
Now that HTTP keywords are no longer automatically registered for HTTP/2, register them manually.
DNS/HTTP2 no longer automatically registers all keywords also for DOH2. Instead, the DNS keywords and HTTP/2 stream keywords are registered for DOH2 explicitly as well. - flow alproto DOH2 + engine DOH2 -> inspect inner DNS - flow alproto DOH2 + engine HTTP2 -> inspect outer HTTP/2 - flow alproto DOH2 + engine UNKNOWN -> inspect outer HTTP/2
For the most part hard coded for HTTP/2 for now.
Register a generic list for each sub state / progress combo.
Otherwise we'd have to re-register the relevant callbacks.
With substate support the fixed table approach for policies was no longer a good fit, so convert to a hash table. The policies are stored per alproto, sub_state, progress and direction.
Now that there is a policy hash, make the alert signature object a member of that to avoid another hash table lookup.
Check that sub state handling is correct.
In preparation of being able to log it.
http2 states were not yet supported even if the built-in states could already work. Clearly error out on a hook w/o substate.
In firewall mode the alproto logic when building prefilter and rule engines is strict, but was out of sync between prefilter and rule app inspect engines. For SIGNATURE_HOOK_TYPE_APP rules the logic is strict, with an exception for HTTP/2 and DOH2. DOH2 is not a full protocol implementation, but rather HTTP/2 with a different alproto. Fixes: d64954a ("detect: don't register unrelated inspect engines")
This is to make the hook use names consistent with the default hooks and other protocols.
DOH2 depends on HTTP/2, so it makes no sense to enable it separately. It would also put the sub state handling in a weird state, as the DOH2 side reuses the registered HTTP/2 callbacks.
When building the per signature app_inspect list, only add engines that match the sub state specified in the hook.
Register HTTP/2 and DOH2 this way.
|
WARNING:
Pipeline = 32497 |
victorjulien
requested review from
a team,
catenacyber and
jasonish
as code owners
Closed
Member
|
Given this rule: ASAN throws a memory leak error on test, for example: Similar leak if you convert the rule to an |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Foundational tx sub state support, with app transactions able to have their own type and state machines (hence sub states). Implemented here for HTTP2/DOH2 with a stream and global type.
SV_BRANCH=OISF/suricata-verify#3168
It's not perfect yet, with some known issues:
Would like to get this in and work more incrementally from here.
https://redmine.openinfosecfoundation.org/issues/8386