IMAP protocol parser, logger and sticky buffers v9

doc/userguide/output/eve/eve-json-format.rst

@@ -3373,3 +3373,24 @@ Example ::
     "min_ttl": 1,
     "max_ttl": 1
   }
+
+Event type: IMAP
+----------------
+
+Fields
+~~~~~~
+
+* "requests": Array of IMAP request lines sent by the client. Each entry is formatted as "tag command arguments".
+* "responses": Array of IMAP response lines from the server.
+
+Example ::
+
+  "imap": {
+    "requests": [
+      "8 UID fetch 1 (UID RFC822.SIZE BODY.PEEK[])"
+    ],
+    "responses": [
+      "* 1 FETCH (UID 1 RFC822.SIZE 452 BODY[] {452})",
+      "8 OK UID FETCH completed"
+    ]
+  }

doc/userguide/rules/email-keywords.rst

@@ -10,6 +10,8 @@ Matches the MIME ``From`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.from; content:"<content to match against>";
@@ -34,6 +36,8 @@ Matches the MIME ``Subject`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.subject; content:"<content to match against>";
@@ -58,6 +62,8 @@ Matches the MIME ``To`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.to; content:"<content to match against>";
@@ -82,6 +88,8 @@ Matches the MIME ``Cc`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.cc; content:"<content to match against>";
@@ -106,6 +114,8 @@ Matches the MIME ``Date`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.date; content:"<content to match against>";
@@ -130,6 +140,8 @@ Matches the MIME ``Message-Id`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.message_id; content:"<content to match against>";
@@ -154,6 +166,8 @@ Matches the MIME ``X-Mailer`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.x_mailer; content:"<content to match against>";
@@ -178,6 +192,8 @@ Matches ``URL`` extracted of an email.
 
 Comparison is case-sensitive.
 
+This keyword works with SMTP only.
+
 Syntax::
 
  email.url; content:"<content to match against>";
@@ -204,6 +220,8 @@ Matches ``Received`` field of an email.
 
 Comparison is case-sensitive.
 
+Works with both SMTP and IMAP protocols.
+
 Syntax::
 
  email.received; content:"<content to match against>";
@@ -223,13 +241,163 @@ Example of a signature that would alert if a packet contains the MIME field ``re
 
   alert smtp any any -> any any (msg:"Test mime email received"; :example-rule-emphasis:`email.received; content:"from [65.201.218.30] (helo=COZOXORY.club)by 173-66-46-112.wash.fios.verizon.net with esmtpa (Exim 4.86)(envelope-from )id 71cF63a9for mirjam@abrakadabra.ch\; Mon, 29 Jul 2019 17:01:45 +0000";` sid:1;)
 
+email.command
+-------------
+
+Matches on the lowercased IMAP command name associated with an email transfer.
+For example, ``append`` when the client uploads a message, or ``fetch`` when a
+message is retrieved from the server.
+
+This keyword works with IMAP only.
+
+Syntax::
+
+ email.command; content:"<command>";
+
+``email.command`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+This keyword maps to the EVE field ``email.command``
+
+Examples
+^^^^^^^^
+
+Example of a signature that would alert if an email is being uploaded via APPEND:
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP APPEND email"; :example-rule-emphasis:`email.command; content:"append";` sid:1;)
+
+Example of a signature that would alert if an email is being fetched:
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP FETCH email"; :example-rule-emphasis:`email.command; content:"fetch";` sid:1;)
+
+email.body
+----------
+
+Matches on the body content of an email transferred over IMAP.
+
+This keyword works with IMAP only.
+
+Syntax::
+
+ email.body; content:"<content to match against>";
+
+``email.body`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+Example
+^^^^^^^
+
+Example of a signature that would alert if the email body contains "confidential":
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP email body match"; :example-rule-emphasis:`email.body; content:"confidential";` sid:1;)
+
+Email Header Normalization
+--------------------------
+
+Email headers are normalized before matching:
+
+- **Header names** are converted to lowercase with hyphens replaced by underscores
+  (e.g. ``Content-Type`` becomes ``content_type``).
+- **Header values** have leading and trailing whitespace trimmed.
+- **Folded headers** (multi-line headers per RFC 5322) are unfolded: the CRLF and
+  leading whitespace of continuation lines are replaced by a single space
+  (e.g. ``Subject: very long\r\n  value`` becomes ``very long value``).
+- When the same header appears multiple times, each occurrence is exposed as a
+  separate buffer.
+
+The ``email.header`` buffer combines these as ``name: value`` (with a colon and
+space separator), so a header originally written as ``Content-Type: text/plain``
+is presented as ``content_type: text/plain``.
+
+email.header
+------------
+
+Matches on email headers in normalized ``name: value`` format.
+
+This keyword works with IMAP only.
+
+Syntax::
+
+ email.header; content:"<content to match against>";
+
+``email.header`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+``email.header`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
+
+Example
+^^^^^^^
+
+Example of a signature that would alert if the email has a subject header:
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP email header match"; :example-rule-emphasis:`email.header; content:"subject";` sid:1;)
+
+email.header.name
+-----------------
+
+Matches on email header names only (normalized: lowercase, hyphens replaced by underscores).
+
+This keyword works with IMAP only.
+
+Syntax::
+
+ email.header.name; content:"<content to match against>";
+
+``email.header.name`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+``email.header.name`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
+
+Example
+^^^^^^^
+
+Example of a signature that would alert if the email contains a "subject" header:
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP email header name match"; :example-rule-emphasis:`email.header.name; content:"subject";` sid:1;)
+
+email.header.value
+------------------
+
+Matches on email header values only (trimmed of leading/trailing whitespace, folded lines unfolded).
+
+This keyword works with IMAP only.
+
+Syntax::
+
+ email.header.value; content:"<content to match against>";
+
+``email.header.value`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+``email.header.value`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
+
+Example
+^^^^^^^
+
+Example of a signature that would alert if an email header value contains a specific address:
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP email header value match"; :example-rule-emphasis:`email.header.value; content:"user@example.com";` sid:1;)
+
 email.body_md5
 --------------
 
 Matches the ``md5`` hash generated from an email body.
-This keyword only works if config option
-``app-layer.protocols.smtp.mime.body-md5`` is enabled or auto.
+This keyword works with both SMTP and IMAP protocols.
+For SMTP, the config option ``app-layer.protocols.smtp.mime.body-md5``
+must be enabled or auto. For IMAP, the config option
+``app-layer.protocols.imap.mime.body-md5`` must be enabled or auto.
 It should be used with ``requires: keyword email.body_md5;``
+For IMAP, the md5 hash is computed on the body data retained by the
+parser, which may be truncated. If the body exceeds the parser's maximum
+body size, the hash will be computed on the truncated data and will not match
+the hash of the complete body.
 
 Syntax::
 
@@ -248,3 +416,7 @@ is generated from an email body:
 .. container:: example-rule
 
   alert smtp any any -> any any (msg:"Test mime email body_md5"; :example-rule-emphasis:`requires: keyword email.body_md5; email.body_md5; content:"ed00c81b85fa455d60e19f1230977134";` sid:1;)
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"Test IMAP email body_md5"; :example-rule-emphasis:`requires: keyword email.body_md5; email.body_md5; content:"ed00c81b85fa455d60e19f1230977134";` sid:2;)

doc/userguide/rules/imap-keywords.rst

@@ -0,0 +1,102 @@
+IMAP Keywords
+=============
+
+.. role:: example-rule-action
+.. role:: example-rule-header
+.. role:: example-rule-options
+.. role:: example-rule-emphasis
+
+imap.request
+------------
+
+Matches on IMAP request lines sent from the client to the server.
+
+Syntax::
+
+ imap.request; content:"<content to match against>";
+
+``imap.request`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+``imap.request`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
+
+This keyword maps to the EVE field ``imap.requests[]``
+
+Examples
+^^^^^^^^
+
+Example of a signature that would alert if an IMAP request contains a LOGIN command:
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP LOGIN request"; :example-rule-emphasis:`imap.request; content:"LOGIN";` sid:1;)
+
+imap.response
+-------------
+
+Matches on IMAP response lines sent from the server to the client.
+
+Syntax::
+
+ imap.response; content:"<content to match against>";
+
+``imap.response`` is a 'sticky buffer' and can be used as a ``fast_pattern``.
+
+``imap.response`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
+
+This keyword maps to the EVE field ``imap.responses[]``
+
+Examples
+^^^^^^^^
+
+Example of a signature that would alert if an IMAP response contains "OK":
+
+.. container:: example-rule
+
+  alert imap any any -> any any (msg:"IMAP OK response"; :example-rule-emphasis:`imap.response; content:"OK";` sid:1;)
+
+Frames
+------
+
+The IMAP parser supports the following frames:
+
+* imap.pdu
+* imap.headers
+* imap.body
+
+imap.pdu
+^^^^^^^^
+
+A single IMAP request or response PDU. Each command from the client or
+response line from the server creates a separate frame.
+
+.. container:: example-rule
+
+  alert imap any any -> any any (\
+  :example-rule-options:`frame:imap.pdu; content:"LOGIN"; startswith;` \
+  sid:1;)
+
+imap.headers
+^^^^^^^^^^^^
+
+The email header section of an email transferred as IMAP literal data (e.g. via
+FETCH or APPEND). The frame spans from the beginning of the email content up to the
+blank line that separates headers from body.
+
+.. container:: example-rule
+
+  alert imap any any -> any any (\
+  :example-rule-options:`frame:imap.headers; content:"Subject|3a|"; startswith;` \
+  sid:1;)
+
+imap.body
+^^^^^^^^^
+
+The email body section of an email transferred as IMAP literal data (e.g. via FETCH
+or APPEND). The frame starts after the blank line that separates headers from body.
+Only created when the body is non-empty.
+
+.. container:: example-rule
+
+  alert imap any any -> any any (\
+  :example-rule-options:`frame:imap.body; content:"Click here";` \
+  sid:1;)

doc/userguide/rules/index.rst

@@ -41,6 +41,7 @@ Suricata Rules
    quic-keywords
    nfs-keywords
    smtp-keywords
+   imap-keywords
    websocket-keywords
    app-layer
    decode-layer