feat: Add support for highlighting CVEs referenced in changelog entries

Phil Roche · Phil Roche · commit a3ec46837911 · 2022-11-05T16:09:32.000+01:00
Some changelog entries can be very long and hard to manually parse, especially when looking for references to CVEs in kernel changelogs.

This commits adds three new arguments. All three are flags.

`--highlight-cves`:
  Highlight the CVEs referenced in each individual changelog entry
`--highlight-cves-only`:
  Highlight the CVEs referenced in each individual changelog entry
  but do NOT show the rest of the content of the changelog entry
`--highlight-cves-show-cve-description`:
  When highlighting CVEs, show the CVE description

  `--highlight-cves` or `--highlight-cve-only`  must also be used for this to take affect.

Examples added to Readme too.
diff --git a/README.rst b/README.rst
@@ -33,6 +33,97 @@ must be given. Eg::
   fips-initramfs (0.0.11~rc5) bionic; urgency=medium
   [...]
 
+
+`ubuntu-package-changelog` can be also used to highlight the CVEs referenced in a changelog for
+a given Ubuntu source package. Note this also shows the CVE priority assigned by the Ubuntu security team. Eg::
+
+    ubuntu-package-changelog --highlight-cves focal Updates linux-azure
+    linux-azure (5.4.0-1094.100) focal; urgency=medium
+
+      CVEs addressed/mitigated in linux-azure version 5.4.0-1094.100:
+        CVE-2022-2602 (high priority)
+        CVE-2022-41674 (medium priority)
+        CVE-2022-42721 (medium priority)
+        CVE-2022-42720 (medium priority)
+
+      [ Ubuntu: 5.4.0-131.147 ]
+
+      * CVE-2022-2602
+        - SAUCE: io_uring/af_unix: defer registered files gc to io_uring release
+        - SAUCE: io_uring/af_unix: fix memleak during unix GC
+      * CVE-2022-41674
+        - SAUCE: wifi: cfg80211: fix u8 overflow in
+          cfg80211_update_notlisted_nontrans()
+        - SAUCE: wifi: cfg80211/mac80211: reject bad MBSSID elements
+        - SAUCE: wifi: cfg80211: ensure length byte is present before access
+        - SAUCE: wifi: mac80211_hwsim: avoid mac80211 warning on bad rate
+        - SAUCE: wifi: cfg80211: update hidden BSSes to avoid WARN_ON
+      * CVE-2022-42721
+        - SAUCE: wifi: cfg80211: avoid nontransmitted BSS list corruption
+      * CVE-2022-42720
+        - SAUCE: wifi: cfg80211: fix BSS refcounting bugs
+
+     -- Thadeu Lima de Souza Cascardo <cascardo@canonical.com>  Sun, 16 Oct 2022 23:55:23 -0300
+
+If you wish to use `ubuntu-package-changelog` to show CVE descriptions when highlighting the CVEs referenced in a changelog for
+a given Ubuntu source package. Eg::
+
+    ubuntu-package-changelog --highlight-cves --highlight-cves-show-cve-description focal Updates linux-azure
+    linux-azure (5.4.0-1094.100) focal; urgency=medium
+
+      CVEs addressed/mitigated in linux-azure version 5.4.0-1094.100:
+        CVE-2022-2602 (high priority):  David Bouman and Billy Jheng Bing Jhong discovered that a race condition existed in the io_uring subsystem in the Linux kernel, leading to a use- after-free vulnerability. A local attacker could use this to cause a denial of service (system crash) or possibly execute arbitrary code.
+        CVE-2022-41674 (medium priority):  Sönke Huster discovered that an integer overflow vulnerability existed in the WiFi driver stack in the Linux kernel, leading to a buffer overflow. A physically proximate attacker could use this to cause a denial of service (system crash) or possibly execute arbitrary code.
+        CVE-2022-42721 (medium priority):  Sönke Huster discovered that the WiFi driver stack in the Linux kernel did not properly handle BSSID/SSID lists in some situations. A physically proximate attacker could use this to cause a denial of service (infinite loop).
+        CVE-2022-42720 (medium priority):  Sönke Huster discovered that the WiFi driver stack in the Linux kernel did not properly perform reference counting in some situations, leading to a use-after-free vulnerability. A physically proximate attacker could use this to cause a denial of service (system crash) or possibly execute arbitrary code.
+
+      [ Ubuntu: 5.4.0-131.147 ]
+
+      * CVE-2022-2602
+        - SAUCE: io_uring/af_unix: defer registered files gc to io_uring release
+        - SAUCE: io_uring/af_unix: fix memleak during unix GC
+      * CVE-2022-41674
+        - SAUCE: wifi: cfg80211: fix u8 overflow in
+          cfg80211_update_notlisted_nontrans()
+        - SAUCE: wifi: cfg80211/mac80211: reject bad MBSSID elements
+        - SAUCE: wifi: cfg80211: ensure length byte is present before access
+        - SAUCE: wifi: mac80211_hwsim: avoid mac80211 warning on bad rate
+        - SAUCE: wifi: cfg80211: update hidden BSSes to avoid WARN_ON
+      * CVE-2022-42721
+        - SAUCE: wifi: cfg80211: avoid nontransmitted BSS list corruption
+      * CVE-2022-42720
+        - SAUCE: wifi: cfg80211: fix BSS refcounting bugs
+
+     -- Thadeu Lima de Souza Cascardo <cascardo@canonical.com>  Sun, 16 Oct 2022 23:55:23 -0300
+
+
+If you wish to use `ubuntu-package-changelog` to only show referenced CVEs in a changelog for
+a given Ubuntu source package. Eg::
+
+    ubuntu-package-changelog --highlight-cves-only focal Updates linux-azure
+    linux-azure (5.4.0-1094.100) focal; urgency=medium
+
+      CVEs addressed/mitigated in linux-azure version 5.4.0-1094.100:
+        CVE-2022-2602 (high priority)
+        CVE-2022-41674 (medium priority)
+        CVE-2022-42721 (medium priority)
+        CVE-2022-42720 (medium priority)
+
+     -- Thadeu Lima de Souza Cascardo <cascardo@canonical.com>  Sun, 16 Oct 2022 23:55:23 -0300
+
+This `--highlight-cves-only` flag can be used in conjunction with `--highlight-cves-show-cve-description` to only show referenced CVEs in a changelog and also include the CVE description. Eg::
+
+    ubuntu-package-changelog --highlight-cves-only --highlight-cves-show-cve-description focal Updates linux-azure
+    linux-azure (5.4.0-1094.100) focal; urgency=medium
+
+      CVEs addressed/mitigated in linux-azure version 5.4.0-1094.100:
+        CVE-2022-2602 (high priority):  David Bouman and Billy Jheng Bing Jhong discovered that a race condition existed in the io_uring subsystem in the Linux kernel, leading to a use- after-free vulnerability. A local attacker could use this to cause a denial of service (system crash) or possibly execute arbitrary code.
+        CVE-2022-41674 (medium priority):  Sönke Huster discovered that an integer overflow vulnerability existed in the WiFi driver stack in the Linux kernel, leading to a buffer overflow. A physically proximate attacker could use this to cause a denial of service (system crash) or possibly execute arbitrary code.
+        CVE-2022-42721 (medium priority):  Sönke Huster discovered that the WiFi driver stack in the Linux kernel did not properly handle BSSID/SSID lists in some situations. A physically proximate attacker could use this to cause a denial of service (infinite loop).
+        CVE-2022-42720 (medium priority):  Sönke Huster discovered that the WiFi driver stack in the Linux kernel did not properly perform reference counting in some situations, leading to a use-after-free vulnerability. A physically proximate attacker could use this to cause a denial of service (system crash) or possibly execute arbitrary code.
+
+     -- Thadeu Lima de Souza Cascardo <cascardo@canonical.com>  Sun, 16 Oct 2022 23:55:23 -0300
+
 Installation
 ============
 
diff --git a/ubuntu_package_changelog/__init__.py b/ubuntu_package_changelog/__init__.py
@@ -1,12 +1,18 @@
 #!/usr/bin/python3
 
+import re
 import sys
 import argparse
 import urllib.parse
+from lazr.restfulclient.errors import NotFound
 from launchpadlib.launchpad import Launchpad
 
 
-def _lp_get_changelog_url(args, lp):
+def _lp_get_changelog_url(sources):
+    return sources.changelogUrl()
+
+
+def _lp_get_sources(args, lp):
     ubuntu = lp.distributions["ubuntu"]
     pocket = args.pocket
     if args.ppa:
@@ -19,6 +25,7 @@ def _lp_get_changelog_url(args, lp):
         archive = ubuntu.main_archive
 
     lp_series = ubuntu.getSeries(name_or_version=args.series)
+
     sources = _get_published_sources(archive,
                                      args.package,
                                      lp_series,
@@ -39,11 +46,7 @@ def _lp_get_changelog_url(args, lp):
                                              source_package_name,
                                              lp_series,
                                              pocket)
-
-    if len(sources) == 1:
-        return sources[0].changelogUrl()
-    else:
-        return None
+    return sources
 
 
 def _get_binary_packages(archive, binary_package_name, pocket):
@@ -79,11 +82,25 @@ def _parser():
         description='Ubuntu package changelog finder')
     parser.add_argument('--lp-user', help='Launchpad username', default=None)
     parser.add_argument('--ppa', help='Search for a package in the given PPA instead'
-                        'of the Ubuntu archive. Given PPA must have the '
-                        'format "owner/ppa-name". Eg. "toabctl/testing"',
+                                      'of the Ubuntu archive. Given PPA must have the '
+                                      'format "owner/ppa-name". Eg. "toabctl/testing"',
                         type=_args_validate_ppa_name)
     parser.add_argument('--entries', help='number of changelog entries to get from the '
-                        'changelog. 0 mean all. Default: %(default)s', type=int, default=1)
+                                          'changelog. 0 mean all. Default: %(default)s', type=int, default=1)
+    parser.add_argument('--highlight-cves',
+                        help='Highlight the CVEs referenced in each individual changelog entry'
+                             '. Default: %(default)s',
+                        action='store_true')
+    parser.add_argument('--highlight-cves-only',
+                        help='Highlight the CVEs referenced in each individual changelog entry '
+                             'but do NOT show the rest of the content of the changelog entry'
+                             '. Default: %(default)s',
+                        action='store_true')
+    parser.add_argument('--highlight-cves-show-cve-description',
+                        help='When highlighting CVEs, show the CVE description. '
+                             '`--highlight-cves` or `--highlight-cve-only` '
+                             'must also be used for this to take affect.. Default: %(default)s',
+                        action='store_true')
     parser.add_argument('series', help='the Ubuntu series eg. "20.04" or "focal"')
     parser.add_argument('pocket',
                         choices=['Release', 'Security', 'Updates', 'Proposed', 'Backports'],
@@ -104,22 +121,111 @@ def main():
             'ubuntu-package-changelog',
             'production', version='devel')
 
-    changelog_url = _lp_get_changelog_url(args, lp)
-    if not changelog_url:
+    sources = _lp_get_sources(args, lp)
+
+    if len(sources) != 1:
         print('no changelog found')
         sys.exit(0)
 
+    changelog_url = _lp_get_changelog_url(sources[0])
+    source_package_name = sources[0].source_package_name
+
     url = lp._root_uri.append(urllib.parse.urlparse(changelog_url).path.lstrip('/'))
     resp = lp._browser.get(url).decode('utf-8')
     entry_count = 0
+    changelog_entry_lines = []
     for line in resp.splitlines():
         line = line.rstrip()
+
+        # this is the first line of a changelog entry. Find the version and reset lists for
+        # changelog entry lines and changelog entry cves
+        if source_package_name in line and line.startswith(source_package_name) and ' (' in line and ')' in line:
+            individual_changelog_entry_lines = []
+            individual_changelog_entry_cves = []
+            individual_changelog_entry_source_package_version = line[line.index('(') + 1: line.index(')')]
+
+        if (args.highlight_cves or args.highlight_cves_only) and 'CVE' in line:
+            cve_pos = [(m.start(), m.end()) for m in re.finditer(r'CVE-\d+-\d+', line)]
+            for start, end in cve_pos:
+                cve = line[start:end].strip()
+                if cve not in [individual_changelog_entry_cve['cve']
+                               for individual_changelog_entry_cve in individual_changelog_entry_cves]:
+                    cve_details = {}
+                    cve_details['cve'] = cve
+                    cve_details_lines = _get_cve_details(cve, lp)
+                    cve_ubuntu_description = ''
+                    cve_description = ''
+                    for cve_details_line in cve_details_lines:
+                        # only get the CVE description if the user has requested it
+                        if args.highlight_cves_show_cve_description and not cve_ubuntu_description and cve_details_line.startswith('Ubuntu-Description:'):
+                            # get the string in the line after the Ubuntu-Description: line
+                            # while the next line is not 'Notes' keep appending to cve_description
+                            while True:
+                                next_line = next(cve_details_lines)
+                                if next_line.startswith('Notes'):
+                                    break
+                                cve_ubuntu_description += next_line
+                        if args.highlight_cves_show_cve_description and not cve_description and cve_details_line.startswith('Description:'):
+                            # get the string in the line after the Ubuntu-Description: line
+                            # while the next line is not 'Notes' keep appending to cve_description
+                            while True:
+                                next_line = next(cve_details_lines)
+                                if next_line.startswith('Ubuntu-Description:'):
+                                    break
+                                cve_description += next_line
+                        cve_details['cve_description'] = cve_ubuntu_description \
+                            if cve_ubuntu_description else cve_description
+                        if 'Priority:' in cve_details_line:
+                            cve_details['cve_priority'] = cve_details_line.split('Priority:')[1].strip()
+                    individual_changelog_entry_cves.append(cve_details)
+
+        individual_changelog_entry_lines.append(line)
+
+        # the '--' string is used to separate entries in the changelog as it is the last line of a changelog entry
+        # example "-- Maintainer Name <maintainer@email.com>  Mon, 17 Oct 2022 10:32:58 -0300"
         if line.startswith(' -- '):
             entry_count += 1
+            if (args.highlight_cves or args.highlight_cves_only) and individual_changelog_entry_cves:
+                individual_changelog_entry_lines.insert(1, '\n  CVEs addressed/mitigated in {} version {}:'.format(
+                    source_package_name, individual_changelog_entry_source_package_version))
+                cve_details_insertion_index_start = 2
+                for individual_changelog_entry_cve in individual_changelog_entry_cves:
+                    individual_changelog_entry_lines.insert(cve_details_insertion_index_start, '    {} ({} priority){}'.format(
+                        individual_changelog_entry_cve['cve'],
+                        individual_changelog_entry_cve['cve_priority'],
+                        ': {}'.format(individual_changelog_entry_cve['cve_description'])
+                        if args.highlight_cves_show_cve_description else ''))
+                    cve_details_insertion_index_start += 1
+
+            if args.highlight_cves_only:
+                changelog_entry_lines.extend(individual_changelog_entry_lines[0:cve_details_insertion_index_start])
+                changelog_entry_lines.extend(individual_changelog_entry_lines[-2:])
+                changelog_entry_lines.append('\n')
+            else:
+                changelog_entry_lines.extend(individual_changelog_entry_lines)
+
+        # If we have parsed all the entries we want, then we can stop parsing the changelog
         if args.entries > 0 and entry_count >= args.entries:
-            print(line)
             break
-        print(line)
+
+    # print the lines in changlog_entry_lines
+    print('\n'.join(changelog_entry_lines))
+
+
+def _get_cve_details(cve, lp):
+    # download the cve details and parse so we can get the CVE description and the CVE priority
+    cve_details_lines = []
+    possible_cve_detail_locations = ['active', 'retired', 'ignored']
+    for possible_cve_detail_location in possible_cve_detail_locations:
+        try:
+            cve_details_url = 'https://git.launchpad.net/ubuntu-cve-tracker/plain/{}/{}'.format(
+                possible_cve_detail_location, cve)
+            cve_details_resp = lp._browser.get(cve_details_url).decode('utf-8')
+            cve_details_lines = iter(cve_details_resp.splitlines())
+            return cve_details_lines
+        except NotFound:
+            pass  # Keep trying until we find the cve details
+    return cve_details_lines
 
 
 if __name__ == '__main__':
