Re: [RFC PATCH 2/2] doc: distros: new document about assessing security vulnerabilities

[Matt Wilson]

On Mon, Mar 11, 2024 at 04:00:54PM +0100, Vegard Nossum wrote:
> On February 13, kernel.org became a CVE Numbering Authority (CNA):
> 
>   https://www.cve.org/Media/News/item/news/2024/02/13/kernel-org-Added-as-CNA
> 
> The kernel.org CNA/CVE team does not provide any kind of assessment of
> the allocated CVEs or patches. However, this is something that many
> distributions want and need.
>
> Provide a new document that can be used as a guide when assessing
> vulnerabilities. The hope is to have a common point of reference that
> can standardize or harmonize the process and hopefully enable more
> cross-distribution collaboration when it comes to assessing bugfixes.
> 
> We deliberately emphasize the difficulty of assessing security impact
> in the wide variety of configurations and deployments.
> 
> Since what most distros probably ultimately want is a type of CVSS score,
> the guide is written with that in mind. CVSS provides its own "contextual"
> modifiers, but these are not accurate or nuanced enough to capture the
> wide variety of kernel configurations and deployments. We therefore focus
> on practical evaluation under different sets of assumptions.

(sending from my msw@xxxxxxxxx account to emphasize that I am speaking
only for myself, not my current employer.)

I'm not sure that Linux distributions particularly *want* a CVSS base
score for kernel CVEs. It is something that downstream _users_ of
software have come to expect, especially those that operate under
compliance regimes that suggest or require the use of CVSS in an
enterprise's vulnerability management function.

Those compliance regimes often suggest using CVSS scores as found in
the NVD in search of an objective third party assessment of a
vulnerability. Unfortunately the text of these regulations suggests
that the base scores generated by the CVSS system, and found in the
NVD, are a measure of "risk" rather than a contextless measure of
"impact".

There have been occurrences where a CVSSv3.1 score produced by a
vendor of software are ignored when the score in the NVD is higher
(often 9.8 due to NIST's standard practice in producing CVSS scores
from "Incomplete Data" [1]). I don't know that harmonizing the
practice of producing CVSSv3.1 base scores across Linux vendors will
address the problem unless scores that are made available in the NVD
match.

But, stepping back for a moment I want to make sure that we are
putting energy into a system that is fit for the Linux community's
needs. CVSS lacks a strong scientific and statistical basis as an
information capture and conveyance system. A study of the distribution
of CVSSv3.1 base scores historically generated [2] shows that while
the system was designed to resemble a normal distribution, in practice
it is anything but.

A guide that helps a practitioner evaluate the legitimate risks that
may be present in a given version, configuration, and use case for the
Linux kernel could be a very helpful thing. This guide is an excellent
start for one! But as you rightly call out, CVSS is not a system that
has an ability to capture all the nuance and context of software the
likes of the Linux kernel, therefore the focus should be on practical
evaluation under common use cases.

> Create a new top-level (admittedly rather thin) "book" for information
> for distros and place the document there as this document is not meant
> for either developers or users.
> 
> See the rendered document at:
> 
>   https://vegard.github.io/linux/2024-03-11/security-assessment.html

[...]

> +
> +CVEs and CVSS scores for the kernel
> +===================================
> +
> +CVSS (`Common Vulnerability Scoring System <https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System>`_)
> +is an open standard for vulnerability scoring and the system which is
> +commonly used by Linux distributions and various industry and government
> +bodies.
> +
> +We won't go into the details of CVSS here, except to give a guide on how
> +it could be applied most effectively in the context of the kernel.

If the guide has something to say about CVSS, I (speaking only for
myself) would like for it to call out the hazards that the system
presents. I am not convinced that CVSS can be applied effectively in
the context of the kernel, and would rather this section call out all
the reasons why it's a fool's errand to try.

--msw

[1] https://nvd.nist.gov/vuln-metrics/cvss
[2] https://theoryof.predictable.software/articles/a-closer-look-at-cvss-scores/#the-distribution-of-actual-scores