Re: [RFC PATCH 2/2] doc: distros: new document about assessing security vulnerabilities

[Vegard Nossum]

On 11/03/2024 18:59, Matt Wilson wrote:
> On Mon, Mar 11, 2024 at 04:00:54PM +0100, Vegard Nossum wrote:
> > Since what most distros probably ultimately want is a type of CVSS score,
> > the guide is written with that in mind. CVSS provides its own "contextual"
> > modifiers, but these are not accurate or nuanced enough to capture the
> > wide variety of kernel configurations and deployments. We therefore focus
> > on practical evaluation under different sets of assumptions.
>
> (sending from my msw@xxxxxxxxx account to emphasize that I am speaking
> only for myself, not my current employer.)
>
> I'm not sure that Linux distributions particularly *want* a CVSS base
> score for kernel CVEs. It is something that downstream _users_ of
> software have come to expect, especially those that operate under
> compliance regimes that suggest or require the use of CVSS in an
> enterprise's vulnerability management function.

Very true.

> Those compliance regimes often suggest using CVSS scores as found in
> the NVD in search of an objective third party assessment of a
> vulnerability. Unfortunately the text of these regulations suggests
> that the base scores generated by the CVSS system, and found in the
> NVD, are a measure of "risk" rather than a contextless measure of
> "impact".
>
> There have been occurrences where a CVSSv3.1 score produced by a
> vendor of software are ignored when the score in the NVD is higher
> (often 9.8 due to NIST's standard practice in producing CVSS scores
> from "Incomplete Data" [1]). I don't know that harmonizing the
> practice of producing CVSSv3.1 base scores across Linux vendors will
> address the problem unless scores that are made available in the NVD
> match.

That link actually says they would use 10.0 for CVEs without enough
detail provided by the filer/CNA (as I understood it).

I wonder what their strategy would be for all of these new kernel CVEs
-- should we expect to see 10.0 or 9.8 for all of them, do you know? I
assume they do NOT have people to evaluate all these patches in detail.

> But, stepping back for a moment I want to make sure that we are
> putting energy into a system that is fit for the Linux community's
> needs. CVSS lacks a strong scientific and statistical basis as an
> information capture and conveyance system. A study of the distribution
> of CVSSv3.1 base scores historically generated [2] shows that while
> the system was designed to resemble a normal distribution, in practice
> it is anything but.

Yes, agreed.

The article was interesting; thanks for that!

> > +CVEs and CVSS scores for the kernel
> > +===================================
> > +
> > +CVSS (`Common Vulnerability Scoring System <https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System>`_)
> > +is an open standard for vulnerability scoring and the system which is
> > +commonly used by Linux distributions and various industry and government
> > +bodies.
> > +
> > +We won't go into the details of CVSS here, except to give a guide on how
> > +it could be applied most effectively in the context of the kernel.
>
> If the guide has something to say about CVSS, I (speaking only for
> myself) would like for it to call out the hazards that the system
> presents. I am not convinced that CVSS can be applied effectively in
> the context of the kernel, and would rather this section call out all
> the reasons why it's a fool's errand to try.

I also heard this concern privately from somebody else.

I am considering replacing the CVSS part with something else. To be
honest, the part that really matters to reduce duplicated work for
distros is the reachability analysis (including the necessary conditions
to trigger the bug) and the potential outcomes of triggering the bug.
Once you have those, scoring for impact, risk, etc. can be done fairly
easily (at least more easily) in different systems and taking
distro-specific constraints (configuration, mitigations, etc.) into account.


Vegard