Re: [RFC PATCH] drm/ci: add helper script update-xfails.py

[Sergi Blanch Torne]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Helen,

On Wed, 2023-09-27 at 19:28 -0300, Helen Koike wrote:
> > > +def get_unit_test_name_and_results(unit_test):
> > > +    if "Artifact results/failures.csv not found" in unit_test:
> > > +        return None, None
> > > +    unit_test_name, unit_test_result =
> > > unit_test.strip().split(",")
> > > +    return unit_test_name, unit_test_result
> > 
> > Suggestion: it is not managing empty lines or comments. By now,
> > there
> > aren't, but they could be found.
> 
> Indeed.

Just add a new if statement to discard if the strings start with # or
strip the line and check the length. Perhaps we can think of other
assertions to sanitise the string.

> > Suggestion: Sometimes tests fails with different status ("Fail" to
> > "Crash" for example) and the expectations should be updated with
> > the
> > newer status.
> 
> The status is only present in the fails and not in the flakes list,
> so I 
> update it with add_unit_test_or_update_result_to_fails_if_present() 
> function below, make sense?

Absolutely, sorry that I didn't see this was a process included in the
last if statement. If it is present in the fails' file (that includes
the test name and its state) you do exactly what's necessary: add if
not present, update if it was already in the file.

> 
Regards,


- -- 
Sergi Blanch Torné
Senior Software Engineer

Collabora Ltd.
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK
Registered in England & Wales, no. 5513718

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQQwWRK68l+taJfhwqAto5bHyTm9RwUCZRUnAgAKCRAto5bHyTm9
R53NAP9T2OCiwbnEjv+H0CQg/eK1xGe7yS/3cqjaPFRvvZPp1wD/V1H9NuhpRR6M
8+QZgbsS/swSPdwYABtcz+75CKpuJwo=
=XRRO
-----END PGP SIGNATURE-----