Review libstdc++ test failures

[ken@…]

I've become diverted into trying to understand why everyone sees the 22_locale/time_get failures (it looks as if glibc changes to locales including en_HK did indeed trigger this) and why I'm the only one who reports 22_locale/numpunct/members/char/3.cc failing.

That later is because I normally install all locales, and the locale in glibc has been changed, but the test has not (and maybe no current locales would pass).

Along the way I started to look at which locales are currently referenced, and it seems to me that there are several more which should probably be added. At the moment I've got a backup just before glibc, and I'm running to the end of the gcc tests changing which locales are used. This will take some time.

I'll also note that sporadically, at least on this machine, I see

FAIL: 30_threads/this_thread/95989.cc (test for excess errors)
UNRESOLVED: 30_threads/this_thread/95989.cc compilation failed to produce executable

which means 2 less expected results, 1 unexpected failure and 1 unresolved testcase).

[Bruce Dubbs]

When I investigated the 22_locale/time_get failures some time ago, I also found that they involved the en_HK locale. I did not delve deeper, but I do note that those failures are regularly reported in the upstream test failures at ​https://gcc.gnu.org/ml/gcc-testresults/.

My conclusion at the time was that upstream knows about the problem and has not made it a priority to fix the test (or the locale).

[ken@…]

Replying to Bruce Dubbs:

When I investigated the 22_locale/time_get failures some time ago, I also found that they involved the en_HK locale. I did not delve deeper, but I do note that those failures are regularly reported in the upstream test failures at ​https://gcc.gnu.org/ml/gcc-testresults/.

My conclusion at the time was that upstream knows about the problem and has not made it a priority to fix the test (or the locale).

I suspect it is a hard problem to fix (comment 1 in ​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71367 suggest that), but also that there is no synchronisation between (random) locale fixes in glibc and the many places where libstdc++ test use locales. To be clear, the locales appear to be correct but libstdc++ tests were correct at the time they were last changed, but have become out of date.

For the tests, I think they are close to "you are in a maze of twisty and fragile tests, all not-quite alike".

I have to admit that I rarely look at upstream gcc test results these days, they tend to be too far away from what we are using.

[ken@…]

The issue with

FAIL: 22_locale/numpunct/members/char/3.cc execution test

has already been reported, ​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96322 blaming a change in glibc-2.31. Only people who installed all locales (or nl_NL@euro) will see this, but based on what was done for Pure LFS I think we should add all locales where there are valid tests.

At the moment I'm adding one locale at a time to the minimum set. For nl_NL@euro I get one more unexpected failure and three more expected passes, so some more tests are succeeding.

I'll note that the number of tests is not always obvious - in g++ we remark about 3 failures for constexpr-52830.C, those are running with different --std=c++ variants and fail on test for excess errors, but also XPASS on each with internal compiler error. My point here is that one source file is counted for at least six tests.

Because most of my runs on this machine have

FAIL: 30_threads/this_thread/95989.cc (test for excess errors)
UNRESOLVED: 30_threads/this_thread/95989.cc compilation failed to produce executable

which nobody else has reported, so presumably something specific to this machine, I'm counting the number of tests as expected passes + unexpected failures + unexpected successes (none so far in libstdc++) + unresolved testcases. I will note here that on one run I saw an extra (second) FAIL from 95989.cc.

There is one additional oddity: two cvt tests use en_US.ISO-8859-15 which glibc used to install as part of 'all locales'. Since (at least) glibc-2.30 that is no-longer the case. If I install that, four more tests run (probably 'does it compile' and 'does it work') for each. I've just raised ​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101234 for that, not sure if it is worth adding en_US.ISO-8859-15 - I'm hoping they'll change the test to use an 8869-15 locale which is already widely used.

Meanwhile, checking other missing locales. On my 'all locales' build I get 352 unsupported tests, that would reduce to 348 if that en_US variant was added, but I've no idea what the other unsupported tests are - probably unrelated to locales.

Still got a couple of locales to try.

[ken@…]

The en_US.iso885915 locale turns out only ever have been installed by fedora and RedHat using a patch. Test should probably be changed, we don't need the locale.

I've got down to 353 unsupported tests (i.e. 1 more than with all locales), but I think that is as close as I'm going to get for the moment. Grepping for dg-requires, apart from namedlocale there are various things including among others -cxa-atexit, debug-mode, a lot of variants of -effective-target : not just c++ variants, some iconv variants, so I guess that explains why not all tests are supported.

This adds the following locales:

de_DE@euro
en_GB (8859-1)
es_ES@euro
is_IS (8859-1)
is_ISiUTF-8
it_IT@euro
nl_NL@euro
se_NO.UTF-8 (Northern Sami)
ta_IN.UTF-8
zh_TW.UTF-8

[ken@…]

Duh, de_DE@euro is already present, I comemnted it early in my testing and then reinstated it.

[ken@…]

Fixed in @cefe50534a0f76741a2113d251992ee36629528f 10.1-106