#3494 closed task (wontfix)
Book lacks any discussion/explanation of binutils 'make check' output
| Reported by: | Joel Kammet | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | 7.5 |
| Component: | Book | Version: | SVN |
| Severity: | normal | Keywords: | |
| Cc: |
Description
Given that (in section 6.13.1) "The test suite for Binutils in this section is considered critical" it would be useful to have some discussion of its expected results and the meaning of output such as " # of expected passes 803 " " # of expected failures 61 " " # of untested testcases 1 " and so on.
Change History (4)
comment:1 by , 11 years ago
comment:2 by , 11 years ago
Thanks. To be honest, by the time I got to section 6.13 I had completely forgotten about the Section 4.6 mention of the test suite logs. So, yes, I am now reassured that my build is OK and proceeding "normally". But it still doesn't satisfy my curiosity as to why particular failures should be expected. I don't find any clues in the Gnu Binutils Documentation at http://sourceware.org/binutils/docs-2.24. I did find the following at http://sourceware.org/binutils/binutils-porting-guide.txt:
"On completion of the binutils test run, a summary of the results will be in the binutils directory in the binutils.sum file. More detailed information will also be available in the binutils/binutils.log file. For the gas testsuite the results are in the gas/testsuite/gas.sum and gas/testsuite/gas.log files and for the linker testsuite they are in the ld/ld.sum and ld/ld.log files."
It was interesting to look at those logs, but they don't seem to give any reason for "expected failures". Since the goal of LFS seems to be primarily educational rather than just about achieving a working system, I would appreciate a bit more explanation about these tests. Also, maybe mention the locations of the log files generated by the testsuites.
comment:3 by , 11 years ago
| Resolution: | → wontfix |
|---|---|
| Status: | new → closed |
Expected failures are different for each package. It's up to the upstream developers to decide what is expected or not. Generally the tests are for the upstream developers, but we run them as a confidence check. We need to take them with a grain of salt.
comment:4 by , 11 years ago
Sorry if I'm beating a dead horse, but after searching some more I found a nice explanation in the automake manual here: https://www.gnu.org/software/automake/manual/html_node/Generalities-about-Testing.html#Generalities-about-Testing
You might want to consider linking or quoting it in Section 4.6.
I don't think this is needed, as you can already look up the results by comparing the testsuite logs linked in the book. I believe the Glibc and GCC pages talk about known failures, etc. because those testsuites do often have some unexpected failures and known errors - which is why make -k is used for both of those - but the Binutils testsuite should generally have no unexpected failures at all.