libpipeline documentation wrongly claims that it does not have a test suite

[Colin Watson]

(I'm the upstream author of libpipeline.)

​http://www.linuxfromscratch.org/lfs/view/development/chapter06/libpipeline.html says that libpipeline does not come with a test suite. This is not true - it does ('make check'), although you need to have the 'check' package installed to use it.

If you don't want to advise users to build and install 'check', I guess that's OK (although it seems wise to take any opportunity to run test suites), but in that case please rephrase the documentation to make clear that LFS is not providing directions on running the test suite rather than saying the package doesn't come with one. Thanks!

[bdubbs@…]

Thanks for the heads up. We don't get maintainers here very often, but it is appreciated.

When we do 'make check', the command completes without doing anything. You mention needed a 'check' package. I could not find a reference to that in the tarball or the home page. In the case of udev, we do grab the testing data from the repository and tell the user how to test it. We can do the same for libpipeline, but it would be much easier for us if we could just get a tarball to tell the users to download.

I did try to cd to the tests directory and run 'make check-TESTS', but got an error:

In file included from basic.c:24:0:
common.h:3:19: fatal error: check.h: No such file or directory

If we can get the directions on how to run the test suite, we will be glad to include them.

[Colin Watson]

Yes, 'make check' does nothing unless you have the separate 'check' package (​http://check.sourceforge.net/) installed before configuring libpipeline. I expect you'll need pkg-config too.

You're right that there's no reference to this in libpipeline's own documentation. I'll amend the README and perhaps the home page to correct this.

[Colin Watson]

Done. The README change is in revision control and will be in the next release tarball; the home page change is visible now (​http://libpipeline.nongnu.org/).

[bdubbs@…]

OK, I was able to make the tests work. This is the process. The user needs to download

​http://sourceforge.net/projects/check/files/check/0.9.8/check-0.9.8.tar.gz

The build is a simple 'configure --prefix=/usr && make && make install'

This package has a 'make check' of it's own that takes quite a long time. I got some errors from that:

Running suite(s): Setup Sig
0%: Checks: 1, Failures: 0, Errors: 1
check_check_fixture.c:139:S:Setup Sig:test_sub_fail:0: (after this point) Received signal 8 (Floating point exception)

Running suite(s): Teardown Sig
0%: Checks: 1, Failures: 0, Errors: 1
check_check_fixture.c:145:S:Teardown Sig:test_sub_pass:0: (after this point) Received signal 8 (Floating point exception)

I can't tell if these are expected or not but at the end, it said "All 5 tests passed"

Then the user needs to do:

cd tests
sed -i 's/^LDFLAGS =/& -lcheck/'
make check
cd ..

If pkg-config and check are present, then libpipeline only needs

./configure --prefix=/usr
make
make check
make install

Right now, I'm inclined to fix this ticket with a comment that the check package and pkg-config need to be installed to run the 'make check' command from libpipeline.

Other opinions?

[Matthew Burgess]

Replying to bdubbs@…:

Right now, I'm inclined to fix this ticket with a comment that the check package and pkg-config need to be installed to run the 'make check' command from libpipeline.

Other opinions?

That sounds reasonable for now. I wish there was 'one true' automated testing software, because now it looks as if we need 'expect', 'dejagnu', 'autotest' and 'check' to get full test coverage in LFS.

Colin, if you're still reading this, was there a particular reason that you chose to use 'check'? Would you be interested if I were to take a shot at converting your test suite to autotest instead? That way, the tests would be self-contained; not depending on any external package.

[Colin Watson]

Bruce, I can't speak for check's test suite errors, but perhaps you should report them as a bug to its maintainers if it isn't discussed in its documentation.

pkg-config is a widespread dependency nowadays, and it's small and of general utility. I realise LFS is pretty minimalist, but I honestly think you'd be much better off using pkg-config rather than advising your users to fiddle about with LDFLAGS or other similar hacks (as I see you advise for man-db's use of libpipeline); that's a distinctly error-prone process and doesn't seem to have much educational value either. The point of pkg-config is that other packages and users shouldn't need to keep track of the specifics of how libraries need to be compiled and linked against.

Matthew, check seemed a pretty good unit testing suite for C. Obviously you're welcome to try converting to Autotest, but Autotest is designed principally for testing things that you can run directly from a shell; it does not look at all suitable for testing C libraries. One could no doubt have it compile little chunks of input data on the fly and run those, but that would be much less readable and I would be unlikely to accept such a change. Maintainability and readability are higher priorities for me than self-containedness.

DejaGNU looks like a somewhat better fit, although it's always struck me as fairly big and complicated. check is much simpler to my eye, and I found it much easier to get my head around.

[Colin Watson]

Oh, and Autotest's manual mentions how DejaGNU itself tended to have portability problems. check may not be already installed everywhere, but at least it seems simple to get going itself (you don't have to get Tcl going first, etc.) ...

[bdubbs@…]

Replying to cjwatson:

Bruce, I can't speak for check's test suite errors, but perhaps you should report them as a bug to its maintainers if it isn't discussed in its documentation.

I checked the source and the 'errors' appear to be intentional and are not classified as failures.

pkg-config is a widespread dependency nowadays, and it's small and of general utility. I realise LFS is pretty minimalist, but I honestly think you'd be much better off using pkg-config rather than advising your users to fiddle about with LDFLAGS or other similar hacks (as I see you advise for man-db's use of libpipeline); that's a distinctly error-prone process and doesn't seem to have much educational value either. The point of pkg-config is that other packages and users shouldn't need to keep track of the specifics of how libraries need to be compiled and linked against.

The problem with pkg-config is that it has proliferated dependencies. It uses glib which now in turn requires PCRE and python. There are only two packages in the base LFS system that use pkg-config: E2fsprogs and man-db; and they have relatively simple work arounds.

I do think that pkg-config should be built early in the BLFS process of fleshing out a system, but it can be delayed until needed there.

[bdubbs@…]

Replying to matthew@…:

That sounds reasonable for now. I wish there was 'one true' automated testing software, because now it looks as if we need 'expect', 'dejagnu', 'autotest' and 'check' to get full test coverage in LFS.

Matt, I'm glad you brought up 'expect' and 'dejagnu'. There is also 'tcl'. We build those as a part of the testing tools in Chapter 5. I'm not opposed to adding 'check' to Chapter 5 (and BLFS) and adding the commands needed to run the libpipeline checks in Chapter 6.

[Matthew Burgess]

Replying to bdubbs@…:

I'm not opposed to adding 'check' to Chapter 5 (and BLFS) and adding the commands needed to run the libpipeline checks in Chapter 6.

That sounds like a good idea to me!

[bdubbs@…]

Added Check-1.2.0 and incorporated it into the libpipeline 'make check' procedure.

Fixed at revision 9526.