Tcl-8.5.4

[Matthew Burgess]

New version. Obviously, the Tcl-8.4.17 release is probably very low risk. Does anyone have any experience in building LFS and/or BLFS with Tcl-8.5.0 yet? Is there an easy way to extract what can/does make use of Tcl in BLFS to test how big an impact this will have?

[randy@…]

I don't think the connection between LFS and BLFS really matters here. Sure it would be nice if the Tcl/Tk versions were the same in both books, but because Tcl is not permanantly installed in LFS, you could actually use 8.5.0 in LFS and BLFS still using the 8.4 series.

I'm not sure what would be a good test for an update of BLFS to 8.5.0, but on the 8.5 'Changes' page, it doesn't appear that any mainline (other than the 'text' widget) functions should have much difference.

Here's a page I found helpful: ​http://wiki.tcl.tk/20361

[Matthew Burgess]

Thanks Randy. I'd forgotten we don't actually install Expect or Tcl in chapter 6. The first problem I encountered is that Expect doesn't build against Tcl-8.5.0. See ​https://sourceforge.net/tracker/?func=detail&atid=113179&aid=1634110&group_id=13179 for the upstream report. The fix is simply:

cp exp_inter.c{,.bak}
sed 's/tcl.h/tclInt.h/' exp_inter.c.bak > exp_inter.c

This is already fixed upstream, so on the next release of Expect, whenever that might be, the workaround can be dropped.

Now I've got a problem in that when compiling Dbg.c I get the following:

gcc -c -I. -I. -I/tools/include -DEXP_VERSION=\"5.43.0\" -DSCRIPTDIR=\"/tools/lib/expect5.43\" -DEXECSCRIPTDIR=\"/tools/lib/expect5.43\" -DTCL_DEBUGGER -DUSE_NON_CONST -DSTTY_BIN=\"/bin/stty\" -DDFLT_STTY="\"sane\"" Dbg.c In file included from /tools/include/tclPort.h:27,

from /tools/include/tclInt.h:3791, from Dbg.c:25:

/tools/include/tclUnixPort.h:109:33: error: ../compat/unistd.h: No such file or directory make[1]: * [Dbg.o] Error 1

This is caused by HAVE_UNISTD_H not being defined (which causes tclUnixPort.h to pull in the compat header rather than the system header). I can't trace through the Makefile/configure magic to see why this is a problem now - tclUnixPort.h hasn't changed in this regard between 8.4.x and 8.5.x.

[bryan@linuxfromscratch.org]

The relevant change was in unix/tcl.m4. In Tcl 8.4.17, the SC_MISSING_POSIX_HEADERS macro checks for a different set of headers than in Tcl 8.5.0 -- in 8.4.17, it's possible for that macro to AC_DEFINE both NO_ERRNO_H and HAVE_UNISTD_H (if the relevant header is either missing or present), but neither of those is possible in 8.5.0. Of course, on Linux, NO_ERRNO_H isn't supposed to be defined (since the file does exist), so this one won't cause a problem. But on any OS that does have a unistd.h, if HAVE_UNISTD_H doesn't get AC_DEFINE'd, but other headers use that symbol (like tclUnixPort.h), then Tcl will have a problem.

The CVS checkin that made this change is ​here. From the log, it sounds like the idea was to "consolidate calls to limit redundancy in configure", although I'm not exactly sure what those calls are to. In any case, it looks like this needs to be fixed somehow or other.

There are a couple solutions I can think of. First, we could patch the unistd.h header check back into unix/tcl.m4. The downside to this is that we'd have to rerun all the autofoo on the package, and since Tcl is one of the first packages in chapter 6, I'm not sure I'd want to rely on the host's autofoo.

Second, we could patch unix/tclUnixPort.h to just assume that HAVE_UNISTD_H is defined, since that seems to be the intent of the change. We'd also want to patch the value of ac_includes_default in unix/configure to either always include <unistd.h>, or never include it -- I'd recommend always including it. I can see broken configure tests if that header is needed but isn't included (in other words, I'm not sure that the 8.5.0 configure script is detecting everything properly).

It may also be possible to add a -DHAVE_UNISTD_H to the compilation command line, but I think we'd have to do that to every compilation command. So it'd be easier to either keep doing the test, or remove any reference to the symbol.

[Matthew Burgess]

Thanks for the quick analysis, Bryan. Is there any chance you could report this upstream please and ask for clarification of the intent of the commit you pointed to? You're certainly more qualified than myself to get an accurate description of the problem to them.

Thanks again!

[bryan@linuxfromscratch.org]

OK, done. We'll see what they say. :-)

[Matthew Burgess]

Even after hacking that #include so it always pulls in <unistd.h>, I get this:

gcc -Wl,--export-dynamic -o expect exp_main_exp.o libexpect5.43.a -L/home/matthew/tools/lib -ltcl8.5 -ldl -lieee -lm -lutil -Wl,-rpath,/home/matthew/tools/lib:/home/matthew/tools/lib libexpect5.43.a(exp_command.o): In function `Exp_CloseObjCmd': exp_command.c:(.text+0x3b93): undefined reference to `Tcl_CloseObjCmd' libexpect5.43.a(exp_command.o): In function `Exp_InterReturnObjCmd': exp_command.c:(.text+0x4bb0): undefined reference to `Tcl_ReturnObjCmd' collect2: ld returned 1 exit status make: * [expect] Error 1

The definitions of both functions are in tclInt.h which is included by exp_command.c. So is this now a missing library on the command line?

[bryan@linuxfromscratch.org]

Yep, that's a classic missing-library type problem ("undefined reference" errors from the linker). According to nm, those symbols are defined in <tcl prefix>/lib/libtcl8.5.so, and my current system (which was built using Tcl 8.4) has the same /tools/lib/libtcl8.4.so file with the same symbols in it. Of course, the command you posted *does* link against -ltcl8.5, so this *should* all be working fine...

Oh, hang on, there is one difference between 8.4 and 8.5 that I see: Running nm on the 8.5 library shows these two symbols with a "t" before them (i.e. they're in the text (code) section), but running it on the 8.4 library shows these symbols with a "T" before them (which also means in the text section). According to the nm manpage, uppercase means the symbol is "global (external)", and lowercase means it's "local"; I assume that "local" means other code can't link to it, and if that's true, that explains the problem with Expect.

Now, as for why Tcl made those symbols "local", and how that flag gets set: I'm pretty sure it's some gcc attribute or other, but I'm not sure which one. Looking at the CVS history for generic/tclInt.h, it looks like Tcl_CloseObjCmd has been declared MODULE_SCOPE since about August 2004 (before that it was EXTERN), so the usage of MODULE_SCOPE itself isn't new. And MODULE_SCOPE is defined in generic/tclInt.h to be "extern" if it's not already defined, and "extern" would not cause the symbol to be local. So I'm guessing that macro is already defined somewhere else in 8.5...

Searching through the rest of the Tcl files for MODULE_SCOPE shows usages all over the place, but only a few definitions; the relevant one is in unix/tcl.m4:

    AS_IF([test $tcl_cv_cc_visibility_hidden = yes], [
	AC_DEFINE(MODULE_SCOPE,
	    [extern __attribute__((__visibility__("hidden")))],
	    [Compiler support for module scope symbols])
    ])

which sets it to use the __visibility__ attribute if that $tcl_cv_cc_... variable is set. That variable gets set earlier, via the autoconf cache and a simple link-test program, so if we need to, we can override this definition by adding:

echo tcl_cv_cc_visibility_hidden=no >config.cache

before running configure. (We may also need to add a --cache-file=config.cache or similar to the configure command line.)

This is all new code as of ​revision 1.180. Before that, only Mac machines would have MODULE_SCOPE defined to anything at all (which means that generic/tclInt.h would set it to just plain "extern" on Linux). Apparently nobody uses Expect on Macs? Hmm...

Anyway, there are probably a couple of options here too. First would be to override the check for the hidden-visibility support (make configure think it's not supported, so it doesn't use it), but that's *really* not what the Tcl people intend. Plus I'm not sure whether anything else in the Tcl package will use that value -- nothing does right now, but that doesn't rule it out in the future either. (Besides, Expect really only needs those two functions.)

Second would be to remove the MODULE_SCOPE from the declaration of those two functions, and replace it with EXTERN (before compiling Tcl). This would also not be what I expect the Tcl people intend, but it would be a lot more minimal of a change than neutering all the MODULE_SCOPE declarations.

Third would be to see if there's a way to patch Expect so that it doesn't need these symbols -- that would probably be what the Tcl team would recommend, but I don't know if it's possible or not. Let me do some more digging in Expect...

[bryan@linuxfromscratch.org]

OK, I can get rid of the reference to Tcl_CloseObjCmd by calling a public function that will return the pointer to that function (and then storing it away) right before the Expect code that registers its own function to handle Tcl "close" commands. (This is wrong -- Expect should be putting its code in its own namespace, IMO -- but they can't do that now that they've already released so many versions that have their "close" in the root namespace. That's unfortunate, but not something we can do anything about.)

Tcl_ReturnObjCmd is different -- the Expect handler that uses this function isn't an override of a builtin Tcl command. But it calls into the private Tcl function anyway, because it wants to use that implementation of "return". That's fairly easy to fix, though: just call Tcl_GetCommandInfo("return") (to get the info on Tcl's "return" command), then call the function given by that function.

Patch is attached. This can probably be merged into the existing Expect spawn patch, since it touches the same file. With this patch (and the munging of tclUnixPort.h from Tcl, and after adding the include of tclInt.h instead of tcl.h), Expect compiles successfully. I have not tried running any testsuites with it yet, though.

[Matthew Burgess]

Thanks again, Bryan. I'm wondering whether this is worth bothering with at this point.

There have been some recent commits to Expect that get it to compile with Tcl-8.5.0, so I think we should try getting it to compile/test correctly ourselves as well, so that when upstream release a new version we know it's a relatively simple job. To that end, I checked out Expect CVS Head and hit the problem described at ​http://sourceforge.net/tracker/index.php?func=detail&aid=1858698&group_id=13179&atid=113179. My analysis is in the comment dated 2008-01-12 04:57.

[dnicholson@…]

The expect problem appears to be in the TEA_PATH_TKCONFIG autoconf macro.

​http://expect.cvs.sourceforge.net/expect/expect/tclconfig/tcl.m4?view=markup

Basically, the whole macro after AC_ARG_WITH(tk,... needs to be wrapped in:

if test x"${with_tkconfig}" != xno ; then
   ...do the tk checks...
fi

And then some sort of conditional in the Makefile so expectk does or doesn't get built.

[bryan@linuxfromscratch.org]

Ah -- I was wondering why I didn't see that error, but of course the Expect-5.43.0 configure script doesn't have that issue, and that's the version I was testing with. Duh.

Anyway, we can always upgrade to Tcl-8.4.17 for now (and leave Expect alone), and save Tcl-8.5.0/new-Expect for later (after they get this stuff figured out).

[Matthew Burgess]

Due to the number of issues with Tcl-8.5.0 & Expect, I'll split this into two tickets. This one can continue to track the 8.5.x series, and the new one can handle 8.4.17.

[Matthew Burgess]

Now 2.5.1. ChangeLog at ​http://sourceforge.net/project/shownotes.php?group_id=10894&release_id=573933

[Jeremy Huntwork]

Now 8.5.2. Changelog at ​http://sourceforge.net/project/shownotes.php?group_id=10894&release_id=587805

[randy@…]

Updated to Tcl-8.5.4 in r8569 and r8571