tcl-9.0.0 (wait for expect/dejagnu adaption)

[Xi Ruoyao]

New major version.

[Xi Ruoyao]

Highlights of Tcl 9.0

• 64-bit Capacity: Data values larger than 2Gb
• Unicode and Encodings: full codepoint range, added encodings, encoding profiles to govern I/O, and more.
• Zip Filesystems: mount zipfiles as filesystems
• Attached Archives: enable starkit-style deployment of apps, with support data in filesystem archives attached to executable or libraries. Build tclsh and wish this way.
• New Notifiers: The central event handling engine in Tcl is now constructed on top of the system calls epoll or kqueue when they are available. The select based implementation also remains for platforms where they are not.
• Many new commands and features

Important Incompatibilities in Tcl 9.0

• Namespace varname resolution: Current namespace, not global.
• I/O malencoding: now raises error by default.
• Tilde (~) in pathnames: no longer interpreted as home directory.
• tcl_precision no longer has effect on number formatting

[Xi Ruoyao]

On my system it seems using /usr/bin/zip. Not sure what will happen in the LFS chroot environment.

[Xi Ruoyao]

On another system w/o zip installed, the build fails because when building the thread sub-package the building system attempts to find "minizip" at a wrong location. It can be worked around with make NATIVE_ZIP=$PWD/minizip (i.e. overriding the relative path NATIVE_ZIP=./minizip in the Makefile).

Maybe we should report this to upstream.

[Xi Ruoyao]

The sed expression

-e "s|$SRCDIR/pkgs/tdbc1.1.9/library|/usr/lib/tcl8.6|"

contains hard-coded "8.6" and it seems just wrong even with tcl-8.6.x (IIRC someone has complained via lfs-support or dev). It looks like /usr/lib/tcl8.6 should be /usr/lib/tdbc1.1.9 (where we can find tdbc.tcl) instead.

[Xi Ruoyao]

12 tests require an Internet connection, so it looks like they'll fail in the chroot.

[Xi Ruoyao]

Expect needs to be ported.

[Xi Ruoyao]

I can get Expect build and its test suite pass. But then DejaGNU test suite blows up with some cryptic message:

UpdateStringProc should not be invoked for type localVarName

I don't know how to resolve.

[Xi Ruoyao]

Replying to Xi Ruoyao:

I can get Expect build and its test suite pass. But then DejaGNU test suite blows up with some cryptic message:

UpdateStringProc should not be invoked for type localVarName

I don't know how to resolve.

It's actually caused by a Use After Free I carelessly introduced when I ported the code. I've fixed it and now with the attached patch (on top of the gcc 14 patch) for expect, and sed s/encoding/translation/ -i lib/utils.exp for dejagnu, all dejagnu tests pass.

[Xi Ruoyao]

Replying to Xi Ruoyao:

Replying to Xi Ruoyao:

I can get Expect build and its test suite pass. But then DejaGNU test suite blows up with some cryptic message:

UpdateStringProc should not be invoked for type localVarName

I don't know how to resolve.

It's actually caused by a Use After Free I carelessly introduced when I ported the code. I've fixed it and now with the attached patch (on top of the gcc 14 patch) for expect, and sed s/encoding/translation/ -i lib/utils.exp for dejagnu, all dejagnu tests pass.

Binutils test suite is fine but GCC test suite blows up with something like

ERROR: tcl error sourcing /home/xry111/sources/lfs/gcc-14.2.0/gcc/testsuite/gcc.dg/cpp/cpp.exp.
ERROR: tcl error code NONE
ERROR: error reading "file8": invalid or incomplete multibyte or wide character
    while executing
"gets $fd cur_line"
    (procedure "grep" line 20)
    invoked from within
"grep $prog "{\[ \t\]\+dg-\[-a-z\]\+\[ \t\]\+.*\[ \t\]\+}" line"
    (procedure "dg-get-options" line 4)
    invoked from within
"dg-get-options $prog"
    (procedure "saved-dg-test" line 75)
    invoked from within
"saved-dg-test /home/xry111/sources/lfs/gcc-14.2.0/gcc/testsuite/c-c++-common/cpp/Winvalid-utf8-1.c { -Wc++-compat } {}"
    ("eval" body line 1)
    invoked from within
"eval saved-dg-test $args "
    (procedure "dg-test" line 4)
    invoked from within
"dg-test $testcase $options ${default-extra-options}"
    (procedure "dg-runtest" line 10)
    invoked from within
"dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cpp/*.{c,S} ]] \
	" -Wc++-compat " """
    (file "/home/xry111/sources/lfs/gcc-14.2.0/gcc/testsuite/gcc.dg/cpp/cpp.exp" line 43)
    invoked from within
"source /home/xry111/sources/lfs/gcc-14.2.0/gcc/testsuite/gcc.dg/cpp/cpp.exp"
    ("uplevel" body line 1)
    invoked from within
"uplevel #0 source /home/xry111/sources/lfs/gcc-14.2.0/gcc/testsuite/gcc.dg/cpp/cpp.exp"
    invoked from within
"catch "uplevel #0 source $test_file_name" msg"

Still needs to investigate.

[Xi Ruoyao]

Replying to Xi Ruoyao:

Binutils test suite is fine but GCC test suite blows up with something like

ERROR: error reading "file8": invalid or incomplete multibyte or wide character

Still needs to investigate.

This just seems the "I/O malencoding: now raises error by default." in the release note.

My current idea is redefining the Tcl open builtin (and maybe others) in expect and override "the default" there.

I've seen no simple way (flag or environment variable) to override it globally.

[Bruce Dubbs]

Fixed at commit 8a9d779178

Update to Python3-3.12.7.
Update to tcl9.0.0.
Update to linux-6.11.1.
Update to libtool-2.5.3.
Update to iproute2-6.11.0.
Update to bash-5.2.37.
Update to bc-7.0.3.

[Xi Ruoyao]

​https://github.com/xry111/expect-tcl9

4 commits after the draft patch. With them GCC test results seem at least "reasonable."

[Xi Ruoyao]

Replying to Xi Ruoyao:

​https://github.com/xry111/expect-tcl9

4 commits after the draft patch. With them GCC test results seem at least "reasonable."

Still not reliable. There are things like

libgomp11/libgomp.sum.sep: no recognised summary line
libgomp11/libgomp.log.sep: no recognised summary line

and libgomp test results are completely gone. coredumpctl shows the /usr/bin/expect process generating libgomp11/libgomp.{log,sum}.sep has crashed.

The crash is in the epoll code newly added in Tcl 9. I guess it's because expect closes some fd too early, while doing so was "harmless" with Tcl 8 but it is now breaking with Tcl 9.

[Xi Ruoyao]

I guess I need strace to debug this issue. Is there an instruction to install strace on BLFS?

[Xi Ruoyao]

Replying to Xi Ruoyao:

I guess I need strace to debug this issue. Is there an instruction to install strace on BLFS?

Forget it. strace won't be so useful before I can find a minimal test case reproducing the issue...

I've had enough of this expect vs. tcl9 thing today. I'll go to sleep and maybe continue the debugging when I have the mood...

Let's also monitor ​https://bugzilla.redhat.com/show_bug.cgi?id=2315280 etc. to see how other distros handle this.

[Bruce Dubbs]

Just FYI, my latest strace build (version 6.6) is:

./configure --prefix=/usr --with-libunwind --enable-mpers=no
make 
make install

Checking upstream, the latest version is 6.11.

[Xi Ruoyao]

We've rolled back to 8.6.15.

[Bruce Dubbs]

I checked what arch says uses tcl.

LFS:

expect
dejagnu (check)
gcc (check)
vim (make)

BLFS

tk
graphviz (optional)
libnewt (optional)
postgresql (make)
sqlite (make)
swig (check)

[Xi Ruoyao]

Hmm I was too stupid to type the subject correct.