Opened 6 years ago

Closed 6 years ago

#8045 closed enhancement (wontfix)

texlive and tlpkg

Reported by: Armin K Owned by: ken@…
Priority: normal Milestone: 7.10
Component: BOOK Version: SVN
Severity: normal Keywords:
Cc:

Description

I was pointed out that some parts of texlive don't work as expected because some perl modules were missing (tlmgr is the program).

The book (and my scripts) manually copy two perl modules to $INSTALLDIR/tlpkg. However, those two files aren't enough.

I have found out that ftp://tug.org/texlive/historic/2016/texlive-20160523-extra.tar.xz contains tlpkg directory and contains the two perl modules plus the additional perl modules required by other parts of texlive (tlmgr works after that though).

I think the extra tarball should be included, too.

Change History (10)

comment:1 by ken@…, 6 years ago

Thanks for passing-on this report. I had assumed it was only the two of us who build texlive from source ;-)

Confirmed: TLPDB, TLPOBJ, TLWinGoo, TLDownload, TLConfFile, TLCrypto are all needed by tlmgr. And for anybody wanting to use tlmgr to install updates, gnupg2 should also be recommended, as I already did for the binary.

I'm out of cycles at the moment - in the unlikely event that anybody else here cares, feel free to take this.

comment:2 by ken@…, 6 years ago

Please define "works". I can now run

tlmgr --version
tlmgr --help

but if I try to use it

tlmgr update --list
/opt/texlive/2016/bin/x86_64-linux/tlmgr: open(/opt/texlive/2016/tlpkg/texlive.tlpdb) failed: No such file or directory at /opt/texlive/2016/tlpkg/TeXLive/TLPDB.pm line 415.

The file texlive.tlpdb would contain the version details for every installed package. I have attempted to use tlmgr update self : that initially gave the same error, but after I touched that file it failed with:

cannot setup TLPDB in /opt/texlive/2016 at /opt/texlive/2016/bin/x86_64-linux/tlmgr line 5753.

I suspect that the tlpdb file, containing the versions shipped by upstream, is needed to be able to sensibly look for updates - and there is no obvious way of creating it.

Also, 'tlmgr --info somepackage' fails similarly.

There are suggestions that tlmgr init-usertree might be useful (intended for where the user cannot write to TEXMFHOME), but the manpage says " In user mode the file usertree"/tlpkg/texlive.tlpdb" contains only the packages that have been installed into the user tree using "tlmgr", plus additional options from the virtual package "00texlive.installation" (similar to the main installation's "texlive.tlpdb"), so since everything is already installed I think that option will not be useful.

Unless somebody provides details of how to make this useful, I cannot fix it. Note that this problem only applies to the from-source build.

comment:3 by ken@…, 6 years ago

Think I've found it.

comment:4 by ken@…, 6 years ago

Owner: changed from blfs-book@… to ken@…
Status: newassigned

Seemed to be so close, but back to obscure errors when TLUtils.pm tries to run config.guess, although I can now run it myself (copied from the texlive source to the expected location). Giving up for the moment.

If I do manage to crack that piece of weirdness, and if it then works, I've still got to document the process for getting the correct version of texlive.tlpdb, otherwise we'll have to do this again next year.

Taking the ticket, even if my current work on it will be just thinking and swearing.

comment:5 by Armin K, 6 years ago

If you want to use tlmgr to install packages system wide (into tl install prefix), then you need to fetch the mentioned file manually. It's available on tug.org or any mirror.

I'd recommend using "user mode", where tlmgr would fetch the db itself, and install packges for specific user, so you don't need to mess with the system wide tree, and can revert in case something goes wrong.

in reply to:  5 comment:6 by ken@…, 6 years ago

Replying to Krejzi:

If you want to use tlmgr to install packages system wide (into tl install prefix), then you need to fetch the mentioned file manually. It's available on tug.org or any mirror.

I'd recommend using "user mode", where tlmgr would fetch the db itself, and install packges for specific user, so you don't need to mess with the system wide tree, and can revert in case something goes wrong.

I haven't managed to make any sense of user mode (see above), but the tlpdb matching the 2016 release is now at ftp://tug.org/historic/systems/texlive/2016/texlive-20160523-tlpdb.tar.gz thanks to a query from one of the SuSe devs.

I've still got the config.guess problem, and I suspect some of the postcode/ (i.e. post install) scripts might be needed by some updates.

I suppose I can take a wider look at user-mode.

comment:7 by ken@…, 6 years ago

I think I've got tlmgr working.

tlmgr option repository http://mirror.ctan.org/systems/texlive/tlnet

and then

tlmgr update --self

gave a lot of messages but appeared to work.

I managed to get win32 binaries by updating xetex (good job I didn't try updating lollipop, which has binary updates for most OS/arch combinations), but fortunately my xetex binary still has a timestamp from 26th June like the rest of my from-source TeX progs. Using tlmgr will update binaries if you let it, but I'm at a loss why it didn't limit itself to x86_64-linux which is what I thought was happening when I last tested it on a binary texlive install.

I saw 'update --list' showed an update for fontspec, which I am using, so I updated that but it appears to be from the same date as what I already had, and comparing the backup to what is now installed shows a slightly different _time_ on 1st Feb (was 19:47, now 23:06) but the same content. An example document which uses fontspec still compiles.

'tlmgr paper' works.

To get to here I had to install config.guess from the texlive source. That needs a change to the source install, but for the rest I am minded to add an optional page for 'tlmgr' to the source install (install perl modules, postcode (i.e. post-install stuff), create some tlmgr directories).

I have not yet looked at user mode, nor have I tried doing the full source install followed by what I think will now work for 'tlmgr' - I had to add one extra directory beyond what was in my script and for the moment I'm not close to updating my scripts for a fresh build - getting tlmgr to this point was blocking that.

comment:8 by ken@…, 6 years ago

I've given this another try, and played with user mode, but I have NO CONFIDENCE that this is doing anything useful, and I intend to close this as WONTFIX, subject to any further comments.

Build changes to get to here:

  1. in texlive itself, at the end (copied from my script, but it should be understandable):
mkdir -p /opt/texlive/${TEXYEAR}/tlpkg/installer
install -v -m644 ../build-aux/config.guess /opt/texlive/${TEXYEAR}/tlpkg/installer/

That is so that the perl modules can use config.guess - that ought to limit tlmgr to x86_64-linux updates in my case, but it does not.

  1. extract the texlive-20160523-extra tarball, cd into it, copy files and create directories
    cp -v tlpkg/TeXLive/* /opt/texlive/${TEXYEAR}/tlpkg/TeXLive/
    cp -av tlpkg/tlpostcode /opt/texlive/${TEXYEAR}/tlpkg/
    mkdir -pv /opt/texlive/${TEXYEAR}/tlpkg/{backups,tlpobj}
    
  1. extract the tlpdb whic h matches the source:
    tar -xf texlive-20160523-tlpdb.tar.gz  -C /opt/texlive/${TEXYEAR}/tlpkg/
    

comment:9 by ken@…, 6 years ago

And now some comments on using it. I have three main reservations: the real packaging files are only installed during the binary install, using tlmgr can install binary programs, and the main purpose of using it in usermode seems to be to add extra packages or collections not already installed - but our source build installed everything except asy, biber, xindy.

I have run the following commands:

  1. tlmgr option repository http://mirror.ctan.org/systems/texlive/tlnet

That successfully pointed it to a local mirror (by default it thinks it installed from the DVD).

  1. tlmgr platform list:
    tlmgr: package repository http://mirror.ox.ac.uk/sites/ctan.org/systems/texlive/tlnet (not verified: gnupg not available)
    Available platforms:
    (i) amd64-freebsd
    (i) amd64-netbsd
        armel-linux
        armhf-linux
    (i) i386-cygwin
    (i) i386-freebsd
    (i) i386-linux
    (i) i386-netbsd
    (i) i386-solaris
        powerpc-linux
    (i) sparc-solaris
    (i) universal-darwin
    (i) win32
    (i) x86_64-cygwin
    (i) x86_64-darwin
    (i) x86_64-linux
    (i) x86_64-solaris
    Already installed platforms are marked with (i)
    You can add new platforms with: tlmgr platform add ARCH1 ARCH2...
    

That was "a bit of a surprise", but I cleaned it up with

3.

tlmgr platform remove amd64-freebsd amd64-netbsd i386-cygwin i386-freebsd i386-linux i386-netbsd i386-solaris sparc-solaris universal-darwin win32 x86_64-cygwin x86_64-darwin x86_64-solaris

That claims to remove a lot of things, takes a VERY long time and reports errors at end. starting with

tlmgr: binary directory /opt/texlive/2016/bin/amd64-freebsd not empty after removal of amd64-freebsd.

Because this is a source build, only x86_64-linux actually exists under bin/.

I then tried running it as a user, after creating a directory I could write to:

  1. tlmgr init-usertree --usertree /scratch/ken/texmf
  1. tlmgr --usermode --usertree /scratch/ken/texmf update --list | less
    tlmgr: backupdir as set in tlpdb
      /scratch/ken/texmf/tlpkg/backups
    is not a directory.
    
  1. mkdir -pv /scratch/ken/texmf/tlpkg/backups

Retry

  1. tlmgr --usermode --usertree /scratch/ken/texmf update --list | less
    tlmgr: no updates available
    

At this point, I gave up on usermode, having already noted that a lot of updates appeared to be available for root.

So, as root:

  1. tlmgr update collection-xetex

That reported that tlmgr itself needed to be updated, so

tlmgr update --self and then repeat tlmgr update collection-xetex

I then looked at the dates on the files, but the only things updated were ls-lR and fmtutil.cnf so I have zero confidence that this process is useful.

comment:10 by ken@…, 6 years ago

Resolution: wontfix
Status: assignedclosed
Note: See TracTickets for help on using tickets.