Archive Python 2 (Wait until BLFS-12.1 is released)

[Pierre Labastie]

Sept 30, 2023

The following references to python2 are in the book. Those marked optional can be replaced with an external reference. Really the only thing preventing archiving Python2 is gimp via pygtk.

multimedia/libdriv/alsa-lib.xml:             optional
multimedia/libdriv/libmusicbrainz.xml:       optional
general/prog/yasm.xml:                       optional
general/graphlib/lcms.xml:                   optional
general/graphlib/opencv.xml:                 optional
x/lib/libglade.xml:                          optional
pst/printing/cups.xml:                       optional
networking/netlibs/ldns.xml:                 optional
server/other/unbound.xml:                    optional

general/prog/python-modules/pycairo2.xml: 
  pygtk
  pygobject2

general/prog/python-modules/pygtk.xml:
  x/wm/openbox.xml:     referenced in configuration
  /netutils/avahi.xml:  optional
  xsoft/other/gimp.xml  recommended

general/prog/python-modules/pygobject2.xml: 
  pygtk

general/prog/python-modules/pyxdg.xml:      
  openbox (optional)

Older comments:

After working on the Python module page and some packages which depend on those module, I've realized that simple tweaks could remove the dependency on Python 2. Sometimes, it is as simple as changing #!/usr/bin/env python to #!/usr/bin/env python3 in some programs, which are otherwise Python 3 compatible. Sometimes, running 2to3 on some specific programs may be required.

This ticket is for showing advances in this respect. It is not expected to be fully completed for 8.4.

[Pierre Labastie]

List of Packages needing Python 2, with dependency status

[Pierre Labastie]

Seems six is not directly needed by libtasn1 and gtk+3.

Furthermore, all the executable installed by gtk-doc have #!/usr/bin/python3. Actually, the NEWS file says:

NEWS:GTK-Doc now requires python3.X. It does not require python-six anymore

[Pierre Labastie]

gtk-doc fixed at r21014

[Pierre Labastie]

NetworkManager can be made P3 only by running:

grep -rl '^#!.*python' | xargs sed -i '1s/python/&3/'

I need to test the Qt examples before committing.

libbytesize is now P3 only: done at r21022.

[Pierre Labastie]

I've committed the fix to NetworkManager at r21025. It seems that the Qt examples are built but not installed. I do not know whether it is a problem with using P3 or with meson/ninja: there are no errors in the log.

[Pierre Labastie]

for volume_key: in any case, python is not required. Python is used for building bindings to the volume_key library, using swig. Both P2 and P3 bindings can be built. those can be disabled by using --without-python and --without-python3 respectively. Note that swig is not tested by configure: if a Python executable is found, and the corresponding --without-python flag has not been passed, swig will be run, and the build will stop if swig is not installed.

The python bindings are not needed in the book (the only user of volume_key is libblockdev, which uses directly the shared library libvolume_key.so, so I think:

P2 and swig should be made optional, and both --without-python and --without-python3 should be added to the configure invocation.

Note that gnupg2 can be optionally used, and should be added to optional deps (it is not required nor recommended for gpgme).

[Pierre Labastie]

Hmm, the configure options for libblockdev have --with-python3. Again, this is just for building python bindings, which are not used in the book, I think (I'll try to check anyway, but I think udisks2 does not use python, and it is the only user of libblockdev).

[Douglas R. Reno]

I think that optional python bindings should be built irregardless of whether or not they are actually used by anything in the book. It simplifies maintenance in the future if they are actually required at some point, and doesn't hurt anything.

Also - Upstream is good at hiding dependencies on python bindings - especially in GNOME. Udisks2 does use Python internally, as does gvfs.

[Pierre Labastie]

On my way to testing udisks2, I've found that asciidoc requires P2 at run time, and that the asciidoc scripts cannot be converted easily to P3 (2to3 is not enough). For now, I'll disable building the docs in btrfs-progs, but the asciidoc page on the book has to be modified to make P2 required (runtime).

[Pierre Labastie]

Replying to renodr:

I think that optional python bindings should be built irregardless of whether or not they are actually used by anything in the book. It simplifies maintenance in the future if they are actually required at some point, and doesn't hurt anything.

Well, in theory, I agree with this. In practice, it is not always simple (python bindings for gpgme require gnupg2 and swig to be present, python bindings for volume_key require swig, etc). I'll try my best to make everything coherent... But I think that hidden things is not what we want in blfs: the whole object of lfs/blfs is to control what is on the machine. We should try not to give up on this, otherwise, there will be no difference with any other distro...

Also - Upstream is good at hiding dependencies on python bindings - especially in GNOME. Udisks2 does use Python internally, as does gvfs.

Actually, Udisks2 tests seem to require pygobject for tests (the gi module). This is not noted in the book (will investigate).

[Pierre Labastie]

Stupid mozilla js requires P2. So P2 will be present anyway when building Udisks2.

[Bruce Dubbs]

Replying to renodr:

I think that optional python bindings should be built irregardless of whether or not they are actually used by anything in the book. It simplifies maintenance in the future if they are actually required at some point, and doesn't hurt anything.

Also - Upstream is good at hiding dependencies on python bindings - especially in GNOME. Udisks2 does use Python internally, as does gvfs.

Unless we know about a P2 dependency, then we can omit it by default if it takes a command to include it. I'd say leave in the P3 bindings as we have P3 already in LFS, but make P2 optional and don't take explicit action to disable the P2 bindings. Looking at ./configure --help for volume_key, it looks like picking up P2 is automatic if it is present, so we probably only need to move P2 to optional.

[Pierre Labastie]

Replying to bdubbs:

Unless we know about a P2 dependency, then we can omit it by default if it takes a command to include it. I'd say leave in the P3 bindings as we have P3 already in LFS, but make P2 optional and don't take explicit action to disable the P2 bindings.

There is an issue with this approach:

• suppose P2 is present. The package will then try to build say, bindings for P2. If in the process, a python module is needed, this module must have been built with P2. This means that all the python modules would have to be built for P2 as well a for P3. I'd rather avoid that.

I think it's better to try to prevent P2 to come into the picture each time it is possible: otherwise, we'll always have to build P2 bindings (and so recommend P2), "just in case".

Looking at ./configure --help for volume_key, it looks like picking up P2 is automatic if it is present, so we probably only need to move P2 to optional.

But the problem is, in this case (for both P2 and P3 actually): if the build system wants to build the python bindings, it'll need swig, and since swig is not tested by configure, it will fail during the "make" stage. OK, let's do it this way:

• P2 optional, swig recommended, others required as already noted. pass --without-python (disable only P2) and comment about the --without-python{,3} in command explanations.

[Bruce Dubbs]

We can prevent the modules from being built but we need to explain the complexities of the inter dependencies in a note (or similar). If we don't need any of the modules in BLFS, I'd rather list swig as optional rather than recommended.

[Pierre Labastie]

Well, let's discuss some sort of policy (I do not have the whole picture yet, so may need adjustments later). From Douglas and Bruce inputs, I'd say it could be:

• Python modules: always build for P3, except when impossible (pygtk). Build for P2 only when really needed by something in the book (normally, already implemented).
• Other applications:
  • Python bindings: don't prevent building bindings for P3 (but do not build them if upstream default is to not build them), and always prevent building them for P2, unless the package itself or a dependent package require P2.
  • Python apps: build only for P3 as much as possible. Build for P2 only when upstream is too lazy has been too busy to take the time to switch to P3.

[Bruce Dubbs]

Sounds good to me.

[ken@…]

And to me.

[Pierre Labastie]

Thanks Ken and Bruce.

For asciidoc, I've found: ​https://github.com/asciidoc/asciidoc-py3. No release yet. There is also ​https://asciidoc3.org/. Looks like they are independent forks. The latter has releases, but the "historical" devs of asciidoc are on the former.

[Pierre Labastie]

Qemu can use P3. Done at r21039

[Pierre Labastie]

nodejs really needs P2. No way to escape...

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

One rustc test needs python2.7 (sysroot-crates-are-unstable). Otherwise, rustc can be made P3 only, by changing any occurrence of ./x.py to python3 ./x.py in the current book instructions (not tried the custom rust page, but I expect the same results, since only the installation dir is changed). In this case, and if gdb is present and built without python, 7 tests fail. I'd say it is still acceptable. Anyway, I won't change rustc instructions unless instructed by Ken or Bruce...

[Pierre Labastie]

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

[ken@…]

Replying to pierre.labastie:

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Can you be specific about which is the fourth, and is it different from the three certain and one possible already specified ?

As to building rustc without python3 - we did, for a while. I reverted back to python2 on 6th January, because of #11520. That was with 1.31.1. As I noted earlier, fedora have a patch for python3 for (I think) only one of the rust files. Somebody, probably Thanos, pointed out that some of the rustc scripts specify 2.7 (I think) - can't find the mail in the archive, it was probably early this month.

[Bruce Dubbs]

I note that Arch is using system llvm for rust, but patches it:

​https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/llvm

For rust they use P2:

​https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/rust

[Pierre Labastie]

Replying to ken@…:

Replying to pierre.labastie:

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Can you be specific about which is the fourth, and is it different from the three certain and one possible already specified ?

No it's not different: the four failures are issues 37131, 49851 and 50993, and sysroot-crates-are-unstable. The latter uses a python script, which explicitly requires python2.7.

As to building rustc without python3 - we did, for a while. I reverted back to python2 on 6th January, because of #11520. That was with 1.31.1. As I noted earlier, fedora have a patch for python3 for (I think) only one of the rust files. Somebody, probably Thanos, pointed out that some of the rustc scripts specify 2.7 (I think) - can't find the mail in the archive, it was probably early this month.

Well, this seems to work for this version of rust. Fedora's patch does not seem to be needed. It is in a lldb_something file, and I do not think our build makes lldb usable. I'll try building firefox and let you know if it works. Note that if changing the python version is enough to generate SIGSEV, it means that there are severe flaws in the build system... The python scripts are supposed to run compilation commands, not to define memory locations...

[ken@…]

Replying to pierre.labastie:

Replying to ken@…:

Replying to pierre.labastie:

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Can you be specific about which is the fourth, and is it different from the three certain and one possible already specified ?

No it's not different: the four failures are issues 37131, 49851 and 50993, and sysroot-crates-are-unstable. The latter uses a python script, which explicitly requires python2.7.

And all my desktop builds/upgrades build python2 at a very early stage, yet I have occasionally seen that failure (and I definitely had it with 1.32.0 and system llvm-7 on my phenom, although that install appeared to be usable).

Also, as to the gdb failures you mentioned earlier, I got those on another machine when updating 8.3, in that case python-2.7.15 was definitely installed when gdb-8.1.1 was installed, so I assumed that gdb-6.1.1 was probably too old for the tests.

I'll add a comment that three gdb tests can fail, but I'm waiting for Bruce to merge his changes.

As to building rustc without python3 - we did, for a while. I reverted back to python2 on 6th January, because of #11520. That was with 1.31.1. As I noted earlier, fedora have a patch for python3 for (I think) only one of the rust files. Somebody, probably Thanos, pointed out that some of the rustc scripts specify 2.7 (I think) - can't find the mail in the archive, it was probably early this month.

Well, this seems to work for this version of rust. Fedora's patch does not seem to be needed. It is in a lldb_something file, and I do not think our build makes lldb usable. I'll try building firefox and let you know if it works. Note that if changing the python version is enough to generate SIGSEV, it means that there are severe flaws in the build system... The python scripts are supposed to run compilation commands, not to define memory locations...

Personally, I have very poor experience of all python build-systems, and I have not found that 3 is necessarily better. Both x.py and mach havefailed for me in odd ways (x.py with both 2 and 3, mach still definitely needs 2).

But I'm not sure that I associate the change from 2 to 3 with SIGSEGV - testing 1.32.0 has been unpleasant, I've forgotten a lot of the details and I'm probably confused about the order in which I tried certain things, but for me the rust tests which failed unexpectedly with SIGSEGV were when using system LLVM-7.

[ken@…]

Replying to bdubbs:

I note that Arch is using system llvm for rust, but patches it:

​https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/llvm

For rust they use P2:

​https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/rust

I'm glad they've now patched their llvm - that was where the problem was first identified. I can't remember for certain whether they ever used P3 to build it.

And once you've merged your changes (I assume you got my mail reply mentioning some details) I'll update the list of possible test failures re gdb and then hopefully not touch rustc for a few weeks!

[ken@…]

Replying to pierre.labastie:

Well, this seems to work for this version of rust. Fedora's patch does not seem to be needed. It is in a lldb_something file, and I do not think our build makes lldb usable.

Forgot to comment on lldb : it is an external llvm tool (like clang, lld) and has at least one extra dep that is not in the book, plus although I thought I had doxygen installed it complained that was missing. But the extra dep made me stop investigating it. The rust-lldb is just a front-end for it.

I'm still *vaguely* interested in trying to build rust with thin-lto (requires clang, llvm-ar, lld) but that would need several variations, and builds of the big programs which use rustc (firefox, thunderbird) to try different settings and to see if any make a meaningful improvement in build times. Seems too much effort to be worth the time in the short term. but I've added lld to my future llvm builds in case I change my mind.

[Pierre Labastie]

Replying to ken@…:

But I'm not sure that I associate the change from 2 to 3 with SIGSEGV - testing 1.32.0 has been unpleasant, I've forgotten a lot of the details and I'm probably confused about the order in which I tried certain things, but for me the rust tests which failed unexpectedly with SIGSEGV were when using system LLVM-7.

The error reported in ticket #11520 was a SIGSEGV. I'm not sure it is the only thing which decided you to switch back to P2, but it is the only thing which is reported in that ticket...

As for using P3 for the build system: my aim with this ticket is to reduce P2 use to a minimum, that is, use P2 only for packages which need it. SO I'd like to see P3 used as much as possible. With rustc in /usr, built with P3, firefox can be built (I'm using it right now). Of course, P2 had to be installed at some point (for building nodejs actually).

I'll take responsibility to put back P3 instructions for rust ;). It really seems to me they do not hurt. It's development after all...

[Pierre Labastie]

git has P2 as recommended, but I do not see why:

• python (either 2 or 3) is almost not used in git. The only script using python outside of the contrib directory is git-p4, which is used to interact with perforce repositories. To me, it seems optional.
• git building does not need python either...

So I'll remove P2 from the recommended deps.

[Pierre Labastie]

Rustc and git done at r21045

[Pierre Labastie]

Poppler is in a somewhat inconsistent state: The only use of Python is for building the the API documentation, but gtk-doc is deemed optional, while P2 is deemed recommended. Actually, P3 can be used if one the header of make-glib-api-doc is changed from #!/usr/bin/env python to #!/usr/bin/env python3. Moved P2 to optional, and added a command explanation for doc regeneration at r21047

[Pierre Labastie]

usbutils does not need P2 anymore. lsusb.py has been ported to P3 (thanks to Douglas for the patch), and it is the only user of Python in usbutils... Removed requirement for P2 at r21049.

[Pierre Labastie]

For QtWebEngine, I do not have the same experience as Ken: it built without Python 2... (Current book instructions for both Qt5 and then QtWebEngine. Will try Falkon tomorrow

[Pierre Labastie]

Replying to pierre.labastie:

For QtWebEngine, I do not have the same experience as Ken: it built without Python 2... (Current book instructions for both Qt5 and then QtWebEngine. Will try Falkon tomorrow

Well, I may have done a mistake here. Let's do it again after really removing Python 2!

[Pierre Labastie]

Replying to pierre.labastie:

Replying to pierre.labastie:

For QtWebEngine, I do not have the same experience as Ken: it built without Python 2... (Current book instructions for both Qt5 and then QtWebEngine. Will try Falkon tomorrow

Well, I may have done a mistake here. Let's do it again after really removing Python 2!

Yeah. Python2 is needed.

[Pierre Labastie]

for tracker, Python is only needed for the "functional tests" which are disabled by default. If those tests are desired, P2 is needed. I'll move P2 to optional, and explain why it may be needed. Also, all the tests pass if in a graphical session with dbus, and tracker is installed (I guess at least the dbus service file is needed).

[Pierre Labastie]

Dpendencies on Python 2, updated

[Douglas R. Reno]

On a systemd system with Tracker pre-installed, I'm still getting the errors. I'm not sure what's causing them, I'll check pre-release

[Pierre Labastie]

Replying to renodr:

On a systemd system with Tracker pre-installed, I'm still getting the errors. I'm not sure what's causing them, I'll check pre-release

For testing, I had only openbox (no other desktop manager). Of course, it was on SysV. A few tests were skipped: maybe they aren't skipped in your environment, and they fail... I fear I have trashed the test logs. But I can redo them (it's not that long) and send them to you.

[Douglas R. Reno]

Replying to pierre.labastie:

Replying to renodr:

On a systemd system with Tracker pre-installed, I'm still getting the errors. I'm not sure what's causing them, I'll check pre-release

For testing, I had only openbox (no other desktop manager). Of course, it was on SysV. A few tests were skipped: maybe they aren't skipped in your environment, and they fail... I fear I have trashed the test logs. But I can redo them (it's not that long) and send them to you.

I would much appreciate it. Thank you!

[Pierre Labastie]

Replying to renodr:

I would much appreciate it. Thank you!

Sent through private mail. Hope I used the right address.

[Douglas R. Reno]

Replying to pierre.labastie:

Replying to renodr:

I would much appreciate it. Thank you!

Sent through private mail. Hope I used the right address.

Thanks Pierre! Alright, running the tests with the package installed results in no FAILs and no ERRORs.

[Pierre Labastie]

Replying to renodr:

Replying to pierre.labastie:

Replying to renodr:

I would much appreciate it. Thank you!

Sent through private mail. Hope I used the right address.

Thanks Pierre! Alright, running the tests with the package installed results in no FAILs and no ERRORs.

Good to know. Feel free to adapt my wording on the tracker page.

[Pierre Labastie]

Just found that libpwquality uses Python (not noted in the deps), and that default binary is python. It can be changed by passing --with-python-binary=python3 to configure. Will modify the book's page.

[Pierre Labastie]

libpwquality done at r21056

[Douglas R. Reno]

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

[Pierre Labastie]

Replying to renodr:

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

Great finding, thanks. Will give it a try later, but right now, I'm struggling through plasma, where P2 is deemed required. Up to now, I could build without P2, but some modules have been installed in /opt/kf5/lib/python3.7/site-packages. That leads to 2 issues:

• the pathappend ... PYTHONPATH in /etc/profile.d/kf5.sh is wrong.
• if P2 is present, it'll be used even if P3 is there too. We must find a way to prevent that, since P3 can be used in all cases.

[Pierre Labastie]

It looks like the cmake modules look for P3 first, then for P2. My guess is that if P3 is present (and it should, since it has been built with LFS), it will be used. So the second item in comment:45 is not true (I should have been less affirmative).

The dependency on P2 has been on the plasma page since the very beginning (almost). At the time, building P3 was optional. I think it is safe to remove dependency on P2, and to move the PYTHONPATH to 3.7 (more exactly to &python3-majorver;).

[Pierre Labastie]

Note that several KF5 and plasma packages have an optional requirement on ​sip.

[Pierre Labastie]

So I've been able to build KF5, plasma, and KF5 applications without P2. At some point, I needed asciidoc (recommended for btrfs-progs, which is itself recommendend for udisks2, which is needed at runtime by KDE applications). So:

• I installed P2
• I moved /usr/bin/{python,python2,python2.7} to /usr/bin/{python,python2,python2.7}nouse
• I changed the headers of a2x and asciidoc from /usr/bin/python to /usr/bin/python2.7nouse

asciidoc is now working, but P2 is invisible to any other package... I wonder if we could not use this kind of approach in the book, for packages which need P2 at run time.

For packages which need P2 at build time, it is often possible to pass the location of the P2 executable to the build system, so passing the nouse name could be done in this case as well. Well, to close to freeze maybe. Let's wait for after 8.4.

[Pierre Labastie]

Well not sure it is practical to change the name of the python executable on a running system. But for building, I suspect it could have some use cases... (Ken has an interesting comment about gimp scripts written in python, but he only sent it to the blfs-book mailing list)

[ken@…]

Replying to pierre.labastie:

Well not sure it is practical to change the name of the python executable on a running system. But for building, I suspect it could have some use cases... (Ken has an interesting comment about gimp scripts written in python, but he only sent it to the blfs-book mailing list)

Now that I'm on a system where current firefox is installed (to login to the wiki!), and also a version of gimp-2.10, I'll give a little more detail here:

Lots of gimp users seem to prefer python (i.e. python2) for scripting. They seem to think that handling the vagaries of whitespace significance is easier than dealing with scheme - actually, they might be onto something there ;-)

I said that probably all of these scripts are external. In fact there are a number shipped as standard in 2.10, including the python console itself (bottom of the filters menu, 'Python-Fu') which is /usr/lib/gimp/2.0/plug-ins/python-console/python-console.py. And all of the shipped scripts start:

#!/usr/bin/env python2

So gimp really does want python2 available at runtime (indeed, it might not even configure without it - I've never tried).

[Pierre Labastie]

Replying to renodr:

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

Actually, Python is not used unless --enable-introspection is passed to configure. The current book instructions don't pass it, and gobject-introspection is deemed optional. We may want to change to recommend g-i, but right now, I'd say the patch is not needed.

[Pierre Labastie]

Replying to pierre.labastie:

Replying to renodr:

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

Actually, Python is not used unless --enable-introspection is passed to configure. The current book instructions don't pass it, and gobject-introspection is deemed optional. We may want to change to recommend g-i, but right now, I'd say the patch is not needed.

Ah, I see we have a sed for libical, although libical is optional too... But trying to use --enable-introspection leads to:

  GISCAN  Abi-3.0.gir
Couldn't find include 'Gsf-1.gir' (search path: '['/usr/share/gir-1.0', '/usr/share/gir-1.0', '/usr/share/gir-1.0', '/usr/share/gir-1.0']' )

So it seems libgsf needs --enable-introspection, too. I do not know how many other packages do not use automatically introspection if it is present...

[Pierre Labastie]

Trying to build Qt5 new version on my P3 only machine gave an error:

sh: python: command not found
Building QtQml requires Python.

The following sed allows to build:

find . -name \*.pro -o -name \*.pri | xargs sed -i 's/python/&3/'

Not sure it would work with qtwebengine...

[Pierre Labastie]

For qtwebengine, configure.pri contains a test, which explicitly requires P2. I'll modify the Qt5 page to add the above sed, and tell that if the sed is applied, qtwebengine cannot be built.

[Pierre Labastie]

Qt5 done at r21106

[Douglas R. Reno]

Pierre,

Can we please get the Python 2 module for PyGObject3 back in the book? So far I've encountered issues with the following packages as a result of it's disappearance:

• Caribou
• libpeas
• Tracker-2.2

I'm certain that there will probably be more.

[Pierre Labastie]

Replying to renodr:

Pierre,

Can we please get the Python 2 module for PyGObject3 back in the book? So far I've encountered issues with the following packages as a result of it's disappearance:

• Caribou
• libpeas
• Tracker-2.2

I'm certain that there will probably be more.

I'll do that, because we are close to release, and we do not have time to look at the details. GTK+3 has moved to P3, so I am almost sure that the issues you see come from something else (maybe the build system finds P2, and then tries to use the pygobject3 module on it. Also, ISTR from our "tracker" discussion, that some tests are P2 only). I think we should tend to have only P3 installed. But that's for after the release... I'll move this ticket to 8.5, and work more on it after the release.

[Pierre Labastie]

P2 build of PyGobject3 reinstated at r21223

[Bruce Dubbs]

Just a comment here. I was upgrading bind and they now need a module 'ply' for dnssec-keymgr.

It is easy enough to install ply with 'sudo pip install ply'

When I did this I got a message:

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.

So this gives some emphasis to trying to minimize python2.

I also noticed that we have a pip3.7, but not a pip3. Perhaps we should add

ln -svf pip3.7 /usr/bin/pip3

when we install python3 in LFS.

[Xi Ruoyao]

Replying to pierre.labastie:

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

Are you sure?

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gdb

[Pierre Labastie]

Replying to xry111:

Replying to pierre.labastie:

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

Are you sure?

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gdb

Hmmm, that was some time ago, so I do not remember the details. I guess I'll have to try again... If I try to recall what I did, I think passing --with-python=/usr/bin/python3 does not prevent the build, but the backend cannot be loaded into gdb. Maybe Archlinux don't care... Actually, I do not understand why they need --with-python=/usr/bin/python3 . Don't they rename python3 to python?

[Xi Ruoyao]

Replying to pierre.labastie:

Replying to xry111:

Replying to pierre.labastie:

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

Are you sure?

​https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gdb

Hmmm, that was some time ago, so I do not remember the details. I guess I'll have to try again... If I try to recall what I did, I think passing --with-python=/usr/bin/python3 does not prevent the build, but the backend cannot be loaded into gdb. Maybe Archlinux don't care...

I just built a GDB-8.2.1 with python3. It seems working well. (At least libstdc++.so.6.0.25-gdb.py works with it).

Actually, I do not understand why they need --with-python=/usr/bin/python3 . Don't they rename python3 to python?

Maybe to prevent issues on systems with /usr/local/bin/python or /opt/anaconda/bin/python? My girlfriend once got stuck because a version conflict between system Python and Anaconda.

[thomas]

Samba 4.10.0 will be the first Python3 version. It is scheduled for Tuesday (March, 19th). 4.11 will be P3-only.

From Release notes (4.10.0rc1):

...
This is the first release of Samba which has full support for Python 3.
Samba 4.10 still has support for Python 2, however, Python 3 will be used by
default, i.e. 'configure' & 'make' will execute using python3.
...

[Pierre Labastie]

Replying to thomas:

Samba 4.10.0 will be the first Python3 version. It is scheduled for Tuesday (March, 19th). 4.11 will be P3-only.

From Release notes (4.10.0rc1):

...
This is the first release of Samba which has full support for Python 3.
Samba 4.10 still has support for Python 2, however, Python 3 will be used by
default, i.e. 'configure' & 'make' will execute using python3.
...

Very good news. Thanks. Note that I have no means to test samba, so hopefully, somebody will be able to test builds with P3 only.

[Xi Ruoyao]

evolution-data-server favors Python 3 since 3.27.91.

[Xi Ruoyao]

Replying to xry111:

evolution-data-server favors Python 3 since 3.27.91.

Now it actually doesn't need python at all (for both 3.30 and 3.32). Removed in r21398.

[Douglas R. Reno]

  * Ported data generator utilities to python3
  * Ported functional tests to python3, reformatted to PEP-8

Tracker no longer needs Python 2.

[Xi Ruoyao]

wget prefers python 3 for test suite now. Fixed at r21424.

[Xi Ruoyao]

For GDB:

1. I'm sure the Python scripts shipped with gdb-8.2.1 can work with Python 3.
2. I'm sure the GDB scripts shipped with libstdc++ (in gcc-8.3.0) and glib-2.60.0 can work with Python 3.

I don't know if there are other BLFS packages shipping Python scripts for GDB.

I think I'll proceed to add --with-python=/usr/bin/python3 for GDB, if nobody disagrees.

[Xi Ruoyao]

Replying to xry111:

I don't know if there are other BLFS packages shipping Python scripts for GDB.

I forgot rustc. Just checked its GDB scripts. They can work with P3 too.

[Xi Ruoyao]

If GDB is built with P3, many scripts would require six (P3 module) to work.

So I decided not to add --with-python=/usr/bin/python3 into the command, but explain it in Command Explanation. And, I appended six to optional dependency list of GDB.

[Pierre Labastie]

About inkscape: If nothing is done and P2 is not installed (it is recommended for scour, but scour is a runtime dep, so P2 can be built after inkscape), inkscape cannot be built... The reason is that "python" is hardcoded in some CMakeLists.txt. It seems that:

find . -name CMakeLists.txt | xargs sed -i 's/python/&3/'

allows the build to pass. Now, I am not sure whether all the functionalities are enabled.

[Pierre Labastie]

Replying to xry111:

If GDB is built with P3, many scripts would require six (P3 module) to work.

So I decided not to add --with-python=/usr/bin/python3 into the command, but explain it in Command Explanation. And, I appended six to optional dependency list of GDB.

Hmm, I am not sure I understand: why not recommend six, rather? It is a small module, and it is in the book.

[Xi Ruoyao]

Now P3 with six is recommended for gdb, as Pierre suggested.

[Xi Ruoyao]

xcb-proto test suite can be ran with P3 now. Removed P2 at r21596.

[Bruce Dubbs]

Milestone renamed

[Xi Ruoyao]

Added the way to run harfbuzz testsuite with P3, at r21803.

[Xi Ruoyao]

krb5 now uses Python 3 for its test suite. Removed P2 at r21913.

[ken@…]

When I was looking at firefox the other week, I took a look at what fedora are doing more generally, and one of the things that cropped up was gimp. Apparently upstream will require python2 for gimp-2.10. The gimp development branch (to be 3.0) already uses python3 but is not yet ready for prime-time, so fedora will be looking for a dispensation for python2.7 and a few modules for fedora32. One of those modules was pygtk, which apparently now has a maintainer but does not expect to release until gtk-4.0 is released.

[ken@…]

For git-2.24.1 I've now installed it on a development box where python2 is already present, using

PYTHON=python3 ./configure ... --with-python=/usr/bin/python3

and

PYTHON=python3 make $MAKEFLAGS test

and on BLFS-svn from 9th December the tests passed and it seems to work as expected (but I've no real idea how much python is used in it, google just gives me results for python-git).

Currently trying on my home server (9.0) which takes a bit longer - that too has now passed.

[ken@…]

On a fresh svn system where python2 has not been installed, git-2.24.1 passed its tests. I did a "clean build", i.e. don't pass PYTHON=python3, merely add '--with-python=/usr/bin/python3'.

Edit: I can't type.

[Bruce Dubbs]

I removed py2 from boost earlier today. It was only used for tests, but all tests pass now with py3:

python3 test_all.py

[Pierre Labastie]

Python 2 dependencies as of April 26th, 2020

[thomas]

mercurial-5.4 compiles without P2 (test with docutils and 'make check' outstanding)

[ken@…]

Just a note that I tried a firefox-76 beta last month on a system without python2, and it absolutely requires python2.7. Whether firefox-78 (next esr) will be any better remains to be seen.

[Douglas R. Reno]

I made mercurial python3 at r23066. There may be a potential performance regression though, but everything seems to be running OK. Only one test failed, and the documentation generated properly too. I ran "hg debuginstall" and "hg" just to verify that those functions worked as well.

[thomas]

Dropped P2 from subversion. For svn-1.14.0 it was required to add py3c as a new (tiny) module. The swig/py3c/subversion combo compiles fine and tested ok without P2 (afaiks).

[Pierre Labastie]

asciidoc-9.0.0 dropped P2.

[Douglas R. Reno]

I don't believe our build uses it, but nspr's buildscript was ported to Python3 with NSPR-4.26.

[Douglas R. Reno]

Seamonkey doesn't seem to want py2 anymore:

checking for Python 3... /usr/bin/python3 (3.8.3)

[Pierre Labastie]

telepathy-mission-control used to require python2 (not noted in the book). It now can be built and tested with python3.

[Douglas R. Reno]

dbus tests were ported to python3 at r23731 in preparation for the removal of pygobject3's python2 module. Note that this does *not* affect pygobject2, which is used for pygtk.

[Douglas R. Reno]

Updated to pygobject3-3.38.0 at r23737, which includes dropping the python2 module for pygobject3

[Douglas R. Reno]

Cleaned up pyatspi2 by removing lingering references to python2 module in r23756

[Douglas R. Reno]

The python2 module for pycryptodome has been removed at r23794. Samba uses python3 now.

[Xi Ruoyao]

js68 archived.

[Xi Ruoyao]

In libevdev only one build script uses python. It's explicitly Python 3 now. P2 reference removed.

In mesa, P2 reference is removed and everyone should sed the test script to use Python 3.

[Douglas R. Reno]

In nfs-utils, several scripts were converted to python3 at the last release (2.5.2)

[ken@…]

I've just noticed that seamonkey still requires python2.7, it is probably (at the moment) picked up via llvm or rustc. The Makefile looks for 2.7. Looking at fedora, they require 2.7.

[Douglas R. Reno]

node.js has been converted to Python3 at r23855

[Douglas R. Reno]

telepathy-glib ported to python3 at r23863

[Douglas R. Reno]

nghttp2's python module was ported to Python3. As part of the update to nghttp2-1.43.0, the python2 dependency was removed at r24176

[Pierre Labastie]

Dependencies on python2 as of March 2nd, 2021

[Pierre Labastie]

it looks like there are several packages where python2 is deemed optional, with the meaning that the package needs python and can use python3, but could use python2 if desired. I propose to remove this type of dependencies: nobody should use python2 if it is possible to use python3 nowadays. Or at least, we shouldn't promote that...

I'll remove those dependencies in the next days, unless somebody speaks up.

[Pierre Labastie]

PYTHON=python3 can be passed to cracklib to force python3. But for configure to enable python, one python include file has to be modified.

cryptsetup python bindings have been removed at version 2.1 (it is presently at 2.3.4).

[Pierre Labastie]

do not advertise python for:

• gpgme
• lipwquality
• volume_key

btrfs-progs cannot use python with version <3.4 anymore.

[Douglas R. Reno]

I caught libqmi attempting to use python2 and ignoring python3 when present. Note that there is no dependency recorded in the book on it, but it is there:

checking for GLIB... yes
checking for python... no
checking for python2... /usr/bin/python2
checking for python version... 2.7
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages

When specifying PYTHON=python3 at the configure line, things work properly. I've added that in at r24333

[Pierre Labastie]

• vim can use python3: it needs --enable-python3interp=dynamic", otherwise, the tests have segmentation faults. Do not advertise python2 anymore.
• brotli can use python3: do not advertise python2.
• for libxml2, the python2 binding is built at another place: remove dependency on P2.
• talloc does not accept python2 anymore: when calling configure with PYTHON=python2, I get:

  Checking for program 'python3'       : python2 
  Checking for program 'python'        : python2 
  Checking for program 'python3'       : /usr/bin/python3 
  Checking for python version >= 3.6.0 : 2.7.18 
  The python version is too old, expecting (3, 6, 0)
  

[Pierre Labastie]

• Little CMS 1.19 can only use python2. Default is to not use it. The only package that reference it is xsane, and it is an optional dependency. Let's keep the ref to python2.
• Newt uses both python2 and python3 if they are present. This can be prevented with --with-python=python3.9 (you have to indicate where the modules are found). If nothing is done, and python2 is installed, you get:

  checking for python versions... python2.7 python3.9
  

[Douglas R. Reno]

Python2 versions of the graphviz bindings were removed in graphviz-2.47.3. Accounted for in 9380d5be50c0b776ca7fa9a20d8b2bd46f76ed2e

[pierre]

Do we need telepathy-logger? It is referenced nowhere in the book, but it requires P2 at build time.

[pierre]

Looks like P2 is only required for some python modules (PyGTK, PyCairo-1, PyGObject-2, libxml2 py2), telepathy-logger as said above, seamonkey, and recommended for texlive. Note that GIMP requires P2 only through dependencies (libxml2 for python2).

PyCairo-1 and PyGobject-2 are only referenced in PyGTK. PyGTK is referenced in nmap (optional, used in zenmap and ndiff), avahi (optional), GIMP (recommended), openbox (optional for a test, but we say there is no testsuite, and required for building obmenu, which is not in the book), keybinder2 (recommended; keybinder2 is only required by lxpanel), and vte2 (optional; vte2 is only required by lxterminal).

Summary: once GIMP-3 is out and if we retire lxde, the only places where P2 would be needed are seamonkey (hopefully, port to P3 should be backported) and texlive.

[ken@…]

For texlive, contributed scripts (including those using python which require python2) are mostly unmaintained. For binary install-tl we describe python2 as recommended at runtime, in source it is listed amongst 'Runtime dependencies'. When we get rid of python2 I think we could easily change it from Recommended to Optional in install-tl and convert both of these to an external link (and describe it as unmaintained) - or just drop it, with a note that some of these scripts no-longer work.

Ideally, we would also then symlink python3 to python so that general scripts expecting python to be python3 work whilst avoiding having to immediately go through both books changing all invocations of python3 and potentially breaking things for people building current stuff on slightly older systems.

[DJ Lucas]

Ideally, we would also then symlink python3 to python

I'm not sure about that last part, at least discuss.

​https://peps.python.org/pep-0394/#recommendation

PEP 394 was updated sometime around 3.3 IIRC, and while now permitted, I think that should be frowned upon. Let's not let the lessons of the past 14 years (yes, it's been 14 years) go by without applying the lesson. Of course, per Guido van Rossum, "NEVER" is when to expect Pytion-4. :-)

[Douglas R. Reno]

telepathy-logger has been archived since it is no longer in use and had a hard dependency on python2

[ken@…]

Replying to ken@…:

For texlive, contributed scripts (including those using python which require python2) are mostly unmaintained.

I revised the texlive text yesterday, now only 3 unmaintained packages have scripts which require python=python2, and pythontex has scripts for both py2 and py3 depending on the system python version.

Therefore, texlive will not hold up removal of python2.

[Xi Ruoyao]

Let's push this a little (like GTK2).

[Xi Ruoyao]

Packages (except Python 2 modules) directly depending on Python 2:

• Seamonkey: Required (really?)
• Subversion: Optional for test
• Doxygen: Text reference, there is a command replacing shebangs to use Python 3 so I guess the ref is outdated.
• yasm: Optional (+swig)
• opencv: Optional
• lcms: Optional (+swig)
• alsa-lib: Optional
• libmusicbrainz: Optional
• ldns: Optional (+swig)
• texruntime: Optional, Runtime, for some scripts
• unbound: Optional (+swig)
• cups: Optional
• docbook-xsl: Optional (libxml2 and libxslt Python 2 modules are referred but AFAIK we don't build libxslt Python 2 module now, and could we switch to Python 3?)
• libglade: Optional

[Xi Ruoyao]

PyGTK:

• gimp: Recommended
• avahi: Optional
• openbox: External (only useful with some external dependencies)

libxml2py2:

• gimp: Optional (for translated help files)

pyxdg:

• openbox: Optional (for openbox-xdg-autostart)

[Douglas R. Reno]

The python2 dependency has been removed from Seamonkey. It was ported in Seamonkey 2.53.17

[Douglas R. Reno]

SANE Backends was using Python2, but has now been adjusted to use Python3 at 437f2d713343e919202d1c6515a7bc4cb4845e1d

[Xi Ruoyao]

It looks like the main blocker is GIMP (again!)

[Xi Ruoyao]

In texlive there is

for F in $TEXLIVE_PREFIX/texmf-dist/scripts/latex-make/*.py ; do
  sed -i 's%/usr/bin/env python%/usr/bin/python3%' $F
done

but with texlive-2023 these files already contains #!/usr/bin/env python3, then the sed makes it "python33"!

I'll remove the command.

[Xi Ruoyao]

Dropped P2 reference in Subversion, Doxygen, and Docbook-xsl.

[Xi Ruoyao]

libxml2py2 is not used now, so it can be archived.

[Xi Ruoyao]

libxml2py2 is archived at r12.0-643-g2f1e634e17.

[Xi Ruoyao]

GIMP 3 will be released in May 2024. But I really don't want to carry Python 2 into another LFS release...

Could we disable Python building GIMP or ship a GIMP 3 release candidate (which should be available for LFS 12.1)? Currently the 2.99.16 beta works fine for me (frankly I only use the basic functionalities of GIMP though).