Opened 4 years ago

Last modified 14 months ago

#11549 new enhancement

Review dependencies on Python 2

Reported by: Pierre Labastie Owned by: allblfseditorsasreq
Priority: normal Milestone: hold
Component: BOOK Version: git
Severity: normal Keywords: Python 2
Cc:

Description

After working on the Python module page and some packages which depend on those module, I've realized that simple tweaks could remove the dependency on Python 2. Sometimes, it is as simple as changing #!/usr/bin/env python to #!/usr/bin/env python3 in some programs, which are otherwise Python 3 compatible. Sometimes, running 2to3 on some specific programs may be required.

This ticket is for showing advances in this respect. It is not expected to be fully completed for 8.4.

Attachments (4)

toto (5.0 KB ) - added by Pierre Labastie 4 years ago.
List of Packages needing Python 2, with dependency status
python-dep-2019-01-28 (4.7 KB ) - added by Pierre Labastie 4 years ago.
Dpendencies on Python 2, updated
python-dep-2020-04-26 (4.0 KB ) - added by Pierre Labastie 3 years ago.
Python 2 dependencies as of April 26th, 2020
python-dep-2021-03-02 (3.1 KB ) - added by Pierre Labastie 22 months ago.
Dependencies on python2 as of March 2nd, 2021

Download all attachments as: .zip

Change History (120)

by Pierre Labastie, 4 years ago

Attachment: toto added

List of Packages needing Python 2, with dependency status

comment:1 by Pierre Labastie, 4 years ago

Seems six is not directly needed by libtasn1 and gtk+3.

Furthermore, all the executable installed by gtk-doc have #!/usr/bin/python3. Actually, the NEWS file says:

NEWS:GTK-Doc now requires python3.X. It does not require python-six anymore

comment:2 by Pierre Labastie, 4 years ago

gtk-doc fixed at r21014

comment:3 by Pierre Labastie, 4 years ago

NetworkManager can be made P3 only by running:

grep -rl '^#!.*python' | xargs sed -i '1s/python/&3/'

I need to test the Qt examples before committing.

libbytesize is now P3 only: done at r21022.

comment:4 by Pierre Labastie, 4 years ago

I've committed the fix to NetworkManager at r21025. It seems that the Qt examples are built but not installed. I do not know whether it is a problem with using P3 or with meson/ninja: there are no errors in the log.

comment:5 by Pierre Labastie, 4 years ago

for volume_key: in any case, python is not required. Python is used for building bindings to the volume_key library, using swig. Both P2 and P3 bindings can be built. those can be disabled by using --without-python and --without-python3 respectively. Note that swig is not tested by configure: if a Python executable is found, and the corresponding --without-python flag has not been passed, swig will be run, and the build will stop if swig is not installed.

The python bindings are not needed in the book (the only user of volume_key is libblockdev, which uses directly the shared library libvolume_key.so, so I think:

P2 and swig should be made optional, and both --without-python and --without-python3 should be added to the configure invocation.

Note that gnupg2 can be optionally used, and should be added to optional deps (it is not required nor recommended for gpgme).

comment:6 by Pierre Labastie, 4 years ago

Hmm, the configure options for libblockdev have --with-python3. Again, this is just for building python bindings, which are not used in the book, I think (I'll try to check anyway, but I think udisks2 does not use python, and it is the only user of libblockdev).

comment:7 by Douglas R. Reno, 4 years ago

I think that optional python bindings should be built irregardless of whether or not they are actually used by anything in the book. It simplifies maintenance in the future if they are actually required at some point, and doesn't hurt anything.

Also - Upstream is good at hiding dependencies on python bindings - especially in GNOME. Udisks2 does use Python internally, as does gvfs.

comment:8 by Pierre Labastie, 4 years ago

On my way to testing udisks2, I've found that asciidoc requires P2 at run time, and that the asciidoc scripts cannot be converted easily to P3 (2to3 is not enough). For now, I'll disable building the docs in btrfs-progs, but the asciidoc page on the book has to be modified to make P2 required (runtime).

in reply to:  7 comment:9 by Pierre Labastie, 4 years ago

Replying to renodr:

I think that optional python bindings should be built irregardless of whether or not they are actually used by anything in the book. It simplifies maintenance in the future if they are actually required at some point, and doesn't hurt anything.

Well, in theory, I agree with this. In practice, it is not always simple (python bindings for gpgme require gnupg2 and swig to be present, python bindings for volume_key require swig, etc). I'll try my best to make everything coherent... But I think that hidden things is not what we want in blfs: the whole object of lfs/blfs is to control what is on the machine. We should try not to give up on this, otherwise, there will be no difference with any other distro...

Also - Upstream is good at hiding dependencies on python bindings - especially in GNOME. Udisks2 does use Python internally, as does gvfs.

Actually, Udisks2 tests seem to require pygobject for tests (the gi module). This is not noted in the book (will investigate).

Last edited 4 years ago by Pierre Labastie (previous) (diff)

comment:10 by Pierre Labastie, 4 years ago

Stupid mozilla js requires P2. So P2 will be present anyway when building Udisks2.

in reply to:  7 ; comment:11 by Bruce Dubbs, 4 years ago

Replying to renodr:

I think that optional python bindings should be built irregardless of whether or not they are actually used by anything in the book. It simplifies maintenance in the future if they are actually required at some point, and doesn't hurt anything.

Also - Upstream is good at hiding dependencies on python bindings - especially in GNOME. Udisks2 does use Python internally, as does gvfs.

Unless we know about a P2 dependency, then we can omit it by default if it takes a command to include it. I'd say leave in the P3 bindings as we have P3 already in LFS, but make P2 optional and don't take explicit action to disable the P2 bindings. Looking at ./configure --help for volume_key, it looks like picking up P2 is automatic if it is present, so we probably only need to move P2 to optional.

in reply to:  11 comment:12 by Pierre Labastie, 4 years ago

Replying to bdubbs:

Unless we know about a P2 dependency, then we can omit it by default if it takes a command to include it. I'd say leave in the P3 bindings as we have P3 already in LFS, but make P2 optional and don't take explicit action to disable the P2 bindings.

There is an issue with this approach:

  • suppose P2 is present. The package will then try to build say, bindings for P2. If in the process, a python module is needed, this module must have been built with P2. This means that all the python modules would have to be built for P2 as well a for P3. I'd rather avoid that.

I think it's better to try to prevent P2 to come into the picture each time it is possible: otherwise, we'll always have to build P2 bindings (and so recommend P2), "just in case".

Looking at ./configure --help for volume_key, it looks like picking up P2 is automatic if it is present, so we probably only need to move P2 to optional.

But the problem is, in this case (for both P2 and P3 actually): if the build system wants to build the python bindings, it'll need swig, and since swig is not tested by configure, it will fail during the "make" stage. OK, let's do it this way:

  • P2 optional, swig recommended, others required as already noted. pass --without-python (disable only P2) and comment about the --without-python{,3} in command explanations.

comment:13 by Bruce Dubbs, 4 years ago

We can prevent the modules from being built but we need to explain the complexities of the inter dependencies in a note (or similar). If we don't need any of the modules in BLFS, I'd rather list swig as optional rather than recommended.

comment:14 by Pierre Labastie, 4 years ago

Well, let's discuss some sort of policy (I do not have the whole picture yet, so may need adjustments later). From Douglas and Bruce inputs, I'd say it could be:

  • Python modules: always build for P3, except when impossible (pygtk). Build for P2 only when really needed by something in the book (normally, already implemented).
  • Other applications:
    • Python bindings: don't prevent building bindings for P3 (but do not build them if upstream default is to not build them), and always prevent building them for P2, unless the package itself or a dependent package require P2.
    • Python apps: build only for P3 as much as possible. Build for P2 only when upstream is too lazy has been too busy to take the time to switch to P3.

comment:15 by Bruce Dubbs, 4 years ago

Sounds good to me.

comment:16 by ken@…, 4 years ago

And to me.

comment:17 by Pierre Labastie, 4 years ago

Thanks Ken and Bruce.

For asciidoc, I've found: https://github.com/asciidoc/asciidoc-py3. No release yet. There is also https://asciidoc3.org/. Looks like they are independent forks. The latter has releases, but the "historical" devs of asciidoc are on the former.

comment:18 by Pierre Labastie, 4 years ago

Qemu can use P3. Done at r21039

comment:19 by Pierre Labastie, 4 years ago

nodejs really needs P2. No way to escape...

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

One rustc test needs python2.7 (sysroot-crates-are-unstable). Otherwise, rustc can be made P3 only, by changing any occurrence of ./x.py to python3 ./x.py in the current book instructions (not tried the custom rust page, but I expect the same results, since only the installation dir is changed). In this case, and if gdb is present and built without python, 7 tests fail. I'd say it is still acceptable. Anyway, I won't change rustc instructions unless instructed by Ken or Bruce...

comment:20 by Pierre Labastie, 4 years ago

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Last edited 4 years ago by Pierre Labastie (previous) (diff)

in reply to:  20 ; comment:21 by ken@…, 4 years ago

Replying to pierre.labastie:

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Can you be specific about which is the fourth, and is it different from the three certain and one possible already specified ?

As to building rustc without python3 - we did, for a while. I reverted back to python2 on 6th January, because of #11520. That was with 1.31.1. As I noted earlier, fedora have a patch for python3 for (I think) only one of the rust files. Somebody, probably Thanos, pointed out that some of the rustc scripts specify 2.7 (I think) - can't find the mail in the archive, it was probably early this month.

in reply to:  21 ; comment:23 by Pierre Labastie, 4 years ago

Replying to ken@…:

Replying to pierre.labastie:

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Can you be specific about which is the fourth, and is it different from the three certain and one possible already specified ?

No it's not different: the four failures are issues 37131, 49851 and 50993, and sysroot-crates-are-unstable. The latter uses a python script, which explicitly requires python2.7.

As to building rustc without python3 - we did, for a while. I reverted back to python2 on 6th January, because of #11520. That was with 1.31.1. As I noted earlier, fedora have a patch for python3 for (I think) only one of the rust files. Somebody, probably Thanos, pointed out that some of the rustc scripts specify 2.7 (I think) - can't find the mail in the archive, it was probably early this month.

Well, this seems to work for this version of rust. Fedora's patch does not seem to be needed. It is in a lldb_something file, and I do not think our build makes lldb usable. I'll try building firefox and let you know if it works. Note that if changing the python version is enough to generate SIGSEV, it means that there are severe flaws in the build system... The python scripts are supposed to run compilation commands, not to define memory locations...

in reply to:  23 ; comment:24 by ken@…, 4 years ago

Replying to pierre.labastie:

Replying to ken@…:

Replying to pierre.labastie:

Confirmed that if gdb is not installed, and P2 is not present, there are 4 test failures in rustc.

Can you be specific about which is the fourth, and is it different from the three certain and one possible already specified ?

No it's not different: the four failures are issues 37131, 49851 and 50993, and sysroot-crates-are-unstable. The latter uses a python script, which explicitly requires python2.7.

And all my desktop builds/upgrades build python2 at a very early stage, yet I have occasionally seen that failure (and I definitely had it with 1.32.0 and system llvm-7 on my phenom, although that install appeared to be usable).

Also, as to the gdb failures you mentioned earlier, I got those on another machine when updating 8.3, in that case python-2.7.15 was definitely installed when gdb-8.1.1 was installed, so I assumed that gdb-6.1.1 was probably too old for the tests.

I'll add a comment that three gdb tests can fail, but I'm waiting for Bruce to merge his changes.

As to building rustc without python3 - we did, for a while. I reverted back to python2 on 6th January, because of #11520. That was with 1.31.1. As I noted earlier, fedora have a patch for python3 for (I think) only one of the rust files. Somebody, probably Thanos, pointed out that some of the rustc scripts specify 2.7 (I think) - can't find the mail in the archive, it was probably early this month.

Well, this seems to work for this version of rust. Fedora's patch does not seem to be needed. It is in a lldb_something file, and I do not think our build makes lldb usable. I'll try building firefox and let you know if it works. Note that if changing the python version is enough to generate SIGSEV, it means that there are severe flaws in the build system... The python scripts are supposed to run compilation commands, not to define memory locations...

Personally, I have very poor experience of all python build-systems, and I have not found that 3 is necessarily better. Both x.py and mach havefailed for me in odd ways (x.py with both 2 and 3, mach still definitely needs 2).

But I'm not sure that I associate the change from 2 to 3 with SIGSEGV - testing 1.32.0 has been unpleasant, I've forgotten a lot of the details and I'm probably confused about the order in which I tried certain things, but for me the rust tests which failed unexpectedly with SIGSEGV were when using system LLVM-7.

in reply to:  22 comment:25 by ken@…, 4 years ago

Replying to bdubbs:

I note that Arch is using system llvm for rust, but patches it:

https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/llvm

For rust they use P2:

https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/rust

I'm glad they've now patched their llvm - that was where the problem was first identified. I can't remember for certain whether they ever used P3 to build it.

And once you've merged your changes (I assume you got my mail reply mentioning some details) I'll update the list of possible test failures re gdb and then hopefully not touch rustc for a few weeks!

in reply to:  23 comment:26 by ken@…, 4 years ago

Replying to pierre.labastie:

Well, this seems to work for this version of rust. Fedora's patch does not seem to be needed. It is in a lldb_something file, and I do not think our build makes lldb usable.

Forgot to comment on lldb : it is an external llvm tool (like clang, lld) and has at least one extra dep that is not in the book, plus although I thought I had doxygen installed it complained that was missing. But the extra dep made me stop investigating it. The rust-lldb is just a front-end for it.

I'm still *vaguely* interested in trying to build rust with thin-lto (requires clang, llvm-ar, lld) but that would need several variations, and builds of the big programs which use rustc (firefox, thunderbird) to try different settings and to see if any make a meaningful improvement in build times. Seems too much effort to be worth the time in the short term. but I've added lld to my future llvm builds in case I change my mind.

in reply to:  24 comment:27 by Pierre Labastie, 4 years ago

Replying to ken@…:

But I'm not sure that I associate the change from 2 to 3 with SIGSEGV - testing 1.32.0 has been unpleasant, I've forgotten a lot of the details and I'm probably confused about the order in which I tried certain things, but for me the rust tests which failed unexpectedly with SIGSEGV were when using system LLVM-7.

The error reported in ticket #11520 was a SIGSEGV. I'm not sure it is the only thing which decided you to switch back to P2, but it is the only thing which is reported in that ticket...

As for using P3 for the build system: my aim with this ticket is to reduce P2 use to a minimum, that is, use P2 only for packages which need it. SO I'd like to see P3 used as much as possible. With rustc in /usr, built with P3, firefox can be built (I'm using it right now). Of course, P2 had to be installed at some point (for building nodejs actually).

I'll take responsibility to put back P3 instructions for rust ;). It really seems to me they do not hurt. It's development after all...

comment:28 by Pierre Labastie, 4 years ago

git has P2 as recommended, but I do not see why:

  • python (either 2 or 3) is almost not used in git. The only script using python outside of the contrib directory is git-p4, which is used to interact with perforce repositories. To me, it seems optional.
  • git building does not need python either...

So I'll remove P2 from the recommended deps.

comment:29 by Pierre Labastie, 4 years ago

Rustc and git done at r21045

Last edited 4 years ago by Pierre Labastie (previous) (diff)

comment:30 by Pierre Labastie, 4 years ago

Poppler is in a somewhat inconsistent state: The only use of Python is for building the the API documentation, but gtk-doc is deemed optional, while P2 is deemed recommended. Actually, P3 can be used if one the header of make-glib-api-doc is changed from #!/usr/bin/env python to #!/usr/bin/env python3. Moved P2 to optional, and added a command explanation for doc regeneration at r21047

comment:31 by Pierre Labastie, 4 years ago

usbutils does not need P2 anymore. lsusb.py has been ported to P3 (thanks to Douglas for the patch), and it is the only user of Python in usbutils... Removed requirement for P2 at r21049.

comment:32 by Pierre Labastie, 4 years ago

For QtWebEngine, I do not have the same experience as Ken: it built without Python 2... (Current book instructions for both Qt5 and then QtWebEngine. Will try Falkon tomorrow

in reply to:  32 ; comment:33 by Pierre Labastie, 4 years ago

Replying to pierre.labastie:

For QtWebEngine, I do not have the same experience as Ken: it built without Python 2... (Current book instructions for both Qt5 and then QtWebEngine. Will try Falkon tomorrow

Well, I may have done a mistake here. Let's do it again after really removing Python 2!

in reply to:  33 comment:34 by Pierre Labastie, 4 years ago

Replying to pierre.labastie:

Replying to pierre.labastie:

For QtWebEngine, I do not have the same experience as Ken: it built without Python 2... (Current book instructions for both Qt5 and then QtWebEngine. Will try Falkon tomorrow

Well, I may have done a mistake here. Let's do it again after really removing Python 2!

Yeah. Python2 is needed.

comment:35 by Pierre Labastie, 4 years ago

for tracker, Python is only needed for the "functional tests" which are disabled by default. If those tests are desired, P2 is needed. I'll move P2 to optional, and explain why it may be needed. Also, all the tests pass if in a graphical session with dbus, and tracker is installed (I guess at least the dbus service file is needed).

by Pierre Labastie, 4 years ago

Attachment: python-dep-2019-01-28 added

Dpendencies on Python 2, updated

comment:36 by Douglas R. Reno, 4 years ago

On a systemd system with Tracker pre-installed, I'm still getting the errors. I'm not sure what's causing them, I'll check pre-release

in reply to:  36 ; comment:37 by Pierre Labastie, 4 years ago

Replying to renodr:

On a systemd system with Tracker pre-installed, I'm still getting the errors. I'm not sure what's causing them, I'll check pre-release

For testing, I had only openbox (no other desktop manager). Of course, it was on SysV. A few tests were skipped: maybe they aren't skipped in your environment, and they fail... I fear I have trashed the test logs. But I can redo them (it's not that long) and send them to you.

in reply to:  37 ; comment:38 by Douglas R. Reno, 4 years ago

Replying to pierre.labastie:

Replying to renodr:

On a systemd system with Tracker pre-installed, I'm still getting the errors. I'm not sure what's causing them, I'll check pre-release

For testing, I had only openbox (no other desktop manager). Of course, it was on SysV. A few tests were skipped: maybe they aren't skipped in your environment, and they fail... I fear I have trashed the test logs. But I can redo them (it's not that long) and send them to you.

I would much appreciate it. Thank you!

in reply to:  38 ; comment:39 by Pierre Labastie, 4 years ago

Replying to renodr:

I would much appreciate it. Thank you!

Sent through private mail. Hope I used the right address.

in reply to:  39 ; comment:40 by Douglas R. Reno, 4 years ago

Replying to pierre.labastie:

Replying to renodr:

I would much appreciate it. Thank you!

Sent through private mail. Hope I used the right address.

Thanks Pierre! Alright, running the tests with the package installed results in no FAILs and no ERRORs.

in reply to:  40 comment:41 by Pierre Labastie, 4 years ago

Replying to renodr:

Replying to pierre.labastie:

Replying to renodr:

I would much appreciate it. Thank you!

Sent through private mail. Hope I used the right address.

Thanks Pierre! Alright, running the tests with the package installed results in no FAILs and no ERRORs.

Good to know. Feel free to adapt my wording on the tracker page.

comment:42 by Pierre Labastie, 4 years ago

Just found that libpwquality uses Python (not noted in the deps), and that default binary is python. It can be changed by passing --with-python-binary=python3 to configure. Will modify the book's page.

comment:43 by Pierre Labastie, 4 years ago

libpwquality done at r21056

comment:44 by Douglas R. Reno, 4 years ago

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

in reply to:  44 comment:45 by Pierre Labastie, 4 years ago

Replying to renodr:

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

Great finding, thanks. Will give it a try later, but right now, I'm struggling through plasma, where P2 is deemed required. Up to now, I could build without P2, but some modules have been installed in /opt/kf5/lib/python3.7/site-packages. That leads to 2 issues:

  • the pathappend ... PYTHONPATH in /etc/profile.d/kf5.sh is wrong.
  • if P2 is present, it'll be used even if P3 is there too. We must find a way to prevent that, since P3 can be used in all cases.

comment:46 by Pierre Labastie, 4 years ago

It looks like the cmake modules look for P3 first, then for P2. My guess is that if P3 is present (and it should, since it has been built with LFS), it will be used. So the second item in comment:45 is not true (I should have been less affirmative).

The dependency on P2 has been on the plasma page since the very beginning (almost). At the time, building P3 was optional. I think it is safe to remove dependency on P2, and to move the PYTHONPATH to 3.7 (more exactly to &python3-majorver;).

comment:47 by Pierre Labastie, 4 years ago

Note that several KF5 and plasma packages have an optional requirement on sip.

comment:48 by Pierre Labastie, 4 years ago

So I've been able to build KF5, plasma, and KF5 applications without P2. At some point, I needed asciidoc (recommended for btrfs-progs, which is itself recommendend for udisks2, which is needed at runtime by KDE applications). So:

  • I installed P2
  • I moved /usr/bin/{python,python2,python2.7} to /usr/bin/{python,python2,python2.7}nouse
  • I changed the headers of a2x and asciidoc from /usr/bin/python to /usr/bin/python2.7nouse

asciidoc is now working, but P2 is invisible to any other package... I wonder if we could not use this kind of approach in the book, for packages which need P2 at run time.

For packages which need P2 at build time, it is often possible to pass the location of the P2 executable to the build system, so passing the nouse name could be done in this case as well. Well, to close to freeze maybe. Let's wait for after 8.4.

comment:49 by Pierre Labastie, 4 years ago

Well not sure it is practical to change the name of the python executable on a running system. But for building, I suspect it could have some use cases... (Ken has an interesting comment about gimp scripts written in python, but he only sent it to the blfs-book mailing list)

in reply to:  49 comment:50 by ken@…, 4 years ago

Replying to pierre.labastie:

Well not sure it is practical to change the name of the python executable on a running system. But for building, I suspect it could have some use cases... (Ken has an interesting comment about gimp scripts written in python, but he only sent it to the blfs-book mailing list)

Now that I'm on a system where current firefox is installed (to login to the wiki!), and also a version of gimp-2.10, I'll give a little more detail here:

Lots of gimp users seem to prefer python (i.e. python2) for scripting. They seem to think that handling the vagaries of whitespace significance is easier than dealing with scheme - actually, they might be onto something there ;-)

I said that probably all of these scripts are external. In fact there are a number shipped as standard in 2.10, including the python console itself (bottom of the filters menu, 'Python-Fu') which is /usr/lib/gimp/2.0/plug-ins/python-console/python-console.py. And all of the shipped scripts start:

#!/usr/bin/env python2

So gimp really does want python2 available at runtime (indeed, it might not even configure without it - I've never tried).

in reply to:  44 ; comment:51 by Pierre Labastie, 4 years ago

Replying to renodr:

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

Actually, Python is not used unless --enable-introspection is passed to configure. The current book instructions don't pass it, and gobject-introspection is deemed optional. We may want to change to recommend g-i, but right now, I'd say the patch is not needed.

in reply to:  51 comment:52 by Pierre Labastie, 4 years ago

Replying to pierre.labastie:

Replying to renodr:

You're going to want this for Abiword (found while trying to find a solution for blanking screen issues)

https://git.archlinux.org/svntogit/packages.git/tree/trunk/python-override.patch?h=packages/abiword

That forces Abiword to use Py3

Actually, Python is not used unless --enable-introspection is passed to configure. The current book instructions don't pass it, and gobject-introspection is deemed optional. We may want to change to recommend g-i, but right now, I'd say the patch is not needed.

Ah, I see we have a sed for libical, although libical is optional too... But trying to use --enable-introspection leads to:

  GISCAN  Abi-3.0.gir
Couldn't find include 'Gsf-1.gir' (search path: '['/usr/share/gir-1.0', '/usr/share/gir-1.0', '/usr/share/gir-1.0', '/usr/share/gir-1.0']' )

So it seems libgsf needs --enable-introspection, too. I do not know how many other packages do not use automatically introspection if it is present...

comment:53 by Pierre Labastie, 4 years ago

Trying to build Qt5 new version on my P3 only machine gave an error:

sh: python: command not found
Building QtQml requires Python.

The following sed allows to build:

find . -name \*.pro -o -name \*.pri | xargs sed -i 's/python/&3/'

Not sure it would work with qtwebengine...

Note that P2 is not in Qt5 dependencies.

Last edited 4 years ago by Pierre Labastie (previous) (diff)

comment:54 by Pierre Labastie, 4 years ago

For qtwebengine, configure.pri contains a test, which explicitly requires P2. I'll modify the Qt5 page to add the above sed, and tell that if the sed is applied, qtwebengine cannot be built.

comment:55 by Pierre Labastie, 4 years ago

Qt5 done at r21106

comment:56 by Douglas R. Reno, 4 years ago

Pierre,

Can we please get the Python 2 module for PyGObject3 back in the book? So far I've encountered issues with the following packages as a result of it's disappearance:

  • Caribou
  • libpeas
  • Tracker-2.2

I'm certain that there will probably be more.

in reply to:  56 comment:57 by Pierre Labastie, 4 years ago

Milestone: x-future8.5

Replying to renodr:

Pierre,

Can we please get the Python 2 module for PyGObject3 back in the book? So far I've encountered issues with the following packages as a result of it's disappearance:

  • Caribou
  • libpeas
  • Tracker-2.2

I'm certain that there will probably be more.

I'll do that, because we are close to release, and we do not have time to look at the details. GTK+3 has moved to P3, so I am almost sure that the issues you see come from something else (maybe the build system finds P2, and then tries to use the pygobject3 module on it. Also, ISTR from our "tracker" discussion, that some tests are P2 only). I think we should tend to have only P3 installed. But that's for after the release... I'll move this ticket to 8.5, and work more on it after the release.

comment:58 by Pierre Labastie, 4 years ago

P2 build of PyGobject3 reinstated at r21223

comment:59 by Bruce Dubbs, 4 years ago

Just a comment here. I was upgrading bind and they now need a module 'ply' for dnssec-keymgr.

It is easy enough to install ply with 'sudo pip install ply'

When I did this I got a message:

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.

So this gives some emphasis to trying to minimize python2.

I also noticed that we have a pip3.7, but not a pip3. Perhaps we should add

ln -svf pip3.7 /usr/bin/pip3

when we install python3 in LFS.

in reply to:  19 ; comment:60 by Xi Ruoyao, 4 years ago

Replying to pierre.labastie:

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

Are you sure?

https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gdb

in reply to:  60 ; comment:61 by Pierre Labastie, 4 years ago

Replying to xry111:

Replying to pierre.labastie:

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

Are you sure?

https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gdb

Hmmm, that was some time ago, so I do not remember the details. I guess I'll have to try again... If I try to recall what I did, I think passing --with-python=/usr/bin/python3 does not prevent the build, but the backend cannot be loaded into gdb. Maybe Archlinux don't care... Actually, I do not understand why they need --with-python=/usr/bin/python3 . Don't they rename python3 to python?

in reply to:  61 comment:62 by Xi Ruoyao, 4 years ago

Replying to pierre.labastie:

Replying to xry111:

Replying to pierre.labastie:

gdb's python backend is P2 only, too. This backend is optional, but this backend must have been built if gdb is present when running rustc tests, otherwise, 3 tests (in the debuginfo series) fail.

Are you sure?

https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gdb

Hmmm, that was some time ago, so I do not remember the details. I guess I'll have to try again... If I try to recall what I did, I think passing --with-python=/usr/bin/python3 does not prevent the build, but the backend cannot be loaded into gdb. Maybe Archlinux don't care...

I just built a GDB-8.2.1 with python3. It seems working well. (At least libstdc++.so.6.0.25-gdb.py works with it).

Actually, I do not understand why they need --with-python=/usr/bin/python3 . Don't they rename python3 to python?

Maybe to prevent issues on systems with /usr/local/bin/python or /opt/anaconda/bin/python? My girlfriend once got stuck because a version conflict between system Python and Anaconda.

Last edited 4 years ago by Xi Ruoyao (previous) (diff)

comment:63 by thomas, 4 years ago

Samba 4.10.0 will be the first Python3 version. It is scheduled for Tuesday (March, 19th). 4.11 will be P3-only.

From Release notes (4.10.0rc1):

...
This is the first release of Samba which has full support for Python 3.
Samba 4.10 still has support for Python 2, however, Python 3 will be used by
default, i.e. 'configure' & 'make' will execute using python3.
...

in reply to:  63 comment:64 by Pierre Labastie, 4 years ago

Replying to thomas:

Samba 4.10.0 will be the first Python3 version. It is scheduled for Tuesday (March, 19th). 4.11 will be P3-only.

From Release notes (4.10.0rc1):

...
This is the first release of Samba which has full support for Python 3.
Samba 4.10 still has support for Python 2, however, Python 3 will be used by
default, i.e. 'configure' & 'make' will execute using python3.
...

Very good news. Thanks. Note that I have no means to test samba, so hopefully, somebody will be able to test builds with P3 only.

comment:65 by Xi Ruoyao, 4 years ago

evolution-data-server favors Python 3 since 3.27.91.

in reply to:  65 comment:66 by Xi Ruoyao, 4 years ago

Replying to xry111:

evolution-data-server favors Python 3 since 3.27.91.

Now it actually doesn't need python at all (for both 3.30 and 3.32). Removed in r21398.

comment:67 by Douglas R. Reno, 4 years ago

  * Ported data generator utilities to python3
  * Ported functional tests to python3, reformatted to PEP-8

Tracker no longer needs Python 2.

comment:68 by Xi Ruoyao, 4 years ago

wget prefers python 3 for test suite now. Fixed at r21424.

comment:69 by Xi Ruoyao, 4 years ago

For GDB:

  1. I'm sure the Python scripts shipped with gdb-8.2.1 can work with Python 3.
  2. I'm sure the GDB scripts shipped with libstdc++ (in gcc-8.3.0) and glib-2.60.0 can work with Python 3.

I don't know if there are other BLFS packages shipping Python scripts for GDB.

I think I'll proceed to add --with-python=/usr/bin/python3 for GDB, if nobody disagrees.

in reply to:  69 comment:70 by Xi Ruoyao, 4 years ago

Replying to xry111:

I don't know if there are other BLFS packages shipping Python scripts for GDB.

I forgot rustc. Just checked its GDB scripts. They can work with P3 too.

comment:71 by Xi Ruoyao, 4 years ago

If GDB is built with P3, many scripts would require six (P3 module) to work.

So I decided not to add --with-python=/usr/bin/python3 into the command, but explain it in Command Explanation. And, I appended six to optional dependency list of GDB.

comment:72 by Pierre Labastie, 4 years ago

About inkscape: If nothing is done and P2 is not installed (it is recommended for scour, but scour is a runtime dep, so P2 can be built after inkscape), inkscape cannot be built... The reason is that "python" is hardcoded in some CMakeLists.txt. It seems that:

find . -name CMakeLists.txt | xargs sed -i 's/python/&3/'

allows the build to pass. Now, I am not sure whether all the functionalities are enabled.

in reply to:  71 comment:73 by Pierre Labastie, 4 years ago

Replying to xry111:

If GDB is built with P3, many scripts would require six (P3 module) to work.

So I decided not to add --with-python=/usr/bin/python3 into the command, but explain it in Command Explanation. And, I appended six to optional dependency list of GDB.

Hmm, I am not sure I understand: why not recommend six, rather? It is a small module, and it is in the book.

comment:74 by Xi Ruoyao, 4 years ago

Now P3 with six is recommended for gdb, as Pierre suggested.

comment:75 by Xi Ruoyao, 4 years ago

xcb-proto test suite can be ran with P3 now. Removed P2 at r21596.

comment:76 by Bruce Dubbs, 3 years ago

Milestone: 8.59.0

Milestone renamed

comment:77 by Xi Ruoyao, 3 years ago

Added the way to run harfbuzz testsuite with P3, at r21803.

comment:78 by Bruce Dubbs, 3 years ago

Resolution: fixed
Status: newclosed

comment:79 by Douglas R. Reno, 3 years ago

Resolution: fixed
Status: closedreopened

comment:80 by Xi Ruoyao, 3 years ago

krb5 now uses Python 3 for its test suite. Removed P2 at r21913.

comment:81 by Xi Ruoyao, 3 years ago

Milestone: 9.0hold

comment:82 by ken@…, 3 years ago

When I was looking at firefox the other week, I took a look at what fedora are doing more generally, and one of the things that cropped up was gimp. Apparently upstream will require python2 for gimp-2.10. The gimp development branch (to be 3.0) already uses python3 but is not yet ready for prime-time, so fedora will be looking for a dispensation for python2.7 and a few modules for fedora32. One of those modules was pygtk, which apparently now has a maintainer but does not expect to release until gtk-4.0 is released.

comment:83 by ken@…, 3 years ago

For git-2.24.1 I've now installed it on a development box where python2 is already present, using

PYTHON=python3 ./configure ... --with-python=/usr/bin/python3

and

PYTHON=python3 make $MAKEFLAGS test

and on BLFS-svn from 9th December the tests passed and it seems to work as expected (but I've no real idea how much python is used in it, google just gives me results for python-git).

Currently trying on my home server (9.0) which takes a bit longer - that too has now passed.

comment:84 by ken@…, 3 years ago

On a fresh svn system where python2 has not been installed, git-2.24.1 passed its tests. I did a "clean build", i.e. don't pass PYTHON=python3, merely add '--with-python=/usr/bin/python3'.

Edit: I can't type.

Last edited 3 years ago by ken@… (previous) (diff)

comment:85 by Bruce Dubbs, 3 years ago

I removed py2 from boost earlier today. It was only used for tests, but all tests pass now with py3:

python3 test_all.py

by Pierre Labastie, 3 years ago

Attachment: python-dep-2020-04-26 added

Python 2 dependencies as of April 26th, 2020

comment:86 by thomas, 3 years ago

mercurial-5.4 compiles without P2 (test with docutils and 'make check' outstanding)

comment:87 by ken@…, 3 years ago

Just a note that I tried a firefox-76 beta last month on a system without python2, and it absolutely requires python2.7. Whether firefox-78 (next esr) will be any better remains to be seen.

comment:88 by Douglas R. Reno, 3 years ago

I made mercurial python3 at r23066. There may be a potential performance regression though, but everything seems to be running OK. Only one test failed, and the documentation generated properly too. I ran "hg debuginstall" and "hg" just to verify that those functions worked as well.

comment:89 by thomas, 3 years ago

Dropped P2 from subversion. For svn-1.14.0 it was required to add py3c as a new (tiny) module. The swig/py3c/subversion combo compiles fine and tested ok without P2 (afaiks).

comment:90 by Pierre Labastie, 3 years ago

asciidoc-9.0.0 dropped P2.

comment:91 by Bruce Dubbs, 2 years ago

Owner: changed from blfs-book to alleditors
Status: reopenednew

comment:92 by Douglas R. Reno, 2 years ago

I don't believe our build uses it, but nspr's buildscript was ported to Python3 with NSPR-4.26.

comment:93 by Douglas R. Reno, 2 years ago

Seamonkey doesn't seem to want py2 anymore:

checking for Python 3... /usr/bin/python3 (3.8.3)

comment:94 by Bruce Dubbs, 2 years ago

Owner: changed from alleditors to all-editors

comment:95 by Bruce Dubbs, 2 years ago

Owner: changed from all-editors to allblfseditors

comment:96 by Pierre Labastie, 2 years ago

telepathy-mission-control used to require python2 (not noted in the book). It now can be built and tested with python3.

comment:97 by Douglas R. Reno, 2 years ago

dbus tests were ported to python3 at r23731 in preparation for the removal of pygobject3's python2 module. Note that this does *not* affect pygobject2, which is used for pygtk.

comment:98 by Douglas R. Reno, 2 years ago

Updated to pygobject3-3.38.0 at r23737, which includes dropping the python2 module for pygobject3

comment:99 by Douglas R. Reno, 2 years ago

Cleaned up pyatspi2 by removing lingering references to python2 module in r23756

comment:100 by Douglas R. Reno, 2 years ago

The python2 module for pycryptodome has been removed at r23794. Samba uses python3 now.

comment:101 by Xi Ruoyao, 2 years ago

js68 archived.

comment:102 by Xi Ruoyao, 2 years ago

In libevdev only one build script uses python. It's explicitly Python 3 now. P2 reference removed.

In mesa, P2 reference is removed and everyone should sed the test script to use Python 3.

comment:103 by Douglas R. Reno, 2 years ago

In nfs-utils, several scripts were converted to python3 at the last release (2.5.2)

comment:104 by ken@…, 2 years ago

I've just noticed that seamonkey still requires python2.7, it is probably (at the moment) picked up via llvm or rustc. The Makefile looks for 2.7. Looking at fedora, they require 2.7.

comment:105 by Douglas R. Reno, 2 years ago

node.js has been converted to Python3 at r23855

comment:106 by Douglas R. Reno, 2 years ago

telepathy-glib ported to python3 at r23863

comment:107 by Douglas R. Reno, 22 months ago

nghttp2's python module was ported to Python3. As part of the update to nghttp2-1.43.0, the python2 dependency was removed at r24176

by Pierre Labastie, 22 months ago

Attachment: python-dep-2021-03-02 added

Dependencies on python2 as of March 2nd, 2021

comment:108 by Pierre Labastie, 22 months ago

it looks like there are several packages where python2 is deemed optional, with the meaning that the package needs python and can use python3, but could use python2 if desired. I propose to remove this type of dependencies: nobody should use python2 if it is possible to use python3 nowadays. Or at least, we shouldn't promote that...

I'll remove those dependencies in the next days, unless somebody speaks up.

comment:109 by Pierre Labastie, 22 months ago

PYTHON=python3 can be passed to cracklib to force python3. But for configure to enable python, one python include file has to be modified.

cryptsetup python bindings have been removed at version 2.1 (it is presently at 2.3.4).

comment:110 by Pierre Labastie, 22 months ago

do not advertise python for:

  • gpgme
  • lipwquality
  • volume_key

btrfs-progs cannot use python with version <3.4 anymore.

comment:111 by Douglas R. Reno, 22 months ago

I caught libqmi attempting to use python2 and ignoring python3 when present. Note that there is no dependency recorded in the book on it, but it is there:

checking for GLIB... yes
checking for python... no
checking for python2... /usr/bin/python2
checking for python version... 2.7
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages

When specifying PYTHON=python3 at the configure line, things work properly. I've added that in at r24333

comment:112 by Pierre Labastie, 21 months ago

  • vim can use python3: it needs --enable-python3interp=dynamic", otherwise, the tests have segmentation faults. Do not advertise python2 anymore.
  • brotli can use python3: do not advertise python2.
  • for libxml2, the python2 binding is built at another place: remove dependency on P2.
  • talloc does not accept python2 anymore: when calling configure with PYTHON=python2, I get:
    Checking for program 'python3'       : python2 
    Checking for program 'python'        : python2 
    Checking for program 'python3'       : /usr/bin/python3 
    Checking for python version >= 3.6.0 : 2.7.18 
    The python version is too old, expecting (3, 6, 0)
    

comment:113 by Pierre Labastie, 21 months ago

  • Little CMS 1.19 can only use python2. Default is to not use it. The only package that reference it is xsane, and it is an optional dependency. Let's keep the ref to python2.
  • Newt uses both python2 and python3 if they are present. This can be prevented with --with-python=python3.9 (you have to indicate where the modules are found). If nothing is done, and python2 is installed, you get:
    checking for python versions... python2.7 python3.9
    

comment:114 by Bruce Dubbs, 19 months ago

Owner: changed from allblfseditors to allblfseditorsasreq

comment:115 by Douglas R. Reno, 18 months ago

Python2 versions of the graphviz bindings were removed in graphviz-2.47.3. Accounted for in 9380d5be50c0b776ca7fa9a20d8b2bd46f76ed2e

comment:116 by pierre, 14 months ago

Version: SVNgit
Note: See TracTickets for help on using tickets.