Python modules dependencies + instructions for building with Python2

[Pierre Labastie]

Some facts:

• several Python modules have instructions for both python2 and python3, but Python2 is not listed as a dependency (or is only listed as optional).

• scour is built only for Python2, but Python2 is not even mentioned as optional

• There is still the following sentence at the beginning of the Python module page: "Installation of the modules shown on this page is meant to follow from top to bottom to handle optional dependencies in each module." But it seems the dependencies have been sorted out, and actually, there are "out of order" recommended dependencies for some modules (e.g. PyGObjects is recommended by PyAtSpi2). So this sentence should not be there (and actually it is nonsense as soon as there is an out of order dep).

[Pierre Labastie]

See also #11316 (new version of pycairo), #11318 (new version of MarkupSafe), #11342 (new version of PyGObject3, and #11408 (new version of scons, which can use Python3).

[Pierre Labastie]

Removed the sentence about building from top to bottom, and added Python2 as recommended for Scour (with a note telling why) at r20786. Please feel free to comment/amend.

[Pierre Labastie]

List of modules with proposed changes (has to be tested, but please, any information is useful):

• D-Bus Python: optional for ibus (can use P3), dbus (for tests, don't know whether it can use P3), avahi (not sure, but uses PyGTK, so my guess is P2), libsecret (for the testsuite, don't know about P2 vs P3). Required for wicd, which requires also P2. So: P2 should be recommended, with P2 and P3 instructions.
• docutils: optional for sshfs, mercurial, dbus-python. According to the book, some of those require P2. Recommend P2, but add P3 instructions.
• PyAtSpi2: optional for GTK+-3 and caribou (runtime). I'd say it needs only P3 instructions, with P2 as optional dep
• PyCairo: Make P2 recommended (since it is a dep of PyGTk), and keep instructions for P2 and P3
• PyCrypto: recommended for samba (only user), which uses P2. I'd guess therefore that the P2 module is needed for samba. If it is so: make P2 recommended and keep both P2 and P3 instructions.
• PyGObject2: Python2 should be required
• PyGObject3: only P3 instructions, P2 as optional dep
• PyGTK: Python2 should be required.
• PyXDG: As far as I can tell, is only optional in the book. Referenced in ibus and openbox. Openbox can use P3, with some tweaks. I'd be inclined to leave only P3 instructions and have P2 as an optional dep.
• funcsigs: see below Mako. funcsigs is only used by Beaker in the book, and is included in core P3.
• Beaker: see below Mako, which is the only user in the book
• libxml2 (for Python2): nothing to be done
• lxml: recommended for inkscape, but as a runtime dep. Same as Scour, which we build as only P2. Make P2 recommended. Keep both P2 and P3 instructions (see below scour)
• MarkupSafe: see below Mako, which is the only user in the book
• Mako: is a required dependency of mesa (and only used for that in the book), with requires also Python2. My guess is that Mako needs therefore to be built for Python2. If it is the case, make P2 recommended, and same for MarkupSafe, Beaker and funcsigs
• Scour: done, but maybe we should add P3 instructions, anticipating P2 removal?
• six: optional for libbytesize, scour, samba (P2 only), gdk-pixbuf,. required for libtasn1, GTK+-3. Required runtime for gtk-doc (which is P2 only AFAICT). Add P2 as recommended, and keep both instructions.

Conclusion of this. There are not so many places where only P3 instructions are needed...

[Pierre Labastie]

According to Brendan L on BLFS-dev (see ​http://lists.linuxfromscratch.org/pipermail/blfs-dev/2018-December/035226.html), mesa 18.3 can now accept P3. If it works, the whole stack of modules for Mako can be made P3 only, and funcsigs can be removed.

[Pierre Labastie]

Replying to pierre.labastie:

According to Brendan L on BLFS-dev (see ​http://lists.linuxfromscratch.org/pipermail/blfs-dev/2018-December/035226.html), mesa 18.3 can now accept P3. If it works, the whole stack of modules for Mako can be made P3 only, and funcsigs can be removed.

Done by bdubbs at r20835

[Pierre Labastie]

Replying to pierre.labastie:

List of modules with proposed changes (has to be tested, but please, any information is useful): [...]

• docutils: optional for sshfs, mercurial, dbus-python. According to the book, some of those require P2. Recommend P2, but add P3 instructions.

Thinking a little mode about this one, although mercurail still needs P2, docutils is only used to generate the man page, so ocutils itself can use P3. So only P3 instructions, and make P2 optional.

[Pierre Labastie]

Information added about PyGTk: optional for avahi, nmap, and vte2; recommended for wicd, keybinder2, and gimp

[Pierre Labastie]

There are some issues with scons+P3:

• first, the installed scripts still have #!/usr/bin/env python. This can be fixed by issuing sed -i 's/env python/&3/' script/*, but

• second, serf does not build:

  pierre [ /sources/serf/serf-1.3.9 ]$ scons PREFIX=/usr
  scons: Reading SConscript files ...
  TypeError: cannot use a string pattern on a bytes-like object:
    File "/sources/serf/serf-1.3.9/SConstruct", line 170:
      re.DOTALL)
    File "/usr/lib/python3.7/re.py", line 183:
      return _compile(pattern, flags).search(string)
  

  the SConstruct file for serf is P2 only...

[Pierre Labastie]

Well, seems that the following sed fixes the issue:

sed -i "/get_contents()/s/,/.decode()&/" SConstruct

[Pierre Labastie]

pycairo and six done at r20958

pygobject3 done at r20951

[Pierre Labastie]

Replying to pierre.labastie:

Replying to pierre.labastie:

List of modules with proposed changes (has to be tested, but please, any information is useful): [...]

• docutils: optional for sshfs, mercurial, dbus-python. According to the book, some of those require P2. Recommend P2, but add P3 instructions.

Thinking a little mode about this one, although mercurail still needs P2, docutils is only used to generate the man page, so ocutils itself can use P3. So only P3 instructions, and make P2 optional.

Well, for mercurial docs, if docutils is only P3, it can work, but it needs:

sed -i '/runrst/s/N)/N)3/' doc/Makefile &&
2to3-3.7 -w doc/hgmanpage.py

before make doc. Note that make PYTHON=python3 doc does not work, because doc/gendoc.py uses mercurial internals, which cannot be converted to P3.

sshfs is OK.

[Pierre Labastie]

dbus-python is OK too, if sphinx is installed only for P3.

[Pierre Labastie]

dbus-python and docutils fixed at r20960

[Pierre Labastie]

Fixed at r20966