seamonkey-2.53.15 FTBFS with python-3.11

[pierre]

When running ./mach configure, I get:

[...]
 0:02.87 checking for Python 3... Creating Python 3 environment
 0:03.08 Using base prefix '/usr'
 0:03.08 New python executable in /sources/seamonkey/seamonkey-2.53.14/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python3
 0:03.08 Also creating executable in /sources/seamonkey/seamonkey-2.53.14/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python
 0:03.08 ERROR: The executable /sources/seamonkey/seamonkey-2.53.14/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python3 is not functioning
 0:03.08 ERROR: It thinks sys.prefix is '/usr' (should be '/sources/seamonkey/seamonkey-2.53.14/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3')
 0:03.08 ERROR: virtualenv is not compatible with this system or executable
[...]

Will upload full log. Downgrading to python-3.10.7, there is no error:

[...]
 0:03.49 checking for Python 3... Creating Python 3 environment
 0:05.96 Using base prefix '/usr'
 0:05.96 New python executable in /sources/seamonkey/seamonkey-2.53.14/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python3
 0:05.96 Also creating executable in /sources/seamonkey/seamonkey-2.53.14/obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/bin/python
 0:05.96 Installing setuptools, pip, wheel...
[...]

Note that Creating Python 3 environment takes at most 0.2 s with python-3.11, while it takes more than 2 s with python-3.10.

[pierre]

log with python-3.11

[ken@…]

I looked at building this with 3.10 a while ago ​https://bugzilla.mozilla.org/show_bug.cgi?id=1756371. I'm sure I posted on -dev or -support around the end of September, but I can't find that post.

At that time the current version of the referenced patch ​https://gitlab.com/frg/seamonkey-253-patches/-/blob/master/mozilla-release/patches/TOP-NOBUG-PLASTER-PY310_support-25314.patch did not build with python-3.10 and fedora were using python-3.9 for their build. In fact they still are, ​https://src.fedoraproject.org/rpms/seamonkey/blob/rawhide/f/seamonkey.spec (lines 161, 162)

# temporary to avoid python-3.11 issues	
BuildRequires:	python3.9

I see that upstream patch was updated 5 days ago, but it is still only targetting 3.10 and I have no idea if 3.10 will now build seamonkey (in any case, we have no more interest in 3.10).

[pierre]

The patch fixes a problem with using exclusively python3. If using python2.7, this is not a problem. But virtualenv builds istelf twice, once with python2 (if configure is run with python2), and once with python3. It is when building the python3 part that I get the error above. Also, I see that part of the TOP-NOBUG-PLASTER-PY310... patch is already included in seamonkey-2.53.14 (the part in virtualenv.py). Actually, as already said, I do not see the error with python-3.10.

[pierre]

The error comes from the third_party/python/virtualenv/ version which is too old. I mean if I do:

pip3 install virtualenv==16.7.8 # the version which is in seamonkey source
virtualenv testenv

I get the same error. Now if I do

yes ""| pip3 uninstall virtualenv
pip3 install virtualenv # the most recent version
virtualenv testenv

No error (don't forget pip3 uninstall virtualenv if you do not want to keep it on your system)

[pierre]

There is not much we can do, except downgrading python...

[pierre]

Note that I don't know whether there are other errors once the virtualenv is correctly installed.

[pierre]

Today, Xi Ruoyao suggested to add an empty pyvenv.cfg in the virtual environment. This can be done with:

sed -i "/if sys.executable !=/i\    open(join(bin_dir, 'pyvenv.cfg'), 'w').close()" third_party/python/virtualenv/virtualenv.py

Then ./mach configure still crashes. But running it a second time, it goes to completion. Then the build is ok and the executable runs. I use ​https://www.seamonkey-project.org/dev/qa-verification

[pierre]

The commit that breaks virtualenv in the python repo is commit b9cdd0fb9c46 "Default to using frozen modules unless running from source tree". This behavior can be changed by passing -Xfrozen_modules=off to python3 command line, but I am not sure how to do this in the seamonkey build system.

[Douglas R. Reno]

Seamonkey-2.53.15 beta 1 is out as of 12/7 and appears to have Python porting related fixes in the release notes (amongst a lot of other things, including rustc-1.65 support and support for clang-15)

Glad to see this project is still getting maintained

[Douglas R. Reno]

Seamonkey-2.53.15 will also have fixes up to Firefox 102.5esr (and probably 102.6esr at the end of it's release) due to it upgrading to the newer ESR rendering engine

[ken@…]

Beta1 fails almost immediately when trying to run mach with python pointing to 3.11.1:

AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?

From ​https://stackoverflow.com/questions/74585622/pyfirmata-gives-error-module-inspect-has-no-attribute-getargspec inspect.getargspec was removed from the standard library in 3.11.

So, perhaps it will build with python-3.10, but it won't build with 3.11.

Obviously, this comment does not address the different issue for those who have both python2 and python3.11 installed and want to build with python2 (Comment 7 above)

[Douglas R. Reno]

Seamonkey has released 2.53.15, let's see if that fixes the problem

[Bruce Dubbs]

The release notes are at ​https://www.seamonkey-project.org/releases/seamonkey2.53.15/

I wonder what "Continue the switch from Python 2 to Python 3 in the build system." means.

[pierre]

Nothing has changed with respect to using python 3.11. I've repeated the procedure in comment:6, and now it fails to build: that is:

• apply the sed
• run ./mach configure: fails
• run ./mach configure again: goes to completion
• run ./mach build: stops very early with:

  RuntimeError: maximum recursion depth exceeded while calling a Python object
  

I'm not inclined to try to build this with python-3.11. And I'd say seamonkey should be removed from blfs... It does not bring more than firefox+thunderbird, except a shorter build time, but at the expanse of a downgrade in the version of python used.

[Douglas R. Reno]

I'm going to have a python3.11 system when I do #17558, and I've got some python2->python3 porting experience so I'll take a look at this.

I do personally understand the reason for keeping Seamonkey around and why some users continue to use it (at least two of my instructors at University of Illinois at least). It's difficult to do things such as moving calendar entries to Thunderbird, and it does have Kompozer built in - a WSYWIG HTML editor which works really well for designing/templating web pages.

I expect this to take me a couple of days to get too though, as I first need to get #17558 done and then install some more dependencies.

In the meantime, I'm also going to adjust the name of the ticket, and then mark it as security related since it's got everything from 102.1 to 102.6ESR. This includes CVE-2022-36319, CVE-2022-36318, CVE-2022-2505, CVE-2022-38472, CVE-2022-38473, CVE-2022-38476, CVE-2022-38477, CVE-2022-38478, CVE-2022-3033, CVE-2022-3032, CVE-2022-3034, CVE-2022-36059, CVE-2022-3266, CVE-2022-40959, CVE-2022-40960, CVE-2022-40958, CVE-2022-40956, CVE-2022-40597, CVE-2022-40692, CVE-2022-39249, CVE-2022-39250, CVE-2022-39251, CVE-2022-39236, CVE-2022-42927, CVE-2022-42928, CVE-2022-42929, CVE-2022-42932, CVE-2022-45403, CVE-2022-45404, CVE-2022-45405, CVE-2022-45406, CVE-2022-45408, CVE-2022-45409, CVE-2022-45410, CVE-2022-45411, CVE-2022-45412, CVE-2022-45416, CVE-2022-45418, CVE-2022-45420, CVE-2022-45421, CVE-2022-46874, CVE-2022-46880, CVE-2022-46872, CVE-2022-46881, CVE-2022-46874, CVE-2022-46882, and CVE-2022-46878.

Note however that some (many likely) of those vulnerabilities probably do not actually affect Seamonkey, but it is implied that it's affeced because of Seamonkey's rendering engine. I pulled that list from all Mozilla security advisories since Firefox 102.1 was released (which also had 91.11's fixes, which is the last version of Firefox that Seamonkey had pulled from prior to this release)

This version will also support ffmpeg5 properly as well, and we might be able to get rid of the gstreamer stack dependency from the dependencies over here as well.

[Douglas R. Reno]

I'm now posting this from a version of Seamonkey that identifies itself as 2.53.16b1pre-20230213.

There has been a lot more work towards moving the build system and it's collection of Python modules to Python3, but it's still not quite where it needs to be yet. Upstream appears to be adding translations in now.

I'm not going to upload the patch to bring 2.53.15 to 2.53.16b1pre-20230213 though, since it's 143MB in size due to significant Python3 compatibility updates to the third party modules, the python folder, the build folder, and updates to all of Seamonkey's bundled libraries (as well as ESR102 updates). It's over 4,000,000 lines in size

The next version of Seamonkey, based off the patches in the queue, will definitely prefer Python3 (and it checks for it first when building). Unfortunately though the xpidl parser in particular hasn't been updated yet, which results in the maximum recursion depth error that Pierre mentioned in comment #13.

I've currently got a build of Seamonkey-2.53.15 running with:

grep -rl \"rU\" | xargs sed -i 's/"rU"/"r"/'

sed -i "/if sys.executable !=/i\ open(join(bin_dir, 'pyvenv.cfg'), 'w').close()" third_party/python/virtualenv/virtualenv.py

export CC=clang CXX=clang++ &&

./mach configure | | ./mach configure &&

./mach build

(Note that there's supposed to be two pipes without a brace there for an OR, but Trac formats it weird).

We're expecting ./mach configure to fail the first time due to a conflict with the virtualenv, but it will bypass it properly on the second attempt.

Seamonkey is currently building now with the above changes and is almost to the linking stage as I type this! :)

If you're interested in building your own Seamonkey from their git repository and patchset, here's a general guide

• install Mercurial
• Turn on the 'mq' extension (edit ~/.hgrc, add:

[extensions]
mq=

• Go to your home directory, or somewhere with ~5GB of space

• Run 'hg clone ​https://foss.heptapod.net/seamonkey/mozilla-release comm-253'

• Change into comm-253

• Run 'hg clone ​https://foss.heptapod.net/seamonkey/comm-release comm'

• Go somewhere outside of the comm-253 directory

• Run 'git clone ​https://gitlab.com/frg/seamonkey-253-patches.git'

• Create directories in comm-253 and comm-253/comm: .hg/patches

• Copy the contents of mozilla-release/patches to comm-253/.hg/patches

• Copy the contents of comm-release/patches to comm-253/comm/.hg/patches

• Inside of comm-253, run: 'hg qpush --all --encodingmode replace'

• Inside of comm-253/comm, run: 'hg qpush --all --encodingmode replace'

That procedure got me a working checkout. After that I needed the two seds described above, but it checked for Python3 first and used it all the way up to the point where it needs the xpidl parser (in the "export" stage - which is also where the xpidl.py change is at).

It's worth noting that the sed for xpidl.py that is in the book right now does break the build, so I will comment it out with my commit.

[Douglas R. Reno]

In terms of experimentation over on the comm-253 side, I did attempt to integrate the virtualenv code from third_party/python/virtualenv from Thunderbird. It did work well for the Python3 side, but crashed immediately on the python2 side (which is down to just three modules - just enough for that xpidl parser!)

I did also attempt to integrate it with the current version of virtualenv but didn't get too far on that either

Overall this is a significant improvement, if all goes well I can submit the update in about 2 hours.

[Douglas R. Reno]

Fixed at 8a71122cdf948e949db73f08cb0edd9f4dc4e905

Security advisory coming in a few minutes

[Douglas R. Reno]

SA-11.2-088 issued