Opened 8 months ago

Closed 8 months ago

#22693 closed defect (fixed)

Gnome 49 problems to tackle before blfs 12.5

Reported by: pierre Owned by: pierre
Priority: normal Milestone: 13.0
Component: BOOK Version: git
Severity: high Keywords:
Cc:

Description

Package freeze for 12.5 is in about three weeks, and there are several problems with gnome 49. We have to make a decision about non-systemd builds for the release, and even on systemd, a few issues remain. Here is a list of the issues and options:

systemd

  • the environment of the gnome system is the one provided by the system (not sure exactly where it is fixed, gnome-session, gdm, ...), not the one set in our /etc/profile.d files. In particular PATH is set to /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin, and programs in /opt cannot be found.
  • the commands for starting gnome on the gnome-session page don't work at all. I'm not sure we want to keep this paragraph, but if needed, I've found a way to start gnome without using gdm with (XDG_SESSION_ID has to be taken form loginctl output):
    systemctl --user set-environment XDG_SESSION_TYPE=wayland XDG_SESSION_ID=1
    systemctl --user start gnome-session-wayland@gnome.target
    

SysV

  • issue (major): nothing works
  • options (no comment here):
    • drop gnome on SysV
    • use openrc as init system
    • use systemd as a user process manager, while keeping sysv as init system

Change History (21)

comment:1 by pierre, 8 months ago

For the systemd on SysV route, I have something that works (running a systemd manager as PID not equal to 1, and starting gdm and gnome, with the same issues as the systemd ones described in the ticket description), but it gives control to systemd for starting dbus, and I am not sure it is acceptable for blfs users. It also replaces elogind, so that users needing only elogind would have to install the whole systemd stack (and systemd-logind would be started by systemd too).

I'll publish a branch (plabs/sysdonv), but it needs a lot of XML editing for dependencies and removing instructions that reject systemd, so don't expect it too soon...

comment:2 by pierre, 8 months ago

For the openrc route, what I see is:

  • we'd have to rely on some third party (gentoo devs) for patching gnome-session (and possibly other packages). This may lag behind most recent version of gnome, and continuity is not guaranteed. And I don't think it would be easy to maintain ourselves.
  • having three init systems could be too much, so SysV would have to be dropped.

Note that zeckma has a working lfs book for openrc, but I am not sure about blfs (please correct me)

comment:3 by pierre, 8 months ago

For the gnome removal route on SysV, renodr has made several objections about difficulty of maintenance, since several gnome packages are needed by other packages in blfs, so that a complete removal cannot be done.

comment:4 by Bruce Dubbs, 8 months ago

These are problems that need to be addressed. There are other issues we are working on right now. We need to update several non-BLFS applications on rivendell, specifically Trac, sympa, and probably clamav and spamassassin. We are trying to test on a new server, balrog, but that is not going quickly.

There are some other options for gnome.

 - revert to gnome-48
 - port the needed portions of systemd to sysV similar to blocaled and elogind
 - delay the release of BLFS 12.5 (BLFS 13.0?) a month to address the issue
 - put a notice in LFS that if the user will want to build/use gnome-49, then
   the sysV version of LFS is not appropriate.

Right now I'm leaning toward that last point.

I'll also add that most gnome applications still work in sysV. Examples are gedit and gnome-connections, but I haven't tried them all.

comment:5 by Douglas R. Reno, 8 months ago

Right now I am leaning towards delaying the release of BLFS 12.5 to examine the possibility of using OpenRC. The other reason is simply that I haven't had enough time lately to take care of everything that needs to happen here, and that will hopefully get better over the course of the week.

One of my concerns about going the route of integrating the systemd components in SysV is that we're trending even farther into unsupported by upstream territory (since the developers of GNOME have merged some changes related to the Shepherd and OpenRC routes), and we're at the mercy of systemd updates potentially breaking things which is even more problematic to me. If you can get it working though, I would be down to at check it out and give it a good consideration.

The reason why I want to use OpenRC is that the user services functionality that we need is already integrated there, and it uses similar enough shell scripts that transitioning wouldn't be too difficult. However, Zeckma will need to get a BLFS branch going, and she (just like me) is really low on time at the moment. The question regarding the unreliability of the gnome-session on OpenRC changes that one of the Gentoo developers is one that I understand, but I think the risk is minimal there since the developer for that is already testing it against GNOME 50 components (heard that on the podcast they did recently). The Gentoo community is also much larger which would at least result in us being able to ask questions, and compared to SysV there are significantly more distros that could help if we needed it as well.

I don't think anybody is doing the systemd on SysV approach - and while I like the idea, I don't want us to have to be the only ones supporting it and then getting into major trouble if one of us needs to step away for a period of time.

Putting a notice in the book isn't going to work. We've seen that be a problem so many times in the past, where people just simply don't read and then bring it to us complaining that it doesn't work. I don't want to end up in a similar situation especially because it feels like we are papering over the problem and I don't want to have to respond to mailing list messages about things not working when we already knew that the issue was a major one. So many other DEs use GNOME infrastructure (even LXQt and Plasma!) that I'd much rather not risk going an unsupported route. Reverting to GNOME 48 is also not an option IMO because it goes EOL when GNOME 50 comes out in March. Eventually we are going to need to do this anyway because Plasma will start depending on this and gvfs may also start depending on user service support being present, which will cause issues for SysV and other DEs too.

At the same time I've been struggling personally with handling all of the tasks and priorities which keep coming my way, especially since everyone keeps saying "I need you to prioritize this" and nothing is getting done because I can't focus for long enough on one task without other people bugging me about another. Right now on top of Balrog I also need to get things straightened out on the CI server for LegacyUpdate so we can do a release over there this week to fix several critical bugs that can corrupt OS installs. In addition I'm being bugged by my parents about the computer that drives the 3D Printer throwing Disk I/O errors, by my brother to get security camera footage because his car was hit and he just now noticed this morning, and I also have my first grad school advisor meeting tomorrow. There's also the OpenJDK update that needs to be done, and I keep having to restart my Minecraft servers that pay me once a day because they keep getting overloaded with requests related to CVE-2026-21945 and CVE-2026-21933. but I haven't had time to update those yet either. I probably need to step away for a few days to prioritize getting everything else under control so that I can focus on *LFS correctly but I feel bad about doing that because I set a date/time with Bruce to have Balrog ready by Tuesday and do our updates to Rivendell on Wednesday, and I could probably still have things ready but it's going to be even more hectic. I also don't want this to run into package freeze, but we desperately need the updates on rivendell - not only for security reasons, but also because Trac gets so unstable at times of more than 1 or 2 people reading/modifying/etc. tickets at the same time because we haven't upgraded it.

All of this to say, I need some time before we go any particular route to make sure I can be here fully to handle testing, proofreading, and implementation. My vote is still on OpenRC, but I could be swayed by the systemd route as long as things are stable.

Now to the GNOME on systemd part...

I think the paragraph should be adjusted for sure regarding starting without gdm. I think it's useful for debugging in particular, so I'd like to keep that if we can. Xi has brought up in the past the changes that need to be made to the Bash Shell Startup Scripts configuration to get the PATH settings to work with gnome-session, but I need to find them and do the work for it when I have time. The underlying cause of this is that the startup script for gnome-shell was rewritten in C from Bash, and we used to use a hack to force a login shell that we can't do anymore. This is part of the reason why I am not on board with just throwing more hacks at the situation because eventually they will break just like this one, and it's back to the drawing board to come up with a solution again. Eventually though again KDE will go to this same approach, very likely with the Plasma release that entirely drops X11 support soon-ish.

comment:6 by zeckma, 8 months ago

I'll chime in tomorrow when I have time. Currently working on other things on the moment.

comment:7 by zeckma, 8 months ago

I want to first address the idea of having OpenRC or Systemd as the process manager, while having SysVinit as the init. I think a reason people don't like Systemd is that it's a complex set of software, with a complex way of handling things, and does a lot to keep things held together. Using something other than Systemd is an attempt to solve at least one of those problems. When you combine OpenRC or Systemd as the process manager and another init system as the init, then you squander one of the benefits of using another init system: simplicity. The main idea is to have the ability to still use init scripts, but its cost is added complexity. There also doesn't seem to be much benefit of doing it this way.

Complexity is one reason, but it raises issues with logind as well. Was elogind present, was anything built against it, is the process manager replacing elogind with another logind variant like systemd-logind? If so, the migration plan would be a long grocery list. You could use OpenRC instead of Systemd in this case, but it still carries a lot of complexity, given that OpenRC already supplies its own init system which is more than good enough. Parallel support could be a bit better but I found it mostly fine.

For dropping GNOME for SysVinit, it eventually won't be just GNOME that will have to be dropped, it will be KDE, and many other DEs. We will just be putting a bandaid on the issue and hope users won't get angry until it will eventually come to a situation where someone, most likely Doug, will have to spend weeks getting everything to work since we put off the issue. It's simply not sustainable unless we abandon DEs altogether, and going scorched earth is not a good solution.

For simplicity, that just leaves two options: drop all alternatives and go Systemd-only like Arch, or go OpenRC for both init and process management for the alternative.

The first option would save us a lot of effort, and we could make things easier for the user who wants to not go Systemd, but we leave them to do things on their own. It's by far the most polarizing.

The second option involves some level of work. Most of it will fall on me initially. Once things are in a good state in BLFS land, things will speed up and people like Doug can test the packages that need specific environments, namely GNOME, Kea, etc.

Beyond that point, it'll be smooth sailing, since all support will then be outsourced to Gentoo, essentially, and developers who have way too much time on their hands. A point talked about is that this is a bad thing, and it certainly can become a bad thing. But OpenRC is basically the most popular Systemd alternative, with a big enough userbase that demands support for projects like GNOME. If it was SysVinit, there'd be no support and we'd be the only ones supporting it till efforts shrivel up. In other words, going OpenRC helps us work less and be ensured that we have support for a long time. We won't need to foster solutions, since everyone else will foster them.

Again, the main issue is the initial work that's needed. But we have a good base, and a foundation for me to go on, and I aim to get things done pretty soon.

That's thinking for the future though. Immediately, my mind is in two places: hurry to get everything done and in place before 12.5/13.0 deadline, or drop GNOME for Systemd alternative for 12.5 then add OpenRC in 13.0 so there's more time.

I can't say I like being rushed but users have now been without GNOME on a Systemd alternative in LFS for a while. So we should probably hurry to get things done.

So my coin put in the pot is to hurry to get OpenRC support in BLFS before 12.5/13.0, drop SysVinit entirely, and ensure GNOME works on OpenRC and can access shell variables before being started automatically. Once that's done, I'll switch from Systemd to OpenRC to be the main or one of the main OpenRC editors.

comment:8 by Bruce Dubbs, 8 months ago

Zeckma, Thank you for your input. I've been thinking about this for a while and have not made up my mind, but I am considering dropping sysV completely and making LFS systemd only.

The reason for this thought is that maintaining two versions of LFS and BLFS is, in aggregate, too much work for the number of editors we have. It's not just the init systems, but the number of packages that have different build instructions for the different systems. That means that editors really need to have both versions of the books to properly test a new package. Many things have to be done twice.

I'll note that the problem really only occurs for workstations. I do not think that there are any packages for a server that need different build instructions for the different systems other than using units instead of boot scripts.

The main reason for using sysV in my opinion is pedagogical. If someone wants to understand the boot process, then reading boot scripts is very easy. The complexity of systemd with it's embedded features like ntp, socket/D-Bus activation, dhcp client, logging, etc make it difficult to see what is going on when looking at details.

One of the earliest features of systemd was supposed to be fast booting through parallelization of tasks when starting. I never thought that was an advantage. On my sysV workstation the boot time from the first boot script to the last is 5 seconds. Does it need to be faster? The bios initialization takes longer than that.

In any case we probably need to delay release of the next stable version of LFS/BLFS for a month or more.

comment:9 by zeckma, 8 months ago

I'm also really tempted to say to just go Systemd-only so we can concentrate all our efforts instead of them being divided. Kea being busted on Systemd was a result of this. Kea was also broken on SysV so it isn't the best example.

The main issue just comes from not showing users how to install elogind and how to make packages adapt for it, as well as making users endure a choice on whether to continue to have an elogind-based system or to switch to Systemd and make them recompile everything that was linked against elogind to libsystemd. I think it'd make a lot of users upset, and while I never met anyone from any universities who use LFS, I imagine plenty who use LFS in a professional setting use elogind.

We would need to have a good migration plan and be very vocal about such a change, and potentially reach out to contacts who have reached out to us in the past. The mailing lists, Reddit, and Discord would be a start, in which I can handle the Reddit and Discord side of things.

There'd be a lot of heat initially. But after it's endured, it'd save a lot of our time, namely Doug's time. I kinda feel bad for him, every couple of months he has to pull out a SysV system to test a package which was broken for an unknown amount of time that no one else is testing, just eats a lot of time that can be spent in more areas.

If we had more editors, having two or more init systems would be something we could better support, but we're no Gentoo. So we may have to scale down. I'll also think a little more since both SysV -> OpenRC and Sysd-only seem like really good options.

comment:10 by pierre, 8 months ago

Just a note about systemd on SysV, but I am not pushing it, just I want to share my findings:

  • in my present setting, there are only two sed's to the systemd code, and there is a need for creating some needed files in /run/systemd, that are only created if running as PID 1. Those are the undocumented parts (I may have to add another sed to allow shutdown). Everything else is documented in systemd's doc, so that it is less likely to break

Now about the whole picture: first a general comment (you can TL;DR it):

  • after spending several weeks dissecting systemd, what occurs to me about the complexity is not the fact that using units is less direct than using bootscripts, but the fact that so many hidden ("automatic" in systemd parlance) dependencies can occur. Those automatic deps depend on the type of the unit, on whether the system is in a start or shutdown phase, and maybe more, so that you cannot know what you are doing when modifying a unit unless you have the doc open on your desk. And more than that, they can never be completely disabled even if using DefaultDependencies=no. That's the real reason for not liking systemd IMO. Anyway, units are mostly provided by packages, so only a few need to be written/modified.

Then about having only one init system (systemd) in LFS:

  • as said, it would simplify maintenance a lot. The problem is not the amount of work it involves but the fact that editors who specialize on some packages may step away for external reasons. I think the kea problem is more due to the stepping away of Thomas Trepl than to a lack of attention to sysv. I am also one of those who had to step away for a while, and I can understand people need to do that at times. If all the openrc stuff for lfs relies on zeckma, what if she has to step away for whatever reason?
  • just a question: if for some reason lfs would start today, what init system would it use? my answer: When LFS started, it was using the most up to date init system, the one that was used by all big distros, that is: sysv (there were others at the time). And lfs at least initially copied/adapted many bootscripts from those distros. So I think nowadays it would be using systemd...
  • zeckma seems to be concerned about users having to move from sysv to systemd without rebuilding from scratch. My personal opinion is that there are no users who don't rebuild everything at least for each new release. The reason is that it is very hard to maintain an lfs system up to date without recompiling everything first. Not only new versions of the toolchain, but also of python, perl, libffi, and several others involve a significant rebuild, and it is easier to rebuild from scratch than to determine which packages need to be rebuilt. So in my opinion, moving to systemd is not technical problem (but may involve a lot of reluctance, of course). But maybe zeckma's concerns are of different nature, sorry if I misinterpreted.

comment:11 by zeckma, 8 months ago

I decided to reach out to the LFS Discord server of our situation for ideas on how we should proceed. The overwhelming message is to drop SysVinit in favor of OpenRC and don't go Systemd-only. We were also asked to consider other init systems like Runit, but with our limited resources, and with our already established OpenRC base, the best idea seems to continue with the Gentoo-backed OpenRC approach.

This won't save us as much time as the Systemd-only route, but will still save us time in the end, is less polarizing, and users will still have an easy resource for a Systemd init alternative.

Considering the input from our users, I suggest we switch out SysVinit for OpenRC.

comment:12 by Xi Ruoyao, 8 months ago

The environment variable issue hacked around at r12.4-1124-gcb30ecc0af (for systemd).

comment:13 by pierre, 8 months ago

While I agree on the conclusion for the direction to take now, I think I'll keep trying the "systemd on other init" (non PID 1 systemd rather) option. If I get to something interesting, I'll publish as a branch. Showing everybody that systemd can run as a system unit manager while not being PID 1 would be great IMO. But my moto is: minimal change. This is not the same as gentoo approach (elogind + gnome-openrc, they'll have to add kde-openrc at some point), which relies on big changes to upstream code. And I don't think they have a plethora of devs. The elogind dev seems pretty alone (almost all commits are from "yamakuzure"). Not sure about the gnome-openrc one.

in reply to:  12 ; comment:14 by pierre, 8 months ago

Replying to Xi Ruoyao:

The environment variable issue hacked around at r12.4-1124-gcb30ecc0af (for systemd).

Shouldn't the file /etc/systemd/user-environment-tenerators/50-profile.sh be made executable to all (chmod a+x)?

in reply to:  14 comment:15 by Xi Ruoyao, 8 months ago

Replying to pierre:

Replying to Xi Ruoyao:

The environment variable issue hacked around at r12.4-1124-gcb30ecc0af (for systemd).

Shouldn't the file /etc/systemd/user-environment-tenerators/50-profile.sh be made executable to all (chmod a+x)?

Oh indeed. I tested the instructions but I didn't remove my draft file at the location which already has the executable bit, so the cat command just writes into the existing +x file and then I failed to find the issue :(.

I'll correct it now.

comment:16 by Bruce Dubbs, 8 months ago

Milestone: 12.5 → 13.0

Milestone renamed

comment:17 by Bruce Dubbs, 8 months ago

Resolution: → overcomebyevents
Status: new → closed

Closing due to decision to go to systemd only.

comment:18 by pierre, 8 months ago

Resolution: overcomebyevents
Status: closed → reopened

The paragraph on starting gnome-session manually is still wrong (this is a systemd issue). Reopening

comment:19 by pierre, 8 months ago

Owner: changed from blfs-book to pierre
Status: reopened → new

comment:20 by pierre, 8 months ago

Status: new → assigned

comment:21 by pierre, 8 months ago

Resolution: → fixed
Status: assigned → closed

gnome start fixed at 337dd6f3863

Note: See TracTickets for help on using tickets.