source: README.BLFS@ fd4a798

ablfs-more legacy trunk
Last change on this file since fd4a798 was fd4a798, checked in by Pierre Labastie <pierre.labastie@…>, 3 years ago

Remove $Id$ comments, they are useless with git

  • Property mode set to 100644
File size: 17.6 KB
Line 
11. INTRODUCTION::
2
3 To automate package builds from the BLFS book instructions is a huge
4 task. Some of the issues are: the BLFS book isn't linear; some package
5 pages use a custom layout; there are circular dependencies; several
6 packages can be installed on a non-default prefix; build commands can
7 change based on what dependencies will be used, etc.
8
9 That being said, the goal of the blfs-tool is to help you solve package
10 dependencies, create build scripts and a Makefile. Not all the auto-generated
11 build scripts and Makefile will work "as is", thus, as a general rule,
12 you will need to review and edit the scripts while reading the book.
13
14 Since version 3.0 of jhalfs, the blfs tools allow also to update packages
15 from the LFS book. LFS packages which may be updated appear in the menu
16 interface. When selected, their scriptlet is generated in the same manner
17 as for BLFS packages.
18
192. PREREQUISITES::
20
21 In addition to a full LFS system, the following packages and their
22 dependencies are needed by this tool:
23 - required: libxml2, libxslt, DocBook XML DTD
24 - recommended: wget (to download the package tarballs) and sudo (to build
25 as a user)
26 - optional: lynx (allows to read the generated linearized book), GPM (to
27 cut and paste commands from the book), subversion (to update the book
28 sources), openssl (used by wget for all https:// sites)
29 Note that the optional dependencies are recommended for ease of use of the
30 tool.
31
32 You should also have the following personal skills:
33 - Ability to write and debug shell scripts: as said in the introduction,
34 not all the generated scripts can be used directly. They need to be
35 edited to produce an error free build.
36 - Ability to debug build failures, like missing dependencies or
37 installation directories not known to the system (when you install in
38 /opt for example).
39 - Ability to choose the tools you need to configure and administrate
40 your system: in the BLFS book, nothing is mandatory, nothing is
41 useless. You are on your own in choosing what to build, but wrong
42 decisions may lead to a non functional system...
43
443. INSTALL::
45
46 There are two ways to install the BLFS tools on an LFS system, described
47 in paragraphs 3.1 and 3.2, respectively:
48
49 3.1 INSTALLATION ON A RUNNING SYSTEM
50
51 Select "Use Book --> Beyond Linux From Scratch" in the jhalfs menu:
52 The tools are installed in $HOME$BLFS_ROOT (the default for $BLFS_ROOT
53 is /blfs_root). The BLFS book is downloaded or copied to its directory.
54 The tracking directory (see below) is initialized but not created: before
55 the installation, you should ensure the tracking directory (default location
56 /var/lib/jhalfs/BLFS) exists and is writable by the user. After the
57 intallation, you should perform the following additional steps:
58
59 - Configure sudo, adding the needed privileges for the user. For
60 newer sudo version, do not forget to add a line `Defaults secure_path='
61 containing /sbin and /usr/sbin (in /etc/sudoers), otherwise some
62 executables are not found.
63
64 - Although it is not strictly necessary, it is recommended to install
65 the bash shell startup files (as per `3.After LFS Configuration
66 Issues' of the BLFS book), as some instructions in BLFS rely on
67 their being present.
68
69 - In this case, the tool has no way to know which version of LFS packages
70 is installed, so that the menu interface will show all the LFS packages,
71 as if they were not installed. If you have a released version of LFS, or
72 the date of your GIT version of LFS is known, you should run the
73 update-lfs.sh script. If you have updated some
74 LFS packages since first installation, or have been using a custom
75 working copy of the LFS book, the only (tedious) way is to create
76 empty files with names <package>-<installed-version> in the tracking
77 directory, and run the tool.
78
79 - If you have also installed some BLFS packages, they are not in the
80 tracking file. The only way is to create empty files with names
81 <package>-<installed-version> in the tracking directory, and run the tool.
82
83 3.2 INSTALLATION ON A JUST BUILT xLFS SYSTEM
84
85 For books that support it (only LFS for jhalfs version 2.4),
86 there is an option to install the BLFS tools right after building
87 the xLFS system: just tick `BOOK Settings/Add blfs-tool support' in
88 jhalfs configuration menu. The tools are installed in $BLFS_ROOT
89 (default /blfs_root) on the xLFS system, and the dependencies are built
90 at the end of the jhalfs run, before the custom tools.
91
92 (TODO: blfs-tools have not been tested with current (version 3.0) of CLFS,
93 and certainly need some adaptation to run)
94
95 After booting the new xLFS system some steps are needed to finish
96 the installation of the automated tools:
97
98 - A user account must be created. You must be logged on that user
99 account to use blfs-tool. This is not strictly necessary,
100 since the packages can be built as root, too, but it is
101 never a good idea to build packages as root.
102
103 - Move /blfs-root to that user's home and change ownership of the
104 directory and files to the user.
105
106 - Give the user read and write privileges over the $TRACKING_DIR
107 directory and the files that it contains.
108
109 - Configure sudo and add the bash shell startup files, as described
110 above
111
112 - Note that the versions of LFS packages are automatically known to
113 the tool in this case, and there is no need to run the update-lfs.sh
114 script.
115
116 We assume that the BLFS tools will be used on a booted xLFS system.
117 Using them to build BLFS packages in a chroot jail is also possible,
118 but not supported.
119
120 3.3 DIRECTORY LAYOUT IN THE $BLFS_ROOT DIRECTORY
121
122 blfs-xml/* GIT tree of the selected BLFS book version
123 lfs-xml/* GIT tree of the selected LFS book version
124 lib/constants.inc functions libraries
125 /func_dependencies for building the dependency tree
126 menu/* lxdialog and menuconfig source code
127 xsl/gen_pkg_list.xsl XSL stylesheet to generate the package database
128 /gen_config.xsl XSL stylesheet to generate the Config.in file
129 for use in the menuconfig system
130 /dependencies.xsl XSL stylesheet to generate the dependency list
131 of a package
132 /make_book.xsl XSL stylesheet to generate the linear book.xml
133 /lfs_make_book.xsl XSL stylesheet to incoporate LFS pages into the
134 linear book.xml
135 /scripts.xsl XSL stylesheet to generate the scriptlets from
136 book.xml
137 /bump.xsl XSL stylesheet to update the tracking file
138 README.BLFS this file
139 TODO developers notes (well, not updated often)
140 gen_pkg_book.sh resolves dependencies and generates linear BLFS
141 books and build scripts
142 gen-makefile.sh generates the target Makefile
143 progress_bar.sh the target Makefile progress bar
144 gen-special.sh Helper script for generating the package database
145 Makefile Used by make to update the package database from
146 the GIT tree, then launch the menuconfig interface,
147 and run gen_pkg_book.sh based on configuration
148 settings
149 packdesc.dtd a simple DTD describing the format of the package
150 database and the tracking file.
151 envars.conf envars needed when running the target build scripts
152
153Working files: several files are generated when first running the tool
154
155 packages.xml auto-generated packages database
156 Config.in input file for the menu driven choices
157 configuration file generated by the menuconfig process
158 dependencies/* files recording the dependency tree
159 book.xml the linearized book
160 book-html/* the linearized book rendered in html
161 scripts/* the scriptlets
162
163 3.4 INSTALLED PACKAGES TRACKING SYSTEM:
164
165 This tool includes a very simple tracking system to log which packages
166 have been installed using the tool. It is used to skip installed packages
167 from target selection menu and to test if an installed package has been
168 updated in the BLFS book. Do not rely on this feature as a package
169 management tool.
170
171 The tracking system itself is an XML file: instpkg.xml. It is
172 initialized when <make> is first run in blfs_root. It resides in a
173 directory, which is created when needed during the process of building
174 custom tools or blfs dependencies, right after xLFS. You can specify
175 that directory location in the blfs-tools sub-menu of jhalfs. You may
176 need to update permissions and/or ownership of this directory before
177 using the blfs tool (see README in jhalfs).
178
179 The default location of the tracking directory is /var/lib/jhalfs/BLFS.
180 NB : after the initial build, that directory is only used to contain
181 instpkg.xml, unless custom tools have been built. In the latter case,
182 it also contains empty files whose name are "$PKG-$VERSION" for each
183 versionned package built. The information about those packages is
184 included into instpkg.xml the next time the tool is run.
185
1864. USAGE::
187
188 From now on, all the work must be done from inside the installation
189 root directory.
190
191 Due to the complexity of the BLFS book, the scripts and Makefile
192 generation is done in several steps:
193
194 4.1 UPDATING BOOK SOURCES::
195
196 If you are using the development book version and you want to update
197 installed packages to the latest version found in that book, you need to
198 update the XML sources and packages database. This is not necessary if
199 you just built xLFS, and you can skip to step 3.4. To do that, run
200 "make update".
201
202 On the next configuration run, packages already installed but listed
203 with a new version in the book will be available for target selection
204 and used to solve dependencies.
205
206 4.2 CONFIGURING AND PARSING THE BOOK::
207
208 The next step is to create a book and build scripts in dependency
209 build order for one or several packages.
210
211 Run <make> to launch the configuration interface. The main menu contains
212 two blocks: individual package selection, and build options.
213
214 In the build options section, the dependencies level and default packages
215 used to solve alternatives are set (currently, only for the MTA). You can
216 also select whether the build will be made as a normal user or as root.
217 Those settings are saved to be reused in future configuration runs.
218
219 Note that you may select as many targets as you want, not just one
220 as in the previous version of this tool. But we suggest to not select
221 too many at a time to be able to sort issues!
222
223 When you are done with the menu, a few checks occur, and the dependency
224 chain is generated. Each dependency appears with its priority (required,
225 recommended, optional, or external), and it's level. There is a root level
226 1. The selected packages have level 2. The dependencies of selected packages
227 have level 3, the dependencies of the dependencies have level 4, and so on.
228 When circular dependencies are found, they appear with a priority of
229 "circular". This means that two (or more) dependency chains arrive at the
230 same package. The algorithm chooses the chain with the highest priority and
231 reorders dependencies to remove the other chain(s). This is not always the
232 solution an user would prefer, but we have found no way to do it better.
233
234 You end up with a book.xml file which contains the linearized book,
235 and a rendered HTML, in the directory book-html, which you can browse with
236 "lynx book-html/index.html" (or with any other browser).
237
238 Furthermore, there is a directory "scripts", which contains the generated
239 scriptlets.
240
241 There is also another directory, "dependencies" that contains files
242 generated while resolving dependencies.
243
244 4.3 EDITING BUILD SCRIPTS::
245
246 Now it is time to review the generated book and scripts, making any
247 changes to the scripts necessary to fix generation bugs or to suit your
248 needs.
249
250 Scripts for additional packages (i.e., for non-BLFS packages) can be
251 easily inserted. For example, if you want to install the external dependency
252 "bar" before "foo" package and the "foo" script is named "064-z-foo", you
253 just need to create a "064-y-bar" build script.
254
255 Remember, the package tracking system isn't a package management tool
256 and knows nothing about packages not in the BLFS book.
257
258 IMPORTANT: Review and edit envars.conf, at least after installing the
259 tool. This file is used to set global envars needed by the build scripts.
260 If you use package management, the variable JH_PACK_INSTALL should point to
261 the directory where the packInstall.sh script resides.
262
263 4.4 CREATING THE MAKEFILE::
264
265 When the build scripts are ready to be run, the Makefile can be
266 created. Create an empty directory (for example "mkdir work") and cd
267 to that directory. Then run ../gen-makefile.sh
268
269 Review the Makefile, and, if all looks sane, start the build by running
270 "make".
271
2725. GENERATED BUILD SCRIPTS ISSUES::
273
274 In this section, known issues with the generated build scripts are
275 discussed. They are due to build procedures and/or BLFS layout
276 particularities that we can't handle. In several cases, editing the
277 build scripts is mandatory.
278 You may also need to insert some build scripts created by you to resolve
279 unhandled dependencies and/or to remove some script installing an unneeded
280 package (unneeded packages may be pulled in the dependency chain, if
281 they occur as an "or" with another package).
282 When there are circular dependencies (only one known in BLFS 8.0 for
283 recommended dependencies), you may need to move around scripts so that they
284 run in the order script-A script-B script-A. This involves copying script-A
285 to another name (using the xxx-a- fields), and possibly renaming the xxx-a-
286 fields of each involved script.
287
288 5.1 BLFS BOOTSCRIPTS::
289
290 Normally, bootscript installation should work. On the other hand, the
291 book does not give instruction for running them, so you might have to
292 manually insert "/etc/init.d/rc.d/<initscript> start" at some place during
293 the build.
294
295 5.2 PACKAGE CONFIGURATION::
296
297 For those packages that have a "Configuration" section, you should
298 edit the build script to fit the needs of your system. Sometimes, the
299 bash startup files are modified. The shipped 'envars.conf' contains a
300 line 'source /etc/profile', which ensures that the proper environment
301 variables are used.
302
303 5.3 PAGES WITH TWO OR MORE PACKAGES::
304
305 For example: sane, poppler, audacious, freetts, which, etc.
306
307 On the pages for those packages, the BLFS book actually has instructions
308 to download and install two or more packages. You must edit the scripts to
309 fix this. A common pitfall is that the variable PACKAGE may be used for
310 several tarballs. Be sure to save the PACKAGE variable to some other
311 name (for example PKG1, PKG2, etc) after each download. The unpacking
312 instructions may need to be repeated for each tarball in turn.
313
314 5.4 XORG7
315
316 The book has special page layouts for the Xorg7 packages. The tool
317 breaks those pages into individual pages for each packages in the linear
318 book. Also, the menu gives the choice to select each package individually.
319
320 To build the whole Xorg7 chapter, select twm. The (recommended)
321 dependency chain brings in the whole set of Xorg packages.
322
323 5.5 PATCHES
324
325 Please, make sure that all scripts have the commands to download/apply
326 the required patches. Due to book layout issues, some patches may be
327 missing (as of BLFS 8.0, all the patches seem to be downloaded).
328
329 5.6 ROOT COMMANDS
330
331 If building as a normal user (the default setting), be sure that all
332 commands that require root privileges are run using sudo. Also make sure
333 necessary root privilege commands are visible in your PATH. Or use
334 the `Defaults secure_path=' in /etc/sudoers.
335
336 For commands necessitating root privileges, the generated scripts wrap
337 them with the construct:
338 sudo -E sh << ROOT_EOF
339 <commands to be executed as root with `$', ``', and `\' escaped>
340 ROOT_EOF
341 The -E switch ensures the whole environment is passed to the
342 commands to be run with root privileges. It is effective only if the
343 /etc/sudoers file contains `Defaults setenv', or SETENV in the user
344 attributes. If you think it is a security issue, you may forbid this
345 flag in /etc/sudoers, but then, you have to un-escape `$' for variables
346 coming from the environment in the instructions.
347 Although this construct is rather strong, it can fail in some corner
348 cases, so carefully review those instructions.
349
350 WARNING: One variable from the environment is not passed through the
351 -E switch, namely PATH. This is because "sudo" always reset the PATH to
352 the default "secure_path". If you need to have the same PATH as the user
353 "root" would have, you may want to add "source /etc/profile" at the
354 beginning of the commands to be executed as root.
355
356 Due to book layout issues, some sudo commands may be missing.
357
358 5.7 OTHERS
359
360 There may be other issues that we are not aware of. If you find
361 any, please report it to <alfs-discuss@linuxfromscratch.org>.
362
Note: See TracBrowser for help on using the repository browser.