source: README.BLFS@ a659e46

ablfs-more legacy trunk
Last change on this file since a659e46 was 1fa0dee, checked in by Pierre Labastie <pierre@…>, 7 years ago

Prepend JH_ to exported variables in jhalfs-blfs, so that collision with
build system variables is unlikely

  • Property mode set to 100644
File size: 17.9 KB
Line 
1$Id$
2
31. INTRODUCTION::
4
5 To automate package builds from the BLFS book instructions is a huge
6 task. Some of the issues are: the BLFS book isn't linear; some package
7 pages use a custom layout; there are circular dependencies; several
8 packages can be installed on a non-default prefix; build commands can
9 change based on what dependencies will be used, etc.
10
11 That being said, the goal of the blfs-tool is to help you solve package
12 dependencies, create build scripts and a Makefile. Not all the auto-generated
13 build scripts and Makefile will work "as is", thus, as a general rule,
14 you will need to review and edit the scripts while reading the book.
15
16 Since version 3.0 of jhalfs, the blfs tools allow also to update packages
17 from the LFS book. LFS packages which may be updated appear in the menu
18 interface. When selected, their scriptlet is generated in the same manner
19 as for BLFS packages.
20
212. PREREQUISITES::
22
23 In addition to a full LFS system, the following packages and their
24 dependencies are needed by this tool:
25 - required: libxml2, libxslt, DocBook XML DTD
26 - recommended: wget (to download the package tarballs) and sudo (to build
27 as a user)
28 - optional: lynx (allows to read the generated linearized book), GPM (to
29 cut and paste commands from the book), subversion (to update the book
30 sources), openssl (used by wget for all https:// sites)
31 Note that the optional dependencies are recommended for ease of use of the
32 tool.
33
34 You should also have the following personal skills:
35 - Ability to write and debug shell scripts: as said in the introduction,
36 not all the generated scripts can be used directly. They need to be
37 edited to produce an error free build.
38 - Ability to debug build failures, like missing dependencies or
39 installation directories not known to the system (when you install in
40 /opt for example).
41 - Ability to choose the tools you need to configure and administrate
42 your system: in the BLFS book, nothing is mandatory, nothing is
43 useless. You are on your own in choosing what to build, but wrong
44 decisions may lead to a non functional system...
45
463. INSTALL::
47
48 There are two ways to install the BLFS tools on an LFS system, described
49 in paragraphs 3.1 and 3.2, respectively:
50
51 3.1 INSTALLATION ON A RUNNING SYSTEM
52
53 Select "Use Book --> Beyond Linux From Scratch" in the jhalfs menu:
54 The tools are installed in $HOME$BLFS_ROOT (the default for $BLFS_ROOT
55 is /blfs_root). The BLFS book is downloaded or copied to its directory.
56 The tracking directory (see below) is initialized but not created: before
57 the installation, you should ensure the tracking directory (default location
58 /var/lib/jhalfs/BLFS) exists and is writable by the user. After the
59 intallation, you should perform the following additional steps:
60
61 - Configure sudo, adding the needed privileges for the user. For
62 newer sudo version, do not forget to add a line `Defaults secure_path='
63 containing /sbin and /usr/sbin (in /etc/sudoers), otherwise some
64 executables are not found.
65
66 - Although it is not strictly necessary, it is recommended to install
67 the bash shell startup files (as per `3.After LFS Configuration
68 Issues' of the BLFS book), as some instructions in BLFS rely on
69 their being present.
70
71 - In this case, the tool has no way to know which version of LFS packages
72 is installed, so that the menu interface will show all the LFS packages,
73 as if they were not installed. If you have a released version of LFS, or
74 the date of your SVN version of LFS is known, you should run the
75 update-lfs.sh script. If you have updated some
76 LFS packages since first installation, or have been using a custom
77 working copy of the LFS book, the only (tedious) way is to create
78 empty files with names <package>-<installed-version> in the tracking
79 directory, and run the tool.
80
81 - If you have also installed some BLFS packages, they are not in the
82 tracking file. The only way is to create empty files with names
83 <package>-<installed-version> in the tracking directory, and run the tool.
84
85 3.2 INSTALLATION ON A JUST BUILT xLFS SYSTEM
86
87 For books that support it (only LFS for jhalfs version 2.4),
88 there is an option to install the BLFS tools right after building
89 the xLFS system: just tick `BOOK Settings/Add blfs-tool support' in
90 jhalfs configuration menu. The tools are installed in $BLFS_ROOT
91 (default /blfs_root) on the xLFS system, and the dependencies are built
92 at the end of the jhalfs run, before the custom tools.
93
94 (TODO: blfs-tools have not been tested with current (version 3.0) of CLFS,
95 and certainly need some adaptation to run)
96
97 After booting the new xLFS system some steps are needed to finish
98 the installation of the automated tools:
99
100 - A user account must be created. You must be logged on that user
101 account to use blfs-tool. This is not strictly necessary,
102 since the packages can be built as root, too, but it is
103 never a good idea to build packages as root.
104
105 - Move /blfs-root to that user's home and change ownership of the
106 directory and files to the user.
107
108 - Give the user read and write privileges over the $TRACKING_DIR
109 directory and the files that it contains.
110
111 - Configure sudo and add the bash shell startup files, as described
112 above
113
114 - Note that the versions of LFS packages are automatically known to
115 the tool in this case, and there is no need to run the update-lfs.sh
116 script.
117
118 We assume that the BLFS tools will be used on a booted xLFS system.
119 Using them to build BLFS packages in a chroot jail is also possible,
120 but not supported.
121
122 3.3 DIRECTORY LAYOUT IN THE $BLFS_ROOT DIRECTORY
123
124 blfs-xml/* SVN tree of the selected BLFS book version
125 lfs-xml/* SVN tree of the selected LFS book version
126 lib/constants.inc functions libraries
127 /func_dependencies for building the dependency tree
128 menu/* lxdialog and menuconfig source code
129 xsl/gen_pkg_list.xsl XSL stylesheet to generate the package database
130 /gen_config.xsl XSL stylesheet to generate the Config.in file
131 for use in the menuconfig system
132 /dependencies.xsl XSL stylesheet to generate the dependency list
133 of a package
134 /make_book.xsl XSL stylesheet to generate the linear book.xml
135 /lfs_make_book.xsl XSL stylesheet to incoporate LFS pages into the
136 linear book.xml
137 /scripts.xsl XSL stylesheet to generate the scriptlets from
138 book.xml
139 /bump.xsl XSL stylesheet to update the tracking file
140 README.BLFS this file
141 TODO developers notes (well, not updated often)
142 gen_pkg_book.sh resolves dependencies and generates linear BLFS
143 books and build scripts
144 gen-makefile.sh generates the target Makefile
145 progress_bar.sh the target Makefile progress bar
146 gen-special.sh Helper script for generating the package database
147 Makefile Used by make to update the package database from
148 the SVN tree, then launch the menuconfig interface,
149 and run gen_pkg_book.sh based on configuration
150 settings
151 packdesc.dtd a simple DTD describing the format of the package
152 database and the tracking file.
153 envars.conf envars needed when running the target build scripts
154
155Working files: several files are generated when first running the tool
156
157 packages.xml auto-generated packages database
158 Config.in input file for the menu driven choices
159 configuration file generated by the menuconfig process
160 dependencies/* files recording the dependency tree
161 book.xml the linearized book
162 book-html/* the linearized book rendered in html
163 scripts/* the scriptlets
164
165 3.4 INSTALLED PACKAGES TRACKING SYSTEM:
166
167 This tool includes a very simple tracking system to log which packages
168 have been installed using the tool. It is used to skip installed packages
169 from target selection menu and to test if an installed package has been
170 updated in the BLFS book. Do not rely on this feature as a package
171 management tool.
172
173 The tracking system itself is an XML file: instpkg.xml. It is
174 initialized when <make> is first run in blfs_root. It resides in a
175 directory, which is created when needed during the process of building
176 custom tools or blfs dependencies, right after xLFS. You can specify
177 that directory location in the blfs-tools sub-menu of jhalfs. You may
178 need to update permissions and/or ownership of this directory before
179 using the blfs tool (see README in jhalfs).
180
181 The default location of the tracking directory is /var/lib/jhalfs/BLFS.
182 NB : after the initial build, that directory is only used to contain
183 instpkg.xml, unless custom tools have been built. In the latter case,
184 it also contains empty files whose name are "$PKG-$VERSION" for each
185 versionned package built. The information about those packages is
186 included into instpkg.xml the next time the tool is run.
187
1884. USAGE::
189
190 From now on, all the work must be done from inside the installation
191 root directory.
192
193 Due to the complexity of the BLFS book, the scripts and Makefile
194 generation is done in several steps:
195
196 4.1 UPDATING BOOK SOURCES::
197
198 If you are using the development book version and you want to update
199 installed packages to the latest version found in that book, you need to
200 update the XML sources and packages database. This is not necessary if
201 you just built xLFS, and you can skip to step 3.4.
202
203 To do that, run "make update". It may happen that the subversion
204 version of your building host is older than the version you just
205 built. This may generate weird errors like "'.' omitted". The easiest
206 thing to do in that case, is to completely remove the blfs-xml directory
207 and run "make update". With recent versions of subversion, you can also
208 run "svn upgrade" from inside the blfs-xml directory.
209
210 On the next configuration run, packages already installed but listed
211 with a new version in the book will be available for target selection
212 and used to solve dependencies.
213
214 4.2 CONFIGURING AND PARSING THE BOOK::
215
216 The next step is to create a book and build scripts in dependency
217 build order for one or several packages.
218
219 Run <make> to launch the configuration interface. The main menu contains
220 two blocks: individual package selection, and build options.
221
222 In the build options section, the dependencies level and default packages
223 used to solve alternatives are set (currently, only for the MTA). You can
224 also select whether the build will be made as a normal user or as root.
225 Those settings are saved to be reused in future configuration runs.
226
227 Note that you may select as many targets as you want, not just one
228 as in the previous version of this tool. But we suggest to not select
229 too many at a time to be able to sort issues!
230
231 When you are done with the menu, a few checks occur, and the dependency
232 chain is generated. Each dependency appears with its priority (required,
233 recommended, optional, or external), and it's level. There is a root level
234 1. The selected packages have level 2. The dependencies of selected packages
235 have level 3, the dependencies of the dependencies have level 4, and so on.
236 When circular dependencies are found, they appear with a priority of
237 "circular". This means that two (or more) dependency chains arrive at the
238 same package. The algorithm chooses the chain with the highest priority and
239 reorders dependencies to remove the other chain(s). This is not always the
240 solution an user would prefer, but we have found no way to do it better.
241
242 You end up with a book.xml file which contains the linearized book,
243 and a rendered HTML, in the directory book-html, which you can browse with
244 "lynx book-html/index.html" (or with any other browser).
245
246 Furthermore, there is a directory "scripts", which contains the generated
247 scriptlets.
248
249 There is also another directory, "dependencies" that contains files
250 generated while resolving dependencies.
251
252 4.3 EDITING BUILD SCRIPTS::
253
254 Now it is time to review the generated book and scripts, making any
255 changes to the scripts necessary to fix generation bugs or to suit your
256 needs.
257
258 Scripts for additional packages (i.e., for non-BLFS packages) can be
259 easily inserted. For example, if you want to install the external dependency
260 "bar" before "foo" package and the "foo" script is named "064-z-foo", you
261 just need to create a "064-y-bar" build script.
262
263 Remember, the package tracking system isn't a package management tool
264 and knows nothing about packages not in the BLFS book.
265
266 IMPORTANT: Review and edit envars.conf, at least after installing the
267 tool. This file is used to set global envars needed by the build scripts.
268 If you use package management, the variable JH_PACK_INSTALL should point to
269 the directory where the packInstall.sh script resides.
270
271 4.4 CREATING THE MAKEFILE::
272
273 When the build scripts are ready to be run, the Makefile can be
274 created. Create an empty directory (for example "mkdir work") and cd
275 to that directory. Then run ../gen-makefile.sh
276
277 Review the Makefile, and, if all looks sane, start the build by running
278 "make".
279
2805. GENERATED BUILD SCRIPTS ISSUES::
281
282 In this section, known issues with the generated build scripts are
283 discussed. They are due to build procedures and/or BLFS layout
284 particularities that we can't handle. In several cases, editing the
285 build scripts is mandatory.
286 You may also need to insert some build scripts created by you to resolve
287 unhandled dependencies and/or to remove some script installing an unneeded
288 package (unneeded packages may be pulled in the dependency chain, if
289 they occur as an "or" with another package).
290 When there are circular dependencies (only one known in BLFS 8.0 for
291 recommended dependencies), you may need to move around scripts so that they
292 run in the order script-A script-B script-A. This involves copying script-A
293 to another name (using the xxx-a- fields), and possibly renaming the xxx-a-
294 fields of each involved script.
295
296 5.1 BLFS BOOTSCRIPTS::
297
298 Normally, bootscript installation should work. On the other hand, the
299 book does not give instruction for running them, so you might have to
300 manually insert "/etc/init.d/rc.d/<initscript> start" at some place during
301 the build.
302
303 5.2 PACKAGE CONFIGURATION::
304
305 For those packages that have a "Configuration" section, you should
306 edit the build script to fit the needs of your system. Sometimes, the
307 bash startup files are modified. The shipped 'envars.conf' contains a
308 line 'source /etc/profile', which ensures that the proper environment
309 variables are used.
310
311 5.3 PAGES WITH TWO OR MORE PACKAGES::
312
313 For example: sane, poppler, audacious, freetts, which, etc.
314
315 On the pages for those packages, the BLFS book actually has instructions
316 to download and install two or more packages. You must edit the scripts to
317 fix this. A common pitfall is that the variable PACKAGE may be used for
318 several tarballs. Be sure to save the PACKAGE variable to some other
319 name (for example PKG1, PKG2, etc) after each download. The unpacking
320 instructions may need to be repeated for each tarball in turn.
321
322 5.4 XORG7
323
324 The book has special page layouts for the Xorg7 packages. The tool
325 breaks those pages into individual pages for each packages in the linear
326 book. Also, the menu gives the choice to select each package individually.
327
328 To build the whole Xorg7 chapter, select twm. The (recommended)
329 dependency chain brings in the whole set of Xorg packages.
330
331 5.5 PATCHES
332
333 Please, make sure that all scripts have the commands to download/apply
334 the required patches. Due to book layout issues, some patches may be
335 missing (as of BLFS 8.0, all the patches seem to be downloaded).
336
337 5.6 ROOT COMMANDS
338
339 If building as a normal user (the default setting), be sure that all
340 commands that require root privileges are run using sudo. Also make sure
341 necessary root privilege commands are visible in your PATH. Or use
342 the `Defaults secure_path=' in /etc/sudoers.
343
344 For commands necessitating root privileges, the generated scripts wrap
345 them with the construct:
346 sudo -E sh << ROOT_EOF
347 <commands to be executed as root with `$', ``', and `\' escaped>
348 ROOT_EOF
349 The -E switch ensures the whole environment is passed to the
350 commands to be run with root privileges. It is effective only if the
351 /etc/sudoers file contains `Defaults setenv', or SETENV in the user
352 attributes. If you think it is a security issue, you may forbid this
353 flag in /etc/sudoers, but then, you have to un-escape `$' for variables
354 coming from the environment in the instructions.
355 Although this construct is rather strong, it can fail in some corner
356 cases, so carefully review those instructions.
357
358 WARNING: One variable from the environment is not passed through the
359 -E switch, namely PATH. This is because "sudo" always reset the PATH to
360 the default "secure_path". If you need to have the same PATH as the user
361 "root" would have, you may want to add "source /etc/profile" at the
362 beginning of the commands to be executed as root.
363
364 Due to book layout issues, some sudo commands may be missing.
365
366 5.7 OTHERS
367
368 There may be other issues that we are not aware of. If you find
369 any, please report it to <alfs-discuss@linuxfromscratch.org>.
370
Note: See TracBrowser for help on using the repository browser.