$Id$ 1. INTRODUCTION:: If you want to add blfs-tool support into an xLFS base system build, read the "BLFS_TOOL SUPPORT" section found in the README and be sure to follow the after-booting installation intructions. To automate package builds from the BLFS book instructions is a huge task. Some of the issues are: the BLFS book isn't linear; some package pages use a custom layout; there are circular dependencies; several packages can be installed on a non-default prefix; build commands can change based on what dependencies will be used, etc. That being said, the goal of the blfs-tool is to help you solve package dependencies, create build scripts and a Makefile. Not all the auto-generated build scripts and Makefile will work "as is", thus, as a general rule, you will need to review and edit the scripts while reading the book. 2. PREREQUISITES:: To use this tool you MUST: - have experience building BLFS packages - know how to edit and write shell scripts - know how a Makefile works - be able to trace build failures and to find what is causing it (user error, package bug, BLFS command bug, or jhalfs code bug) If you do not have the above skills, please don't use this tool. 3. USAGE:: Due to the complexity of the BLFS book, the scripts and Makefile generation is done in several steps: 3.1 INSTALLED PACKAGES TRACKING SYSTEM:: This tool includes a very simple tracking system to log which packages have been installed using the tool. It is used to skip installed packages from target selection menu and to test if an installed package has been updated in the BLFS book. Do not rely on this feature as a package management tool. The tracking system itself is an XML file: instpkg.xml. It is initialized when is first run in blfs_root. It resides in a directory, which is created when needed during the process of building custom tools or blfs dependencies, right after xLFS. You can specify that directory location in the blfs-tools submenu of jhalfs. You may need to update permissions and/or ownership of this directory before using the blfs tool (see README in jhalfs). The default location of the tracking directory is /var/lib/jhalfs/BLFS. NB : after the initial build, that directory is only used to contain instpkg.xml, unless custom tools have been built. In the latter case, it also contains empty files whose name are $PKG-$VERSION for each versionned package built. The information about those packages is included into instpkg.xml the next time the tool is run. 3.2 BLFS_TOOL INSTALLATION:: 3.2.1 Normal install The tools are installed just after the building of xLFS, if the appropriate options have been selected in the building menu, as per jhalfs README. If you forgot to select the options and xLFS has been built, it is possible to go back to selecting the appropriate BLFS tools options in the jhalfs menu, then tick `Run makefile' and not `Rebuild files'. You obtain a /blfs_root directory in the root directory of the new xLFS system, which contains the followings: blfs-xml/* SVN tree of the selected BLFS book version lib/constants.inc functions libraries /func_dependencies for building the dependency tree menu/* lxdialog and menuconfig source code xsl/gen_pkg_list.xsl XSL stylesheet to generate the package database /gen_config.xsl XSL stylesheet to generate the Config.in file for use in the menuconfig system /dependencies.xsl XSL stylesheet to generate the dependency list of a package /make_book.xsl XSL stylesheet to generate the linear book.xml /scripts.xsl XSL stylesheet to generate the scriptlets from book.xml /bump.xsl XSL stylesheet to generate to update the tracking file README.BLFS this file TODO developers notes (well, not updated often) gen_pkg_book.sh resolves dependencies and generates linear BLFS books and build scripts gen-makefile.sh generates the target Makefile progress_bar.sh the target Makefile progress bar gen-special.sh Helper script for generating the package database Makefile Used by make to update the package database from the SVN tree, then launch the menuconfig interface, and run gen_pkg_book.sh based on configuration settings packdesc.dtd a simple DTD describing the format of the package database and the tracking file. envars.conf envars needed when running the target build scripts 3.2.2 Install to an already running LFS/BLFS system If you forgot to install the tools when building xLFS, or want to try the tools, you can just run the install-blfs-tools.sh script. It will create the above hierarchy in your home directory and intialize the tracking file. You have first to make sure that the tracking dir exists and is writable by the user. You may also populate it with (empty) files whose names are of the form package-version, for installed packages, so that they are included into the tracking file. 3.3.3 Working files Several files are generated during the process: packages.xml auto-generated packages database Config.in input file for the menu driven choices configuration file generated by the menuconfig process dependencies/* files recording the dependency tree book.xml the linearized book book-html/* the linearized book rendered in html scripts/* the scriptlets From now on, all the work must be done from inside the installation root directory. You may move that directory to the $HOME of a non root user, or build as root from that directory. 3.3 UPDATING BOOK SOURCES:: If you are using the development book version and you want to update installed packages to the latest version found in that book, you need to update the XML sources and packages database. This is not necessary if you just built xLFS, and you can skip to step 3.4. To do that, run "make update". It may happen that the subversion version of your building host is older than the version you just built. This may generate weird errors like "'.' omitted". The easiest thing to do in that case, is to completely remove the blfs-xml directory and run "make update". With recent versions of subversion, you can also run "svn upgrade" from inside the blfs-xml directory. On the next configuration run, packages already installed but listed with a new version in the book will be available for target selection and used to solve dependencies. 3.4 CONFIGURING AND PARSING THE BOOK:: The next step is to create a book and build scripts in dependency build order for one or several packages. Run to launch the configuration interface. The main menu contains two blocks: individual package selection, and build options. In the build options section, the dependencies level and default packages used to solve alternatives are set (currently, only for the MTA). You can also select whether the build will be made as a normal user or as root. Those settings are saved to be reused in future configuration runs. Note that you may select as many targets as you want, not just one as in the previous version of this tool. But we suggest to not select too many at a time to be able to sort issues! When you are done with the menu, a few checks occur, and the book is generated. When circular dependencies are found, a 3 line message is printed: A is a dependency of B C is a dependency of A A is a dependency of C and a question: Do you want to build A first? This means that the system has found the dependency chain: B->A->C->A. You have therefore to choose whether A is built before C, or C before A: the system cannot make that choice (well, maybe in a few year, with an AI system able to understand the book). If you answer no, C is built first. If you answer yes, C is put in place of A as a dependency of B, then the tree dependency restarts from there, that is with the layout B->C->... You may then hit the case B->C->A->C, for which you should answer no, unless you want to enter an infinite (human driven) loop;-) You end up with a book.xml file which contains the linearized book, and a rendered HTML, in the directory book-html, which you can browse with "lynx book-html/index.html" (or with any other browser). Furthermore, there is a directory "scripts", which contains the generated scriptlets. There is also another directory, "dependencies" that contains files generated while resolving dependencies. 3.5 EDITING BUILD SCRIPTS:: Now it is time to review the generated book and scripts, making any changes to the scripts necessary to fix generation bugs or to suit your needs. Scripts for additional packages (i.e., for non-BLFS packages) can be easily inserted. For example, if you want to install the external dependency "bar" before "foo" package and the "foo" script is named "064-z-foo", you just need to create a "064-y-bar" build script. Remember, the package tracking system isn't a package management tool and knows nothing about packages not in the BLFS book. Also, review and edit envars.conf. This file is used to set global envars needed by the build scripts. 3.6 CREATING THE MAKEFILE:: When the build scripts are ready to be run, the Makefile can be created. Create an empty directory (for example "mkdir work") and cd to that directory. Then run ../gen-makefile.sh Review the Makefile, and, if all looks sane, start the build by running "make". 4. GENERATED BUILD SCRIPTS ISSUES:: In this section, known issues with the generated build scripts are discussed. They are due to build procedures and/or BLFS layout particularities that we can't handle. In several cases, editing the build scripts is mandatory. You may also need to insert some build scripts created by you to resolve unhandled dependencies and/or to remove some script installing the affected package by hand. 4.1 BLFS BOOTSCRIPTS:: Normally, bootscript installation should work. On the other hand, the book does not give instruction for running them, so you might have to manually insert /etc/init.d/rc.d/ at some place during the build. 4.2 PACKAGE CONFIGURATION:: For those packages that have a "Configuration" section, you should edit the build script to fit the needs of your system. Sometimes, the bash startup files are modified (see for example the instructions for llvm). The shipped 'envars.conf' contains a line 'source /etc/profile', which ensures that the proper environment variables are used. 4.3 GCC, JDK, Sane, and KDE-multimedia, freetype2, MesaLib and others On the pages for those packages, the BLFS book actually has instructions to download and install two or more packages. You must edit the scripts to fix this. We will try to fix some of them, but this may not be possible. 4.4 XORG7 The generated scripts for Xorg7 packages have $SRC_ARCHIVE support for individual packages, but not for patches nor *.wget and *.md5 files. If you have previously downloaded the patches, you must edit the scripts to use your local packages. The *.wget and *.md5 files should be downladed always from inside the scripts to be sure that the most current individual packages are used. Thus don't reuse previously existing ones. In the script for xorg7-font, be sure to move the fonts directories symlinks creation to after the "for ... done" loop. 4.5 PATCHES Please, make sure that all scripts have the commands to download/apply the required patches. Due to book layout issues, some patches may be missing. 4.6 ROOT COMMANDS If building as a normal user (the default setting), be sure that all commands that require root privileges are run using sudo. Also make sure necessary root privilege commands are visible in your PATH. Or use the `Defaults secure_path=' in /etc/sudoers. For commands necessitating root privileges, the generated scripts wrap them with the construct: sudo -E sh << ROOT_EOF ROOT_EOF The -E switch ensures the whole environment is passed to the commands to be run with root privileges. It is effective only if the /etc/sudoers file contains `Defaults setenv', or SETENV in the user attributes. If you think it is a security issue, you may forbid this flag in /etc/sudoers, but then, you have to un-escape `$' for variables coming from the environment in the instructions. Although this construct is rather strong, it can fail in some corner cases, so carefully review those instructions. Due to book layout issues, some sudo commands may be missing. 4.7 OTHERS There may be other issues that we are not aware of. If you find any, please report it to .