llvm-3.8.0

[bdubbs@…]

New minor version

[Pierre Labastie]

This is going to take some time, as I am trying to have cmake do the same thing as configure before. And I have not yet found the equivalent of --sysconfdir, for example. I think:

• --prefix=/usr is translated to -DCMAKE_INSTALL_PREFIX=/usr (that's the easy one ;-)
• --enable-libffi becomes -DLLVM_ENABLE_FFI=ON
• --enable-optimized + --disable-assertions becomes -DCMAKE_BUILD_TYPE=Release
• --enable-shared becomes -DBUILD_SHARED_LIBS=ON (not sure yet)
• --enable-targets=host,r600 becomes -DLLVM_TARGETS_TO_BUILD="host;AMDGPU" (to be tested)

but I still need to find the equivalent of --docdir and --sysconfdir

[bdubbs@…]

Did you see what 'cmake -L .' from the top of the source gives?

[Chris Staub]

Replying to pierre.labastie:

This is going to take some time, as I am trying to have cmake do the same thing as configure before. And I have not yet found the equivalent of --sysconfdir, for example. I think:

• --prefix=/usr is translated to -DCMAKE_INSTALL_PREFIX=/usr (that's the easy one ;-)
• --enable-libffi becomes -DLLVM_ENABLE_FFI=ON
• --enable-optimized + --disable-assertions becomes -DCMAKE_BUILD_TYPE=Release
• --enable-shared becomes -DBUILD_SHARED_LIBS=ON (not sure yet)
• --enable-targets=host,r600 becomes -DLLVM_TARGETS_TO_BUILD="host;AMDGPU" (to be tested)

but I still need to find the equivalent of --docdir and --sysconfdir

There isn't any equivalent for --sysconfdir. Previous versions of LLVM, even when using --sysconfdir, don't install anything in /etc anyway, and a search of the 3.8 source dir seems to indicate that this version also won't need anything in /etc.

I'm pretty sure --docdir really did nothing in previous LLVM versions anyway. What's used for the docs install location is PROJ_docsdir in Makefile.config.h, which is why BLFS instructions have a sed command to modify that file, and it looks like that would just stay the same for 3.8.

[Pierre Labastie]

@bdubbs: It took some time, but yes, I have found the command cmake -L. It sdoes not work in the top directory though. You have to make a subdir and type cmake -L ..

@chris: thanks. This is my first go with llvm, and since I focused on the "cmake transition", I had not noticed the sed.

I have built a first try, and I seem to have the other variables OK. Now, what I am worried about, is that there is no pkgconfig files. It may be a problem for packages using llvm and not built with cmake. I'll investigate that.

[Chris Staub]

Replying to pierre.labastie:

@bdubbs: It took some time, but yes, I have found the command cmake -L. It sdoes not work in the top directory though. You have to make a subdir and type cmake -L ..

@chris: thanks. This is my first go with llvm, and since I focused on the "cmake transition", I had not noticed the sed.

I have built a first try, and I seem to have the other variables OK. Now, what I am worried about, is that there is no pkgconfig files. It may be a problem for packages using llvm and not built with cmake. I'll investigate that.

Previous LLVM versions also have no .pc files installed either. Looking at mesa source, it seems to use the "llvm-config" program to check the existence of LLVM and its installed prefix, libs, flags, etc. I assume other packages using LLVM also use llvm-config.

[Pierre Labastie]

Thanks again Chris. I'll try mesa later, since there is something else, which puzzles me: using -DBUILD_SHARED_LIBS=ON builds all the libraries as shared. This is a difference with the previous behavior, where libraries were built static, and concatenated into a big shared library. I would say it is better than before, but the documentation (​http://llvm.org/docs/CMake.html) says:

BUILD_SHARED_LIBS is only recommended for use by LLVM developers. If you want
to build LLVM as a shared library, you should use the LLVM_BUILD_LLVM_DYLIB
option

Using -DLLVM_BUILD_LLVM_DYLIB=ON is equivalent to the former --enable-shared switch. The size of the libclang.so.3.8 library is 35 MB.

I think I'll try to build mesa atop llvm built with -DBUILD_SHARED_LIBS=ON. If it seems to work, I'll put it in the book. This is development after all...

[bdubbs@…]

Replying to pierre.labastie:

This is development after all...

BLFS is (or should be) closer to a distro than development. In general, we should probably go with upstream recommendations.

There is nothing wrong with trying things out, but we need a good reason to put these types of things into the book.

[Pierre Labastie]

Replying to bdubbs@…:

There is nothing wrong with trying things out, but we need a good reason to put these types of things into the book.

I'd say that the "good reason" is that we prefer shared libraries. OTOH, there is no way to build individual libraries, so having everything in one big shared library is not so bad. Actually I am not sure whether mesa links to the shared library or the static ones, when there are static libraries. More things to try...

[Chris Staub]

Replying to pierre.labastie:

Thanks again Chris. I'll try mesa later, since there is something else, which puzzles me: using -DBUILD_SHARED_LIBS=ON builds all the libraries as shared. This is a difference with the previous behavior, where libraries were built static, and concatenated into a big shared library. I would say it is better than before, but the documentation (​http://llvm.org/docs/CMake.html) says:

BUILD_SHARED_LIBS is only recommended for use by LLVM developers. If you want
to build LLVM as a shared library, you should use the LLVM_BUILD_LLVM_DYLIB
option

Using -DLLVM_BUILD_LLVM_DYLIB=ON is equivalent to the former --enable-shared switch. The size of the libclang.so.3.8 library is 35 MB.

I think I'll try to build mesa atop llvm built with -DBUILD_SHARED_LIBS=ON. If it seems to work, I'll put it in the book. This is development after all...

I've built LLVM with -DBUILD_SHARED_LIBS=ON and Mesa works fine. It does link to the individual LLVM shared libs, as it gets the list of libs from the llvm-config program.

[Chris Staub]

I don't see how to install docs in this version, aside from manually copying. Apparently specifying PROJ_docsdir doesn't work, and I can't find any make target or variable to use to install them.

[Pierre Labastie]

Confirmed for mesa with -DBUILD_SHARED_LIBS=ON. I've also tried -DLLVM_BUILD_LLVM_DYLIB=ON, and mesa does not build (at least with present book instructions). The error message is:

configure: error: Could not find llvm shared libraries:
        Please make sure you have built llvm with the --enable-shared option
        and that your llvm libraries are installed in /usr/lib
        If you have installed your llvm libraries to a different directory you
        can use the --with-llvm-prefix= configure flag to specify this directory.
        NOTE: Mesa is attempting to use llvm shared libraries by default.
        If you do not want to build with llvm shared libraries and instead want to
        use llvm static libraries then add --disable-llvm-shared-libs to your configure
        invocation and rebuild.

And looking at configure, it seems that mesa either looks for libLLVM-<version>.so or libLLVMTarget.so. The latter one is built (with many others) when using -DBUILD_SHARED_LIBS=ON.. The former one is never built with the options I've tried up to now. Note that when using --disable-llvm-shared-libs, there is a warning telling that using static libraries may cause compilation issues (I've not seen any). So I've to investigate whether it is possible to have libLLVM-<version>.so built.

[Pierre Labastie]

Replying to chris@…:

I don't see how to install docs in this version, aside from manually copying. Apparently specifying PROJ_docsdir doesn't work, and I can't find any make target or variable to use to install them.

Actually, the book already copies the man pages manually, and does not do anything else. With the present instructions (I mean for LLVM-3.7.1), the sed is not needed, because it is not used.

[Pierre Labastie]

Replying to pierre.labastie:

Confirmed for mesa with -DBUILD_SHARED_LIBS=ON. I've also tried -DLLVM_BUILD_LLVM_DYLIB=ON, and mesa does not build (at least with present book instructions).

Forget this! I made a typo LLVM_BUILD_LLVM_DYIB instead of LLVM_BUILD_LLVM_DYLIB... Sorry, retrying.

[Fernando de Oliveira]

Knowing that the build would switch to cmake, I tested before, only remember some results.

I have used:

cmake -G "Unix Makefiles"         \
      -DCMAKE_INSTALL_PREFIX=/usr \
      -DCMAKE_BUILD_TYPE=Release  \
      -DBUILD_SHARED_LIBS=ON      \
      -DLLVM_ENABLE_FFI=ON        \
      -DLLVM_TARGETS_TO_BUILD="host;AMDGPU;" \
      -Wno-dev ..

The documents gave much trouble, I didn't solve, or I copied? Will test soon, if this matters for you both. I am glad Chris increased the contribution.

From llvm-3.7.1-simulation-2016.01.05-20h04m08s.log.xz:

Total Seconds          = 1999 s
Approximate build time = 13.3266666666667 SBU 


END - Building llvm-3.7.1 - Ter Jan  5 20:37:56 BRT 2016


real	33m48.718s
user	124m36.956s
sys	4m53.665s


before tests
167M	../../DEST-llvm-3.7.1
875M	../../llvm-3.7.1.src
1,1G	total

llvm-3.7.1-simulation-2016.01.06-01h27m11s.log.xz

check-all
...

--

********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Testing Time: 333.64s
********************
Failing Tests (154):

...

  Expected Passes    : 18162
  Expected Failures  : 75
  Unsupported Tests  : 5537
  Unexpected Failures: 154


...

real	10m48.719s
user	52m14.981s
sys	7m37.151s

167M	../../DEST-llvm-3.7.1
6,0G	../../llvm-3.7.1.src
6,2G	total

llvm-3.7.1-make-k-check-all-2016.01.06-01h27m11s.log.xz

    Download MD5 sum: bf8b3a2c79e61212c5409041dfdbd319

    Download size: 14 MB

    Estimated disk space required: 1.1 GB (with Clang - additional 5.1
    GB for the tests)

    Estimated build time: 13 SBU (with Clang - 4.3 SBU for the tests -
    all with parallelism = 4)

 Optional Downloads
Clang

    Download: http://llvm.org/releases/3.7.0/cfe-3.7.0.src.tar.xz

    Download MD5 sum: 0acd026b5529164197563d135a8fd83e

    Download size: 8.7 MB

Compiler RT

    Download:
    http://llvm.org/releases/3.7.0/compiler-rt-3.7.0.src.tar.xz

    Download MD5 sum: 1c6975daf30bb3b0473b53c3a1a6ff01

    Download size: 1.2 MB

bugpoint, clang (symlink), clang-3.7, clang-check, clang-cl (symlink),
clang-format, clang++ (symlink), git-clang-format, llc, lli, llvm-ar,
llvm-as, llvm-bcanalyzer, llvm-c-test, llvm-config, llvm-cov,
llvm-cxxdump, llvm-diff, llvm-dis, llvm-dsymutil, llvm-dwarfdump,
llvm-extract, llvm-lib (symlink), llvm-link, llvm-lto, llvm-mc,
llvm-mcmarkup, llvm-nm, llvm-objdump, llvm-pdbdump, llvm-profdata,
llvm-ranlib (symlink), llvm-readobj, llvm-rtdyld, llvm-size,
llvm-stress, llvm-symbolizer, llvm-tblgen, macho-dump, obj2yaml, opt,
scan-build (symlink), scan-view (symlink), verify-uselistorder, and
yaml2obj


Over 80 shared libraries and, under /usr/lib/clang/3.7.1/lib/linux, some
static libraries.

[Pierre Labastie]

Thanks Fernando. I get roughly the same results with version 3.8.0 as you with version 3.7.1. I have not used -Wno-dev, but I agree this could be needed. As of the -G option, it does not seem to be needed, because the "Unix Makefiles" generator is found automatically on my system, but maybe I am missing something because I do not have much experience (well, I have no experience) with cmake.

For building documentation, if I understand correctly what is written in ​http://llvm.org/docs/CMake.html, we need to pass -DLLVM_ENABLE_SPHINX=ON. This creates targets for building the documentation, but does not build it by default. Then it is possible to issue make docs-llvm-html, make docs-clang-html, make docs-lld-html, and make docs-llvm-man. All of this not tested yet.

[Pierre Labastie]

Forgot: now scan-view and scan-build are built and installed by the build system.

For docs, as Chris said, there is no target to install them. The targets cited in comment:16 work, though.

[Fernando de Oliveira]

Yes, Pierre, good catch, Chris. Just 'make install' (xdemos, of course)!

I've reinstalled mesa-11.1.2 and everything seems OK.

from porg point of view, everything installed by mesa using llvm-3.7.1 (autotools) is also installed by 3.8.0 (cmake), correspondence 1 to 1.

I am using it already now.

[Fernando de Oliveira]

Sorry, mixed mesa (xdemos) and llvm.

[Pierre Labastie]

I'm almost done editing. I decided to use -DBUILD_SHARED_LIBS=ON, since all the reports we have is that mesa compiles and runs OK. I have an hesitation to list all the shared libraries though.

I still have to write some short descriptions.

For documentation, I end up with: For building llvm doc:

cmake -DLLVM_ENABLE_SPHINX=ON         \
      -DSPHINX_WARNINGS_AS_ERRORS=OFF \
      -Wno-dev ..                     &&
make docs-llvm-html  docs-llvm-man

For clang doc, I add:

make docs-clang-html docs-clang-man

Then for installing llvm doc:

install -v -m644 docs/man/* /usr/share/man/man1             &&
install -v -d -m755 /usr/share/doc/llvm-3.8.0/llvm-html     &&
cp -Rv docs/html/* /usr/share/doc/llvm-3.8.0/llvm-html

and for clang (there is only one manual page):

install -v -m644 tools/clang/docs/man/* /usr/share/man/man1 &&
install -v -d -m755 /usr/share/doc/llvm-3.8.0/clang-html    &&
cp -Rv tools/clang/docs/html/* /usr/share/doc/llvm-3.8.0/clang-html

[bdubbs@…]

Replying to pierre.labastie:

I have an hesitation to list all the shared libraries though.

That's not required. If there are some major libs, you can list them or just say something like e.g. libreoffice.

[Pierre Labastie]

Fixed at r17119. Thanks to all who contributed.

[ken@…]

That says it is 3.7.2 :) The md5sums match 3.8.0, I'll fix it up.

[Pierre Labastie]

Replying to ken@…:

That says it is 3.7.2 :) The md5sums match 3.8.0, I'll fix it up.

Updated to Mercurial-3.7.2 on the same day and mixed things up... Sorry.