source: extras/farce@ b7faa5a

1.0 2.3 2.3.x 2.4 ablfs ablfs-more legacy new_features trunk
Last change on this file since b7faa5a was 45f82718, checked in by Manuel Canales Esparcia <manuel@…>, 19 years ago

Merged ICA/farce support from experimental branch.
  • Property mode set to 100755
File size: 24.6 KB
RevLine 
[45f82718]1#!/bin/bash
2
3# Acknowledgment:
4# The following code is a no-modified version (except for this comment
5# and the inline_doc fragment) of an original work written by
6# Ken Moffat and is included here with his permission.
7#
8
9# FARCE: Farce Assists Rebuild Comparison Evaluation ;)
10#
11# to answer the question "can it rebuild itself?"
12#
13# We expect four arguments - first directory path, filelist
14# containing the files in this directory which we wish to compare,
15# second directory path, filelist for second directory.
16#
17# Yes, we could just compare everything in each tree, but the
18# filelist script knows about files it can reasonably ignore,
19# and this also allows us to build a sytem, boot it and get a
20# list of files, build a full desktop environment, and only then
21# build and boot the "can it build itself" test system and get
22# _its_ filelist.
23#
24# What this script aims to do:
25# ____________________________
26#
27# First, report files not in both builds.
28#
29# Then, confirm symlinks point to same targets.
30#
31# After that, compare individual files -
32# if different, run the file name through 'expected'
33# to pick out files that are unlikely to match (logs,
34# pids, fstab [assumes '/' is a different device each time],
35# count these as 'expected'.
36#
37# For whatever is left, check the file type - ar archives
38# have their members extraced and compared (every member has
39# a timestamp), gzipped files are compared beyond their
40# timestamp, binaries, at least those using shared libs or
41# which are shared objects, are copied and subjected to
42# --strip-debug. If files match at this stage, count them as
43# 'accepted'.
44#
45# As a last step for any file that doesn't match, copy it
46# through some perl regexps to "process" it (convert any
47# date, time, kernel-version information from standard formats
48# into tokens, then see if the tokensi match.
49#
50# For details of the regexps, see the tokenize function.
51# Those files that match after this are also counted as
52# 'accepted'. Note that I don't always start from the kernel
53# version that I'm going to build, so this copes with e.g. perl
54# files that hardcode the kernel version.
55#
56# We now have files that don't match. A few of these seem to be
57# common to all builds - some (members of) c++ libraries or ar
58# archives, a few programs which perhaps use some sort of c++ code).
59# The file name # is passed to the 'failure' function - these
60# recognized filenames are labelled as 'predictable FAIL:',
61# anything else is labelled as 'unexpected FAIL:'.
62#
63# output:
64# stderr - files only in one of the builds, failure messages,
65# and totals.
66#
67# farce-results - more details, including which files were treated
68# as expected differences, files where neither copy could be read,
69# files treated as accepted, with the reason (and member for ar
70# archives). This data is typically up to 100 characters wide -
71# sometimes it's a bit more, but it doesn't wrap too badly in a
72# 100 character xterm.
73#
74# farce-extras - diffs for the files, or members, that didn't
75# match. This file is to establish new regexps for picking up
76# date/time/kernel-version formats.
77#
78# farce-identical - the names of the files which are identical
79#
80# farce-substitutions - whenever using tokenizeanddiff results in a
81# difference being accepted, for both versions diff the before and
82# after versions to show what got changed. If the file is a binary,
83# the output may still be hard to read. Note that I _know_ glibc
84# version strings pass one of the regexps looking for a kernel version
85# - since I expect you to use the same version of glibc for each
86# build, this is not a problem.
87#
88# farce-differ - the names of the files which could not be treated
89# as matching (whether or not I regard the failure as predictable)
90# for possible input to ICA processing.
91#
92# Copyright (C) 2005, 2006 Ken Moffat <ken@linuxfromscratch.org>
93#
94# All rights reserved.
95#
96# This program is free software; you can redistribute it and/or modify
97# it under the terms of the GNU General Public License as published by
98# the Free Software Foundation; either version 2 of the License, or (at
99# your option) any later version.
100#
101# This program is distributed in the hope that it will be useful, but
102# WITHOUT ANY WARRANTY; without even the implied warranty of
103# MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
104# NON INFRINGEMENT. See the GNU General Public License for more
105# details.
106#
107# You should have received a copy of the GNU General Public License
108# along with this program; if not, write to the Free Software
109# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
110#
111
112: <<inline_doc
113 desc: do farce analisys and report
114 usage: farce -directory $FARCELOGDIR/dir path1 list1 path2 list2
115 input vars: $1 farce log dir for this comparation
116 $2 full path to previous iteration
117 $3 full path to filelist for previos iteration
118 $4 full path to current iteration
119 $5 full path to filelist for current iteration
120 externals: --
121 modifies: --
122 returns: --
123 on error:
124 on success:
125inline_doc
126
127
128VERSION="002"
129
130# variables for output files
131RESULT=farce-results
132EXTRAS=farce-extras
133IDENTICAL=farce-identical
134SUBS=farce-substitutions
135DIFFER=farce-differ
136
137# documenting the variables
138# C1, C2 temp files to hold disassembled code
139# D1, D2 temp directories for extracting member of ar archive
140# DIFF temp file for diffing the filelists
141# F1, F2 temp files for tokenizeanddiff
142# FTYPE obsolete, commented out
143# M1, M2 temp files to hold members of an ar archive
144# MEMBERS temp file to list members of ar archive
145# OP1, OP2 the original $PWD, needed when extracting members of ar
146# archives
147# P1, P2 paths to first and second build
148# S1, S2 temp files for shared objects
149
150# functions
151function dohelp() {
152 echo "`basename $0`: compare trees of files from a build"
153 echo "and lists of the files they contained"
154 echo ""
155 echo "`basename $0` [ -help | -version ] || path1 list1 path2 list2"
156}
157
158function emessage() {
159 # write a string to both stderr and $RESULT
160 echo "$@" >&2
161 echo "$@" >&5
162}
163
164function expected() {
165 # if we expect it to differ because of its name,
166 # allow it and report, return true ; else return false
167 case $1 in
168 /boot/grub/menu.lst)
169 # just in case somebody puts this into the main filesystem
170 true;;
171 /etc/aliases.db)
172 # some sort of database for postfix, not parsable
173 true;;
174 /etc/blkid.tab)
175 # includes dev name for rootfs
176 true;;
177 /etc/fstab)
178 # fstab, e.g. ' / ' will differ
179 true;;
180 /etc/group*)
181 true;;
182 /etc/hosts)
183 # with dhcp client, I add current ip address to this in a hook
184 true;;
185 /etc/ld.so.*)
186 # .conf and .cache can vary,
187 # particularly if one system has a full build when I run this
188 true;;
189 /etc/lilo.conf|/etc/yaboot.conf)
190 # bootloader control, I assume grub will all be on a separate
191 true;;
192 /etc/mtab)
193 # at a minimum, different '/'
194 true;;
195 /etc/ntp.drift)
196 true;;
197 /etc/passwd*)
198 true;;
199 /etc/shadow*)
200 true;;
201 /etc/ssh/*key|/etc/ssh/*pub)
202 # openssh keys
203 true;;
204 /misc/*)
205 # where I put buildscripts (which mostly won't change)
206 # and stamps containing name/time/space which will differ in the times
207 true;;
208 /root/*)
209 # expect .bash_history etc to differ - if we can read them
210 true;;
211 /usr/bin/lynx)
212 # part of my inital builds, I guess this uses anonymous namespaces
213 true;;
214 /usr/include/c++/*/*/bits/stdc++.h.gch/*)
215 # precompiled headers
216 true;;
217 /usr/lib*/libstdc++.a|/usr/lib*/libstdc++.so*|/usr/lib*/libsupc++.a)
218 # probably, anonymous namespaces
219 # libstdc++.a, libstdc++.so.n.n.n, libsupc++.a
220 true;;
221 /usr/share/info/dir)
222 # if one system has had extra stuff built, this will likely be bigger
223 true;;
224 /usr/share/man/whatis)
225 # if one system has had extra stuff built, this will likely be bigger
226 true;;
227 /var/lib/locate/locatedb)
228 # if one system has had extra stuff built, this will likely be bigger
229 true;;
230 /var/lib/nfs/*)
231 # allow nfs bookkeeping
232 true;;
233 /var/log/*)
234 true;;
235 /var/run/utmp)
236 true;;
237 /var/spool/fcron*)
238 true;;
239 /var/state/*)
240 # allow dhcp leases
241 true;;
242 /var/tmp/random-seed)
243 true;;
244 # following start with wildcards
245 *Image*|*.PPCBoot*|*vmlinuz*|*lfskernel*)
246 # compressed kernels, sometimes just building at a different
247 # date/time is enough to change the length of them, because the
248 # long format date and time is part of the compressed data
249 true;;
250 *pid*)
251 # pids, including e.g. /var/spool/postfix/pid/*
252 true;;
253 *)
254 # nothing else is expected to be different
255 false;;
256 esac
257 if [ $? -eq 0 ]; then
258 message "expected difference in $1"
259 let expected=$expected+1
260 case $TYPE in
261 AR)
262 let EXPAR=$EXPAR+1
263 ;;
264 ELF)
265 let EXPELF=$EXPELF+1
266 ;;
267 UNK)
268 let EXPUNK=$EXPUNK+1
269 ;;
270 # so far, no other valid types, so don't accumulate them
271 *)
272 emessage "internal error, expected difference for $1 of type $TYPE not allowed"
273 exit 2
274 ;;
275 esac
276 true
277 else
278 false
279 fi
280}
281
282function failure() {
283 # first parm is filename or token
284 # second parm is the error message
285 # update the appropriate total
286 # and write to both stderr and the results
287 # by using emessage
288
289 let different=$different+1
290 case $TYPE in
291 AR)
292 let DIFAR=$DIFAR+1
293 ;;
294 ELF)
295 let DIFELF=$DIFELF+1
296 ;;
297 GZ)
298 let DIFGZ=$DIFGZ+1
299 ;;
300 SYM)
301 let DIFSYM=$DIFSYM+1
302 ;;
303 UNK)
304 let DIFUNK=$DIFUNK+1
305 ;;
306 *)
307 emessage "internal error in failure() for TYPE $TYPE"
308 exit 2
309 ;;
310 esac
311 test -f ${P1}$1 && echo $1 >&9
312 emessage "FAIL: $2"
313}
314
315function fatal() {
316 # unrecoverable error
317 echo $*
318 exit 1
319}
320
321function filetype() {
322 TYPE=`file ${P1}${FILE}`
323 case $TYPE in
324 *'current ar archive'*)
325 let TOTAR=$TOTAR+1
326 TYPE=AR
327 ;;
328 *' ELF '*)
329 let TOTELF=$TOTELF+1
330 TYPE=ELF
331 ;;
332 *'gzip compressed data'*)
333 let TOTGZ=$TOTGZ+1
334 TYPE=GZ
335 ;;
336 *)
337 let TOTUNK=$TOTUNK+1
338 TYPE=UNK
339 ;;
340 esac
341}
342
343function message() {
344 # write a string to $RESULT
345 echo $* >&5
346}
347
348function onlyone() {
349 #report files only in one build
350 # text should go to both stderr and the results,
351 # but blank lines only go to the results
352 if [ $1 == '<' ]; then
353 emessage "File(s) only in the first build"
354 else
355 emessage "File(s) only in the second build"
356 fi
357 message ""
358 FILES=`cat $DIFF | grep "^$1" | cut -d ' ' -f 2`
359 for F in $FILES; do
360 emessage $F
361 let only=$only+1
362 done
363 message ""
364}
365
366# 'test' functions are called with three arguments:
367# the two pathes and the filename
368# - we know the file is of this type, so see if we
369# can get it to match by reasonalbe means.
370# if not, treat it as different.
371#
372# NB if pathes are absolute, we need to prefix them
373# with the original $PWD to access the .a files
374#
375function testar() {
376 # ar archives include timestamps for the members,
377 # but diff doesn't show file timestamps unless the data differs
378 # put out a message to help locate which archive any messages
379 # about the members refer to.
380
381 # try just stripping them U1,2 undebuggable
382 U1=`mktemp` || fatal "cannot create a temporary file"
383 U2=`mktemp` || fatal "cannot create a temporary file"
384 cp ${1}${3} $U1
385 cp ${2}${3} $U2
386 strip --strip-debug $U1
387 strip --strip-debug $U2
388 cmp -s $U1 $U2
389 rm $U1 $U2
390 if [ $? -eq 0 ]; then
391 let accepted=$accepted+1
392 let ACCAR=$ACCAR+1
393 message "archive $3 matches after strip --strip-debug"
394 return
395 fi
396 # rest of this function retained primarily for pathologically bad builds
397 # put out a message in the log to help identify which archive has issues.
398 message "examining ar archive $3"
399 D1=`mktemp -d` || fatal "cannot create a temporary directory"
400 D2=`mktemp -d` || fatal "cannot create a temporary directory"
401 cd $D1
402 ar -x ${OP1}${1}${3}
403 cd $D2
404 ar -x ${OP2}${2}${3}
405 cd
406 # diff the members - true means they match
407 diff -Na $D1 $D2 >/dev/null
408 if [ $? -eq 0 ]; then
409 message "accept: $3 after diffing the members"
410 let accepted=$accepted+1
411 let ACCAR=$ACCAR+1
412 else
413 # process individual members to eliminate date/time/kernel-version
414 # first, check the members are the same
415 M1=`mktemp` || fatal "cannot create a temporary file"
416 M2=`mktemp` || fatal "cannot create a temporary file"
417 cd $D1
418 MEMBERS=
419 for F in *; do
420 MEMBERS="$MEMBERS $F"
421 done
422 cd
423 echo $MEMBERS | sort >$M1
424 cd $D2
425 MEMBERS=
426 for F in *; do
427 MEMBERS="$MEMBERS $F"
428 done
429 cd
430 echo $MEMBERS | sort >$M2
431 cmp -s $M1 $M2
432 if [ $? -ne 0 ]; then
433 # oh dear, different members
434 echo "list of members differs for archive $3" >&6
435 diff $M1 $M2 >&6
436 failure $3 "$3 list of members differs"
437 else
438 # members (names) are same,
439 # process each one
440 STATUS=0
441 for M in $MEMBERS; do
442 #avoid firing up perl on matching members
443 cmp -s $D1/$M $D2/$M
444 if [ $? -ne 0 ]; then
445 tokenizeanddiff $D1/$M $D2/$M $FILE:$M
446 if [ $? -eq 0 ]; then
447 message "member $M matches after processing"
448 else
449 message "member $M DIFFERS after processing"
450 STATUS=1
451 fi
452 fi
453 done
454 if [ $STATUS -eq 0 ]; then
455 let accepted=$accepted+1
456 let ACCAR=$ACCAR+1
457 else
458 let different=$different+1
459 let DIFAR=$DIFAR+1
460 echo $3 >&9
461 emessage "FAIL: in $3"
462 fi
463 fi
464 rm $M1 $M2
465 fi
466 rm -rf $D1 $D2
467}
468
469function testgzip() {
470 # bytes 4,5,6,7 are the timestamp, so ignore these
471 cmp -s -i 8 ${1}${3} ${2}${3}
472 if [ $? -eq 0 ]; then
473 message "accept: $3 after ignoring gzip timestamp"
474 let accepted=$accepted+1
475 let ACCGZ=$ACCGZ+1
476 else
477 failure $3 " $3 even after ignoring gzip timestamp"
478 fi
479}
480
481function testso() {
482 # shared object - first try stripping it
483 # in fact, this now handles ALL ELF files
484 S1=`mktemp` || fatal "cannot create a temporary file"
485 S2=`mktemp` || fatal "cannot create a temporary file"
486 cp ${1}${3} $S1
487 strip --strip-debug $S1
488 cp ${2}${3} $S2
489 strip --strip-debug $S2
490 cmp -s $S1 $S2
491 if [ $? -eq 0 ]; then
492 message "accept: $3 after --strip-debug"
493 let accepted=$accepted+1
494 let ACCELF=$ACCELF+1
495 else
496 tokenizeanddiff $S1 $S2 $3
497 if [ $? -ne 0 ]; then
498 failure $3 " $3 differs after stripping and processing"
499 else
500 message "accept: $3 after --strip-debug and processing"
501 let accepted=$accepted+1
502 let ACCELF=$ACCELF+1
503 fi
504 fi
505 rm $S1 $S2
506}
507
508function tokenize() {
509 # use regexes to replace date/time/kernel-version text
510 # with tokens which may allow files to match even though
511 # they have hardcoded date/time/kernel-version.
512 # arguments are file to process, and where to put it.
513 # these regexes are somewhat long, and the order they
514 # are applied in is important (to stop short ones being
515 # used when a longer version would match).
516 # KV00 linux version date (e.g. as in the kernel itself)
517 # allow 2 or 3 groups of three alphas here - optional smp, with day, mon
518 # KV01 kernel version, including possible cpu details (that is for cdda2wav)
519 # KV02 just the version, in quotes e.g. "2.6.12.6" or '2.6.13', for perl stuff
520 # except that "|' gives me grif, so try a boundary
521 # also, it might need local version on the end, I really want
522 # quote2.\d+.\d+.{0,32}quote - it is the quotes that don't work.
523 # DT00 Day Mon .d+ hh:mm:ss TZN CCYY variations include non-caps and 'mon d'
524 # DT01 Mon .d+ CCYY hh:mm:ss
525 # DT02 hh:mm:ss Mon .d CCYY
526 # DT03 Mon .d CCYY
527 # DT04 Day Mon { ,d}d hh:mm:ss CCYY - for groff example postscript files
528 # (somewhat similar to DT00, but ' d' or ' dd' for day of month and no TZN )
529 # DT05 hh:mm:ss
530 # DT06 ISO date using space as separator
531 # DT07 ISO date using dash as separator
532 # DT08 ISO date using slash as separator
533 # DT09 fullmonth (capitalised), day number, comma, 4-digit year (groff 1.18.1 ps)
534 # DT10 dd, fullmonth (capitalised), 4-digit year (groff 1.18.1 manpages)
535 # DT11 '(xample comma space digit(s) backslash ) in groff memef.ps which is
536 # quite clearly the day of the month when it was compiled, preceded by 'example'
537 # with something weird between the e and the x.
538
539 if [ $# -ne 2 ]; then
540 fatal "tokenizing called with $# arguments : $*"
541 fi
542
543 cat $1 | perl -p \
544 -e 's/(L|l)inux.*\d\.\d\.\d+.* \#\d+( [A-Za-z][a-z]{2}){2,3} \d+ \d\d:\d\d:\d\d [A-Za-z]{3} \d{4}\b/%KV00%/g;' \
545 -e 's/(L|l)inux( (\w|_)+)?(-| |_)\d\.\d(\.\d+){1,2}((-|_)?(\w|_)+)?( |\000)*/%KV01%/g;' \
546 -e 's/\W2(\.\d+){2,3}(-|_)?((\w|_)+)?\s*\W/%KV02%/g;' \
547 -e 's/\b([A-Za-z][a-z]{2} ){2}( |\d)?\d \d\d:\d\d:\d\d [A-Za-z]{3} \d{4}\b/%DT00%/g;' \
548 -e 's/\b[A-Z][a-z]{2} ( |\d)\d \d{4} \d\d:\d\d:\d\d\b/%DT01%/g;' \
549 -e 's/\b\d\d:\d\d:\d\d [A-Z][a-z]{2} ( |\d)\d \d{4}\b/%DT02%/g;' \
550 -e 's/\b[A-Z][a-z]{2} ( |\d)\d \d{4}\b/%DT03%/g;' \
551 -e 's/\b([A-Z][a-z]{2} ){2}( |\d)\d \d\d:\d\d:\d\d \d{4}/%DT04%/g;' \
552 -e 's/\b\d\d:\d\d:\d\d\b/%DT05%/g;' \
553 -e 's/\b\d{4} \d\d \d\d\b/%DT06%/g;' \
554 -e 's/\b\d{4}-\d\d-\d\d\b/%DT07%/g;' \
555 -e 's/\b\d{4}\/\d\d\/\d\d\b/%DT08%/g;' \
556 -e 's/\b[A-Z][a-z]{2,} \d{1,2}, \d{4}/%DT09%/g;' \
557 -e 's/\b\d\d [A-Z][a-z]{2,} \d{4}/%DT10%/g;' \
558 -e 's/\(xample, \d{1,2}\\\)/%DT11%/g;' \
559 >$2
560}
561
562function tokenizeanddiff() {
563 # Call tokenize for the inputs, then compare the results
564 # Input arguments are path/filename for old and new versions
565 # third parm is readable name (filename, or archivename:member)
566 # to help understand what is in the extras output.
567 # - sometimes called for files, but other times called for
568 # members of ar archives extracted into temporary directories
569 #message tokenizeanddiff called for $1 $2 $3
570 F1=`mktemp` || fatal "cannot create a temporary file"
571 F2=`mktemp` || fatal "cannot create a temporary file"
572 tokenize $1 $F1
573 tokenize $2 $F2
574
575 # actually, cmp is probably more efficient
576 # but for picking up the pieces it will be better to
577 # use diff to see what got through.
578 cmp -s $F1 $F2
579 TOKENRESULT=$?
580 if [ $TOKENRESULT -ne 0 ]; then
581 echo "failure in $3..." >&6
582 diff -a $F1 $F2 >&6
583 rm $F1 $F2
584 false
585 else
586 # show what we did
587 echo "substitutions for $3" >&8
588 echo "build one" >&8
589 diff -a $1 $F1 >&8
590 echo "build two" >&8
591 diff -a $2 $F2 >&8
592 rm $F1 $F2
593 true
594 fi
595}
596
597function validateargs() {
598# validate the arguments
599BAD=0
600if ! [ -d $1 ]; then
601 echo "Error: first argument is not a directory" >&2
602 let BAD=$BAD+1
603fi
604NAME=`basename ${2%%-*}`
605if [ $NAME != filelist ]; then
606 echo "Error: second argument is not a recognized filelist" >&2
607 let BAD=$BAD+1
608fi
609if ! [ -d $3 ]; then
610 echo "Error: third argument is not a directory" >&2
611 let BAD=$BAD+1
612fi
613NAME=`basename ${4%%-*}`
614if [ $NAME != filelist ]; then
615 echo "Error: fourth argument is not a recognized filelist" >&2
616 let BAD=$BAD+1
617fi
618for I in $1 $2 $3 $4; do
619 if ! [ -r $I ]; then
620 echo "Error: cannot read $I" >&2
621 let BAD=$BAD+1
622 fi
623done
624if [ $1 == $3 ]; then
625 echo "Error: directory pathes are identical" >&2
626 let BAD=$BAD+1
627fi
628if [ $2 == $4 ]; then
629 echo "Error: filelist names are identical" >&2
630 let BAD=$BAD+1
631fi
632if [ $BAD -eq 0 ]; then
633 ARGS=valid
634fi
635}
636
637# Mainline
638ARGS=unproven
639OUTDIR=
640if [ $# -eq 1 ]; then
641 case $1 in
642 -version|--version)
643 echo "`basename $0` version $VERSION"
644 exit 0
645 ;;
646 -help|--help)
647 dohelp
648 exit 0
649 ;;
650 esac
651fi
652if [ $1 = "--directory" ]; then
653 OUTDIR=$2
654 shift 2
655 grep '/$' $OUTDIR >/dev/null 2>&1 || OUTDIR=`echo $OUTDIR | sed 's%$%/%'`
656 echo "creating directory $OUTDIR"
657 mkdir -p $OUTDIR
658 if [ $? -ne 0 ]; then
659 echo "cannot mkdir $OUTDIR"
660 exit 1
661 fi
662fi
663if [ $# -eq 4 ]; then
664 validateargs $*
665fi
666if ! [ $ARGS == valid ]; then
667 dohelp
668 fatal "`basename $0`: error in arguments"
669fi
670
671# ok, we're happy, lets hit these files
672exec 5>${OUTDIR}$RESULT
673exec 6>${OUTDIR}$EXTRAS
674exec 7>${OUTDIR}$IDENTICAL
675exec 8>${OUTDIR}$SUBS
676exec 9>${OUTDIR}$DIFFER
677
678>${OUTDIR}$RESULT
679if [ $? -ne 0 ]; then
680 fatal "cannot write to ${OUTDIR}$RESULT"
681fi
682
683emessage "will compare:"
684emessage " first build at $1 with files listed in $2"
685emessage "second build at $3 with files listed in $4"
686
687let accepted=0
688let different=0
689let expected=0
690let matched=0
691let only=0
692let predictable=0
693let unreadable=0
694let total=0
695
696# break down the accepted
697let ACCAR=0
698let ACCELF=0
699let ACCGZ=0
700let ACCUNK=0
701
702# break down definitely different
703let DIFAR=0
704let DIFELF=0
705let DIFGZ=0
706let DIFSYM=0
707let DIFUNK=0
708
709# break down the expected differences
710let EXPAR=0
711let EXPELF=0
712let EXPGZ=0
713let EXPUNK=0
714
715# break down the identical files
716let MATAR=0
717let MATELF=0
718let MATGZ=0
719let MATSYM=0
720let MATUNK=0
721
722# break down how many of each type
723let TOTAR=0
724let TOTELF=0
725let TOTGZ=0
726let TOTSYM=0
727let TOTUNK=0
728
729# now identify differences between the two trees
730DIFF=`mktemp` || fatal "cannot create a temporary file"
731diff $2 $4 >$DIFF
732
733for RUN in '<' '>' ; do
734 grep -q "$RUN" $DIFF && onlyone "$RUN"
735done
736
737rm $DIFF
738
739# and compare them
740message "Results of file comparison:"
741message ""
742
743# Strip any trailing slash from the path for tidyness,
744# because the filenames all start with a slash. Unfortunately,
745# unfortunately, '/' becomes empty, which breaks subroutines,
746# so special case it.
747# also, to process ar archives we need to extract them in temp
748# directories - that means that after cd'ing we've broken any
749# relative path, so save original pwd as necessary.
750P1=`echo $1 | sed 's%/$%%'`
751echo $1 | grep '^/' >/dev/null
752if [ $? -ne 0 ]; then
753 # relative path
754 OP1=${PWD}/
755 #echo "setting OP1 to $OP1"
756else
757 OP1=
758 #echo "$1 is an absolute path"
759fi
760test -z "$P1" && P1='/'
761P2=`echo $3 | sed 's%/$%%'`
762echo $3 | grep '^/' >/dev/null
763if [ $? -ne 0 ]; then
764 # relative path
765 OP2=${PWD}/
766 #echo "setting OP2 to $OP2"
767else
768 OP2=
769 #echo "$3 is an absolute path"
770fi
771test -z "$P2" && P2='/'
772
773echo "about to read $2"
774while read FILE ; do
775#echo "process $FILE"
776#echo "test existence of ${P2}${FILE}"
777 # confirm it exists in second build
778 # we have already reported files only in one build
779 if [ -f ${P2}"${FILE}" ]; then
780 let total=$total+1
781 # check we can read both of them
782 # or count as unreadable - I used to separate only-one-unreadable,
783 # but if you compre '/' and a _copy_ of /mnt/lfs that assumption
784 # breaks, so be less picky.
785 if ! [ -r "${P1}${FILE}" ] || ! [ -r "${P2}${FILE}" ]; then
786 message "cannot read one or both versions of $FILE"
787 let unreadable=$unreadable+1
788 continue
789 fi
790 if [ -h "${P1}${FILE}" ]; then
791 # for symlink, look at what it points to
792 # exceptionally, do not call filetype
793 TYPE=SYM
794 let TOTSYM=$TOTSYM+1
795 SL1=`ls -l "${P1}${FILE}" | awk '{ print $11 }'`
796 SL2=`ls -l "${P2}${FILE}" | awk '{ print $11 }'`
797 if [ "$SL1" = "$SL2" ]; then
798 echo "symlink $FILE matches for $SL1" >&5
799 let matched=$matched+1
800 let MATSYM=$MATSYM+1
801 else
802 failure TARGET " symlink $FILE points to $SL1 and $SL2"
803 echo $FILE >&9
804 fi
805 else
806 # regular file, start by typing it for accounting,
807 # then compare it
808 filetype ${P1}${FILE}
809 cmp -s "${P1}${FILE}" "${P2}${FILE}"
810 if [ $? -eq 0 ]; then
811 let matched=$matched+1
812 case $TYPE in
813 AR)
814 let MATAR=$MATAR+1
815 ;;
816 ELF)
817 let MATELF=$MATELF+1
818 ;;
819 GZ)
820 let MATGZ=$MATGZ+1
821 ;;
822 UNK)
823 let MATUNK=$MATUNK+1
824 ;;
825 *)
826 echo "unexpected TYPE of $TYPE for $FILE" >&2
827 exit 2
828 ;;
829 esac
830 echo ${FILE} >&7
831 else
832 # seems different, can we do better ?
833 # test if we expect it to differ
834 expected $FILE
835 if [ $? -ne 0 ]; then
836 case $TYPE in
837 GZ)
838 testgzip $P1 $P2 $FILE ;;
839 AR)
840 testar $P1 $P2 $FILE ;;
841 ELF)
842 testso $P1 $P2 $FILE ;;
843 *)
844 # long-stop - strip dates from text files
845 tokenizeanddiff "${P1}${FILE}" "${P2}${FILE}" "$FILE"
846 if [ $? -eq 0 ]; then
847 message "accepted $FILE after processing"
848 let accepted=$accepted+1
849 let ACCUNK=$ACCUNK+1
850 else
851 failure "$FILE" " $FILE is different"
852 fi
853 ;;
854 esac
855 fi
856 fi
857 fi
858 fi
859done < $2
860
861message ""
862# write totals to stderr as well as the results file
863emessage "$only files in only one of the builds"
864emessage "$total files compared, of which"
865emessage "$unreadable files could not be read, skipped"
866emessage "$matched files are identical"
867emessage "$expected files differed as expected"
868emessage "$accepted files had allowable differences"
869#emessage "$predictable files differed as they normally do"
870emessage "$different files differed"
871
872# totals of different file types
873emessage ""
874emessage "$TOTAR ar archives"
875emessage " of which $MATAR are identical"
876emessage " of which $ACCAR are accepted after strip-debug or extracting, diffing, tokenizing"
877emessage " of which $EXPAR differed as expected"
878emessage " of which $DIFAR differed"
879emessage "$TOTELF ELF executables or shared libraries"
880emessage " of which $MATELF are identical"
881emessage " of which $ACCELF are accepted after stripping and tokenizing"
882emessage " of which $EXPELF differed as expected"
883emessage " of which $DIFELF differed"
884emessage "$TOTGZ gzipped files"
885emessage " of which $MATGZ are identical"
886emessage " of which $ACCGZ are accepted after comparing beyond timestamp"
887emessage " of which $DIFGZ are different"
888emessage "$TOTSYM symbolic links"
889emessage " of which $MATSYM are identical"
890emessage " of which $DIFSYM have different targets"
891emessage "$TOTUNK other files"
892emessage " of which $MATUNK are identical"
893emessage " of which $ACCUNK are accepted after tokenizing"
894emessage " of which $EXPUNK differed as expected"
895emessage " of which $DIFUNK differed"
896
Note: See TracBrowser for help on using the repository browser.