TipsAndTricks/DebuggingOddFailure

We noticed one of the Penang QA autobuilders would fail to build tar with a failure in do_install:

NOTE: make -j 16 DESTDIR=/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-opensuse423/build/build/tmp/work/core2-64-poky-linux/tar/1.30-r0/image install
Making install in doc
make[1]: Entering directory '/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-opensuse423/build/build/tmp/work/core2-64-poky-linux/tar/1.30-r0/build/doc'
restore=: && backupdir=".am$$" && \
am__cwd=`pwd` && CDPATH="${ZSH_VERSION+.}:" && cd ../../tar-1.30/doc && \
rm -rf $backupdir && mkdir $backupdir && \
if (/bin/sh /home/ab/autobuilder/yocto-worker/nightly-oe-selftest-opensuse423/build/build/tmp/work/core2-64-poky-linux/tar/1.30-r0/tar-1.30/build-aux/missing makeinfo --version) >/dev/null 2>&1; then \
  for f in ../../tar-1.30/doc/tar.info ../../tar-1.30/doc/tar.info-[0-9] ../../tar-1.30/doc/tar.info-[0-9][0-9] ../../tar-1.30/doc/tar.i[0-9] ../../tar-1.30/doc/tar.i[0-9][0-9]; do \
    if test -f $f; then mv $f $backupdir; restore=mv; else :; fi; \
  done; \
else :; fi && \
cd "$am__cwd"; \
if /bin/sh /home/ab/autobuilder/yocto-worker/nightly-oe-selftest-opensuse423/build/build/tmp/work/core2-64-poky-linux/tar/1.30-r0/tar-1.30/build-aux/missing makeinfo  -DDISTRIB -I ../../tar-1.30/doc \
 -o ../../tar-1.30/doc/tar.info ../../tar-1.30/doc/tar.texi; \
then \
  rc=0; \
  CDPATH="${ZSH_VERSION+.}:" && cd ../../tar-1.30/doc; \
else \
  rc=$?; \
  CDPATH="${ZSH_VERSION+.}:" && cd ../../tar-1.30/doc && \
  $restore $backupdir/* `echo "./../../tar-1.30/doc/tar.info" | sed 's|[^/]*$||'`; \
fi; \
rm -rf $backupdir; exit $rc
../../tar-1.30/doc/tar.texi:5480: Unknown command `atchar'.
../../tar-1.30/doc/tar.texi:5480: Misplaced {.
../../tar-1.30/doc/tar.texi:5480: Misplaced }.
makeinfo: Removing output file `../../tar-1.30/doc/tar.info' due to errors; use --force to preserve.
Makefile:1345: recipe for target '../../tar-1.30/doc/tar.info' failed
make[1]: *** [../../tar-1.30/doc/tar.info] Error 1
make[1]: Leaving directory '/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-opensuse423/build/build/tmp/work/core2-64-poky-linux/tar/1.30-r0/build/doc'
Makefile:1394: recipe for target 'install-recursive' failed
make: *** [install-recursive] Error 1
ERROR: oe_runmake failed

This is caused by the fact that its trying to use the host's makeinfo command to regenerate the tar.info file and makeinfo is too old on the host. This raises the question why does this fail here but not anywhere else.

Adding "--trace -d" to the oe_runmake_call() function in WORKDIR/temp/run.do_install meant we could find out why make wanted to rebuild this:

          Finished prerequisites of target file '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/tar.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/tar.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/version.texi' is newer than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/dumpdir.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/tar-snapshot-edit.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/fdl.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/freemanuals.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/genfile.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/header.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/intern.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/parse-datetime.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/recipes.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/rendition.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/snapshot.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/sparse.texi' is older than target '../../tar-1.30/doc/tar.info'.
          Prerequisite '../../tar-1.30/doc/value.texi' is older than target '../../tar-1.30/doc/tar.info'.
         Must remake target '../../tar-1.30/doc/tar.info'.

i.e. version.texi had changed. A look through the other logs showed the compile log:

make[2]: Entering directory '/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-opensuse423/build/build/tmp/work/core2-64-poky-linux/tar/1.30-r0/build/doc'
Updating ../../tar-1.30/doc/version.texi

which does not appear on a working build. Looking at the makefiles:

doc/Makefile.in:$(srcdir)/version.texi:  $(srcdir)/stamp-vti
doc/Makefile.in:$(srcdir)/stamp-vti: tar.texi $(top_srcdir)/configure

and stamp-vti is a generated file which is compared to version.texi and replaces it if its different. The contents of the file in breaking build:

@set UPDATED 17 December 2017
@set UPDATED-MONTH December 2017
@set EDITION 1.30
@set VERSION 1.30

verses a working one:

@set UPDATED 16 December 2017
@set UPDATED-MONTH December 2017
@set EDITION 1.30
@set VERSION 1.30

so the date is different. Running the command from the makefile to look at this directly:

$ ../build-aux/mdate-sh tar.texi
16 December 2017
$ TZ=Asia/Kuala_Lumpur ../build-aux/mdate-sh tar.texi
17 December 2017

so the issue is the timezone the autobuilder is in changes the timestamp, causing the file to regenerate and break if and only if makeinfo is too old!