[Nix-dev] Building OpenWrt: mysterious segfaults

[Kirill Elagin]

I’m trying to build OpenWrt on my Gentoo laptop. I never had any issues
with this, but now I have some and the problem clearly is that I don’t have
system-wide Git, only the one installed into my user profile.

It all started at the very beginning, when the Makefile was looking for the
tools it needs. The part that fails goes something like this:

~~~~
$(eval $(call SetupHostCommand,git,Please install Git (git-core) >= 1.6.5, \
    git clone 2>&1 | grep -- --recursive))
~~~~

(Basically, this runs `git clone 2>&1 | grep -- --recursive` and tells me
to install git if that fails.)

And here is the relevant part of SetupHostCommand (enhanced with debug
output by me):

~~~~
            bin="$$$$$$$$(PATH="$(subst $(space),:,$(filter-out
$(STAGING_DIR_HOST)/%,$(subst :,$(space),$(PATH))))" \
                which "$$$$$$$${cmd%% *}")"; \
            echo bin= "$$$$$$$$bin" >> /tmp/foo ; \
            echo cmd= "$$$$$$$$cmd" >> /tmp/foo ; \
            echo EVAL >> /tmp/foo ; \
            eval "$$$$$$$$cmd" 2>&1 >> /tmp/foo ; \
            if [ -x "$$$$$$$$bin" ] && eval "$$$$$$$$cmd" >/dev/null
2>/dev/null; then \
                echo WIN >> /tmp/foo; \
                mkdir -p "$(STAGING_DIR_HOST)/bin"; \
                ln -sf "$$$$$$$$bin" "$(STAGING_DIR_HOST)/bin/$(strip
$(1))"; \
                exit 0; \
            fi; \
            echo FAIL >> /tmp/foo ; \

~~~~

and the relevant part of output is:

~~~~
bin= /home/kirrun/.nix-profile/bin/git
cmd= git clone 2>&1 | grep -- --recursive
EVAL
FAIL
~~~~

So I spent some time trying to figure out what was going on, but thought,
screw it, and hacked the Makefile to skip the actual execution of the
program. Well, the build failed =).

~~~~
echo "Checking out files from the git repository..."; mkdir -p
/home/kirrun/proj/openwrt/openwrt/tmp/dl && cd
/home/kirrun/proj/openwrt/openwrt/tmp/dl && rm -rf libubox-2015-05-08 && [
\! -d libubox-2015-05-08 ] && git clone
http://git.openwrt.org/project/libubox.git libubox-2015-05-08 --recursive
&& (cd libubox-2015-05-08 && git checkout
a8e70c6d361967a23977417fb7d6cf56234f8b81 && git submodule update) && echo
"Packing checkout..." && rm -rf libubox-2015-05-08/.git &&       tar czf
/home/kirrun/proj/openwrt/openwrt/tmp/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz
libubox-2015-05-08 && mv
/home/kirrun/proj/openwrt/openwrt/tmp/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz
/home/kirrun/proj/openwrt/openwrt/dl/ && rm -rf libubox-2015-05-08;
Checking out files from the git repository...
bash: line 1: 12429 Segmentation fault      git clone
http://git.openwrt.org/project/libubox.git libubox-2015-05-08 --recursive
Makefile:95: recipe for target
'/home/kirrun/proj/openwrt/openwrt/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz'
failed
make[3]: ***
[/home/kirrun/proj/openwrt/openwrt/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz]
Error 139
~~~~

Segmentation fault! So I put the execution of the test back, and looked at
the kernel log. Indeed, git segfaults.

~~~~
git[23091]: segfault at 100000018 ip 0000003abb621ae7 sp 00007ffc36ce1210
error 4 in libc-2.19.so[3abb600000+19a000]
~~~~

This reminded me that when I tried doing the same in `nix-shell`, I saw
`make` segfaulting even earlier. Here is what happens (with a basic
do-nothing `shell.nix`):

~~~~
[nix-shell:~/proj/openwrt/openwrt]$ make -n
OPENWRT_BUILD= QUIET=0 make V=s -r -s prereq
make: *** [world] Segmentation fault

[nix-shell:~/proj/openwrt/openwrt]$ make V=s -r -s prereq -n
make -r -s staging_dir/host/.prereq-build OPENWRT_BUILD= QUIET=0
make: *** [prepare-tmpinfo] Segmentation fault

[nix-shell:~/proj/openwrt/openwrt]$ make -r -s
staging_dir/host/.prereq-build OPENWRT_BUILD= QUIET=0 -n
mkdir -p tmp
rm -f tmp/.host.mk
export MAKEFLAGS= ;make V=s -j1 -r -s -f
/home/kirrun/proj/openwrt/openwrt/include/prereq-build.mk prereq
2>/dev/null || { \
        echo "Prerequisite check failed. Use FORCE=1 to override."; \
        false; \
}
touch staging_dir/host/.prereq-build

[nix-shell:~/proj/openwrt/openwrt]$ make -r -s
staging_dir/host/.prereq-build OPENWRT_BUILD= QUIET=0
make: *** [staging_dir/host/.prereq-build] Segmentation fault

[nix-shell:~/proj/openwrt/openwrt]$ file tmp
tmp: cannot open `tmp' (No such file or directory)
[nix-shell:~/proj/openwrt/openwrt]$ mkdir -p tmp

[nix-shell:~/proj/openwrt/openwrt]$ file tmp
tmp: directory
~~~~

As you can see, even `mkdir` is segfaulting when invoked by Make:

~~~~
mkdir[23757]: segfault at 100000018 ip 0000003abb621ae7 sp 00007fff12ee38a0
error 4 in libc-2.19.so[3abb600000+19a000]
~~~~

but is working fine when I invoke it from the same shell.

Does anyone have an idea what is going on? It seems that their Makefile
does something weird that breaks libc, but I’m completely lost, and I don’t
know what to do now.

In case someone might want to reproduce this, their repository is
git://git.openwrt.org/openwrt.git, all the interesting makefiles are in
/include, and the failure happens very early during the build, so you won’t
have to wait for more than a few seconds.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20150520/cdeb53fb/attachment-0001.html