[Nix-dev] Building OpenWrt: mysterious segfaults

[Kirill Elagin]

By carefully stracing everything I figured out that they were setting
`LD_LIBRARY_PATH` to `/lib` which resulted in incompatible libc being
loaded, huh. OpenWrt just finished building successfully.

On Wed, May 20, 2015 at 11:06 AM, Kirill Elagin <kirelagin at gmail.com> wrote:

> I’m trying to build OpenWrt on my Gentoo laptop. I never had any issues
> with this, but now I have some and the problem clearly is that I don’t have
> system-wide Git, only the one installed into my user profile.
>
> It all started at the very beginning, when the Makefile was looking for
> the tools it needs. The part that fails goes something like this:
>
> ~~~~
> $(eval $(call SetupHostCommand,git,Please install Git (git-core) >= 1.6.5,
> \
>     git clone 2>&1 | grep -- --recursive))
> ~~~~
>
> (Basically, this runs `git clone 2>&1 | grep -- --recursive` and tells me
> to install git if that fails.)
>
> And here is the relevant part of SetupHostCommand (enhanced with debug
> output by me):
>
> ~~~~
>             bin="$$$$$$$$(PATH="$(subst $(space),:,$(filter-out
> $(STAGING_DIR_HOST)/%,$(subst :,$(space),$(PATH))))" \
>                 which "$$$$$$$${cmd%% *}")"; \
>             echo bin= "$$$$$$$$bin" >> /tmp/foo ; \
>             echo cmd= "$$$$$$$$cmd" >> /tmp/foo ; \
>             echo EVAL >> /tmp/foo ; \
>             eval "$$$$$$$$cmd" 2>&1 >> /tmp/foo ; \
>             if [ -x "$$$$$$$$bin" ] && eval "$$$$$$$$cmd" >/dev/null
> 2>/dev/null; then \
>                 echo WIN >> /tmp/foo; \
>                 mkdir -p "$(STAGING_DIR_HOST)/bin"; \
>                 ln -sf "$$$$$$$$bin" "$(STAGING_DIR_HOST)/bin/$(strip
> $(1))"; \
>                 exit 0; \
>             fi; \
>             echo FAIL >> /tmp/foo ; \
>
> ~~~~
>
> and the relevant part of output is:
>
> ~~~~
> bin= /home/kirrun/.nix-profile/bin/git
> cmd= git clone 2>&1 | grep -- --recursive
> EVAL
> FAIL
> ~~~~
>
> So I spent some time trying to figure out what was going on, but thought,
> screw it, and hacked the Makefile to skip the actual execution of the
> program. Well, the build failed =).
>
> ~~~~
> echo "Checking out files from the git repository..."; mkdir -p
> /home/kirrun/proj/openwrt/openwrt/tmp/dl && cd
> /home/kirrun/proj/openwrt/openwrt/tmp/dl && rm -rf libubox-2015-05-08 && [
> \! -d libubox-2015-05-08 ] && git clone
> http://git.openwrt.org/project/libubox.git libubox-2015-05-08 --recursive
> && (cd libubox-2015-05-08 && git checkout
> a8e70c6d361967a23977417fb7d6cf56234f8b81 && git submodule update) && echo
> "Packing checkout..." && rm -rf libubox-2015-05-08/.git &&       tar czf
> /home/kirrun/proj/openwrt/openwrt/tmp/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz
> libubox-2015-05-08 && mv
> /home/kirrun/proj/openwrt/openwrt/tmp/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz
> /home/kirrun/proj/openwrt/openwrt/dl/ && rm -rf libubox-2015-05-08;
> Checking out files from the git repository...
> bash: line 1: 12429 Segmentation fault      git clone
> http://git.openwrt.org/project/libubox.git libubox-2015-05-08 --recursive
> Makefile:95: recipe for target
> '/home/kirrun/proj/openwrt/openwrt/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz'
> failed
> make[3]: ***
> [/home/kirrun/proj/openwrt/openwrt/dl/libubox-2015-05-08-a8e70c6d361967a23977417fb7d6cf56234f8b81.tar.gz]
> Error 139
> ~~~~
>
> Segmentation fault! So I put the execution of the test back, and looked at
> the kernel log. Indeed, git segfaults.
>
> ~~~~
> git[23091]: segfault at 100000018 ip 0000003abb621ae7 sp 00007ffc36ce1210
> error 4 in libc-2.19.so[3abb600000+19a000]
> ~~~~
>
> This reminded me that when I tried doing the same in `nix-shell`, I saw
> `make` segfaulting even earlier. Here is what happens (with a basic
> do-nothing `shell.nix`):
>
> ~~~~
> [nix-shell:~/proj/openwrt/openwrt]$ make -n
> OPENWRT_BUILD= QUIET=0 make V=s -r -s prereq
> make: *** [world] Segmentation fault
>
> [nix-shell:~/proj/openwrt/openwrt]$ make V=s -r -s prereq -n
> make -r -s staging_dir/host/.prereq-build OPENWRT_BUILD= QUIET=0
> make: *** [prepare-tmpinfo] Segmentation fault
>
> [nix-shell:~/proj/openwrt/openwrt]$ make -r -s
> staging_dir/host/.prereq-build OPENWRT_BUILD= QUIET=0 -n
> mkdir -p tmp
> rm -f tmp/.host.mk
> export MAKEFLAGS= ;make V=s -j1 -r -s -f
> /home/kirrun/proj/openwrt/openwrt/include/prereq-build.mk prereq
> 2>/dev/null || { \
>         echo "Prerequisite check failed. Use FORCE=1 to override."; \
>         false; \
> }
> touch staging_dir/host/.prereq-build
>
> [nix-shell:~/proj/openwrt/openwrt]$ make -r -s
> staging_dir/host/.prereq-build OPENWRT_BUILD= QUIET=0
> make: *** [staging_dir/host/.prereq-build] Segmentation fault
>
> [nix-shell:~/proj/openwrt/openwrt]$ file tmp
> tmp: cannot open `tmp' (No such file or directory)
> [nix-shell:~/proj/openwrt/openwrt]$ mkdir -p tmp
>
> [nix-shell:~/proj/openwrt/openwrt]$ file tmp
> tmp: directory
> ~~~~
>
> As you can see, even `mkdir` is segfaulting when invoked by Make:
>
> ~~~~
> mkdir[23757]: segfault at 100000018 ip 0000003abb621ae7 sp
> 00007fff12ee38a0 error 4 in libc-2.19.so[3abb600000+19a000]
> ~~~~
>
> but is working fine when I invoke it from the same shell.
>
> Does anyone have an idea what is going on? It seems that their Makefile
> does something weird that breaks libc, but I’m completely lost, and I don’t
> know what to do now.
>
> In case someone might want to reproduce this, their repository is
> git://git.openwrt.org/openwrt.git, all the interesting makefiles are in
> /include, and the failure happens very early during the build, so you won’t
> have to wait for more than a few seconds.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20150521/7aa19efc/attachment-0001.html