[Nix-dev] A few questions about ARM support and NixOS on a Chromebook

[James Haigh]

On 28/01/15 07:42, Luke Clifton wrote:
> Hi Bjørn,
>
> I have read that thread. I agree with you 100% that native builds (on
> real or virtual hardware) is the only way this can work. Upstream
> doesn't usually care if their software can cross compile, and they
> can't maintain it themselves even if they did. Sometimes it isn't even
> an option, e.g. GHC still can't cross compile template Haskell yet.
I don't understand why cross-compilation is even a thing, other than
decades of false assumptions being baked into compilers.
    As I understand, if a compiler (and by ‘compiler’ I'm referring to
the whole toolchain required for compilation) is taking source code and
compilation options as input, and giving object code for the specified
platform as output, it is called ‘cross-compiling’ if the specified
target platform is different to the platform that the compiler is
running on. If GCC is running on ARM, compiling code ‘natively’ to ARM
successfully, it is counterintuitive that it would fail to build for ARM
if GCC is running on x86. And vice versa. A compiler should produce
object code for a target platform that implements the source code – it
may not have the same efficiency as the output of other compilers (or
with other compilation options), but should have the same correctness
when execution completes. If the source code being compiled is a
specific version of the GCC source code itself, and it is compiled for
both x86 and ARM, then if the compilation is computationally correct,
both compilations of GCC should produce programs that, although will
compute in a different way and with different efficiency, should give
the exact same object code when given the same source code and
parameters. So if the target platform parameter is ARM, they should both
build exactly the same ARM machine code program.
    However, evidently this is not the case unfortunately. So the
compilers or their toolchains are, in essence, receiving the platform
that they are running on as ‘input’ to the build, and making assumptions
that this build platform has something to do with the target platform.
I.e. they are _aware_ of the platform that they're building on, whereas
theoretically, they shouldn't be. Apparently this has a lot to do with
configure scripts.
    So when a compiler can build ‘natively’ to ARM, but can not
‘cross-compile’ to ARM, it seems that the problem is with the compiler
rather than the program being compiled. If a program fails to compile to
ARM natively as well, then it may be considered either an issue with the
program being compiled or with the compiler, because theoretically, it
should be possible to translate any machine specific code to the target
platform, providing that the target platform is Turing-complete and has
whatever I/O devices are needed in the program.
    Okay, so I can live with “Yes, compilers are too broken to be fixed,
so we should avoid cross-compilation in general.” as a good enough
reason to aim for native builds for the majority of packages. Well, I'm
not experienced with compilers, but at FOSDEM on the Saturday, Nicolas
Pierron explained to me a 3-stage approach which he'd take; (cut; see
below).
> I once used to think that full system cross compilation was the only
> way you should do things (I was an idealist). But it is a dead end.
> Unless you have thousands of hours to throw at the problem, it won't work.
> [...]
Was? Hey, just because the ideal approach is out-of-reach in a specific
scenario, doesn't mean to say that you cannot still be an idealist by
nature! ;-)

On 28/01/15 15:29, Bjørn Forsman wrote:
> [...]
> So far I've been thinking that we need to cross compile a fully
> working (although minimal) nixos system for QEMU, but I'm not sure
> this is strictly needed. Maybe we can bootstrap nix inside QEMU from a
> tiny set of packages (gcc, libc, make)? But I'm not sure how to do it.
> @nbp has some ideas:
> https://github.com/NixOS/nixpkgs/issues/4963#issuecomment-64956636.
Aah brilliant, so he's already documented it in that comment. I was
going to try to pass this suggestion on as best as I can remember, but
it's much better first-hand! :-)
    It has since crossed my mind that this could theoretically be
improved by a virtual heterogeneous computing platform. If Qemu is able
to emulate a heterogeneous ARM+x86 platform, a bit like the platform AMD
is debuting in its Opteron A1100 processor, but pass-through the x86
processing, then it's theoretically possible to have core parts of
compilers be compiled for x86 and run at native speed, yet have the
compiler as a whole only ‘see’ the ARM platform (emulated by Qemu). By
‘core parts’, I mean parts that do the bulk of the computation and which
are hopefully less broken (in terms of assumptions) than the peripheral
tools in the toolchain. Hence you are then compiling natively _and_ at
native speeds, if done right. And if that's feasible, that would fit in
with Hydra running on existing x86 hardware at normal efficiency, energy
consumption, cost, etc.. However, this whole Qemu-based approach seems
like a universal fix for compilers that can build natively to an
architecture but cannot cross-compile to that architecture, so perhaps
it should be implemented upstream in the compilers themselves.

Best regards,
James Haigh.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20150209/9fdf55f6/attachment.html