[Nix-dev] nix for reproducible and self-updating developer environments

[zimbatm]

Lots of good points, definitely experiencing most of these issues myself
(especially the SSL issues lately).

In terms of language support, it seems that we're slowly getting there.
That's what I got from NixCon, lots of people interested in helping out.
But yeah, it takes time. I think that first the foundation needed to be
built before that kind of thing could be tackled properly.

Quick question: have you tried the -P option as in `nix-env -iPf
packages.nix` ?

On Thu, 26 Nov 2015 at 02:25 Alexander Schmolck <a.schmolck at gmail.com>
wrote:

> Hi,
>
> I've been using nix for providing developers on linux and os x with a
> reproducible development environment that is automatically kept up to
> date. I'd like to give a little bit of feedback about the experience
> and ask for some advice how to improve certain aspects (I'm pretty
> sure what I'm doing isn't optimal, so any help is greatly
> appreciated).
>
> The basic setup is that I have a file packages.nix in our main
> development repo that looks roughly like this:
>
> # packages.nix -----
> let
>      # override versions and options for a few system packages, e.g.
> to unbreak php on darwin
>      config = import ./config.nix;
>      # pin to a particular version of the nix-channel
>      pinnedNixpkgs = fetchTarball
> https://github.com/NixOS/nixpkgs-channels/archive/SOMEHASH.tar.gz;
>      pkgs = import pinnedNixpkgs {inherit config};
>      callPackage = pkgs.lib.callPackageWith (pkgs // self);
>      self =  {
>        pep8 = pkgs.pythonPackages.pep8;
>        yapf = callPackage ./yapf.nix { };
>        # ... lots more packages, tools, languages etc.
>      };
> } in  self
> # packages.nix ----
>
> I then added a git hook for branch changes, merges etc. that does
> nix-env -if packages.nix. That way, for all the development tools and
> dependencies managed by nix, a developer should automatically always
> have the right version when switching to a particular branch or
> merging the latest master release into their branch.  On CI, I run the
> build in a nix-shell with a default.nix that roughly looks like so:
>
> # default.nix ---
> let stdenv = (fetchTarball
> https://github.com/NixOS/nixpkgs-channels/archive/SOMEHASH.tar.gz).stdenv;
> in
> {
>    ourDevEnv = stdenv.mkDerivation rec {
>       name = "our-dev-env";
>       buildInputs = builtins.attrValues (import ./packages.nix {});
>    };
> }
> # default.nix ---
>
> which should hopefully provide isolation for parallel build agent
> runs. After a successful build of master on the CI server I also do
> nix-push --dest /tmp/cache $(nix-build packages.nix) and upload to S3,
> to create a binary cache for our derivations.
>
>
> The positives:
>
> - a comprehensive cross-platform and cross-language solution! In
> particular, it covers the most popular deployment (linux) and
> development (os x, linux) platforms and can handle the whole stack,
> unlike e.g. virtualenv/pip or rvm/bundler. I've been bitten badly and
> frequently in the past by python and ruby problems caused by subtle
> shared library differences between different machines even when all
> the python or ruby dependencies where fully pinned.
> - nix-env -if package.nix is extremely fast in the no-op or switching
> back to a previous version case (under a sec), this is what makes it
> possible to run it all the time in a git hook without slowing down git
> use. That's really helpful for ensuring everyone has an up-to-date
> environment.
> - good coverage and recency of packages, especially given the
> community size relative to ubuntu, centos etc.
> - not only is nix(os) conceptually years ahead, I've been very
> impressed by the design of the nix language. Normally when I encounter
> a DSL I just wish someone had used scheme or similar but there are
> very few things I don't like about the nix language, it's very well
> thought out.
> - in many cases, adding a package that's not in nix yet (e.g. yapf
> above) or overriding the version is pretty simple and straightforward
> and the majority of derivations in nixpkgs look quite nice (much more
> so than e.g. debian package defs).
> - well written reference documentation (although certain parts could
> use a bit more coverage)
> - it's pretty easy to setup a cache in S3
>
> The not so positives:
>
> - selecting and pinning, and later bumping a version of nixkpkgs seems
> more work than it should be (maybe I'm doing it the wrong way)
> - neither nix-env or nix-shell really feels quite right for managing
> developer environments. Nix-shell is great for developing derivations
> or CI but it's too high friction for normal development (and I believe
> wouldn't support the git-hook-auto-update trick). Nix-env, on the
> other hand, is a tad too stateful. I'd like a way to make sure that
> *only* the packages currently in packages.nix (and maybe some optional
> user defined personal.nix) are active, so that removing something from
> our packages.nix really deletes it from the user environment. I tried
> something along the lines of keeping a a reference to the /nix/store
> path of the current nix-env binary, nix-env -e '*' and then use the
> absolute path to install packages.nix. But that appears tricky to get
> right and isn't super-elegant. Is there a better solution I'm missing?
> - os x support felt pretty alpha a lot of the time. Several packages
> are linux only and amongst those that are not important packages like
> php have been consistently broken. The El Captain upgrade caused us
> lots of woes and all sorts of fundamental clang compilation breakage.
> Of course major changes between xcode and os x versions made that a
> tough nut to crack and it seems the situation has improved quite a bit
> now, probably thanks to the switch to pure darwin.
> - The reproducible bit turned out harder in practice than I had hoped,
> despite having our own cache and despite trying to pin to a particular
> version of the channel (see above), I ran into at least two problems:
> the above nix-push didn't seem to be sufficient to really put all the
> required artefacts into our cache. One thing that's evidently missing
> is the nixpkgs tarball. I'd like to include it (what's a good way of
> doing so?), but a bigger problem is missing binaries, especially in os
> x where . In one case I had to resort to nix-serve to get an os x
> developer back to a working build environment, because his clang
> compile on nix was hosed and although the nix-env -if got some
> packages out of our cache, it tried to install some other stuff from
> source. The second problem I had, is that try as I might, I couldn't
> convince nix not to use cache.nixos.org, even if I removed it from
> /etc/nix/nix.conf.
> - Other devs experienced various nix-related breakage that needed my
> help to debug in order to get them back to a functioning development
> environment again (e.g. the nix install via curl sometimes appeared to
> be broken, there was at least one instance of corrupted hydra
> packages, nix1.9 became unable to read nixpkgs in a relatively short
> timeframe, I had some instances were people had problems with stuff
> that completely puzzled me, e.g. different conflicting versions of the
> same package installed (despite our pinning attempts), ~/.nix-defexpr
> going missing, apparent inconsistencies in the nix store for no
> obvious reasons – "waiting for locks or build slots" etc.). Maybe some
> of these were caused by attempts to fix things themselves that were
> based on mistaken notions how nix works, but either way it was more of
> a problem than I had anticipated.
> - overriding stuff can be quite challenging in certain cases, for
> example php uses an older and mostly undocumented override mechanism
> - private repos are a headache (fetchGitPrivate is awkward enough that
> I decided to  avoid it completely because I saw no way to robustly
> make it work in an automated fashion. That necessitated some
> additional complexity)
> - ssl authentication has also been a cause of headaches; in the end I
> had to remove curl and git from packages.nix because ssl validation
> failures caused to many problems (this was compounded by some
> particularities of our build process, such as enforcing a well
> controlled set of environment variables for reproducibility, which
> means SSL_CERT_FILE set by nix.sh is not preserved in builds). Also,
> whilst SSL_CERT_FILE works in principle on linux, because there is an
> authoritative file in the required format, I've had various issues on
> OS X where no canonical such file exists.
> - although nix is a nice language, it suffers a bit on the tooling
> side (the combination of lack of type-checking and laziness can result
> in confusing error messages, pretty printing and editor support could
> be better, and as far as I'm aware there is no tooling that could )
> - the quality of programming language infrastructure is a bit uneven
> and documentation is generally sparse (e.g. there seem to be two
> different and incompatible versions of npm2nix and for languages other
> than Haskell documentation on how to do development with language X
> and nix is pretty sparse). What I tried to do with python and ruby
> worked pretty well, but I've had for example no luck with getting
> node-based utilities with a plugin architecture to work (e.g eslint,
> with more than one plugin at the same time).
> - although there is no lack of terrible language packaging
> infrastructure (python or go come to mind) outside of the Haskell camp
> (cursed with cabal and blessed with an unusual appetite for obscure
> tech) nix seems to have seen little uptake as a dev dependency
> management solution by language communities.
>
> One frustration I have with nix is that I sometimes get the impression
> that after solving 95% of the hard problems in an elegant and original
> way it fails to leverage this innovation to provide clearly outlined
> go-to solutions for problems for which people struggle with.  Even
> after investing a fair amount of time into learning it, it's often not
> obvious how to use to accomplish particular tasks in a way that
> represents a major win. Compare that to e.g. docker were I can find
> and teach others in minutes how to solve concrete problems that would
> otherwise be real pain-points with a minimum of fuzz (e.g. run a bunch
> of integration tests in parallel with port and process isolation, test
> a build with a different linux version, bring up a self contained
> service like a code search engine for your repo with a single shell
> line etc.). Docker has it's share of design problems and pitfalls, but
> it's possible to get some real value out of it amazingly quickly. I
> don't think that's true of nix yet to the same extent, although I find
> it far more interesting than docker.
>
> For example the nix homepage touts reproducibility and that "Nix makes
> it trivial to set up and share build environments for your projects,
> regardless of what programming languages and tools you’re using." It
> certainly feel like that ought to be the case, because that would be a
> natural killer feature for nix. However, based on my experiences, that
> statement just doesn't strike me as accurate for any reasonable
> definition of "trivial" (at least if one goes by what can easily found
> in the official docs).
>
> What would be great to see in this context are clear step to step
> instructions for:
>
> - bootstrapping a fixed version of nix, ideally from a self-contained
> archive
> - a simple way to specify (and later bump) an exact known good version
> of nixpkgs and also to downgrade or upgrade individual packages
> relative to that.
> - self-hosting all vital components, so that e.g. an outage (or
> disappearance) of nixos.org or one of the upstream providers of src
> tarballs does not mean that it's suddenly no longer possible to set up
> a developer machine or do a deploy. It's also needed so that
> non-public artifacts can be installed efficiently rather than
> recompiled on every dev's machine. In practice I think that means an
> easy way to set up a binary and, ideally, upstream src tar.gz cache in
> S3 (other options are good as well, of course, but S3 seems to be the
> winning solution for hosting a bunch of files easily, cost-efficiently
> and with sufficient reliability, access controls and bandwidth).
> - a robust way to set additional configuration options where required
> in an automated fashion (think 'git config set'/install flags rather
> than sed and pray)
> - a good way to integrate private packages
> - some easy steps and concrete examples how to integrate utilities and
> library dependencies for the key programming languages. What the key
> languages are is of course both open to debate and a moving target.
> However, by any reckoning javascript,  python, ruby, go, c/c++ are
> definitely up there (and java and php probably too). Mainly because of
> the sheer amount of tooling they underpin, in addition to their
> popularity as languages. Also, since it's unreasonable to expect for
> example every frontend dev to master a relatively obscure lazy
> functional programming language and package system before they can add
> or bump some library dependency, it would really be invaluable to have
> a very streamlined process for updating say npm or gem dependencies.
> Ideally it would just involve editing a package.json and issuing  a
> single command (e.g. `nix bump`) to get an updated nix expression.
> Bundix, npm2nix are great steps in this direction but they're still
> too hard to use, IMO. If nix can't be used successfully in teams were
> the majority have little time to invest in learning it, it's unlikely
> to ever see wide adoption.
>
> I think a git repo with an evolving, complete worked example how to
> provide a reproducible dev environment for a multi-language project
> would be super helpful. If there's interest I'd be more than willing
> to contribute to this. As noted, there are shortcomings with the
> approach I outlined above but maybe it can serve as a starting point
> for something more polished with the aid of more experienced community
> members.
>
> alexander
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20151126/4dd2d5b3/attachment-0001.html