[Nix-dev] Reminder about nixpkgs branches

[Eelco Dolstra]

Hi,

On 24/07/14 21:32, Mathnerd314 wrote:

> So let me describe a little of my recent activities. I wanted the firefox from
> https://github.com/NixOS/nixpkgs/commit/872860e6de8e460fb11a79c46b8092bdcff35da8, the
> quassel from ~/nixpkgs, and the rest of the system from the latest hydra
> channel; I ended up making nixpkgs checkouts for each of those, hard-coding
> their paths in import statements, and manually adding each package (and some of
> their dependencies) to packageOverrides. 

Here it's really more appropriate to use nix-env (i.e. the imperative package
management style), which makes it easy to mix packages from different Nixpkgs
revisions.

The declarative style (environment.systemPackages) doesn't work very well for
this. It's doable if you only want a few packages from one other branch, but as
you point out, it doesn't scale very well.

> Note that, throughout this process, I was *creating* and *picking*, but I never
> needed to revert a commit. I just checked out older versions. I did end up
> trying out a single rollback, when I was looking at Quassel's NixOS module;
> however, it was completely useless, as the problem was in systemd's unit
> dependencies and thus *all* of my versions were broken.

Yes, mixing modules from different NixOS versions doesn't really work. Here it
is easier to have a local branch (e.g. based/rebased on release-14.04) onto
which you can cherry-pick the changes you need.

> NixOS is inherently a
> state-centric system; it's not "this change broke my system" but "version V of
> thing X is broken".

I don't know what this means.

> However, to set up my system the way I wanted it (i.e., so that all my programs
> worked), I did need 4 different nixpkgs checkouts. And I've only been using
> NixOS for a few weeks; as I set up more programs and services I'm probably going
> to need even more than that. Looking at
> https://github.com/MarcWeber/nixpkgs/branches/active it seems he has almost 40
> branches. His workflow is slightly different from mine because he uses TopGit
> instead of letting Nix do most of the work, but the branch-explosion is similar.
> When we start using nixpkgs-monitor, the problem is going to get even worse: a
> steady, near-realtime stream of patches that bump versions, URL's, and hashes,
> with no QA or error checking at all. 

Well, nixpkgs-monitor is intended to help package maintainers, not make them
redundant :-) Maintainers should still do testing before they push a patch from
the monitor.

> The problem is that, currently, Nix expressions are used for two purposes -
> building packages and specifying systems. Due to integration in the nixpkgs
> repository, the coupling between these has grown strong. But, as I hope my
> example has shown, these are orthogonal and independent activities. Sometimes I
> want to build packages, e.g. when I want to "build the latest version of Firefox
> from mozilla-central". Other times I want to specify my system, e.g., "use this
> known-good version of Firefox". If you look closer, you can see the difference:
> in the first case, I am referring to a dynamic, changing process. In the second
> case, I just need a store path. However, currently Nix is using the extensional
> model which intermingles these two.

I don't see the connection with the extensional model. It allows you to specify
a specific version just fine, e.g.

  environment.systemPackages = [ (builtins.storePath
/nix/store/gv67fwcwbyha77kxsmgcs41baqxh2ysi-firefox-31.0) ];

(Not the prettiest syntax, but NixOS wasn't really intended for this kind of
binary-only package management.)

The main advantage of the intensional model is that it allows untrusted users to
fetch binaries from untrusted binary caches.

> On the other side, the release process and picking process becomes easier. Hydra
> fetches all of the store paths, and it's just a matter of examining the results
> and picking the builds that seem to work and pass tests. 

This is pretty much the furthest you can get from declarative package
management. It makes a release the sum of a bunch of packages from different
Nixpkgs revisions, all built against whatever version of the dependencies
happened to be the most recently built at the time. Reproducing such a
configuration is pretty much impossible. (This is in fact the classic Linux
package model, where it is typically doubtful whether all binary packages can be
reproduced from their specifications.)

> After this, my vision is that the nixpkgs repository will only contain very
> high-level instructions, for example "to install firefox: got to mozilla ftp
> directory; load a folder (branch); download tarball; install to $out }". 

I assume you mean "fetch the latest version". That's fundamentally incompatible
with the idea of purely functional package management. It would mean that
running "nix-build -A firefox" on a specific Nixpkgs revision would give a
different result at different points in time.

> Anyways, I filed https://github.com/NixOS/nix/issues/296 a week ago, and
> received no response besides Dolstra tagging it as "feature" and "Nix 2.0". So
> my conclusion is that nobody actually cares enough for this to be implemented
> now, besides possibly me (I'm already doing a GSOC; two projects at once is
> probably not a good idea). 

Switching to the intensional model would be a huge undertaking which I don't see
happening soon.

-- 
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/