[Nix-dev] What is the end game?

[Tim Cuthbertson]

On Mon, Aug 31, 2015 at 1:34 AM, Daniel Peebles <pumpkingod at gmail.com> wrote:
> I'm trying to avoid making this conversation about Hydra (or services)
> specifically, but agree with the points you're making.
>
> To try to steer the discussion in the direction I'm most interested in, what
> if binutils had a release.nix in its repo? How about gcc or clang? What
> would the expressions look like in nixpkgs? The options I can see:
>
> 1) nixpkgs is full of e.g., binutils = (import "${fetch* { url =
> "url/to/binutils"; sha256 = "abc"; }}/release.nix").build
> 2) Duplicate the release.nix of binutils in nixpkgs
> 3) Some combination of 1 and 2, where metadata, name, dependencies are
> expressed in nixpkgs, but build instructions are in the repo's release.nix
> 4) ???
> 5) profit
>
> Actually, I'm not sure #4 and #5 are relevant, but I couldn't think of other
> options.
>
> #1 seems impractical because a single nix-env would result in thousands of
> fetchurls
> #2 seems undesirable due to duplicated information
> #3 seems like our best bet, but I don't know what that would look like


As someone who's been using nixpkgs out-of-tree (i.e. writing
individual expressions for nix-build, but not yet integrating them
into nixpkgs) for a while, this interests me.

I thought (1) as Daniel described would have been be a good solution,
and seems to work fine for _building_ software. But I tried this out
with a new package I'm adding:
https://github.com/NixOS/nixpkgs/pull/9653

nix-env fails - and rightly so, because it can't figure out the name
(and presumably other details) of this package without downloading it,
which would indeed be silly.

The other approach (as cabal2nix is doing elsewhere in this thread) is
to have the primary expression in nixpkgs, and have the upstream
project extend it with overrideDerivation or similar. But that doesn't
make any sense for new package - there's nothing in nixpkgs yet to
extend!

So it seems like doing it this way would involve a few different stages:

 1) write a local .nix file in my package's repo
 2) once it seems to be working, move / copy it to nixpkgs
 3) once it's reached one or more channels, convert my package's .nix
file to override the "downstream" nixpkgs expression. It's not clear
when I should do this, as suddenly anyone using a "too old" nixpkgs
won't be able to build it anymore.

Further, I could imagine adding overrides in my repo for packaging
changes which should then be taken out once those same changes have
been merged into nixpkgs. But again, that's not a concrete point in
time - it depends what channel you're on, and when you last updated.

So... that's not very appealing.

Is there some variant of the "main expression in upstream repo,
metadata in nixpkgs" which would allow nix-env to get enough
information, while keeping the bulk of the logic upstream (and
non-duplicated)?

Cheers,
 - Tim.