[Nix-dev] Publish All of Hackage

[Shea Levy]

The problem with doing this with import-from-derivation is we still 
need the hashes of every tarball ahead of time (though that's much 
smaller than all of hackage, and we really just need the hash of the 
file that contains all the hashes in nixpkgs itself). If we have that, 
then we don't need to generate all of hackage every time, just the bits 
needed to build the specific requested package.

I have an unmerged attempt [1] at making import-from-derivation work 
better for cases like this.

Honestly, though, IMO the nix/nixpkgs model of "determine the build 
exactly from the expressions" simply does not match the modern 
language-specific package manager model of "dynamically query a web 
service for the latest versions that meet my constraints". I think 
something leveraging nix-exec [2] might be a good short-term solution, 
and in the long term better management of impurities (at the expression 
level, resulting in still-pure *builds*) is needed.

[1] https://github.com/NixOS/nix/pull/31
[2] https://github.com/shlevy/nix-exec

On 2015-11-20 08:50, Daniel Peebles wrote:
>> The downside of this approach is that generating the entire Haskell
>> package set is actually kind of expensive, and we probably wouldn't 
>> want
>> to impose those costs onto random users who just wants to have 
>> XMonad
>> for their window manager
>
> Couldn't the derivation we're importing from be cached like any other
> "binary" cache value? I've never checked, but intuitively it seems
> like it would make sense.
>
> On Fri, Nov 20, 2015 at 2:47 PM, Peter Simons <simons at cryp.to> wrote:
>
>> Hi Oliver,
>>
>>  > [You really want to commit] the scripts that *generate* the 50mb 
>> file
>>  > as some sort of Nix expression. Then, when I as a user choose to
>>  > evaluate the set of Haskell packages, I will be forced to 
>> generate
>>  > all the Nix expressions - or, this being Nix, ask a binary
>>  > substitution server for that.
>>
>> you are right -- that is totally what we should do, and we *can* do 
>> it
>> with Nix today, because we can import from a store derivation during
>> evaluation. Thank you for the suggestion!
>>
>> Now, I suppose this begets the question of why we check the "small"
>> hackage-packages.nix file into Nixpkgs? Shouldn't that be generated
>> during evaluation, too?
>>
>> The downside of this approach is that generating the entire Haskell
>> package set is actually kind of expensive, and we probably wouldn't 
>> want
>> to impose those costs onto random users who just wants to have 
>> XMonad
>> for their window manager, but still ... it's tempting to argue that
>> generated files should not be part of Nixpkgs. Instead, the 
>> generation
>> process should be.
>>
>> Best regards,
>> Peter
>>
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev [1]
>
>
>
> Links:
> ------
> [1] http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev