[Nix-dev] garbage collection and nix-push

[Mathijs Kwik]

With my limited perl knowledge, looking at nix-push's source shows me
that it builds expressions that I can (ab)use to generate these nar
dirs.

I can try to use these myself to get what I want.

On Wed, Oct 3, 2012 at 7:42 AM, Mathijs Kwik <mathijs at bluescreen303.nl> wrote:
> Michael Raskin <7c6f434c at mail.ru> writes:
>
>>>Hi all,
>>>
>>>The hardware I use for building my channel is not very powerful. It
>>>does a good job of building derivations, but packing nar files with xz
>>>takes ages.
>>>
>>>My channel is nothing special, It just builds the configurations for a
>>>few home machines and laptops and some packages that hydra doesn't
>>>build with the optional dependencies I like.
>>>
>>>By registering the outputs as gc roots, I made sure no extensive
>>>rebuilds are performed when I don't change much.
>>>However, after a garbage collector run, all nar outputs get removed,
>>>leading to long push times on the next run.
>>>
>>>So for now, I just lowered the garbage-collection cronjob to once a week.
>>>
>>>Does anyone have a suggestion on how to work around this?
>>>For example registering the nar outputs as (temp/indirect) gc roots or
>>>steering the garbage collector not to remove outputs that were built
>>>less than 48 hours ago or something?
>>
>> I am not too familiar with precise nix-push inside. Does the idea of
>> building a directory with symlinks of NARs (wo it depends on them all),
>> calling it something that includes current date, and removing
>> 100-hour-old link sets in cron look hard ot easy to implement?
>
> That's the direction I was thinking in indeed.
> However, apart from using "find" on the store to find the nar outputs, I
> could not find an easy way.
>
> nix-push outputs all nar files into 1 directory.
> However, in the store, each file is in its own subdirectory and the name
> of that dir is not easily guessable.
>
> nix-push also outputs .narinfo files, but they don't contain info on
> that dirname either.
>
> So it will get scripty =)
> First just find all .nar.xz files in /nix/store and store the listing in
> a temporary file, then grep that file for every narfile in nix-push's
> output dir and symlink to that.
>
> It's not hard to do, but I was hoping for a somewhat easier way.