[Nix-dev] Zero Hydra Failures (ZHF) project for NixOS

[Michael Raskin]

>- IRC bot that reports build failures for a range of commits once
>nixos-combined jobset is done

Would be nice

>- email to commiters that broke the the build (with a range of commits and
>list of builds failed)

Would be nice

>- nixos channel updates only when there are zero failures on jobset (this
>would mean reverts will happen often - and I believe that's the correct way
>to go instead of blocking people and punishing good testers)

Back to the good old times of any failure blocking channel update, m-m-m.

But we now have binary cache, so that doesn't matter that much.

Can we also have a branch where bot would push the zero-failure channel
commits, but no one else can push?

>- encouragement of "nox-review wip" use. This will bulid also all
>reverse-dependencies (good old Gentoo times)
>
>- ease of bisecting failures (for 99% of use cases this could be a range of
>commits done for jobset with test script respecting exit code of nix-build)
>- it could be even done as a separate service or part of hydra or just a
>copy/paste command for local testing

I hope there would be bisect-wide failure caching (most of the time you
simply need to find the first commit that changed anything at all in the
build).

>All that being said, there are still number of false positives that will
>drive people crazy. Mostly due to networking issues and transient failures
>in build systems (mostly during testing phase). We should address them one
>by one and reduce hard unpurities. I've already done so as part of ZHF, but
>much more work is needed.

Well, I think giving long-time committers minimal access to Hydra jobs 
(restarting failures as transient), that could already be a step
forward. I am not sure whether it is possible to give out GC forcing
rights without creating madness…