[Nix-dev] Hydra and security updates

[Leo Gaspard]

On 06/03/2017 01:55 AM, Frank wrote:
> Op 3-6-2017 om 0:59 schreef Leo Gaspard:
>> On 06/02/2017 06:54 PM, Frank wrote:
>>> Op 1-6-2017 om 23:32 schreef Leo Gaspard:
>>>> Hi all,
>>>>
>>>> I just wanted to point out an issue with hydra: it doesn't make any
>>>> distinction between security updates and normal changes.
>>> Why is this an issue? Security-updates are just as likely to introduce
>>> bugs as every other update.
>> If I have to choose between having a security vulnerability and having
>> some installer tests that don't build (as these seem to be the source of
>> most test failures)... I know what I'd rather have (especially given
>> install images aren't generated from every commit of nixpkgs), don't you
>> think?
> You mean al the tests that didn't catch the bug in the first place? Or
> the tests that assure the fix will be installed without problems?
> 
> If the testing is a problem for distributing the software, the tests are
> probably wrong. You can't fix things by testing, so don't try to repeat
> and improve the upstream testing (not during distribution at least).
> 
> The focus of the distribution is, distributing software, that installs
> well on all target systems. And if your fix breaks some systems it
> doesn't matter how important it is for security.
> 
> I really agree, it's important to roll out security fixes fast. But I
> don't see why other updates should be very time consuming.

OK, I think I failed explaining what I think the issue is.

In my mind, the issue is not having a security fix that breaks tests*,
as fixes are(/should be) tested by upstream to not change any observable
behavior except the actual security flaw.

However, the issue is in having security fixes being delayed by
unrelated commits that break tests. Because those other packages are way
more disruptive than a security fix, and can (often) break tests, as
there is no enforced "must pass tests on hydra" before merging a PR.


* Even though I'd bet that may happen with transient test failures --
and I'd still want that patch, so that anyone can't break in my system,
even though it may mean some features not working perfectly as intended:
time for tests is when preparing the patch, patching systems should be
done within a few hours at most to consistently avoid attacks, and a few
hours is hardly enough to even rebuild the system and get people to
patch. Like, major distros get an ahead-of-time notification of serious
flaws and prepare and pre-build the patch before it's even known to us,
just to get the patch out faster... But it's not my main point, as this
should actually just never happen, the choice of behavior in this case
is irrelevant.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: OpenPGP digital signature
URL: <https://mailman.science.uu.nl/pipermail/nix-dev/attachments/20170603/6e451c6d/attachment.sig>