[Nix-dev] Improving security updates

[Nicolas Pierron]

On Sun, Apr 12, 2015 at 8:56 AM, James Cook <james.cook at utoronto.ca> wrote:
> On 10 April 2015 at 14:16, Nicolas Pierron <nicolas.b.pierron at gmail.com> wrote:
>> Hi,
>>
>> On Fri, Apr 10, 2015 at 7:12 PM, CodeHero <codehero at nerdpol.ch> wrote:
>>> So, after this huge update delay for nixos-unstable I think we should
>>> talk about improving the way security updates are handled. One can
>>> currently install security upgrades by using the instructions on this
>>> page (https://nixos.org/wiki/Security_Updates), but it's a lot of work
>>> to find all the libs that need those updates; and flagging packages as
>>> security updates will most likely not work without a dedicated security
>>> team.
>>>
>>> We've been brainstorming a little bit on the irc
>>> (https://botbot.me/freenode/nixos/2015-04-10/?msg=36316600&page=4), and
>>> we came up with a few ideas. I personally like the extra security branch
>>> idea, but i'm not sure how it would work out
>>> (https://botbot.me/freenode/nixos/2015-04-10/?msg=36318539&page=5), so
>>> that's why I'm asking here. Maybe somebody has some ideas and the
>>> know-how to make things better.
>>>
>>> The question is: who has suggestions on how to improve the installation
>>> of critical security updates; who knows how to implement the best
>>> suggestion; and who will maintain it?
>>
>> I want to enumerate a few points that we should keep in mind:
>>  1. This should be done as part of nixpkgs, and not as part of nixos
>>  2. This should be transparent for the end user.
>>  3. We should provide tools to watch for security updates. (polling /
>> push-notifications)
>>
>> Currently the only way to provide fast security updates is to
>> substitute in-place the packages which are vulnerable.  I do not think
>> that we have better alternative here.
>> Still, I think there is a problem with the approach suggested on the
>> wiki (https://nixos.org/wiki/Security_Updates  ), which is that there
>> is no automatic relation between the original package and the fix.
>>
>> The problem I can see with `replaceDependency`, is that we replace
>> `hash-vul` by `hash-fix` in one derivation, but not in all derivations
>> which are depending on `hash-vul`.
>> This leads to 2 questions:
>>  - How do we know that `hash-vul` is vulnerable ?
>>  - How do we know that one derivation depends on `hash-fix` ?
>>
>> Nixpkgs does not know about anything about runtime dependecies (except
>> for cross-compilation), it only know things about build dependencies.
>> This implies that one package can depend on a dependency of its
>> dependencies.  Thus we have to carry over the hash substitution of the
>> dependencies.
>>
>> A: buildInputs = [ B ];
>> B: buildInputs = [ C ];
>> C: [ { vul = ; fix = ; } ]
>>
>> A: [ /* A */ { vul = ; fix = ; } /* B */ { vul = ; fix = ; } /* C */ {
>> vul = ; fix = ; } ]
>> B: [ /* B */ { vul = ; fix = ; } /* C */ { vul = ; fix = ; } ]
>> C: [ /* C */ { vul = ; fix = ; } ]
>>
>> So basically, we want a channel, which is constructed by merging the
>> `last`, with the `small`.  Basically, the merge will flag all `small`
>> (fix) packages as being a replacement of the `last` (vulnerable).
>> Then we replace the `stdenv.mkDerivation` function to do the
>> substitution if the original derivation depends on a fixed small
>> package.
>>
>> This method implies that we should never bump the version of small
>> packages across ABI-incompatible changes.  Which means that we would
>> have to ensure that any small package is versioned in nixpkgs, at
>> least until the next `last` channel update.
>>
>> At the end, the small channel should mirror the latest channel, with
>> delta binary patches for all fixes of the small packages.
>>
>> I think this is all :)
>
> Dependency replacement has me pretty confused. If someone will indulge
> me, I want to make sure I understand the above point, or at least how
> replace-dependency.nix works (assuming that's what you're talking
> about).
>
> First of all, am I correct in assuming that replace-dependency.nix
> works by taking the output of the old derivation and doing some
> relatively fast post-processing, looking for symlinks to the old
> package and replacing with the new?

Almost, basically this looks like doing a search and replace of the
hash dependency by another in one output.
Redefining mkDerivation is just a simple way to collect all "patches"
which have to be done to the output.

> Now, let's work with an example: wget depends on gnutls and gnutls
> depends on libtasn1. libtasn1 needs to be bumped (GitHub issue 7333).
>
> We stalt by building everything with the vulnerable version of libtasn1, right?

If you were to use the --fallback option while using the
nixpkgs-security channel, yes.
I guess we can easily restrict this substitution if the derivation
match one of the pre-built packages of the "lastest" channel (on-top
of which the nixpkgs-security is build) such that non-prebuilt
binaries can be build from the latest sources only.

> Then, because of some magic (you're suggesting in
> stdenv.mkDerivation), a new version of everything that depends
> directly or indirectly on libtasn1 (or any other fixed package) is
> built. This building is fast, since it just involves taking the
> original output and doing some post-processing. (Does this happen by
> renaming pkgs to pkgsBeforeFixes, and setting pkgs =
> fancyNewApplyFixesFunction pkgsBeforeFixes in all-packages.nix?)
>
> Is that correct?

This is correct.  The only problem that we have as opposed to mutation
based system is that we have to iterate over every package to make
sure that we no longer have references to the bad one.

> Side questions:
> - Why does stdenv.mkDerivation need to be clever? Why not just blindly
> apply all the fixes to every package?

The substitution is not as simple as doing a "mapAttrs", as this would
lead to infinite loops, and would imply that you have to download all
the binary version of the fixed packages.

> - What are the `small` and `last` you rever to above?

I use small (or quick-fix) to refer to the tiny channel of quick fixes.
I use last (or stable) to refer to the last good known state of the channel.

The security channel is made out of these two, such that we inherit
all packages from the stable channel while doing something similar to
a mutation based approach on pre-build stable packages with the ABI
compatible changes from the quick-fix channel.

-- 
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/