[Nix-dev] Re: [Nix-commits] SVN commit: nix - r25375 - in

[Evgeny Egorochkin]

On Thursday 06 January 2011 19:37:13 7c6f434c at mail.ru wrote:
> >> >    nixpkgs/trunk/pkgs/tools/networking/knetworkmanager/
> >> >    nixpkgs/trunk/pkgs/tools/networking/knetworkmanager/live.nix
> >> 
> >> I've disabled this in all-packages.nix because the fetchgitrevision call
> >> breaks "nix-env -q".  This is because "nix-env -q" won't perform
> >> derivations during evaluation.
> >> 
> >> Also, perhaps these packages should be kept in a separate tree of
> >> "bleeding edge" developer/maintainer-targeted Nix expressions, since
> >> Nixpkgs should have at least some minimum level of stability :-)
> >
> >However, I disagree that this is the right way to address the problem.
> >
> >KNetworkManager doesn't really have any releases to speak of. Distros use
> >snapshots. The idea was to have a live build and then a stable snapshot of
> >it: knetworkmanager = knetworkmanager_live.override { revision = "..." };
> 
> We do have quite a lot of snapshot builds in Nixpkgs. Unfortunately, live
> builds go one step further...
> 
> We lose stability in the sense of a configuration.nix + /etc/nixos/*/
> revisions being enough to determine everything thagt gets built uniquely.
> 
> >As to fetchgitrevision in general, this function was intended to be
> >overridden by the end user. For example, some people would prefer live
> >builds to be only rebuilt once a day or once a week, and not everytime
> >there's a new revision. Some other people, would rather have
> >fetchgitrevision use a manifest file with repository+revision pairs
> >that's generated by an external script.
> >
> >If the default trunk-following implementation breaks stuff, let's replace
> >it with a dummy that would output something that would not break
> >evaluation and would throw an exception if you try to instantiate it.
> >Users of live builds would override gitfetchrevision dummy with
> >gitfetchrevisiondaily, gitfetchrevisionweekly or whatever it is they
> >want.
> 
> I think that the good middle ground would be specifying some default state
> for those who do want to keep their systems reproducible.
> 
> By the way, as far as I understand, live builds will not work with chroot
> builds enabled. Generally, I'd say that if you want live build, it seems
> easier to override src and just run "git pull" / "mtn pull; mtn up" /
> whatever on the repository itself - that works with chroot builds etc.
> 
> >As to stability, say we have kde4.6 in trunk, but the stable kde version
> >is clearly and conveniently named as kde4. I don't see any problems with
> >this or how live builds are different. You can't install the live build
> >by accident afaik, or if you can, we can always lower the priority to fix
> >this. And of course, someone using a live build should at least expect to
> >face the issues that arise, which in many cases means no issues, because
> >many projects try and maintain trunk in a working state, doing
> >development in branches.
> >
> >Oher projects use branches to efficiently release patches. One such
> >package is redmine(which I'm packaging now). They have 0.* branches and
> >0.x.* releases. However, you can get fixes faster if you follow 0.*
> >branch, instead of waiting for a maintenance release. All comits in the
> >branch are maintenance only, keeping full compatibility. For a web app,
> >getting security fixes faster is essential. The only nice way to
> >implement this is trunk-following live builds. The user would do
> >nixos-rebuild boot and if any of essential services are updated, activate
> >the configuration as soon as they possibly can(eg avoiding downtime
> >during peak hours).
> >
> >Of course, in theory, you could wrap nixos-checkout and add some code to
> >check for new revisions. But which solution is more practical? Which
> >solution is more automated and solves more problems? Which solution
> >requires the user to delve deeper into release/fix/patch approach of the
> >app they use?
> 
> Unfortunately, it can not only solve problems, but also create other ones;
> the question is which is better. Currently, after succesful "nixos-rebuild
> build", it is very hard for "nixos-rebuild boot" to fail. The simplest
> problem I can imagine is "disk full". With your patch, loss of
> connectivity or an upstream bug fix that needs some tweaking to work with
> our directory layout can make this fail.
> 
> Another problem is that if I update, write down all revisions and back up
> configuration.nix, I have near-certain chances of recovering _exactly_ same
> system configuration if a need arises. It always seemed to me that being
> able bisect problems is a large source of confidence. Having auto-updating
> expressions in trunk may break this.
> 
> In general, I think that building something to update specified set of
> repos into nixos-rebuild (just like it can update manifests before
> building) would be very nice and useful. Live builds directly as
> implemented now put many very desirable properties of Nixpkgs/NixOS at
> risk while giving benefits that I cannot call groundbreaking.
> 
> Just for the record: currently I do use a set of wrapper scripts to update
> repositories of four packages in Nixpkgs (the repositories are used as src
> overrides), and I do think that trunk builds are useful - but I do not
> think that they are worth what it would cost to adopt current solution
> widely as-is.

I don't propose we adopt my solution widely. In fact, downloading a tarball is 
so much better and faster. I propose to adopt it only where other solutions 
fail or are inferior.

Now, the solution itself ctually consists of several losely-related parts.

One part is essentially a scm build approach which makes it easier to override 
the source and which abstracts away the potentially dirty code that decides 
which revision is getting built.

Another part is one of the possible implementations of the revision decision 
function, and the dirtiest of the possible implementations. I agree with the 
criticism that it fails at reproducibility department making bisecting system 
configs a problem.

What bothers me is that  ALL criticism and discussion so far was centered 
aroung this small bit of code(calling it the solution btw), ignoring the 
proposed scm build format and the fact that the revision logic is pluggable.

What I DO agree with is that the dirtiest fetchgitrevision approach should not 
be made the default, it was a mistake on my part.

I see the dirty revision approach as an easy way to run a trunk build of some 
program by the end user, not necessarily the best and only approach, such as: 
try a dirty live build, if it works, pin the revision which is has just 
printed in console and forget about it; or as in case with redmine, follow the 
maintenance branch.

The manifest-based approach is what you would use if you used many live 
builds, or wanted to bisect the config. And I do plan implementing this as 
well. This entails having a manifest file put under revision control and an 
external script to fetch current revisions.

-- 
Evgeny