[Nix-dev] Anarchy

[Shea Levy]

Hi Michi,

On Jun 27, 2012, at 8:02 AM, Michael Weiss <wm at borasi.de> wrote:

> Greetings,
> 
> I'd like to derail the discussion about policy and focus on the parts 
> about how to reduce the need to agree on policy. As others have pointed 
> out, nix is great for having your own cake and I want to know how to 
> make the most of it. Unfortunately, I'm incapable of really grasping nix 
> as a language and nixpkgs in particular, so all I can do at this point 
> is sketch some visions. As Helmut Schimidt said, people with visions 
> should go see a doctor and given that many among you have a doctorate 
> degree, I think this list is a valid place to turn to.
> 

Not a PhD, but maybe I can treat your symptoms anyway ;)

> Packaging software with nix can be easy. Pick a name, point to the 
> source, define the dependencies, done. The difficult part is that I'm 
> not really done at this point. Ignorant of nix, I then try to find out 
> how others have done before me and insert a line in haskell-packages.nix 
> and maybe in all-packages.nix, too. This works out until somebody 
> upstream adds a package with an adjacent name in the lexicographic 
> order, that I chose to respect. Now that's a strange conflict. 
> Intuitively, adding something to a collection should never break what's 
> already there.
> 

Well, expecting the insert algorithm (i.e. upstream committers) for an ordered list to respect elements it doesn't know about (i.e. your local changes) is a bit too much to ask of the current state-of-the-art in computer science :).

> Is it possible to define package names only once by placing the 
> expression in the filesystem hierarchy?

I don't think it's currently possible, but that doesn't mean it never could be. The main problem right now is that nix doesn't have any primitives for enumerating the files in a given directory. As a result, we have things like manually listing in all-packages.nix and, if you look at the nixos sources, a manually maintained modules.nix file.

That being said, I'm not sure there's an algorithm that would do the job right even if we did have those primitives. There are a few decisions to be made when adding a package to all-packages above and beyond simply following a lexicographic order: What name should I give the attribute? Should I use import, callPackage, or something else? Do I need to override the default argument filled in by callPackage? There are some reasonable default assumptions but I think in the end there'd still be a fair bit of manual work there.

> It would then be impossible to 
> have null bytes or slashes in names but I could live with that. Having 
> to deal with different hierarchies (attributes, filesystem, a flat 
> namespace of names?) and naming restrictions at once is somewhat 
> challenging, invites misconfiguration and requires more agreement.
> 
> Let's talk about names. The most advanced naming system I know of is 
> Content Centric Networking (http://www.ccnx.org/). The basic principle 
> is that publishers assign hierarchically structured names to content and 
> secure that binding by signing it. Consumers retrieve content by asking 
> the network for its name and may optionally specify restrictions on what 
> content is acceptable. For example I might ask for enumeration of names 
> and subsequently content under ccnx:/nixos.org/pkgs/stdenv signed by 
> Eelco and ccnx:/cryp.to/pkgs/haskell signed by Peter or 
> ccnx:/InFactWeDon'tRelyOnDNS/home encrypted and signed by me.
> 
> The protocol itself does not assign any meaning to names but there are 
> naming conventions 
> (http://www.ccnx.org/releases/latest/doc/technical/NameConventions.html)
> that are useful to follow to get versioning out of the box. It is not to 
> be confused with a source control system as it is not concerned with 
> merging, just with naming and getting stuff.
> 

Wow. That seems really powerful and really way too much for what we want. Do we really need to get signing involved to fix our admittedly ad-hoc naming situation?

> Executive summary: Punk works best with pull semantics and preferably 
> only one hierarchy.
> 

Ah, it turns out I do need a doctorate to help you, as you've completely lost me here.

> Cheers,
> michi

Cheers,
Shea