[Nix-dev] Language integration (Was: NiJS package manager)

[Matthew Sackman]

On Tue, Apr 01, 2014 at 06:03:13PM +0000, Marc Weber wrote:
> Please think the other way round:
> - most linux distributions (exceptions LFS I know about) want to install
>   packages
> 
>   Some packages (such as gnome) have dependency information, but only
>   encoded in "configure" scripts or READMEs.

I'm not sure how that the above relates to the rest of your email.

To my mind:
1. The incumbants will not be turned: there is no way a) the market has
the appetite to switch to a new distribution as a whole (at least,
top-down); b) any distribution is going to switch away from their
existing packaging strategy.

2. Making debs and rpms is really hard, error prone, and generally an
unpleasant experience that no one is going to bother with. Nix is vastly
simpler and nicer to use to generate packages - having a language which
actually has the act of building a package as a built in primitive
operation is a *huge* step forwards.

3. The fact that no one likes making debs and rpms is one (of several)
reason why almost all new programming languages reinvent the wheel and
have their own packaging systems for language libraries an so forth.
(It's also why much "deployment" consists of "git clone ...".) No one
bats an eyelid these days at RANDOM(language de jour) having its own way
of expressing, downloading and installing libraries. This works to Nix's
advantage if you promote a "if you want to deploy your code, this is
what you need to do" approach rather than a "you should abandon
everything you know about linux and switch to this system instead" (much
though I agree the world would be a better place if the latter
happened). Adoption is driven by developers, and the hordes of hipster
developers really don't care very much about linux and technical purity.
Gone are the days when we all spent a few days reading howtos on setting
up smail, uw-imapd and writing xf86-config files by hand. I think you
can appeal to developers by stressing the potential benefits of
developing with nix as part of their tool-chain: making the
code-package-deploy-test cycle much nicer - more predictable, quite
possibly language-agnostic, and with nicely abstracted deployment
details.

> Rubyforge and PyPi (don't know what's current in python community) do
> have some limited readable information. At least my nixpkgs-ruby-overlay
> is based on it.
> 
> Thus maybe there is a way to convert dependency information rather than
> duplicating the work.

I think it's more: duplicating information is not viable. Manually
curating is also not viable. Such information has to be "owned" by the
package author(s) and it must be essential for the author(s) to maintain
that information for their package to work at all, on any platform.
Thankfully, this is the case: the authors must produce a gemfile, a
pom.xml, a rebar.config. In Go, you just have "import" lines directly in
your code which reference Git - there's even less overhead for the
author. "All" that's needed is preprocessing tools that can read that
information, transitively follow chains of deps, and spit out nix
expressions for everything that has been discovered.

But I'm not sure we're disagreeing, or actually talking about the same
thing. So if I've misunderstood, please do say!

> Even omitting "dependency information' - just knowing about the latest
> stable version - would make it easier to keep nixpkgs/NiJS/guix up to
> date IMHO.

Yes indeed, and that information has always historically been manually
curated by the distributions AIUI. The other issue of course is rapid
notifications of security issues and fixes. Someday, we'll have a better
world... ;)

Matthew