[Nix-dev] On npm2nix and the NPM package set in Nixpkgs

Thu Jun 23 01:39:21 CEST 2016

Hi Sander,

awesome stuff.

I would say, change name to something like node2nix and let's merge the
thing as it looks very good.

Pros:
- backward compatibility
- process of merging will be lot faster (IMHO) as it will not collide with
anything and probably this will limit non productive discussions out

Big thanks,
Tom

2016-06-22 19:24 GMT+01:00 Sander van der Burg <svanderburg at gmail.com>:

> Hello Nix and Node.js users,
>
> I have been absent for a while in this discussion, but as far as I know
> the state of the NPM packages in Nixpkgs is still quite bad and despite
> some discussions on the mailing list we have not really come to any
> consensus yet.
>
> As some of you may know, I have my own re-engineered version of npm2nix
> that lives in a specific branch in my own personal fork (
> https://github.com/svanderburg/npm2nix/tree/reengineering2). A few months
> ago, I did some major efforts in getting npm 3.x's behaviour supported,
> which I have documented in this blog post:
> http://sandervanderburg.blogspot.com/2016/02/managing-npm-flat-module-installations.html
>
> I have been using this reengineering2 branch for all my public and some of
> my private projects since the beginning of this year, and for me it seems
> to work quite well, despite the fact that some of npm 3.x's flat module
> installation oddities are still not accurately supported yet.
>
> I also received a couple of reports from other people claiming that their
> projects work and even encountered some people saying that it should
> replace the current npm2nix. :)
>
> Obviously, I do not want to claim that my implementation is the perfect
> solution as it (for example) is much slower than the vanilla npm2nix, and
> it composes the entire set of dependencies in one derivation as opposed to
> generating a Nix store path per NPM dependency. (I do this for a very good
> reason. For more details, please read my blog post).
>
> Furthermore, I have also spoken to people that suggested completely
> different kinds of approaches in getting NPM supported in a Nix environment.
>
> Something that I have not done yet is investigating whether this
> reengineered solution could be a potential replacement for the NPM packages
> set in Nixpkgs.
>
> Today, I have been working on an integration pattern, and the good news
> is: it seems that I was able to generate Nix expressions for almost all
> packages that are in pkgs/top-level/node-packages.json. The only exceptions
> were the node-xmpp-* and bip-* packages, but some of them seem to have
> broken dependencies, which is not npm2nix's fault.
>
> If we would proceed integrating, we have a number of practical
> implications:
>
> - I believe it is desired to have both Node.js 4.x and Node.js 5.x, 6.x
> supported (I actually need all of them). To support all of these, we need
> two different sets of generated Nix expressions. The former uses npm 2.x
> with the classic dependency addressing approach and the latter uses npm 3.x
> with flat module installations.
> - I think most library packages should be removed from node-packages.json:
> as explained in my blog post: how a package gets composed and to which
> version a range resolve depends on the state of the includer. When somebody
> wants their own NPM project to be deployed, he should use npm2nix directly
> on package.json, and not refer to any NPM libraries in Nixpkgs.
> - Some NPM packages must be overridden to provide native dependencies. The
> mechanisms that the reengineering2 branch use are different. It would
> probably take a bit of effort to get these migrated.
>
> For example, this is how I override the webdrvr package to provide
> phantomjs and the Selenium webdriver:
>
> {pkgs, system}:
>
> let
>   nodePackages = import ./composition-v4.nix {
>     inherit pkgs system;
>   };
> in
> nodePackages // {
>   webdrvr = nodePackages.webdrvr.override (oldAttrs: {
>     buildInputs = oldAttrs.buildInputs ++ [ pkgs.phantomjs ];
>
>     preRebuild = ''
>       mkdir $TMPDIR/webdrvr
>
>       ln -s ${pkgs.fetchurl {
>         url = "
> https://selenium-release.storage.googleapis.com/2.43/selenium-server-standalone-2.43.1.jar
> ";
>         sha1 = "ef1b5f8ae9c99332f99ba8794988a1d5b974d27b";
>       }} $TMPDIR/webdrvr/selenium-server-standalone-2.43.1.jar
>       ln -s ${pkgs.fetchurl {
>         url = "
> http://chromedriver.storage.googleapis.com/2.10/chromedriver_linux64.zip";
>         sha1 = "26220f7e43ee3c0d714860db61c4d0ecc9bb3d89";
>       }} $TMPDIR/webdrvr/chromedriver_linux64.zip
>
>     '';
>   });
> }
>
>
> Although we have some practical issues, I think none of them would impose
> a serious problem.
>
> Then about npm2nix itself: Obviously, we could say that my version
> replaces the upstream npm2nix and gets "blessed" into the new "official"
> version, but I don't know whether everybody likes it.
>
> Alternatively, we could be a bit more pragmatic: I stop calling my
> reengineering2 version npm2nix, I give it a different name and I release it
> as a different package. This makes it possible for those who want it, to
> still use the 'vanilla' npm2nix alongside my version.
>
> Then in Nixpkgs we can decide to:
>
> - to keep npm2nix the default and provide my tool as a package
> - or to make the reengineering2 version the default, and provide npm2nix
> as a package
> - in theory: support both package sets, but this might be a bit overkill :)
>
> For those who don't know: although my repository is a fork of npm2nix, the
> reengineering2 version is basically a rewrite of npm2nix and quite
> different than the upstream version. It is written in JavaScript (as
> opposed to CoffeeScript), has a different modular structure and different
> command-line interface, so that's why I'm very careful in proposing to
> replace the upstream npm2nix.
>
> Moreover, it also does not share any git revision history with the
> upstream npm2nix. :)
>
> As a final note: for those who do not know about this: the reengineering2
> tool can already be used outside Nixpkgs and this is what I have been doing
> for all my projects. The expressions that it generates are based on the
> principles I have described in this blog post:
> http://sandervanderburg.blogspot.com/2014/07/managing-private-nix-packages-outside.html
>
> My apologies for this very long email, but I'd like to have your feedback
> and I don't want my preferences to disrupt other people's workflows.
>
> What do you think?
>
> Best,
>
> Sander
>
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>

-- 
Tomasz Czyż
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20160623/5f10d291/attachment-0001.html>