[Nix-dev] On npm2nix and the NPM package set in Nixpkgs

Tue Jul 5 08:36:25 CEST 2016

+1 for just keeping the name npm2nix and bumping up the version.

i'm not using it on any active project, but i'm going to in the near future.

On Mon, Jul 4, 2016 at 10:11 PM, Tobias Pflug <tobias.pflug at gmx.net> wrote:
> Hi Sander,
>
> sorry for my very late response. I'll make this one brief as I am sadly on
> my phone.
>
> I  belong to one of those who tried your new npm2nix and in fact am already
> using it regularly. I am very much in favor of having your re-engineeering2
> branch replacing npm2nix as the de-facto node integration tool.
>
> I also definitely want to see the current set of auto-generated node
> packages removed from nix. They are almost exclusively *totally* outdated.
>
> Thank you a lot for your continued efforts on this. Working with npm/node is
> annoying but we are better off with your contributions.
>
> cheers,
> Tobi
>
> On 22 Jun 2016, at 20:24, Sander van der Burg <svanderburg at gmail.com> wrote:
>
> Hello Nix and Node.js users,
>
> I have been absent for a while in this discussion, but as far as I know the
> state of the NPM packages in Nixpkgs is still quite bad and despite some
> discussions on the mailing list we have not really come to any consensus
> yet.
>
> As some of you may know, I have my own re-engineered version of npm2nix that
> lives in a specific branch in my own personal fork
> (https://github.com/svanderburg/npm2nix/tree/reengineering2). A few months
> ago, I did some major efforts in getting npm 3.x's behaviour supported,
> which I have documented in this blog post:
> http://sandervanderburg.blogspot.com/2016/02/managing-npm-flat-module-installations.html
>
> I have been using this reengineering2 branch for all my public and some of
> my private projects since the beginning of this year, and for me it seems to
> work quite well, despite the fact that some of npm 3.x's flat module
> installation oddities are still not accurately supported yet.
>
> I also received a couple of reports from other people claiming that their
> projects work and even encountered some people saying that it should replace
> the current npm2nix. :)
>
> Obviously, I do not want to claim that my implementation is the perfect
> solution as it (for example) is much slower than the vanilla npm2nix, and it
> composes the entire set of dependencies in one derivation as opposed to
> generating a Nix store path per NPM dependency. (I do this for a very good
> reason. For more details, please read my blog post).
>
> Furthermore, I have also spoken to people that suggested completely
> different kinds of approaches in getting NPM supported in a Nix environment.
>
> Something that I have not done yet is investigating whether this
> reengineered solution could be a potential replacement for the NPM packages
> set in Nixpkgs.
>
> Today, I have been working on an integration pattern, and the good news is:
> it seems that I was able to generate Nix expressions for almost all packages
> that are in pkgs/top-level/node-packages.json. The only exceptions were the
> node-xmpp-* and bip-* packages, but some of them seem to have broken
> dependencies, which is not npm2nix's fault.
>
> If we would proceed integrating, we have a number of practical implications:
>
> - I believe it is desired to have both Node.js 4.x and Node.js 5.x, 6.x
> supported (I actually need all of them). To support all of these, we need
> two different sets of generated Nix expressions. The former uses npm 2.x
> with the classic dependency addressing approach and the latter uses npm 3.x
> with flat module installations.
> - I think most library packages should be removed from node-packages.json:
> as explained in my blog post: how a package gets composed and to which
> version a range resolve depends on the state of the includer. When somebody
> wants their own NPM project to be deployed, he should use npm2nix directly
> on package.json, and not refer to any NPM libraries in Nixpkgs.
> - Some NPM packages must be overridden to provide native dependencies. The
> mechanisms that the reengineering2 branch use are different. It would
> probably take a bit of effort to get these migrated.
>
> For example, this is how I override the webdrvr package to provide phantomjs
> and the Selenium webdriver:
>
> {pkgs, system}:
>
> let
>   nodePackages = import ./composition-v4.nix {
>     inherit pkgs system;
>   };
> in
> nodePackages // {
>   webdrvr = nodePackages.webdrvr.override (oldAttrs: {
>     buildInputs = oldAttrs.buildInputs ++ [ pkgs.phantomjs ];
>
>     preRebuild = ''
>       mkdir $TMPDIR/webdrvr
>
>       ln -s ${pkgs.fetchurl {
>         url =
> "https://selenium-release.storage.googleapis.com/2.43/selenium-server-standalone-2.43.1.jar";
>         sha1 = "ef1b5f8ae9c99332f99ba8794988a1d5b974d27b";
>       }} $TMPDIR/webdrvr/selenium-server-standalone-2.43.1.jar
>       ln -s ${pkgs.fetchurl {
>         url =
> "http://chromedriver.storage.googleapis.com/2.10/chromedriver_linux64.zip";
>         sha1 = "26220f7e43ee3c0d714860db61c4d0ecc9bb3d89";
>       }} $TMPDIR/webdrvr/chromedriver_linux64.zip
>
>     '';
>   });
> }
>
>
> Although we have some practical issues, I think none of them would impose a
> serious problem.
>
> Then about npm2nix itself: Obviously, we could say that my version replaces
> the upstream npm2nix and gets "blessed" into the new "official" version, but
> I don't know whether everybody likes it.
>
> Alternatively, we could be a bit more pragmatic: I stop calling my
> reengineering2 version npm2nix, I give it a different name and I release it
> as a different package. This makes it possible for those who want it, to
> still use the 'vanilla' npm2nix alongside my version.
>
> Then in Nixpkgs we can decide to:
>
> - to keep npm2nix the default and provide my tool as a package
> - or to make the reengineering2 version the default, and provide npm2nix as
> a package
> - in theory: support both package sets, but this might be a bit overkill :)
>
> For those who don't know: although my repository is a fork of npm2nix, the
> reengineering2 version is basically a rewrite of npm2nix and quite different
> than the upstream version. It is written in JavaScript (as opposed to
> CoffeeScript), has a different modular structure and different command-line
> interface, so that's why I'm very careful in proposing to replace the
> upstream npm2nix.
>
> Moreover, it also does not share any git revision history with the upstream
> npm2nix. :)
>
> As a final note: for those who do not know about this: the reengineering2
> tool can already be used outside Nixpkgs and this is what I have been doing
> for all my projects. The expressions that it generates are based on the
> principles I have described in this blog post:
> http://sandervanderburg.blogspot.com/2014/07/managing-private-nix-packages-outside.html
>
> My apologies for this very long email, but I'd like to have your feedback
> and I don't want my preferences to disrupt other people's workflows.
>
> What do you think?
>
> Best,
>
> Sander
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>

-- 
Rok Garbas
http://www.garbas.si
rok at garbas.si