[Nix-dev] On npm2nix and the NPM package set in Nixpkgs
Sander van der Burg
svanderburg at gmail.com
Thu Jul 7 22:57:19 CEST 2016
Good point! I just also made a change that adds a small disclaimer comment
on top the generated expressions.
On Tue, Jul 5, 2016 at 4:17 PM, Graham Christensen <graham at grahamc.com>
wrote:
> You all are great! Thank you so much!
>
> Graham
>
> On Tue, Jul 5, 2016 at 8:14 AM Kamil Chmielewski <kamil.chm at gmail.com>
> wrote:
>
>> +1.. I'll do this in go2nix.
>>
>> --
>> Kamil
>>
>> 2016-07-05 15:10 GMT+02:00 Rok Garbas <rok at garbas.si>:
>>
>>> +1 ... i did just that recently for pypi2nix. but i'll also add a link
>>> to the project home.
>>>
>>> [1]
>>> https://github.com/garbas/pypi2nix/commit/339aee3b149909430ebe7e3e27b8cf158addaef1
>>>
>>> On Tue, Jul 5, 2016 at 2:47 PM, Graham Christensen <graham at grahamc.com>
>>> wrote:
>>> > I've found myself confused by multiple projects using the same lang2nix
>>> > name, and big changes in format. One consistent complaint I have is
>>> the top
>>> > of the file usually says:
>>> >
>>> > // Generated by lang2nix
>>> >
>>> > but having more information like a version number and a URL to the
>>> project
>>> > would have saved hours of searching and trying different tools.
>>> Something
>>> > like:
>>> >
>>> > // Generated by lang2nix v0.1.0
>>> > // See more at https://github.com/myuser/lang2nix
>>> >
>>> > would be a really nice usability adjustment.
>>> >
>>> > On Tue, Jul 5, 2016 at 7:36 AM Rok Garbas <rok at garbas.si> wrote:
>>> >>
>>> >> we can still keep and old version of npm2nix in nixpkgs for ppl who
>>> use
>>> >> it.
>>> >> and also a branch with old code could be created, for people that want
>>> >> pudh bugfixes or develop further (very unlikely).
>>> >>
>>> >>
>>> >> On Tue, Jul 5, 2016 at 11:16 AM, Tomasz Czyż <tomasz.czyz at gmail.com>
>>> >> wrote:
>>> >> > Rok,
>>> >> >
>>> >> > what about people who are already using previous solution? Why break
>>> >> > their
>>> >> > workflows?
>>> >> >
>>> >> > 2016-07-05 7:36 GMT+01:00 Rok Garbas <rok at garbas.si>:
>>> >> >>
>>> >> >> +1 for just keeping the name npm2nix and bumping up the version.
>>> >> >>
>>> >> >> i'm not using it on any active project, but i'm going to in the
>>> near
>>> >> >> future.
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Mon, Jul 4, 2016 at 10:11 PM, Tobias Pflug <
>>> tobias.pflug at gmx.net>
>>> >> >> wrote:
>>> >> >> > Hi Sander,
>>> >> >> >
>>> >> >> > sorry for my very late response. I'll make this one brief as I am
>>> >> >> > sadly
>>> >> >> > on
>>> >> >> > my phone.
>>> >> >> >
>>> >> >> > I belong to one of those who tried your new npm2nix and in fact
>>> am
>>> >> >> > already
>>> >> >> > using it regularly. I am very much in favor of having your
>>> >> >> > re-engineeering2
>>> >> >> > branch replacing npm2nix as the de-facto node integration tool.
>>> >> >> >
>>> >> >> > I also definitely want to see the current set of auto-generated
>>> node
>>> >> >> > packages removed from nix. They are almost exclusively *totally*
>>> >> >> > outdated.
>>> >> >> >
>>> >> >> > Thank you a lot for your continued efforts on this. Working with
>>> >> >> > npm/node is
>>> >> >> > annoying but we are better off with your contributions.
>>> >> >> >
>>> >> >> > cheers,
>>> >> >> > Tobi
>>> >> >> >
>>> >> >> > On 22 Jun 2016, at 20:24, Sander van der Burg <
>>> svanderburg at gmail.com>
>>> >> >> > wrote:
>>> >> >> >
>>> >> >> > Hello Nix and Node.js users,
>>> >> >> >
>>> >> >> > I have been absent for a while in this discussion, but as far as
>>> I
>>> >> >> > know
>>> >> >> > the
>>> >> >> > state of the NPM packages in Nixpkgs is still quite bad and
>>> despite
>>> >> >> > some
>>> >> >> > discussions on the mailing list we have not really come to any
>>> >> >> > consensus
>>> >> >> > yet.
>>> >> >> >
>>> >> >> > As some of you may know, I have my own re-engineered version of
>>> >> >> > npm2nix
>>> >> >> > that
>>> >> >> > lives in a specific branch in my own personal fork
>>> >> >> > (https://github.com/svanderburg/npm2nix/tree/reengineering2). A
>>> few
>>> >> >> > months
>>> >> >> > ago, I did some major efforts in getting npm 3.x's behaviour
>>> >> >> > supported,
>>> >> >> > which I have documented in this blog post:
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> http://sandervanderburg.blogspot.com/2016/02/managing-npm-flat-module-installations.html
>>> >> >> >
>>> >> >> > I have been using this reengineering2 branch for all my public
>>> and
>>> >> >> > some
>>> >> >> > of
>>> >> >> > my private projects since the beginning of this year, and for me
>>> it
>>> >> >> > seems to
>>> >> >> > work quite well, despite the fact that some of npm 3.x's flat
>>> module
>>> >> >> > installation oddities are still not accurately supported yet.
>>> >> >> >
>>> >> >> > I also received a couple of reports from other people claiming
>>> that
>>> >> >> > their
>>> >> >> > projects work and even encountered some people saying that it
>>> should
>>> >> >> > replace
>>> >> >> > the current npm2nix. :)
>>> >> >> >
>>> >> >> > Obviously, I do not want to claim that my implementation is the
>>> >> >> > perfect
>>> >> >> > solution as it (for example) is much slower than the vanilla
>>> npm2nix,
>>> >> >> > and it
>>> >> >> > composes the entire set of dependencies in one derivation as
>>> opposed
>>> >> >> > to
>>> >> >> > generating a Nix store path per NPM dependency. (I do this for a
>>> very
>>> >> >> > good
>>> >> >> > reason. For more details, please read my blog post).
>>> >> >> >
>>> >> >> > Furthermore, I have also spoken to people that suggested
>>> completely
>>> >> >> > different kinds of approaches in getting NPM supported in a Nix
>>> >> >> > environment.
>>> >> >> >
>>> >> >> > Something that I have not done yet is investigating whether this
>>> >> >> > reengineered solution could be a potential replacement for the
>>> NPM
>>> >> >> > packages
>>> >> >> > set in Nixpkgs.
>>> >> >> >
>>> >> >> > Today, I have been working on an integration pattern, and the
>>> good
>>> >> >> > news
>>> >> >> > is:
>>> >> >> > it seems that I was able to generate Nix expressions for almost
>>> all
>>> >> >> > packages
>>> >> >> > that are in pkgs/top-level/node-packages.json. The only
>>> exceptions
>>> >> >> > were
>>> >> >> > the
>>> >> >> > node-xmpp-* and bip-* packages, but some of them seem to have
>>> broken
>>> >> >> > dependencies, which is not npm2nix's fault.
>>> >> >> >
>>> >> >> > If we would proceed integrating, we have a number of practical
>>> >> >> > implications:
>>> >> >> >
>>> >> >> > - I believe it is desired to have both Node.js 4.x and Node.js
>>> 5.x,
>>> >> >> > 6.x
>>> >> >> > supported (I actually need all of them). To support all of
>>> these, we
>>> >> >> > need
>>> >> >> > two different sets of generated Nix expressions. The former uses
>>> npm
>>> >> >> > 2.x
>>> >> >> > with the classic dependency addressing approach and the latter
>>> uses
>>> >> >> > npm
>>> >> >> > 3.x
>>> >> >> > with flat module installations.
>>> >> >> > - I think most library packages should be removed from
>>> >> >> > node-packages.json:
>>> >> >> > as explained in my blog post: how a package gets composed and to
>>> >> >> > which
>>> >> >> > version a range resolve depends on the state of the includer.
>>> When
>>> >> >> > somebody
>>> >> >> > wants their own NPM project to be deployed, he should use npm2nix
>>> >> >> > directly
>>> >> >> > on package.json, and not refer to any NPM libraries in Nixpkgs.
>>> >> >> > - Some NPM packages must be overridden to provide native
>>> >> >> > dependencies.
>>> >> >> > The
>>> >> >> > mechanisms that the reengineering2 branch use are different. It
>>> would
>>> >> >> > probably take a bit of effort to get these migrated.
>>> >> >> >
>>> >> >> > For example, this is how I override the webdrvr package to
>>> provide
>>> >> >> > phantomjs
>>> >> >> > and the Selenium webdriver:
>>> >> >> >
>>> >> >> > {pkgs, system}:
>>> >> >> >
>>> >> >> > let
>>> >> >> > nodePackages = import ./composition-v4.nix {
>>> >> >> > inherit pkgs system;
>>> >> >> > };
>>> >> >> > in
>>> >> >> > nodePackages // {
>>> >> >> > webdrvr = nodePackages.webdrvr.override (oldAttrs: {
>>> >> >> > buildInputs = oldAttrs.buildInputs ++ [ pkgs.phantomjs ];
>>> >> >> >
>>> >> >> > preRebuild = ''
>>> >> >> > mkdir $TMPDIR/webdrvr
>>> >> >> >
>>> >> >> > ln -s ${pkgs.fetchurl {
>>> >> >> > url =
>>> >> >> >
>>> >> >> >
>>> >> >> > "
>>> https://selenium-release.storage.googleapis.com/2.43/selenium-server-standalone-2.43.1.jar
>>> ";
>>> >> >> > sha1 = "ef1b5f8ae9c99332f99ba8794988a1d5b974d27b";
>>> >> >> > }} $TMPDIR/webdrvr/selenium-server-standalone-2.43.1.jar
>>> >> >> > ln -s ${pkgs.fetchurl {
>>> >> >> > url =
>>> >> >> >
>>> >> >> >
>>> >> >> > "
>>> http://chromedriver.storage.googleapis.com/2.10/chromedriver_linux64.zip
>>> ";
>>> >> >> > sha1 = "26220f7e43ee3c0d714860db61c4d0ecc9bb3d89";
>>> >> >> > }} $TMPDIR/webdrvr/chromedriver_linux64.zip
>>> >> >> >
>>> >> >> > '';
>>> >> >> > });
>>> >> >> > }
>>> >> >> >
>>> >> >> >
>>> >> >> > Although we have some practical issues, I think none of them
>>> would
>>> >> >> > impose a
>>> >> >> > serious problem.
>>> >> >> >
>>> >> >> > Then about npm2nix itself: Obviously, we could say that my
>>> version
>>> >> >> > replaces
>>> >> >> > the upstream npm2nix and gets "blessed" into the new "official"
>>> >> >> > version,
>>> >> >> > but
>>> >> >> > I don't know whether everybody likes it.
>>> >> >> >
>>> >> >> > Alternatively, we could be a bit more pragmatic: I stop calling
>>> my
>>> >> >> > reengineering2 version npm2nix, I give it a different name and I
>>> >> >> > release
>>> >> >> > it
>>> >> >> > as a different package. This makes it possible for those who
>>> want it,
>>> >> >> > to
>>> >> >> > still use the 'vanilla' npm2nix alongside my version.
>>> >> >> >
>>> >> >> > Then in Nixpkgs we can decide to:
>>> >> >> >
>>> >> >> > - to keep npm2nix the default and provide my tool as a package
>>> >> >> > - or to make the reengineering2 version the default, and provide
>>> >> >> > npm2nix
>>> >> >> > as
>>> >> >> > a package
>>> >> >> > - in theory: support both package sets, but this might be a bit
>>> >> >> > overkill
>>> >> >> > :)
>>> >> >> >
>>> >> >> > For those who don't know: although my repository is a fork of
>>> >> >> > npm2nix,
>>> >> >> > the
>>> >> >> > reengineering2 version is basically a rewrite of npm2nix and
>>> quite
>>> >> >> > different
>>> >> >> > than the upstream version. It is written in JavaScript (as
>>> opposed to
>>> >> >> > CoffeeScript), has a different modular structure and different
>>> >> >> > command-line
>>> >> >> > interface, so that's why I'm very careful in proposing to
>>> replace the
>>> >> >> > upstream npm2nix.
>>> >> >> >
>>> >> >> > Moreover, it also does not share any git revision history with
>>> the
>>> >> >> > upstream
>>> >> >> > npm2nix. :)
>>> >> >> >
>>> >> >> > As a final note: for those who do not know about this: the
>>> >> >> > reengineering2
>>> >> >> > tool can already be used outside Nixpkgs and this is what I have
>>> been
>>> >> >> > doing
>>> >> >> > for all my projects. The expressions that it generates are based
>>> on
>>> >> >> > the
>>> >> >> > principles I have described in this blog post:
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> http://sandervanderburg.blogspot.com/2014/07/managing-private-nix-packages-outside.html
>>> >> >> >
>>> >> >> > My apologies for this very long email, but I'd like to have your
>>> >> >> > feedback
>>> >> >> > and I don't want my preferences to disrupt other people's
>>> workflows.
>>> >> >> >
>>> >> >> > What do you think?
>>> >> >> >
>>> >> >> > Best,
>>> >> >> >
>>> >> >> > Sander
>>> >> >> >
>>> >> >> > _______________________________________________
>>> >> >> > nix-dev mailing list
>>> >> >> > nix-dev at lists.science.uu.nl
>>> >> >> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>> >> >> >
>>> >> >> >
>>> >> >> > _______________________________________________
>>> >> >> > nix-dev mailing list
>>> >> >> > nix-dev at lists.science.uu.nl
>>> >> >> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>> >> >> >
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Rok Garbas
>>> >> >> http://www.garbas.si
>>> >> >> rok at garbas.si
>>> >> >> _______________________________________________
>>> >> >> nix-dev mailing list
>>> >> >> nix-dev at lists.science.uu.nl
>>> >> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Tomasz Czyż
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Rok Garbas
>>> >> http://www.garbas.si
>>> >> rok at garbas.si
>>> >> _______________________________________________
>>> >> nix-dev mailing list
>>> >> nix-dev at lists.science.uu.nl
>>> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>
>>>
>>>
>>> --
>>> Rok Garbas
>>> http://www.garbas.si
>>> rok at garbas.si
>>> _______________________________________________
>>> nix-dev mailing list
>>> nix-dev at lists.science.uu.nl
>>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>
>>
>>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20160707/e001da8d/attachment-0001.html>
More information about the nix-dev
mailing list