[Nix-dev] Simplify node packages?

[Profpatsch]

On 17-04-23 01:10pm, Benno Fünfstück wrote:
> * A simple approach to packaging NPM applications would be to split the
> package for an NPM application into two:
>   - fetch: let NPM download all the dependencies. it should(?) be possible
> to make this a fixed-output derivation, as long as NPM downloads the same
> set of dependencies each time
>   - build: use the cache from the fetch phase, should require no network
> access
> 
> The fetch phase should be deterministic if we freeze the NPM index (is such
> a thing possible?) and use shrinkwrap.
> Or is this not the case? In what cases will it fail to be deterministic?
> 
> * Alternatively, `yarn` claims to be more deterministic. Can we use `yarn`
> to build all NPM packages? Or are there NPM packages that are incompatible?

I’m in the process of writing yarn2nix.

After resolving dependencies yarn writes a lockfile,
example: https://github.com/facebook/react/blob/master/yarn.lock
As you can observe this has everything we need (even the hash!)

Sadly it is not a known format, so I wrote a simple-stupid parser:
https://hackage.haskell.org/package/yarn-lock-0.1.0/docs/Yarn-Lock.html

The pre-alpha-version of converting yarn.lock files to nix expressions
can be found here:
https://github.com/Profpatsch/yarn2nix

nix-shell
$ hpack
$ cabal build
$ ./dist/build/yarn2nix/yarn2nix

should do the trick.

It works for simple yarn.lock files, but the generated output
cannot be nicely checked in yet, since it’s way too verbose.
I need to change the internal data representation a bit,
to a multi-keyed map.

When trying to build our production (ember) app I stumbled on
another major problem: npm package dependencies and yarn.lock
files have cycles.
No sane package manager would allow cycles, but here we are.
So a cycle-breaking function is still needed, not very much
work but not trivial either.

The most interesting property is, that all dependency trees
are linked together with symlinked, so they are cached by nix
on a package level:
https://github.com/Profpatsch/yarn2nix/blob/master/buildNodePackage.nix#L33

I expect yarn2nix to be ready in a few days, most certainly
in the next two weeks.

-- 
Proudly written in Mutt with Vim on NixOS.
May take up to five days to read your message. If it’s urgent, call me.