[Nix-dev] Nix for internal projects and monorepos

[Bas van Dijk]

Hi Daniel,

At LumiGuide we use the following structure for our monorepo:

  https://github.com/LumiGuide/lumi-example

Most of our packages are written in Haskell and these are stored in
the <lumi/hs-pkgs> directory (we have around 50 of them). Each package
contains a default.nix file where the src attribute is set to
`lib.sourceByRegex ./. [...]`. This ensure that only the needed
sources are build and that build artefacts like ./dist don't pollute
the sources. Also note that each Haskell package contains a shell.nix
file which is used when we want to work on a package using:

  make hs-pkgs.lumi-example-hs-pkg.shell

In the resulting shell we can invoke cabal to build the package. Our
emacs editor is configured to also invoke nix-shell to compile the
package:

  (require 'haskell-mode)
  (setq haskell-compile-cabal-build-command
    "cd %s && nix-shell --command 'cabal configure --enable-tests &&
cabal build --ghc-option=-ferror-spans'")

We use nixops to deploy to production. We don't call nixops directly
but use the appropriate targets in the Makefile. For example: make
facility-1.deploy.

We have been using this setup for over 2 years and it works well. Let
me know if you have any questions.

Bas

On 24 March 2017 at 21:01, Daniel Peebles <pumpkingod at gmail.com> wrote:
> Hi all,
>
> I've been ruminating about the best way to deal with an internal (to a
> company or otherwise private) package set with Nix, especially with a
> monorepo.
>
> My first inclination would be to leave Nix expressions (as default.nix)
> alongside the project source code, and use path references in those
> expressions (e.g., src = ./.;) to actually build the project.
>
> The difficulty arises when you want to deploy this stuff to production.
> Unless you want to ship your entire internal source tree to each box with
> Nix on it (no, I don't in practice want to be building my apps in
> production, and would use a Hydra/binary cache), you're going to want to
> "lift out" the nix hierarchy from your source code hierarchy, replace the
> src links with fixed-output derivations somehow, and ship the whole thing as
> a channel. I don't know of any tooling to do that, or how to do that
> effectively. It seems like it would require some tooling on the Nix side to
> work nicely.
>
> The alternative would be to adopt a more nixpkgs-like approach, and maintain
> a separate tree of packages that point into fixed-output references into the
> actual codebase. That unfortunately means moving the build spec farther away
> from the code it builds, and although it might be necessary in the case of
> nixpkgs (because we don't control all FOSS projects), it seems like we can
> do better.
>
> How do other people tackle this problem? Does anyone maintain a reasonably
> large internal Nix package set?
>
> Thanks,
> Dan
>
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>