[Nix-dev] nixos-container networking

[Michael Walker]

Tomasz,

I have declarative container networking set up and working on a VPS,
but I wrote most of the configuration as I was learning things, so it
may not be the best way.

Here's the configuration.nix for the VPS:
https://github.com/barrucadu/nixfiles/blob/master/hosts/innsmouth.nix
Each container has a config file here:
https://github.com/barrucadu/nixfiles/tree/master/containers

Containers have ports forwarded to them via NAT; each container is
running a web server on port 80 with the host reverse-proxying via
nginx; the host also does https and letsencrypt for all the proxied
containers.

At the top of the innsmouth.nix file, I have a "containerSpecs" record
which has all the details for each container. The relevant bits of the
config are:

1. Set up the networking and NAT:

networking.nat.enable = true;
networking.nat.internalInterfaces = ["ve-+"];
networking.nat.externalInterface = "enp0s4";

2. Forward ports to containers:

networking.nat.forwardPorts = concatMap
    ( {num, ports, ...}:
        map (p: { sourcePort = p; destination =
"192.168.255.${toString num}:${toString p}"; }) ports
    ) containerSpecs';

3. Define all the containers:

containers = mapAttrs
    (_: {num, config, ...}:
        { autoStart = true
        ; privateNetwork = true
        ; hostAddress = "192.168.254.${toString num}"
        ; localAddress = "192.168.255.${toString num}"
        ; config = config
        ; }
    ) containerSpecs;

4. Reverse-proxy HTTPS to HTTP in each container, manage letsencrypt
certificates, and forward HTTP to HTTPS.

This is a little complex as I have a fairly custom nginx config (see
the services/nginx.nix file in the repository), but the
reverse-proxying is fairly straightfoward. Here is the generated
nginx.conf: https://misc.barrucadu.co.uk/nginx.txt

On 13 March 2017 at 02:12, Tomasz Czyż <tomasz.czyz at gmail.com> wrote:
> Hey,
>
> could anyone using nixos-container (declarative style) share how you setup
> networking?
>
> I'm trying to setup few containers with private network and http proxy at
> the front. Each container potentially could run application on port 80 and I
> would like to expose them through proxy.
>
> I tried to set this up with
>
> privateNetwork=true;
> hostAddress
> localAddress
>
> and I tried to also run nat on the host with (just to enable outbound
> traffic)
> internalInterfaces = ["ve-+"];
> externalInterfaces = "eth0";
>
> but no luck.
> My next try will be creating bridge on the host and add containers to that
> bridge. Is that how you do stuff or are better ways of doing container
> networking?
>
> Tom
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>



-- 
Michael Walker (http://www.barrucadu.co.uk)