[Nix-dev] Fwd: nixos-container networking

Tomasz Czyż tomasz.czyz at gmail.com
Tue Mar 14 19:12:37 CET 2017 

---------- Forwarded message ----------
From: Tomasz Czyż <tomasz.czyz at gmail.com>
Date: 2017-03-14 18:12 GMT+00:00
Subject: Re: [Nix-dev] nixos-container networking
To: Danylo Hlynskyi <abcz2.uprola at gmail.com>


Hey Danylo,

yup, I hit 13 char limit and because I was removing - I thought it's that.

I also had the issue with recreating containers, but this happened only
sometimes and didn't happen when I restart the machine so I was not sure
why is that. Thanks for your mail, it's very useful.

Would you share your bridget networking?
I was trying that but I'm wondering if you have one shared bridge or you
have bridge per container and how you access containers from host (or route
traffic to them).

Cheers,
Tom


2017-03-14 6:01 GMT+00:00 Danylo Hlynskyi <abcz2.uprola at gmail.com>:

> Strange, I have lot's of containers with "-" and experience no problems.
> But maybe you've exceeded by accident limit 13 symbols per container name?
>
> Also, last time I tried "veth" networking, I was struggling from
> https://github.com/NixOS/nixpkgs/issues/16330. My container experience
> was awful when I tried container renames. That's why I've already switched
> to bridged networking
>
> ---
>
> BTW, I highly recommend patch to switch-to-configuration.pl
> <https://github.com/NixOS/nixpkgs/pull/3021/commits/6e36619b277f78ece1bb81b79b5651897e46a2bf#diff-0a057d6ff3f6f83f68b859178484f4fe>
> from https://github.com/NixOS/nixpkgs/pull/3021/commits/6e36619b2
> 77f78ece1bb81b79b5651897e46a2bf
>
> It isn't clear from commit message, but it does the following: makes
> declarative containers truly reloadable (when you change
> container config, it activates new configuration for container). The
> culprit is *it should be* default behavior, because of
>
> 1. https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/
> virtualisation/containers.nix#L225-L230
> 2. https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/
> virtualisation/containers.nix#L676
>
> I'd like to PR this, but got no time to test properly other parts of Nixos.
>
> 2017-03-14 4:42 GMT+02:00 Tomasz Czyż <tomasz.czyz at gmail.com>:
>
>> Michael, Ian, thank you for your answers.
>>
>> Looks like my problem was with the container name. I tried bunch of
>> different setups which didn't work and I discovered that when I'm using "-"
>> in container name it doesn't work (I had impression that worked one or two
>> times when I started machine from scratch, but most of the time didn't).
>>
>> After I removed "-" from the name, looks like private network is working
>> (I can access private IP of container) so I don't need NAT actually.
>>
>> Tom
>>
>> 2017-03-13 23:54 GMT+00:00 Ian-Woo Kim <ianwookim at gmail.com>:
>>
>>> I've recently made nixos-container port forwarding easier (both
>>> imperative and declarative) and it's now merged into master.
>>>
>>> https://github.com/NixOS/nixpkgs/pull/20869
>>>
>>> Hope that this helps.
>>>
>>> Ian
>>>
>>> On Sun, Mar 12, 2017 at 7:52 PM, Michael Walker <mike at barrucadu.co.uk>
>>> wrote:
>>> > Tomasz,
>>> >
>>> > I have declarative container networking set up and working on a VPS,
>>> > but I wrote most of the configuration as I was learning things, so it
>>> > may not be the best way.
>>> >
>>> > Here's the configuration.nix for the VPS:
>>> > https://github.com/barrucadu/nixfiles/blob/master/hosts/innsmouth.nix
>>> > Each container has a config file here:
>>> > https://github.com/barrucadu/nixfiles/tree/master/containers
>>> >
>>> > Containers have ports forwarded to them via NAT; each container is
>>> > running a web server on port 80 with the host reverse-proxying via
>>> > nginx; the host also does https and letsencrypt for all the proxied
>>> > containers.
>>> >
>>> > At the top of the innsmouth.nix file, I have a "containerSpecs" record
>>> > which has all the details for each container. The relevant bits of the
>>> > config are:
>>> >
>>> > 1. Set up the networking and NAT:
>>> >
>>> > networking.nat.enable = true;
>>> > networking.nat.internalInterfaces = ["ve-+"];
>>> > networking.nat.externalInterface = "enp0s4";
>>> >
>>> > 2. Forward ports to containers:
>>> >
>>> > networking.nat.forwardPorts = concatMap
>>> >     ( {num, ports, ...}:
>>> >         map (p: { sourcePort = p; destination =
>>> > "192.168.255.${toString num}:${toString p}"; }) ports
>>> >     ) containerSpecs';
>>> >
>>> > 3. Define all the containers:
>>> >
>>> > containers = mapAttrs
>>> >     (_: {num, config, ...}:
>>> >         { autoStart = true
>>> >         ; privateNetwork = true
>>> >         ; hostAddress = "192.168.254.${toString num}"
>>> >         ; localAddress = "192.168.255.${toString num}"
>>> >         ; config = config
>>> >         ; }
>>> >     ) containerSpecs;
>>> >
>>> > 4. Reverse-proxy HTTPS to HTTP in each container, manage letsencrypt
>>> > certificates, and forward HTTP to HTTPS.
>>> >
>>> > This is a little complex as I have a fairly custom nginx config (see
>>> > the services/nginx.nix file in the repository), but the
>>> > reverse-proxying is fairly straightfoward. Here is the generated
>>> > nginx.conf: https://misc.barrucadu.co.uk/nginx.txt
>>> >
>>> > On 13 March 2017 at 02:12, Tomasz Czyż <tomasz.czyz at gmail.com> wrote:
>>> >> Hey,
>>> >>
>>> >> could anyone using nixos-container (declarative style) share how you
>>> setup
>>> >> networking?
>>> >>
>>> >> I'm trying to setup few containers with private network and http
>>> proxy at
>>> >> the front. Each container potentially could run application on port
>>> 80 and I
>>> >> would like to expose them through proxy.
>>> >>
>>> >> I tried to set this up with
>>> >>
>>> >> privateNetwork=true;
>>> >> hostAddress
>>> >> localAddress
>>> >>
>>> >> and I tried to also run nat on the host with (just to enable outbound
>>> >> traffic)
>>> >> internalInterfaces = ["ve-+"];
>>> >> externalInterfaces = "eth0";
>>> >>
>>> >> but no luck.
>>> >> My next try will be creating bridge on the host and add containers to
>>> that
>>> >> bridge. Is that how you do stuff or are better ways of doing container
>>> >> networking?
>>> >>
>>> >> Tom
>>> >>
>>> >> _______________________________________________
>>> >> nix-dev mailing list
>>> >> nix-dev at lists.science.uu.nl
>>> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Michael Walker (http://www.barrucadu.co.uk)
>>> > _______________________________________________
>>> > nix-dev mailing list
>>> > nix-dev at lists.science.uu.nl
>>> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>
>>
>>
>>
>> --
>> Tomasz Czyż
>>
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>
>>
>


-- 
Tomasz Czyż



-- 
Tomasz Czyż
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170314/413684e6/attachment.html>

More information about the nix-dev mailing list