[Nix-dev] nixos-container networking
Danylo Hlynskyi
abcz2.uprola at gmail.com
Wed Mar 15 09:58:14 CET 2017
I think it's possible. It is even done for imperative containers:
https://github.com/NixOS/nixpkgs/blob/98a9d815e05dd56aba0f9040d96092335cd90444/pkgs/tools/virtualization/nixos-container/nixos-container.pl#L131
(but I think 11 chars is too restrictive - veth name should be 16 chars max
minus 3 for "ve-" or "vb-")
Another possibility to remove this (unintuitive IMO) restriction is to
generate veth name uniquely by some hashing (like it's done in
https://github.com/chrisfarms/nixos-shell ). Yet another possibility -
expose veth name as configurable container parameter (break abstraction).
2017-03-14 16:48 GMT+02:00 Thomas Hunger <tehunger at gmail.com>:
> Would it be possible to add an assert if there are any restrictions on the
> naming? I don't know enough about this to be of much help though.
>
> On 14 March 2017 at 06:01, Danylo Hlynskyi <abcz2.uprola at gmail.com> wrote:
>
>> Strange, I have lot's of containers with "-" and experience no problems.
>> But maybe you've exceeded by accident limit 13 symbols per container name?
>>
>> Also, last time I tried "veth" networking, I was struggling from
>> https://github.com/NixOS/nixpkgs/issues/16330. My container experience
>> was awful when I tried container renames. That's why I've already switched
>> to bridged networking
>>
>> ---
>>
>> BTW, I highly recommend patch to switch-to-configuration.pl
>> <https://github.com/NixOS/nixpkgs/pull/3021/commits/6e36619b277f78ece1bb81b79b5651897e46a2bf#diff-0a057d6ff3f6f83f68b859178484f4fe>
>> from https://github.com/NixOS/nixpkgs/pull/3021/commits/6e36619b2
>> 77f78ece1bb81b79b5651897e46a2bf
>>
>> It isn't clear from commit message, but it does the following: makes
>> declarative containers truly reloadable (when you change
>> container config, it activates new configuration for container). The
>> culprit is *it should be* default behavior, because of
>>
>> 1. https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/
>> virtualisation/containers.nix#L225-L230
>> 2. https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/
>> virtualisation/containers.nix#L676
>>
>> I'd like to PR this, but got no time to test properly other parts of
>> Nixos.
>>
>> 2017-03-14 4:42 GMT+02:00 Tomasz Czyż <tomasz.czyz at gmail.com>:
>>
>>> Michael, Ian, thank you for your answers.
>>>
>>> Looks like my problem was with the container name. I tried bunch of
>>> different setups which didn't work and I discovered that when I'm using "-"
>>> in container name it doesn't work (I had impression that worked one or two
>>> times when I started machine from scratch, but most of the time didn't).
>>>
>>> After I removed "-" from the name, looks like private network is working
>>> (I can access private IP of container) so I don't need NAT actually.
>>>
>>> Tom
>>>
>>> 2017-03-13 23:54 GMT+00:00 Ian-Woo Kim <ianwookim at gmail.com>:
>>>
>>>> I've recently made nixos-container port forwarding easier (both
>>>> imperative and declarative) and it's now merged into master.
>>>>
>>>> https://github.com/NixOS/nixpkgs/pull/20869
>>>>
>>>> Hope that this helps.
>>>>
>>>> Ian
>>>>
>>>> On Sun, Mar 12, 2017 at 7:52 PM, Michael Walker <mike at barrucadu.co.uk>
>>>> wrote:
>>>> > Tomasz,
>>>> >
>>>> > I have declarative container networking set up and working on a VPS,
>>>> > but I wrote most of the configuration as I was learning things, so it
>>>> > may not be the best way.
>>>> >
>>>> > Here's the configuration.nix for the VPS:
>>>> > https://github.com/barrucadu/nixfiles/blob/master/hosts/innsmouth.nix
>>>> > Each container has a config file here:
>>>> > https://github.com/barrucadu/nixfiles/tree/master/containers
>>>> >
>>>> > Containers have ports forwarded to them via NAT; each container is
>>>> > running a web server on port 80 with the host reverse-proxying via
>>>> > nginx; the host also does https and letsencrypt for all the proxied
>>>> > containers.
>>>> >
>>>> > At the top of the innsmouth.nix file, I have a "containerSpecs" record
>>>> > which has all the details for each container. The relevant bits of the
>>>> > config are:
>>>> >
>>>> > 1. Set up the networking and NAT:
>>>> >
>>>> > networking.nat.enable = true;
>>>> > networking.nat.internalInterfaces = ["ve-+"];
>>>> > networking.nat.externalInterface = "enp0s4";
>>>> >
>>>> > 2. Forward ports to containers:
>>>> >
>>>> > networking.nat.forwardPorts = concatMap
>>>> > ( {num, ports, ...}:
>>>> > map (p: { sourcePort = p; destination =
>>>> > "192.168.255.${toString num}:${toString p}"; }) ports
>>>> > ) containerSpecs';
>>>> >
>>>> > 3. Define all the containers:
>>>> >
>>>> > containers = mapAttrs
>>>> > (_: {num, config, ...}:
>>>> > { autoStart = true
>>>> > ; privateNetwork = true
>>>> > ; hostAddress = "192.168.254.${toString num}"
>>>> > ; localAddress = "192.168.255.${toString num}"
>>>> > ; config = config
>>>> > ; }
>>>> > ) containerSpecs;
>>>> >
>>>> > 4. Reverse-proxy HTTPS to HTTP in each container, manage letsencrypt
>>>> > certificates, and forward HTTP to HTTPS.
>>>> >
>>>> > This is a little complex as I have a fairly custom nginx config (see
>>>> > the services/nginx.nix file in the repository), but the
>>>> > reverse-proxying is fairly straightfoward. Here is the generated
>>>> > nginx.conf: https://misc.barrucadu.co.uk/nginx.txt
>>>> >
>>>> > On 13 March 2017 at 02:12, Tomasz Czyż <tomasz.czyz at gmail.com> wrote:
>>>> >> Hey,
>>>> >>
>>>> >> could anyone using nixos-container (declarative style) share how you
>>>> setup
>>>> >> networking?
>>>> >>
>>>> >> I'm trying to setup few containers with private network and http
>>>> proxy at
>>>> >> the front. Each container potentially could run application on port
>>>> 80 and I
>>>> >> would like to expose them through proxy.
>>>> >>
>>>> >> I tried to set this up with
>>>> >>
>>>> >> privateNetwork=true;
>>>> >> hostAddress
>>>> >> localAddress
>>>> >>
>>>> >> and I tried to also run nat on the host with (just to enable outbound
>>>> >> traffic)
>>>> >> internalInterfaces = ["ve-+"];
>>>> >> externalInterfaces = "eth0";
>>>> >>
>>>> >> but no luck.
>>>> >> My next try will be creating bridge on the host and add containers
>>>> to that
>>>> >> bridge. Is that how you do stuff or are better ways of doing
>>>> container
>>>> >> networking?
>>>> >>
>>>> >> Tom
>>>> >>
>>>> >> _______________________________________________
>>>> >> nix-dev mailing list
>>>> >> nix-dev at lists.science.uu.nl
>>>> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Michael Walker (http://www.barrucadu.co.uk)
>>>> > _______________________________________________
>>>> > nix-dev mailing list
>>>> > nix-dev at lists.science.uu.nl
>>>> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>>
>>>
>>>
>>>
>>> --
>>> Tomasz Czyż
>>>
>>> _______________________________________________
>>> nix-dev mailing list
>>> nix-dev at lists.science.uu.nl
>>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>>
>>>
>>
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170315/7e1b943c/attachment.html>
More information about the nix-dev
mailing list