[Nix-dev] NFS performance issues after upgrade to 16.09

[4levels]

Hi all,

I removed the vers=3 and some other NFS v3 related mount options since my
host supports NFS versions up to 4.1 and the issues seem to have
disappeared!

As this is only happening (and NFS is only needed) in my local VM, this
works for me ;-)

If the issue comes back, I'll continue debugging NFS and report back here.

Kind regards,

Erik

On Mon, 17 Oct 2016, 06:47 Wout Mertens, <wout.mertens at gmail.com> wrote:

Maybe it's something else?
https://www.clearos.com/clearfoundation/social/community/solved-nfs-causing-high-load

To further troubleshoot, I'd look into NFS logging to see what is being
read/written, and snooping network traffic might be a quick way into that.
Maybe in the new versions one of the apps started doing lots of mini writes
with fsync?

Wout.

On Thu, Oct 13, 2016, 11:57 PM zimbatm <zimbatm at zimbatm.com> wrote:

For the channels look into https://github.com/NixOS/nixpkgs-channels. The
branches here map directly to the channels that you can fetch publicly. If
you figure out which files are being used by nfs then it's a matter of
running a diff between the different branches.

https://github.com/NixOS/nixpkgs-channels/blob/nixos-16.09/nixos/modules/services/network-filesystems/nfsd.nix
doesn't seem to have changed since 16.03.
It seems to be using the nfs-utils which you can find in the
pkgs/top-level/all-packages.nix and follow from there. nfs-utils seems to
have changed:
https://github.com/NixOS/nixpkgs-channels/blob/nixos-16.09/pkgs/os-specific/linux/nfs-utils/default.nix
There is also the kernel module that you might want to dig into.

I know it's not a direct solution but hopefully it will give you some more
things to try out.

Cheers,
z



On Thu, 13 Oct 2016 at 15:18 4levels <4levels at gmail.com> wrote:

Hi Jonas,

disabling fail2ban didn't help, now the Nginx webserver keeps crashing as
soon as a php error occurs.
Do you know by any chance if with the upgrade to nixos-16.09 a newer
version of NFS is included than in 16.03?  Maybe some default parameters in
NFS have changed causing this havoc.
As a more general question: how can I see which version of package x is
used in nixos version y?

Kind regards,

Erik

On Thu, Oct 13, 2016 at 11:05 AM 4levels <4levels at gmail.com> wrote:

Hi,

thank you for your suggestions.  I've disabled the firewall and fail2ban
services, let's hope that does the trick!
I'll report back when things are better..

Kind regards,

Erik

On Thu, Oct 13, 2016 at 9:09 AM zimbatm <zimbatm at zimbatm.com> wrote:

Hi,

What happens if you disable fail2ban? Maybe the behaviour has changed.

Or try to change the kernel and NFS versions.

I know it's not much help, all I can recommend is to try and replace each
component to reduce the error.

On Wed, 12 Oct 2016, 10:51 4levels, <4levels at gmail.com> wrote:

Dear Nix'ers,

I've a permanent issue since the upgrade to nixos-16.09 in my local vm
(with kvm-qemu from an SSD partition).  The load of the vm is increasing
over time without any signs in the output of top.  Bash completion when
traversing directories stalls and the whole system becomes unresponsive
after about 5 to 10 minutes with top showing a load > 30.  Even rebooting
fails with several services failing to stop (eg. fail2ban, phpfpm, ).

This has everything to do with NFS: as soon as I disable the NFS mounts,
the system maintains normal operation.  Nginx / phpfpm are using NFS
mounted folders for local development.

These are the filesystem declarations in the nixops expression:

  fileSystems."/data/dev" = {
    device = "d01:/data/dev";
    fsType = "nfs";
    options = [ "defaults" "noatime" "nolock" "noacl" "vers=3" "udp"
"actimeo=1" ];
  };
  fileSystems."/extra/Documents" = {
    device = "d01:/extra/Documents";
    fsType = "nfs";
    options = [ "defaults" "noatime" "nolock" "noacl" "vers=3" "udp"
"actimeo=1" ];
  };

with d01 being declared in extrahosts

      networking.extraHosts = "192.168.121.1 d01 d01.local";

Has anyone an idea how this could be related to the upgrade to 16.09?  On
16.03 this all worked normally..

Kind regards,

Erik aka 4levels

_______________________________________________
nix-dev mailing list
nix-dev at lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

_______________________________________________
nix-dev mailing list
nix-dev at lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20161017/f3c8bda5/attachment-0001.html>