[Nix-dev] Re: Performance impact of Nix

[Tony White]

2009/8/9 Ludovic Courtès <ludo at gnu.org>:
> Hello!
>
> ludo at gnu.org (Ludovic Courtès) writes:
>
>> In a recent discussion, Ralf Wildenhues [0] rightfully wondered about
>> the performance impact of Nix' file system layout and/or that of the
>> symlink indirection in profiles.
>
> I did a quick experiment with the attached Guile script (run as "sudo
> guile test-symlink.scm"), which shows these timings for stat(2) of a
> regular file vs. stat(2) of a symlink pointing to that file:
>
>  * regular file
>  clock utime stime cutime cstime gctime
>   1.14  0.74  0.38   0.00   0.00   0.21
>  * symlink
>  clock utime stime cutime cstime gctime
>   1.46  0.86  0.59   0.00   0.00   0.30
>
> Here it's 28% slower to stat(2) a symlink that to stat a plain regular
> file.
>
> I'm not sure how much can be extrapolated from this result, but it's
> probably safe to say that symlinks do add noticeable overhead.
>
> As for possible solutions, I don't have any idea.  Hard links won't work
> across file systems, and it's not obvious that some unionfs would
> perform better.  (GNU/Hurd has "firm links", which would be handy
> here...)
>
> Thanks,
> Ludo'.
>
>
> (use-modules (ice-9 time))
>
> (define (drop-caches)
>  (catch 'system-error
>    (lambda ()
>      (with-output-to-file "/proc/sys/vm/drop_caches"
>        (lambda ()
>          (format #t "3~%"))))
>    (lambda (key proc fmt . args)
>      (format (current-error-port) "can't drop caches (need to be root?) [~A]~%"
>              (car args)))))
>
> (define %regular-file ",,test")
> (define %symlink ",,link")
>
> (define (remove-files)
>  (false-if-exception (delete-file %regular-file))
>  (false-if-exception (delete-file %symlink)))
>
> (remove-files)
> (with-output-to-file %regular-file newline)
> (symlink %regular-file %symlink)
>
> (define (how-long? str thunk)
>  (format #t "* ~A~%" str)
>  (drop-caches)
>  (time (let loop ((i 300000))
>          (and (> i 0)
>               (begin
>                 (thunk)
>                 (loop (1- i)))))))
>
> (how-long? "regular file" (lambda () (stat %regular-file)))
> (how-long? "symlink"      (lambda () (stat %symlink)))
>
> (remove-files)
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at cs.uu.nl
> https://mail.cs.uu.nl/mailman/listinfo/nix-dev
>
>
Hi,
I know patchelf is not part of this discussion but does it not speed
up the finding of libraries like prelink does :
http://en.wikipedia.org/wiki/Prelink
so therefore if there is a definate time lag using symlinks (Soft)
won't the fact that patchelf makes things load quicker (I don't mean
symlinks) Cancel out some or lots of the time lag in comparison to
systems without prelink. So there isn't exactly a noticeable speed
difference using nix even though it uses symlinks.
I don't think that prelink is used by default in many Linux distributions.

Of course none of that really solves the symlink speed, it just kind
of means it's not problem, more of a yeah; lets make it go faster!
If there was a way to make symlinks or something (Else?) Better
perform as fast as regular files; NixOs would be blindingly fast?
It goes way beyond my knowledge but the only solution that springs to
my mind is a nixfs or nixosfs which deals with links differently and
speeds up the use of symlinks (Or whatever method is used.) It would
involve developing a kernel module and I imagine that would be quite
difficult. Not impossible, however.

Thanks,
Tony