[Nix-dev] Two approaches to kmod migration

[Shea Levy]

On 04/04/2012 02:42 PM, Yury G. Kudryashov wrote:
> Shea Levy wrote:
>
> First of all, let me stress the main difference between module-init-tools
> and kmod. The former is a collection of command line utilities, while the
> latter is a *library* plus a collection of command line utilities provided
> for backwards compatibility.
>
> This difference was the main reason for me to try another approach.
>
> BTW, I moved kmod-MODULE_DIR to kmod-lib-modules. You're welcome to create
> another branch with your kmod migration code.
>
>> On 03/27/2012 01:07 PM, Yury G. Kudryashov wrote:
>>> Hi!
>>>
>>> * Easier to catch buggy kernel module buildsystem.
>>> | Just use chroot builds. If you're not root, read the makefile(s)
>>> | carefully.
>> By this argument, we should just fully follow FHS.
> No. I didn't say this.
>
> On the other hand, I see nothing bad in following FHS for *system-wide* one-
> instance services. We already have /etc, /bin/sh and /usr/bin/env, why
> /lib/modules is worse than /etc? Non-chroot builds can look into /etc, and
> make strange decisions at configure time.

Sure, and I don't like those either. But one important difference is 
that these impurities are important for non-automated end-user uses. An 
end user needs some writeable place to modify system configuration (even 
if the entire system were configured in configuration.nix, it would need 
to be located _somewhere_), an end user needs to be able to write 
scripts with shebangs. Why does an end-user ever need to care where the 
modules are kept?

>> But bugs DO happen,
> See below (after your item 3).
>>> | We'll need the same patches to buildsystem in both cases. The patches
>>> | can be accepted upstream and/or shared with mainstream distros.
>> Not necessarily. If we use MODULE_DIR, yes, but if we switch to the new
>> command line flags that kmod's modprobe supports we won't need to patch
>> kmod. We'll need to patch udev if it doesn't supply a similar command
>> line flag, of course. The kernel build system can remain as it is now.
> See above (about library vs utilities).
>>> /lib/modules:
>>> =============
>>> * No need to patch:
>>>     - Kernel buildsystem (removing '-b' key);
>>> | I'm pretty sure that the patch will never be accepted upstream
>>>     - kmod;
>> We won't need to patch kmod.
> OK, we'll have to patch either kmod, or every package that links to
> libkmod.so and every package that calls `modprobe`.

OK, but if we need to patch kmod the patch is extremely simple. Just 
modify kmod_new so that when dirname is NULL use MODULE_DIR instead of 
/lib/modules. Problem solved for all utilities that use kmod, no need 
for wrapper scripts or anything.

>>> | Same here
>>>     - udev;
>>> | Same here, if patch needed (see below)
>>>     - more packages to come later.
>> If you're allowed to say 'more packages to come later', then I'm allowed
>> to say 'more kernelPackages that will hard-code /lib/modules'
> The kernelPackages that hard-code /lib/modules cause no problems if you use
> chroot builds. The build will fail, and you'll have to patch the
> buildsystem.

Not everyone can use chroot builds. Not everyone wants to use chroot 
builds. Chroot builds can be modified (e.g. I could choose to include 
/lib in my chroot). Unless Eelco says otherwise, I think it's a really 
bad idea to say that ANY package in nixpkgs or any nixos evaluation 
requires chroot dirs. There are many specialized features that we 
purposely keep out of nixpkgs (e.g. dynamic import, use of <> path 
resolution, etc.) to keep it working on the widest variety of systems, 
and I don't think a minor patch to a single library is enough reason to 
break compatibility for some systems.

>
>>> | I'm not sure if we need to patch every package, but we definitely need
>>> | to ensure that every package using libkmod has MODULE_DIR env var set
>>> | to the desired value.
>>>     - every shell script that has $(find /lib/modules/`uname -r`)
>> Yes, but patching these is a GOOD THING since most of them occur in
>> kernelPackage build scripts.
> I'm not talking about build scripts, I'm talking about runtime. Sorry that I
> haven't stated this explicitly.

Can you point me to a script that uses $(find /lib/modules/`uname -r`) 
that's actually doing the right thing?

>
> We'll have to patch build scripts in both cases.
>>> | I have no well-known example of such script, but kmod has no
>>> | `modprobe -l` (this switch was deprecated in module-init-tools).
>>>
>>> * Easy to switch all running processes to a new module dir.
>>> | Rarely needed. It it's hard (impossible?) to reload many modules on the
>>> | fly, and it's rather easy to manually `insmod` a few modules.
>>
>> Some further thoughts:
>>       1. Purity as a general principle has proven value. I believe the
>> burden of proof of value is on those who wish to add an impurity.
> I think that having /lib/modules/`uname -r` invisible to build scripts adds
> no impurity. The result of each chroot build will depend only on its
> buildInputs, and the runtime behaviour will depend only on its /nix/store
> dependencies and system-wide settings like /lib/modules/`uname -r` symlink,
> /etc, e.t.c..

Again, I don't accept this dependency on chroot builds. This is 
absolutely an impurity, otherwise why would you want it? MODULE_DIR 
provides a way to explicitly pass the location of the modules to 
programs that need it, and using /lib/modules lets those programs make 
assumptions about system state that aren't explicitly passed to them. If 
/lib/modules truly resulted in no impurities then it would have no 
advantages to MODULE_DIR.

>
>>       2. The MODULE_DIR approach has precedent and has not, to my
>> knowledge, caused any significant problems
> As I've stated above, libkmod is a shared library. This means that the
> number of binaries that will need to know $MODULE_DIR will grow as more
> binaries will link to libkmod.so.

Unless we just modify kmod, then all such binaries will work correctly 
unless they stupidly call kmod_new with "/lib/modules" instead of NULL.

>
>>       3. Problems in the MODULE_DIR approach generally come in the form
>> of early failures: Build failure, or a boot fails completely on the
>> first try. Failures due to a /lib/modules could hide a long time before
>> surfacing, such is the nature of impure state
> Could you please provide an example of a package such that:
> * it leads to an early failure with MODULE_DIR approach;
> * it doesn't lead to an early failure with /lib/modules approach with chroot
> builds?

I don't accept the second half of your second constraint. I've already 
pointed to a package that would fail without a patch without chroot builds.

>
> I agree to add "doesn't require chroot builds for early buildsystem errors
> detection" to MODULE_DIR pro's.
>
> BTW, I can provide (currently, theoretical) example of a situation when
> MODULE_DIR approach leads to a hidden failure. Suppose that `foo` executable
> links to libkmod.so, and we've forgotten to wrap `foo` to set MODULE_DIR in
> the wrapper. If `foo` is not critical for boot, then nobody will notice the
> failure. Let me repeat that this situation was not possible with module-
> init-tools, but it is possible with libkmod.

True, so the solution here is to patch libkmod and be done with it.

>>       4. The 'burden' of patching build systems is inherent in a system
>> as radical as nix, and is not really that high. We patch shebangs, gcc,
>> ld.so, etc. etc... Why is it suddenly a problem for udev, the kernel,
>> etc.?
> Because we can proceed without these extra patches.

But at what cost? Those patches carry very small development cost and 
almost nil compile-time cost, and for the end user the cost is nil.