[Nix-dev] How to plug systemd-bootchart into kernel args?

[Mateusz Czaplinski]

Thanks!

Although IIUC, this doesn't seem to be the culprit (now I see that
apparently init= is what's run as the 2nd stage of init, with root fs
already mounted), your hint let me find the actual 1st stage init
script (https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1-init.sh),
and go on exploring from that.

With that, for now I managed to "hijack" the stage-2 init script from
my configuration.nix, and replace it with a custom wrapper, which let
me capture stderr from bootchart. The first error messages were very
easy to squash, the ability to see the stderr was invaluable. Still,
then I've got something weirder, where I see an error that "dmesg"
could not be found, I assume by shell. But grepping stage-2 script for
dmesg didn't seem to show anything. I suppose I'll have to try
investigating further.

Anyway, in case anybody is interested, the important parts of what I
have now look like this:

{ config, pkgs, ... }:
let
  systemd-bootchart = pkgs.lib.callPackageWith pkgs ./systemd-bootchart.nix {};
  boottest = pkgs.writeScript "boottest" ''
    #! ${pkgs.bash}/bin/bash
    set -x
    echo "hello in FAKE stage 2!"
    fsuffix="$(${pkgs.coreutils}/bin/date -Iminutes)"
    exec 3>&1 4>&2 1> >(${pkgs.coreutils}/bin/tee "/mcout.$fsuffix"
>&3) 2> >(${pkgs.coreutils}/bin/tee "/mcerr.$fsuffix" >&4)
    echo "hello n.2 in FAKE stage 2!"
    echo "schedstat?"
    ${pkgs.coreutils}/bin/cat /proc/config.gz | ${pkgs.gzip}/bin/gzip
-d | ${pkgs.gnugrep}/bin/grep SCHEDSTAT
    echo "please enter something..."
    read line
    ${systemd-bootchart}/lib/systemd/systemd-bootchart -i
/nix/store/b1b08c4n50pybhkmnh20x997sqxijzxm-nixos-system-nixos-16.09pre84674.fbde3a7/init
    echo "rebooting..."
    ${pkgs.systemd}/bin/reboot -f
  '';
in {
....
    boot.kernelParams = [
      "initcall_debug"
      "printk.time=y"
      "init=${boottest}"
    ];
    # kernel config option required by systemd-bootchart, see:
https://lists.ostroproject.org/pipermail/ostro-dev/2016-June/000537.html
    # NOTE: adding it triggers ~1h+ rebuild of kernel!
    nixpkgs.config = {
      packageOverrides = pkgs: {
        stdenv = pkgs.stdenv // {
          platform = pkgs.stdenv.platform // {
            kernelExtraConfig = "SCHEDSTATS y";
          };
        };
      };
    };
....
}

/Mateusz.

On Mon, Jul 11, 2016 at 2:35 AM, Tomasz Czyż <tomasz.czyz at gmail.com> wrote:
> Just a blind guess. Make sure that this binary is in initrd (inspect initrd
> image or boot with debug flags to jump into initrd shell).
>
> If you don't have the binary there you could add something like here:
> https://github.com/NixOS/nixos/issues/208#issuecomment-22529169
>
>  extraUtilsCommands =
>          ''
>            cp -v ${kernel.zfs}/sbin/zfs $out/sbin
>          ''
>
> to copy files you need.
>
> 2016-07-11 1:31 GMT+01:00 Mateusz Czaplinski <czapkofan at gmail.com>:
>>
>> I've read that for detailed boot time instrumentation, one is advised
>> to use 'init=/usr/lib/systemd/systemd-bootchart' kernel param.
>>
>> I've already:
>> - found out that with systemd-230, systemd-bootchart was moved out to
>> a separate project (github.com/systemd/systemd-bootchart);
>> - built a derivation for it and made it to compile successfully;
>> (running systemd-bootchart --help prints help info);
>> - tried to pass it into /etc/nixos/configuration.nix as:
>>
>>   kernelParams = [
>>     "init=${systemd-bootchart}/lib/systemd/systemd-bootchart"
>>   ];
>>
>> - but I'm getting a kernel panic on boot.
>>
>> What should I do? Also, I don't know how to retrieve the exact
>> contents of the panic message from the "previous boot" (i.e. the
>> panicked one), after I hard-reboot and pick a last-working
>> configuration in GRUB menu. Dmesg seems to show only the successful
>> one, journalctl doesn't seem to be aware of the panic either with -b
>> -1.
>>
>> I've also noticed, that /boot/grub/grub.cfg actually contains two
>> init= arguments, as the new one is only appended (the old one is still
>> kept); can this be the reason? If yes, is there some reasonable hack I
>> could try to fix that?
>>
>> Also, I've read that I should maybe pass -i option to bootchart, to
>> inform it what init script it should pass control to. But I don't know
>> how to retrieve the path dynamically in Nix expression? it looks to me
>> it's available only temporarily as a let-expression, which is used in
>> some advanced expression and thus doesn't escape into "global scope".
>> Still, when I tried to hardcode it as:
>> "init=${systemd-bootchart}/.../systemd-bootchart -i
>> /nix/store/.../init", it panicked anyway (that said, in grub.cfg, it
>> seems the spaces are not escaped around -i here; is that ok?), so I
>> assume I haven't even reached this phase yet.
>>
>> I'd be grateful for any help!
>>
>> Thanks & Best Regards,
>> /Mateusz Czapliński.
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>
>
>
> --
> Tomasz Czyż