[Nix-dev] string vs path hell

[Sergey Mironov]

Wow, Nix strings are not that simple at all. I'll now go and make my
own copy of splitString with '==' replaced by 'lib.eqSttring' as you
have suggested.

Thanks for the explanation!
Sergey

2014-06-10 13:27 GMT+04:00 Kirill Elagin <kirelagin at gmail.com>:
> So, here is what I found.
>
> Strings in nix have contexts:
> https://github.com/NixOS/nix/blob/master/src/libexpr/value.hh#L45.
> When you compare strings with  `==`, those contexts also get compared.
>
> So, `lib.splitString`, obviously, uses `==` internally to compare substrings
> with the separator, and since `"/"` doesn't have any context the comparison
> will never yield true.
>
> I'm not really sure what is the proper workaround here. I guess, in your
> case this might be the right thing to do:
>
> ~~~
> nix-repl> lib.splitString "/" a
> [ "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar" ]
>
> nix-repl> lib.splitString "/" (builtins.unsafeDiscardStringContext a)
>
> [ "" "nix" "store" "nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1" "foo"
> "bar" ]
> ~~~
>
> Another option is changing `splitString` to use `lib.eqString` which
> basically concatenates contexts before doing `==`. But I'm not sure, that
> might break something.
>
> So, as another workaround, you can concatenate the contexts yourself:
>
> ~~~
> nix-repl> lib.splitString "/" a
> [ "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar" ]
>
> nix-repl> lib.splitString (lib.substring 0 0 a + "/") a
>
> [ "" "nix" "store" "nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1" "foo"
> "bar" ]
> ~~~
>
>
> --
> Кирилл Елагин
>
>
> On Tue, Jun 10, 2014 at 1:11 PM, Kirill Elagin <kirelagin at gmail.com> wrote:
>>
>> Going further:
>>
>> ~~~~~~~~~~~
>> nix-repl> a = "${pkgs.busybox}/foo/bar"
>>
>> nix-repl> b =
>> "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar"
>>
>> nix-repl> a
>> "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar"
>>
>> nix-repl> b
>> "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar"
>>
>> nix-repl> lib.substring 0 1 a
>> "/"
>>
>> nix-repl> lib.substring 0 1 b
>> "/"
>>
>> nix-repl> lib.substring 0 1 a == "/"
>> false
>>
>> nix-repl> lib.substring 0 1 b == "/"
>> true
>> ~~~~~~~~~~~~~
>>
>>
>> --
>> Кирилл Елагин
>>
>>
>> On Tue, Jun 10, 2014 at 1:04 PM, Kirill Elagin <kirelagin at gmail.com>
>> wrote:
>>>
>>> Oh yeah, I've managed to reproduce this:
>>>
>>> ~~~~~
>>> nix-repl> :l <nixpkgs>
>>> Added 3863 variables.
>>>
>>> nix-repl> lib.splitString "/" "${pkgs.busybox}/foo/bar"
>>> [ "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar" ]
>>>
>>> nix-repl> lib.splitString "/" (toString "${pkgs.busybox}/foo/bar")
>>> [ "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar" ]
>>>
>>> nix-repl> lib.splitString "/"
>>> "/nix/store/nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1/foo/bar"
>>> [ "" "nix" "store" "nn00kn1fa0154qa1szszijw825cha8ps-busybox-1.22.1"
>>> "foo" "bar" ]
>>> ~~~~~
>>>
>>> Total weirdness.
>>>
>>>
>>> --
>>> Кирилл Елагин
>>>
>>>
>>> On Tue, Jun 10, 2014 at 12:58 PM, Kirill Elagin <kirelagin at gmail.com>
>>> wrote:
>>>>
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>> let lib = (import <nixpkgs> {}).lib;
>>>>     exe = {
>>>>       foo = "/some/path/to/foo";
>>>>       bar = "/other/path/to/bar";
>>>>     };
>>>> in
>>>> {
>>>>   contents =
>>>>
>>>>     map (v :
>>>>       let
>>>>         takeFileName = p :
>>>>           let fl = lib.splitString "/" p; in lib.last fl;
>>>>         fn = takeFileName "${v}";
>>>>         dummy = takeFileName "/a/string/encoded/path/dummy";
>>>>       in
>>>>
>>>>         #builtins.trace fn
>>>>         #builtins.trace dummy
>>>>
>>>>
>>>>         {
>>>>           symlink = "/bin/${fn}";
>>>>           object = v;
>>>>         }) (lib.attrValues exe);
>>>> }
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>> ~~~~~~~~~~~~~~~~
>>>> nix-repl> :l test.nix
>>>> Added 1 variables.
>>>>
>>>> nix-repl> :p contents
>>>> [ { object = "/other/path/to/bar"; symlink = "/bin/bar"; } { object =
>>>> "/some/path/to/foo"; symlink = "/bin/foo"; } ]
>>>> ~~~~~~~~~~~~~~~~~
>>>>
>>>> Dunno, you should probably inspect your `exe` set closer in case
>>>> something's wrong with values there.
>>>>
>>>>
>>>> --
>>>> Кирилл Елагин
>>>>
>>>>
>>>> On Tue, Jun 10, 2014 at 10:41 AM, Sergey Mironov <grrwlf at gmail.com>
>>>> wrote:
>>>>>
>>>>> OK, you are right. My code snippet really works fine. So I feel I must
>>>>> post a part of my real code which still doesn't work for me. It is a
>>>>> bit long, but maybe you will be able to help me find a mistake. Here
>>>>> it is (below). This expression takes a set of paths to the executables
>>>>> (exe // setuids) and builds the initrd image. Every attribute of (exe
>>>>> // setuids) is either a string like [adduser =
>>>>> "${busybox}/sbin/adduser";] or the result of mkDerivation producing
>>>>> the single binary in it's $out.
>>>>>
>>>>> Note the "dummy" expression. If I uncomment "builtins.trace dummy"
>>>>> line, it will show me the expected result (string "dummy" for every
>>>>> list item). But in the same time, "builtins.trace fn" shows that fn is
>>>>> evaluated incorrectly. Every time it contains full Nix path
>>>>> ("/nix/strote/...", see example at the end of the letter) instead of
>>>>> the filename part. It looks like 'splitString' doesn't recognize it's
>>>>> argument as a string.
>>>>>
>>>>> Please, comment!
>>>>> Sergey
>>>>>
>>>>>
>>>>> --
>>>>> the code
>>>>>
>>>>>       ...
>>>>>
>>>>>       img = makeInitrd {
>>>>>
>>>>>         compressor = "${gzip}/bin/gzip --fast";
>>>>>
>>>>>         contents = with all.ipkgs;
>>>>>           [{ symlink = "/init";
>>>>>             object = exe.init;
>>>>>           }
>>>>>           { symlink = "/bin/sh";
>>>>>             object = exe.shell;
>>>>>           }] ++
>>>>>           map (v :
>>>>>
>>>>>             let
>>>>>
>>>>>               takeFileName = p : let
>>>>>                   fl = lib.splitString "/" p;
>>>>>                 in lib.last fl;
>>>>>
>>>>>               fn = takeFileName "${v}";
>>>>>
>>>>>               dummy = takeFileName "/a/string/encoded/path/dummy";
>>>>>
>>>>>             in
>>>>>
>>>>>             # builtins.trace fn   # <---------------- see example
>>>>> output below
>>>>>             # builtins.trace dummy
>>>>>
>>>>>             {
>>>>>               symlink = "/bin/${fn}";
>>>>>               object = v;
>>>>>             }) (lib.attrValues (exe // setuids));
>>>>>       };
>>>>>
>>>>> --
>>>>> example output of "builtins.trace fn":
>>>>>
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/sbin/addgroup
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/sbin/adduser
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/false
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/true
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/awk
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/cat
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/chmod
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/chown
>>>>> trace:
>>>>> /nix/store/sx19dvnv3wi2j8pvyra8npl9vdwdbmrk-busybox-1.21.1/bin/cp
>>>>> trace:
>>>>> /nix/store/h8rs91fjivsh6wakmwmpkca31wfp38g0-msr-tools-1.3/bin/cpuid
>>>>
>>>>
>>>
>>
>