[Nix-dev] Dynamic substituteInPlace

[stewart mackenzie]

hey zimbatm,

On Tue, Feb 16, 2016 at 6:04 AM, zimbatm <zimbatm at zimbatm.com> wrote:
> It's not super clear to me what you are trying to achieve. Maybe you've
> zoomed into that specific solution but with more context there might be
> others available.

I suspect this might be the case.

> Is the file compiled or is it interpreted ?

rust src files are compiled, subnets are used to link (not a GNU ld
linker type link, but as in graph theory, a node links to another node
via an edge) rust shared objects at runtime, these links carry
messages, where messages are structured using capnproto contracts. We
have features to adjust the execution graph at runtime, but we're
currently not making use of them.

>
> I understand that you want to provide some information at runtime and
> resolve the path to a derivation. How is that information provided to the
> runtime

subnets are programmed by humans at development time, the programmer
knows the names of derivations and ask for them as a surgeon asks a
nurse for an instrument by name.
Nix then rewrites those names to the full path just before compile time.
Nix then makes a new derivation for the compiled executable and links
in all the dependencies (via hierarchies of subnets).
when you run the executable all the subnets have already had their
derivation names substituted and are ready to be processed by the fvm
parser. Each component (a subnet and *.so is a component) has their
own derivation in the /nix/store/

> and is the name of the derivation in a limited set ?

the name of the derivation is in a limited set
(https://github.com/sjmackenzie/fractalide/blob/fractalide/components/default.nix),
if the name is not found I want the compilation to abort.
I extract the derivations names from a subnet here:
https://github.com/sjmackenzie/fractalide/blob/fractalide/build-support/buildFractalideSubnet.nix#L11

Then check [1] to see if the component is a *.so or subnet here:
https://github.com/sjmackenzie/fractalide/blob/fractalide/build-support/buildFractalideSubnet.nix#L19

then I want to substituteInPlace the name for the full path to the
/nix/store/<hash>/lib/lib.subnet or
/nix/store/<hash>/lib/libcomponent.so
The parser will then read the subnet file and the allocator will load
that shared object into memory, or if it's another subnet then it'll
load that and add some more nodes and edges to the message passing
execution graph.

we can have deep hierarchies of subnets, but they all end in a rust
components (ie libcomponent.so), it's at this point, nix lazily stop
evaluating that subnet branch.

> Maybe all you need is patchelf if you're compiling a program dynamically but
> want to update it later to point to the .so object.
>
>
This point opens unthought out pathways, in the near future we want a
hypercard type program (built using fractalide components), I was
thinking how to introduce new components into an environment
dynamically.
It would be interesting to patchElf a running hypercard program
dynamically and introduce these new .so objects... I don't know yet.
:-)


[1] It's at this point I get this error:

[stewart at rivergod:~/dev/fractalide/fractalide]$ nix-build -A
components.maths_boolean_and fvm.nix
error: string ‘/nix/store/y206xxi12nf961m8m2svhbf5x3w34nqh-maths_boolean_nand/lib/lib.subnet’
cannot refer to other paths, at
/home/stewart/dev/fractalide/fractalide/build-support/buildFractalideSubnet.nix:19:14
(use ‘--show-trace’ to show detailed location information)

maths_boolean_and is a subnet:
https://github.com/sjmackenzie/fractalide/blob/fractalide/components/maths/boolean/and/lib.subnet
which references maths_boolean_nand (a shared library:
https://github.com/sjmackenzie/fractalide/blob/fractalide/components/maths/boolean/nand/src/lib.rs)
and maths_boolean_not (another subnet:
https://github.com/sjmackenzie/fractalide/blob/fractalide/components/maths/boolean/not/lib.subnet)

Eventually the all of these testing/example maths_boolean_* will point
to maths_boolean_nand (except, obviously, maths_boolean_print)

---------------

Alternatively we could do away with humans developing in lib.subnets
and get them to write the subnets *in* a nix file, then we can quite
easily make use of things like this:

" 'text' -> input_port var_name_instance(${(builtins.tryEval
components."a_dynamic_component_name").value}) output_port "

I could pull that out into a function

" 'text' -> input_port var_name_instance(${find
"a_dynamic_component_name"}) output_port "

but now it's becoming super annoying and the programmer cannot easily
and quickly write subnets, also there is no subnet syntax highlighting
in nix files.