[Nix-dev] is there something like unsafeImpureEnvVars?

[Ben Franksen]

Hi Shea

thanks for taking the time to explain this to me. I see now that what I 
wanted is not possible in a clean way.

I think what confused me was that Nix does not enforce purity of builds by 
default. Out of the box, i.e. without chroot builds, getting sources for 
instance via darcs or plain file path seems to "just work" so I wondered why 
not via ssh. I got the false impression that it is okay to develop Nix 
packages in this setting, while in fact the default is probably only okay 
for installing (existing) packages that have already been purity tested in a 
chroot environment.

I will now read up on how to enable chroot builds in multi-user configured 
Nix...

Cheers
Ben

Shea Levy wrote:
> On Mon, Apr 14, 2014 at 04:12:00AM +0200, Ben Franksen wrote:
>> Shea Levy wrote:
>> > Fetching source over the network is the main reason fixed output
>> > derivations even exist. When chroot builds are enabled, networking is
>> > not allowed for non-fixed output derivations.
>> 
>> Interesting, I did not know this.
>> 
>> I agree that this policy makes sense for stuff that gets downloaded from
>> the internet, and especially if you base a complete linux distribution on
>> it (security considerations: you want to make sure that the source has
>> not been tampered with).
>> 
> 
> It's not just security considerations, it's also that the invariants nix
> maintains require that a derivation contains *all* of the information
> about the output, and if you have general access to global resources
> (even the filesystem, let alone the network) the only way nix can
> guarantee that is if the derivation specifies exactly what the output
> will look like (thus, the hash in fixed derivation outputs).
> 
>> 
>> > Why is this case special?
>> 
>> The repo server is in-house, and we talk about ssh access, limited to the
>> trusted maintainers of the code base. The level of security here is
>> arguably better than using an NFS share (which would appear as a local
>> path to Nix and thus allow unrestricted (non-fixed output) derivations
>> even with chroot builds if I understood you correctly... or maybe not,
>> depending on whether the share gets bind-mounted into the builder's
>> environment or not).
>> 
> 
> When adding files to the store from the local filesystem, they are
> hashed as they are added and thus nix's invariants are maintained. The
> build should not (and in chroot mode does not) have access to any paths
> outside of the nix store, so if you're keeping sources on the local FS
> they are copied to the store (with a path that corresponds to their
> contents) first.
> 
>> 
>> We have many modules in many versions. I was hoping I could avoid to list
>> each and every one of them in our "local" nix expression tree, and
>> instead (mostly) rely on functions that get the version number as a
>> parameter.
>> 
> 
> I think the best you can do here without hacks (and not just hacks of
> implementation but hacks around nix's core model) is to have your
> functions take the version and the hash.
> 
>> 
>> But I must admit: I am half convinced that relying on module name and
>> version alone to correctly identify a source may not be strong enough to
>> actually *guarantee* consistent and repeatable builds, even in our
>> limited setting.
>> 
> 
> Yeah, content hashing is a much sounder general basis for repeatable
> builds IMO.
> 
> ~Shea
> 
>> 
>> Cheers
>> Ben
>> 
>> >> On Apr 13, 2014, at 16:03, Ben Franksen <ben.franksen at online.de>
>> >> wrote:
>> >> 
>> >> Hello
>> >> 
>> >> Is there *any* way (besides hacking the Nix source code) to circumvent
>> >> the limitation on impureEnvVars, i.e. that one has to provide the hash
>> >> up front, making the derivation fixed-output? What I am asking for is
>> >> something akin to unsafePerformIO in Haskell (which exists for similar
>> >> situations, and motivated the name unsafeImpureEnvVars in the
>> >> subject).
>> >> 
>> >> Effectively I want to be able to say to Nix: "I swear these variables
>> >> are used in a pure way, just believe me."
>> >> 
>> >> Background:
>> >> 
>> >> I want to use fetchdarcs for remote repositories via ssh; we use darcs
>> >> to version control our software internally (even stuff we get from the
>> >> outside, to track patches for customisation and bug fixes). We use
>> >> tags to identify versions and have lots of darcs repos, all on a
>> >> single server, accessible to developers via ssh.
>> >> 
>> >> Unfortunately, fetchdarcs fails in this case because, to avoid being
>> >> asked for a passphrase, ssh (in this case called via darcs) needs to
>> >> know how to access a runnig ssh agent, and this knowledge is encoded
>> >> in two environment variables: SSH_AGENT_PID and SSH_AUTH_SOCK. But of
>> >> course these are not defined in the builder, its environment gets
>> >> cleared before it starts.
>> >> 
>> >> Now, there is the attribute impureEnvVars. But that works only if you
>> >> pass the expected hash to the builder (so the derivation becomes
>> >> fixed-output). That means either I hard-code the expected hash into
>> >> the derivation, or pass it as an argument. In both cases, I need to
>> >> find out the hash before-hand. This makes it very inconvenient for
>> >> users to add new version of packages.
>> >> 
>> >> Note that this is *only* because of the two environment variables. I
>> >> can use fetchdarcs just fine with any local path or remote http url
>> >> without providing any hash.
>> >> 
>> >> Note also that use of these variables does not imply any impurity,
>> >> they are only used for authentication.
>> >> 
>> >> Cheers
>> >> Ben
>> >> --
>> >> "Make it so they have to reboot after every typo." -- Scott Adams
>> >> 
>> >> _______________________________________________
>> >> nix-dev mailing list
>> >> nix-dev at lists.science.uu.nl
>> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>> --
>> "Make it so they have to reboot after every typo." -- Scott Adams
>> 
>> 
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
-- 
"Make it so they have to reboot after every typo." -- Scott Adams