[Nix-dev] is there something like unsafeImpureEnvVars?

Tue Apr 15 15:31:04 CEST 2014

Hey Ben,

On Mon, Apr 14, 2014 at 09:05:25PM +0200, Ben Franksen wrote:
> Hi Shea
> 
> thanks for taking the time to explain this to me. I see now that what I 
> wanted is not possible in a clean way.
> 

No problem, sorry if anything wasn't clear!

> 
> I think what confused me was that Nix does not enforce purity of builds by 
> default. Out of the box, i.e. without chroot builds, getting sources for 
> instance via darcs or plain file path seems to "just work" so I wondered why 
> not via ssh. I got the false impression that it is okay to develop Nix 
> packages in this setting, while in fact the default is probably only okay 
> for installing (existing) packages that have already been purity tested in a 
> chroot environment.
> 

Yeah, unfortunately chroot builds require a bit of extra OS support as
well as administrative setup, so we can't have it on by default. I
personally recommend it if you can get it set up though.

> 
> I will now read up on how to enable chroot builds in multi-user configured 
> Nix...
> 
> Cheers
> Ben
> 
> Shea Levy wrote:
> > On Mon, Apr 14, 2014 at 04:12:00AM +0200, Ben Franksen wrote:
> >> Shea Levy wrote:
> >> > Fetching source over the network is the main reason fixed output
> >> > derivations even exist. When chroot builds are enabled, networking is
> >> > not allowed for non-fixed output derivations.
> >> 
> >> Interesting, I did not know this.
> >> 
> >> I agree that this policy makes sense for stuff that gets downloaded from
> >> the internet, and especially if you base a complete linux distribution on
> >> it (security considerations: you want to make sure that the source has
> >> not been tampered with).
> >> 
> > 
> > It's not just security considerations, it's also that the invariants nix
> > maintains require that a derivation contains *all* of the information
> > about the output, and if you have general access to global resources
> > (even the filesystem, let alone the network) the only way nix can
> > guarantee that is if the derivation specifies exactly what the output
> > will look like (thus, the hash in fixed derivation outputs).
> > 
> >> 
> >> > Why is this case special?
> >> 
> >> The repo server is in-house, and we talk about ssh access, limited to the
> >> trusted maintainers of the code base. The level of security here is
> >> arguably better than using an NFS share (which would appear as a local
> >> path to Nix and thus allow unrestricted (non-fixed output) derivations
> >> even with chroot builds if I understood you correctly... or maybe not,
> >> depending on whether the share gets bind-mounted into the builder's
> >> environment or not).
> >> 
> > 
> > When adding files to the store from the local filesystem, they are
> > hashed as they are added and thus nix's invariants are maintained. The
> > build should not (and in chroot mode does not) have access to any paths
> > outside of the nix store, so if you're keeping sources on the local FS
> > they are copied to the store (with a path that corresponds to their
> > contents) first.
> > 
> >> 
> >> We have many modules in many versions. I was hoping I could avoid to list
> >> each and every one of them in our "local" nix expression tree, and
> >> instead (mostly) rely on functions that get the version number as a
> >> parameter.
> >> 
> > 
> > I think the best you can do here without hacks (and not just hacks of
> > implementation but hacks around nix's core model) is to have your
> > functions take the version and the hash.
> > 
> >> 
> >> But I must admit: I am half convinced that relying on module name and
> >> version alone to correctly identify a source may not be strong enough to
> >> actually *guarantee* consistent and repeatable builds, even in our
> >> limited setting.
> >> 
> > 
> > Yeah, content hashing is a much sounder general basis for repeatable
> > builds IMO.
> > 
> > ~Shea
> > 
> >> 
> >> Cheers
> >> Ben
> >> 
> >> >> On Apr 13, 2014, at 16:03, Ben Franksen <ben.franksen at online.de>
> >> >> wrote:
> >> >> 
> >> >> Hello
> >> >> 
> >> >> Is there *any* way (besides hacking the Nix source code) to circumvent
> >> >> the limitation on impureEnvVars, i.e. that one has to provide the hash
> >> >> up front, making the derivation fixed-output? What I am asking for is
> >> >> something akin to unsafePerformIO in Haskell (which exists for similar
> >> >> situations, and motivated the name unsafeImpureEnvVars in the
> >> >> subject).
> >> >> 
> >> >> Effectively I want to be able to say to Nix: "I swear these variables
> >> >> are used in a pure way, just believe me."
> >> >> 
> >> >> Background:
> >> >> 
> >> >> I want to use fetchdarcs for remote repositories via ssh; we use darcs
> >> >> to version control our software internally (even stuff we get from the
> >> >> outside, to track patches for customisation and bug fixes). We use
> >> >> tags to identify versions and have lots of darcs repos, all on a
> >> >> single server, accessible to developers via ssh.
> >> >> 
> >> >> Unfortunately, fetchdarcs fails in this case because, to avoid being
> >> >> asked for a passphrase, ssh (in this case called via darcs) needs to
> >> >> know how to access a runnig ssh agent, and this knowledge is encoded
> >> >> in two environment variables: SSH_AGENT_PID and SSH_AUTH_SOCK. But of
> >> >> course these are not defined in the builder, its environment gets
> >> >> cleared before it starts.
> >> >> 
> >> >> Now, there is the attribute impureEnvVars. But that works only if you
> >> >> pass the expected hash to the builder (so the derivation becomes
> >> >> fixed-output). That means either I hard-code the expected hash into
> >> >> the derivation, or pass it as an argument. In both cases, I need to
> >> >> find out the hash before-hand. This makes it very inconvenient for
> >> >> users to add new version of packages.
> >> >> 
> >> >> Note that this is *only* because of the two environment variables. I
> >> >> can use fetchdarcs just fine with any local path or remote http url
> >> >> without providing any hash.
> >> >> 
> >> >> Note also that use of these variables does not imply any impurity,
> >> >> they are only used for authentication.
> >> >> 
> >> >> Cheers
> >> >> Ben
> >> >> --
> >> >> "Make it so they have to reboot after every typo." -- Scott Adams
> >> >> 
> >> >> _______________________________________________
> >> >> nix-dev mailing list
> >> >> nix-dev at lists.science.uu.nl
> >> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
> >> --
> >> "Make it so they have to reboot after every typo." -- Scott Adams
> >> 
> >> 
> >> _______________________________________________
> >> nix-dev mailing list
> >> nix-dev at lists.science.uu.nl
> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
> -- 
> "Make it so they have to reboot after every typo." -- Scott Adams
> 
> 
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev