[Nix-dev] package test set up and tear down

[Manuel Pages]

1. Application should be tested in a clean VM;
2. The tests in questions should then be refactored.
When it comes to building the package, I second James Cook's opinion —
tests have a bug (making assumptions about the global state), so those
should be disabled till (2) happens.
Or, shorter,
In your build expression doCheck = false;
in your CI expression, spin off a VM.

On Wed, Apr 15, 2015 at 4:20 AM, James Cook <james.cook at utoronto.ca> wrote:

> On 14 April 2015 at 05:54, Tomas Hlavaty
> <tomas.hlavaty at knowledgetools.de> wrote:
> > Hi all,
> >
> > I have a nix package expression with "doCheck = true;".  The tests
> > require a RDBMS (postgresql for now).  preCheck creates postgresql roles
> > and database and postCheck drops the roles and database.  However, if
> > make check fails, postCheck is not executed and the database remains
> > polluted (and needs manual intervention).  Is there a way to revert the
> > changes made by preCheck reliably even if make check fails?  Something
> > like postCheckFinally, maybe?
> >
> > What is the recommended way of testing such packages?  The only one I
> > found was nixos/tests/trac.nix which somehow runs nixos specific tests
> > and creates a VM for the database, which is then discarded somehow.  If
> > this is the way to go, do you have some pointers or examples I could
> > follow about how to create and run these tests?
> >
> > I also found
> >
> https://nixos.org/wiki/Kiberpipa_sprint_2014_topics_wiki_backup#Nixtest_-_testing_for_nixpkgs
> > so it seems that there is a demand better package testing.  Have there
> > been any progress on this front?
> >
> > Thank you,
> >
> > Tomas
>
> Touching a global DB from the build process sounds risky in terms of
> reproducibility. E.g. building two versions at the same time might not
> work. Ideally, some way of isolating the test (e.g. the VM method you
> mentioned) sounds like the right path, though I'm afraid I have no
> advice to offer there.
>
> IMO disabling tests for the time being is preferrable to running tests
> that touch global state.
>
> James
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>



-- 
Fly safe,
Sloz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20150415/063fbb3e/attachment.html