[Nix-dev] Travis false-positives

[Peter Simons]

Hi Georges,

 > I think of it more as proof of concept of what CI on the pull
 > requests can bring us than a final testing workflow.

unfortunately, users submitting pull requests cannot see that the
travis-ci job isn't supposed to be considered "reliable". I've run into
cases where people submit totally untested patches as a PR, because they
assume travis-ci will test the patch for them. Travis, however, comes
back with some kind of system failure that gives no indication whether
the patch is good or not.

This means that users are more likely to submit broken patches because
they rely on a safety net that actually doesn't exist. Because of this
phenomenon, the travis-ci job actually decreases the quality of our
submissions!

Is it possible to add a note to the PR status pages that draws people's
attention to the fact that the travis-ci service is experimental?

Best regards,
Peter