[Nix-dev] Tracking upstream releases

[Tony White]

2009/7/22 Ludovic Courtès <ludo at gnu.org>:
> Hi Nicolas,
>
> Nicolas Pierron
> <nicolas.b.pierron at gmail.com> writes:
>
>> Many commits until now have been made to to bump the version number.
>> This auto-update task can be seen as the patch-shebang function, some
>> things have to be done automatically because we don't want to lose
>> time on them.  We should focus our work on more valuable (without
>> expiration) features.  Thus, a tool which generates & maintains Nix
>> expressions could be extremely valuable.
>
> While I agree in principle, I think there's a number of issues that
> would need to be solved.  Often, updating a package takes more than just
> taking the new tarball:
>
>  * At least you may need to fetch and check its digital signature or
>    cryptographic hash.
>
>  * Sometimes, new dependencies must be added, which you find out
>    manually when trying to build it.
>
>  * When the package has specific patches, they may have to be updated.
>
>  * Last but not least, as Isaac suggested, you'd want to check the
>    aftermath of upgrading the package---how many packages that depend
>    on it are broken by the upgrade?
>
> I think these constraints rule out a fully automated mechanism.
>
> However, we could surely have tools to help us track upstream releases.
> Debian has a tool ("watches").  For example, the `watch' file for
> GCC 4.4 looks like this:
>
> --8<---------------cut here---------------start------------->8---
> version=2
> ftp://gcc.gnu.org/pub/gcc/releases/gcc-(4\.4[\d\.]*)    debian  uupdate
> --8<---------------cut here---------------end--------------->8---
>
> (See
> http://www.debian.org/doc/maint-guide/ch-update.en.html#s-newupstream
> for details.)
>
> I guess we could have a `meta' attribute to contain this kind of
> information.  Then `nix-build' could be changed to actually use this
> information; or there could be a dedicated tool.
>
> What do you think?
>
> Thanks,
> Ludo'.
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at cs.uu.nl
> https://mail.cs.uu.nl/mailman/listinfo/nix-dev
>

Hi Ludo,
I'm not sure the constraints you mention are all that constrictive.

>  * At least you may need to fetch and check its digital signature or
>    cryptographic hash.

Yes and of course that would be a big thing but I'm sure very possible.
hydra uses perl right? I think a perl script may be able to do that
and then write out the new expression.

>  * Sometimes, new dependencies must be added, which you find out
>    manually when trying to build it.

There are already a few packages in the nixpkg collection (I can see
in trunk) Which already suffer that. qt4 for example. As long as the
build doesn't fail problems like that can be detected when certain
features aren't available in dependent software.
The no openssl support in qt4 for example, which means you can't
browse the nix svn repo using konqueror.

A failing package usually gives away it's missing dep just before it
errors out so inspecting the build log would help solve missing deps
in case failure.

>  * When the package has specific patches, they may have to be updated.

If the patch doesn't fail the build it would be OK, It happens. Also
any requires patch packages could be flagged as manually update and
"Stashed" (Put to one side) Instead.

>  * Last but not least, as Isaac suggested, you'd want to check the
>    aftermath of upgrading the package---how many packages that depend
>    on it are broken by the upgrade?

Using a separate repo for and calling it something like bleeding edge
then experience will likely be useful to create rules for various
dependency chains.
For example the qt4 and pyqt dependency means that if qt4 is updated,
pyqt needs to be updated if kde is being rebuilt and the pyqt release
usually happens maybe about a week after the qt update, so in that
example, enforcing a rule of, don't update without the other would be
required.

The successful results of the bleeding edge repo could be filtered
down to say trunk manually by doing a scripted merge and then tested
so the bleeding edge repo would never generally be recommended to be
used for much else but instead manipulated into trunk in stages as
seen fit.

I'm sure there would even be heaps more stuff to consider along the
way and plenty of pitfalls, it's a very ambitious idea.

I think watches is a good idea. debian unstable is usually very
current, so if they are using that, it might be a very, very useful
tool. Liking the idea. Didn't know it existed.

Thanks,
Tony