[Nix-dev] Auto-updating sources: handling the general case

[Michael Raskin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

		Hello

What features I want:
1) Automatically maintain version correspondence between source name and
package name.
2) Update package to given (as URL or otherwise) source version.
3) Automatically update packages to the latest version (if a fresher
version is available).

What properties I want or understand others wanting:
1) One file per expression.
2) Not too much implied formatting restrictions that cannot be expressed
in Nix terms. I.e. comments, attribute ordering, formatting etc. are not
supposed to matter in Nix expressions and making them matter sometimes
may be not nice.
3) An approach that can cover fetchsvn etc. if needed.

What options are there:
1) X.org/KDE/Gnome: update scripts for specific recurseIntoAttributes
worthy things.
2) gnupdate: update all GNU software: relies on specific way of
publishing releases.
3) my update-upstream-data: uses three files per expression and
sometimes exposes too much internal information.

What can be done:
For general case there is update-upstream data. There are some fixes
that can be made:
1) We have "callPackage" now. That allows evaluating package (or some of
its attributes) without guessing how it is called in all-packages.nix.
2) It looks like "nix-instantiate --eval-only --xml --strict" is stable
now. So there is no need to build some temporary derivations for robust
getting of attributes from attrsets.
3) The resulting source data (url etc.) can be put back right into Nix
expression if we require marking the generated attribute set by putting
something like ``autoGeneratedClause = "auto-update";''. The hard part
(writing regexp for finding such attrSet in the expression text) is
already done (in CL-PPCRE tree syntax - debugging this as a
string-expressed regexp is too hard). Or maybe we want to fix
"--parse-only --xml"? In that case we will need a lot of position
information, though.
4) The attributes should be designed from the beginning for different
methods in different cases. I.e. there is a "method" where we read
sourceforge page and find the latest release (optionally: filter by
filename, process nested groups); there is a method where project
news/download page contains a direct link to the latest version; etc.
5) Version extraction should also be split into different handling methods.

I guess the thing requiring most synchronization is naming. What should
a mark of the generated expression look like? How should names for
source obtaining methods (fetchurl/fetchgit/..), updating latest data
(sourceforge/by link from download page/..), extracting version (by
regexp/from dirname in URL/..) be chosen?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJMbZV6AAoJEE6tnN0aWvw3unYIALi1JG+FOO4y03H59ZCNYGoX
Rg4MNTa7/ZyuVk4sheQJ8QQ/AhYrEjuxsf3s7toI2GyWkWSxXhxe1xMbheaYJxKy
XpJO0VQq1iRfve5ZwA0dHZhlKDG7NWBzPR8Ok+gpimD5eVbAvDNpD8a4TO96TkVA
vGh3dZ7d6bYH53fM3o+KNPHFz5E0hgPtPaeSKUq2wTQTPXJhxFmdIyTzSCHwqtrf
uNnIJBkcxvya59zLB3xsRxhOKEY3lRiOi6wtZGOmWVhu65AoQyzMPMr5JehYoRcg
ywtwImH9GtI1kw/RorJe/lZYiKQjylh0bGlWTqs3rb3Vn9ERhFJ8e2zvrV1RiZ4=
=rcmo
-----END PGP SIGNATURE-----