[Nix-dev] Re: ‘nix-prefetch-git’ optimizations

[Nicolas Pierron]

2010/1/25 Ludovic Courtès <ludo at gnu.org>:
> Hello,
>
> Nicolas Pierron <nicolas.b.pierron at gmail.com> writes:
>
>> Author: NicolasPierron
>> Date: 2010-01-18 08:28:38 +0000 (Mon, 18 Jan 2010)
>> New Revision: 19513
>>
>> You can view the changes in this commit at:
>>    https://svn.nixos.org/viewvc/nix?rev=19513&view=rev
>>
>> Modified:
>>    nixpkgs/trunk/pkgs/build-support/fetchgit/nix-prefetch-git
>>
>> Log:
>> nix-prefetch-git:
>> * Optimized download size in case of references.
>> * Add support for submodules.
>
> [...]
>
>> +# Fetch only a branch/tag and checkout it.
>> +checkout_ref(){
>> +    local hash="$1";
>> +    local ref="$2";
>> +
>> +    if test -z "$ref"; then
>> +        ref=$(ref_from_hash $hash);
>> +    fi;
>> +
>> +    if test -n "$ref"; then
>> +        # --depth option is ignored on http repository.
>> +        git fetch --depth 1 origin +"$ref" || return 1
>
> People tried ‘--depth 1’ in the past (see r16047) and reported problems,
> because the ‘--depth’ argument asks Git to fetch the last revisions of
> the given on *any* branch, which is often useless (see r18438).  Does
> your code specifically work around that?

Yes it does.

git remote add ...

is used to specify which branch is mention, so this is equivalent to
creating a snapshot of one branch.

When the server is an http server, this cannot work and the system
fallback to the previous code.  In addition, this implementation
support submodules which is similar to svn externals.

> Besides, this technique prevents “git describe” from working, probably
> because it can’t access all the tags and history.  This causes problems
> to most projects at http://hydra.nixos.org/project/gnu because they
> derive their version string from the output of “git describe” – a very
> useful feature because it leads to source tarballs with a git commit
> hash in their names.  In some cases having a wrong version string can
> even lead to test failures.
>
> Any idea how to restore that functionality?

May be you can add an environment variable which causes the function
"checkout_ref" to return 1.  The system will fallback to
"checkout_hash", which does the same as the previous code.  Currently,
I haven't found any optimized solution to make "git describe" working.

>> +clone(){
>> +    local top=$(pwd)
>> +    local dir="$1"
>> +    local url="$2"
>> +    local hash="$3"
>> +    local ref="$4"
>> +
>> +    cd $dir;
>> +
>> +    # Initialize the repository.
>> +    init_remote "$url";
>> +
>> +    # Download data from the repository.
>> +    checkout_ref "$hash" "$ref" ||
>> +    checkout_hash "$hash" "$ref" || (
>> +        echo 1>&2 "Unable to checkout $hash$ref from $url.";
>> +        exit 1;
>> +    )
>> +
>> +    # Checkout linked sources.
>> +    init_submodules;
>> +
>> +    cd $top;
>> +}
>
> [...]
>
>> -    git clone "$url" $tmpFile 1>&2
>> -    if test -n "$rev"; then
>> -      cd $tmpFile
>> -      echo $tmpFile >&2
>> -      git checkout $rev 1>&2
>> -    fi
>> +    case "$rev" in
>> +        HEAD|refs/*)
>> +            clone "$tmpFile" "$url" "" "$rev" 1>&2;;
>
> Why not just use ‘git clone "$url" "$tmpFile"’ here?  I would find it
> simpler and easier to understand.
>
> Thanks,
> Ludo’.
>



-- 
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
Lars Rasmussen (Google IO 2009) - Icland is an icland.