[Nix-dev] Download all of hackage

[John Ericson]

I agree that we over use overrides, and agree with Peter's proposals. But I
think the first step needs to be scraping (not necessarily building!) all of
package.

  {
    "foo" = {
      "0_1_2_3" = <whatever route we choose, e.g. function for callPackage>;
      "1_2_3_4" = <ditto>;
      ...
    }
    "bar" = ...;
    ...
  }

This would allow conservative users to override their nixpkgs with a newer
scrape rather than be forced to upgrade.

I'm not sure what sort of auto-generated overrides we do now, but I'd guess
those ought to be implemented as a big map over the scrape.

Configurations would have a "type" of something like:

  {
    versions:        Map package-name ([version] -> Nullable version);
    extraConfigs:    Map package-name (fixed-pkg-set -> extra-config-set);
    defaultFallback: [version] -> Nullable version)
  }

the idea is use the pkg-specific, else fallback function to pick the right
version, where null means don't include in final set. Then use the extra
config function to provide/override args to extraConfig if there is
one.[Exactly how we want to do overriding/exposed config/etc I'll leave to
the other thread.]

Some examples. A bleeding-edge configuration would have `defaultFallback =
max`, while `haskellPackages` would have fixed versions for everything, so
as not to break on re-scrapes.