[Nix-dev] CPU optimized packages

[Shea Levy]

On 02/07/2013 10:09 AM, Danny Wilson wrote:
> Great!
> Maybe instead of depending on time, it could use `md5sum 
> /proc/cpuinfo`  or something that identifies the machine?
>
> I get the concepts but I'm very new to the nix language. How would I 
> modify an existing package to use this in my ~/.nixpkgs/config.nix ?

Unfortunately, this isn't currently easy to do, we'd need to add some 
hooks in stdenv to allow adding some custom CFLAGS. Can you open an 
issue on github.com/nixos/nixpkgs for this so I can have a reminder 
about adding this?

>
>
>
>> Shea Levy <mailto:shea at shealevy.com>
>> 7 februari 2013 16:01
>> OK, well technically that is possible, with something like this:
>>
>> extra_cflags = import (pkgs.runCommand "cflags" {} ''
>> mkdir $out
>> echo "" | gcc -O3 -march=native -mtune=native -v -E - 2>&1 |grep cc1 
>> |sed -r 's/.*? - -(.*)$/-\1/' > $out/flags
>> echo "builtins.readFile ./flags" > $out/default.nix
>> '');
>>
>> Basically, what that does is build a derivation which creates a nix 
>> expression, then imports that nix expression. One problem with this 
>> is that the result is saved in the nix store, meaning if you share 
>> nix stores between machines it won't work (because it will run once 
>> on one machine and the same result will be used on different 
>> machines). To get around this, you can make the derivation depend on 
>> the current time, which will force it to calculate the cflags every 
>> time you build something. Also, it's possible that in the future 
>> builds run in chroot will be inside namespaces that block how gcc 
>> detects the host system, so we should make this not run in chroot:
>>
>> extra_cflags = import (pkgs.runCommand "cflags" {time = 
>> builtins.currentTime; __noChroot = true;} ''
>> mkdir $out
>> echo "" | gcc -O3 -march=native -mtune=native -v -E - 2>&1 |grep cc1 
>> |sed -r 's/.*? - -(.*)$/-\1/' > $out/flags
>> echo "builtins.readFile ./flags" > $out/default.nix
>> '');
>>
>> I'm not sure of your level of understanding of nix, so please let me 
>> know if this needs more explanation.
>>
>> Thanks,
>> Shea
>>
>> On 02/07/2013 09:53 AM, Danny Wilson wrote:
>>
>> Danny Wilson <mailto:danny at prime.vc>
>> 7 februari 2013 15:53
>> Well in this case I am the user. And I don't want to fill this in for 
>> every different server I might have.
>>
>> My suggestion was to make -march=native pure, by expanding it on the 
>> system configuring itself at nix expression evaluation time:
>>
>> $ echo "" | gcc -O3 -march=native -mtune=native -v -E - 2>&1 |grep 
>> cc1 |sed -r 's/.*? - -(.*)$/-\1/'
>> -march=corei7 -mcx16 -msahf -mno-movbe -maes -mno-pclmul -mpopcnt 
>> -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-tbm 
>> -mno-avx -msse4.2 -msse4.1 --param l1-cache-size=32 --param 
>> l1-cache-line-size=64 --param l2-cache-size=4096 -mtune=generic -O3
>>
>>
>>
>>
>> Shea Levy <mailto:shea at shealevy.com>
>> 7 februari 2013 15:33
>> Hi Danny,
>>
>> Unfortunately, there's not a good answer to this because 
>> -march=native is inherently impure. Is there any reason you can't 
>> tell users "Find out what -march=native means on your system and then 
>> fill in nixpkgs.config.extra_cflags with the result?" or some such 
>> (Note that config.extra_cflags doesn't currently exist AFAIK, but could)?
>>
>> ~Shea
>>
>> On 02/07/2013 07:36 AM, Danny Wilson wrote:
>>
>> Danny Wilson <mailto:danny at prime.vc>
>> 7 februari 2013 13:36
>> With help of the gentoo wiki ( 
>> http://en.gentoo-wiki.com/wiki/Hardware_CFLAGS )
>> I found this produces a nice GCC option list:
>>
>> echo "" | gcc -O3 -march=native -mtune=native -v -E - 2>&1 |grep cc1 
>> |sed -r 's/.*? - -(.*)$/-\1/'
>>
>>
>>
>>
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>> Danny Wilson <mailto:danny at prime.vc>
>> 7 februari 2013 12:37
>> I'd like some packages to be optimized for my server CPU. GCC has
>> -march=native for this, but that would not generate a cpu specific nixos
>> hash.
>>
>> Is there a nix expression available which adds something like this:
>> gcc -march=native -Q --help=target |grep enabled
>>
>> to CFLAGS and correctly hashing it?
>>
>>
>>
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>> ------------------------------------------------------------------------
>
>
>
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130207/6489fd20/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: postbox-contact.jpg
Type: image/jpeg
Size: 1178 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130207/6489fd20/attachment.jpg 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compose-unknown-contact.jpg
Type: image/jpeg
Size: 770 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130207/6489fd20/attachment-0001.jpg