Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Safe CFLAGS
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
brendlefly62
Tux's lil' helper
Tux's lil' helper


Joined: 19 Dec 2009
Posts: 93

PostPosted: Mon Jan 23, 2017 9:59 pm    Post subject: Safe CFLAGS Reply with quote

Just figured I'd toss this out there (feedback welcome) --

The script below automates the procedure described at the beginning of https://wiki.gentoo.org/wiki/Safe_CFLAGS to determine safe CFLAGS to set for the current (arbitrary) machine. The meat of this is pretty simple. Most of the script below is formatting for friendly interaction. I don't normally include that directly in my scripts and prefer to source my "script_header" instead, but I've directly included the formatting definitions and functions used in this script to avoid having to post the whole script_header separately.

Code:
#!/bin/bash
# getmycflags.sh
# Author: brendlefly 23 January 2017
# Implements the procedure specified at https://wiki.gentoo.org/wiki/Safe_CFLAGS
#   to determine safe CFLAGS to set for this (arbitrary) machine

#---[ local definitions ]--------------------------------------------------------
BUILD=0.0.1
DEBUG="false"
#DEBUG="true"

old_dir=$(pwd)
user_home_dir="$HOME"

ROOT_UID=0
E_ROOT="Please run this script as user, not root."

#--[ Easy ANSI Escape sequences to put color in my scripts ]---
#  see http://en.wikipedia.org/wiki/ANSI_escape_code
#  see also http://ascii-table.com/ansi-escape-sequences.php
CSI="\033["             # control sequence initiator == hex "\x1b["
#---[ Select Graphics Rendition on/off ]---------------------------
BOLD="1"       # bold on
SGRoff="0"     # Bold off (reset all SGR (e.g. blink, underline)
#---[ Set Text Color, Foreground ]---------------------------------
RED="31"       # foreground red
GREEN="32"     # foreground green
YELLOW="33"    # foreground yellow
BLUE="34"      # foreground blue
MAG="35"       # foreground magenta (it's like fucia)
LBLUE="36"     # foreground light blue (cyan)
WHITE="37"     # foreground white (cyan)
#---[ My Favorite Colors (terminate with ${Boff} ]-----------------
BRon="${CSI}${RED};${BOLD}m"
BGon="${CSI}${GREEN};${BOLD}m"
BYon="${CSI}${YELLOW};${BOLD}m"
BBon="${CSI}${BLUE};${BOLD}m"
BMon="${CSI}${MAG};${BOLD}m"
LBon="${CSI}${LBLUE};${BOLD}m"
BWon="${CSI}${WHITE};${BOLD}m"
Boff="${CSI}${SGRoff}m"          # Bold off (reset all SGR (e.g. blink, underline)

#---[ function blocks ]----------------------------------------------------------
checknotroot()     # Run as root, of course.
{
  if [ "$UID" -eq "$ROOT_UID" ]; then E_message "${E_ROOT}"; echo; exit 1; else return 0; fi
}

message()       # echo a simply formatted message (arg $1) to the screen
{ echo -e "${BGon}*${Boff} ${1}" && return 0 || return 1; }

E_message()     # echo a simply formatted error message (arg $1) to the screen
{ echo -e "${BRon}*${Boff} ${1}" && return 0 || return 1; }

repeat()        # output a repeated string of character (arg $1) of length (arg $2)
{
  local i thing limit
  thing="$1"; limit=$2; out_str=""; i=0
  while [ $i -lt $limit ]; do out_str="${out_str}${thing}"; let "i++"; done
  echo -en "$out_str" && return 0 || return 1
}

termwidth()     # calculate and output the width of the terminal
{ echo -n $(stty size | sed 's/[0-9]* *//') && return 0 || return 1; }

termheight()    # calculate and output the height of the terminal
{ echo -n $(stty size | cut -d' ' -f1) && return 0 || return 1; }

separator()     # draw a horizontal line with a simple title (arg $1) and preface (arg $2)
{
  # to facilitate separation of portions of the output of various scripts
  # include a title preface (arg $2 or $(hostname) if $2 is not provided
  local msg preface msg_len
  [ ! -z "$2" ] && preface="$2" || preface=$(hostname)
  msg="${BYon}---[${BRon} $preface ${LBon}$1 ${BYon}]"
  msg_len=$(( ${#msg} - $(( ${#BYon} + ${#BRon} + ${#BBon} + ${#BYon} + ${#Boff} )) ))
  echo -en "$msg" && \
  echo -n $(repeat "-" $(( $(termwidth) - $(( $msg_len + ${#Boff} )) )) ) && echo -e ${Boff} && \
  return 0 || return 1
}

#---[ main script ]--------------------------------------------------------------
separator "getmycflags.sh-$BUILD"

checknotroot

if [ "$old_dir" != "$user_home_dir" ]
then
  message "Changing from [${old_dir}] to [${user_home_dir}] and initializing..."
  cd ${user_home_dir}
else
  message "Already in [${user_home_dir}]. Initializing..."
fi

LANG="en"
rm -v native.* 2>/dev/null; rm -v march.* 2>/dev/null
echo "" > native.cc
echo "" > march.cc
message "Testing compiler with -march=native" &&
gcc -fverbose-asm -march=native native.cc -S
message "Here is the result: "
result=$(grep march native.s)
march=$(for word in $result; do echo $word; done | grep march | cut -d= -f2)
message "  result: [$result]"
message "  extracting march: [$march]"
message "Now testing with -march=${march}"
gcc -fverbose-asm -march=${march} march.cc -S
message "Editing output files with sed to check selected options"
sed -i 1,/options\ enabled/d march.s
sed -i 1,/options\ enabled/d native.s
message "Comparing output files..."
message "  ${BYon}If no output between separators below, then use -march=${march}${Boff} ,"
message "   otherwise use the procedure at https://wiki.gentoo.org/wiki/Safe_CFLAGS"
message "   to determine what additional switches you must set. (see further below)"
separator "diff march.s native.s"
diff march.s native.s
separator "diff march.s native.s"
message "${Byon}If not blank between separators above, then${Boff}"
message "  (1) Identify <switch1> switches that were enabled by -march=${march},"
message "      but were NOT enabled by -march=native, and"
message "  (2) Identify <switch2> switches that were enabled by -march=native,"
message "      but were NOT enabled by -march=${march}, and"
message "  (3) Run the following commands, with those switches disabled/enabled (respectively)"
message "      to determine what else might need to be disabled or enabled:"
echo
message "  $ gcc -fverbose-asm -march=${march} -mno-<switch1> -m<switch2> march.cc -S"
message "  $ sed -i 1,/options\ enabled/d march.s"
message "  $ diff march.s native.s"
echo
message "I will now try to do that automatically..."
march_s_has=$(diff march.s native.s | grep '< #' | sed -e 's/#//' -e 's/<//')
native_s_has=$(diff march.s native.s | grep '> #' | sed -e 's/#//' -e 's/>//')
message "  Using march.s switches: $march_s_has"
message "  Using native.s switches: $native_s_has"
# go through each in march.s and see if there's a match in native.s
march_s_has_out=$march_s_has
native_s_has_out=$native_s_has
for m_swx in $march_s_has
do
  [[ "$DEBUG" == "true" ]] && message "  Checking native_s_has for a match to [$m_swx]"
  for n_swx in $native_s_has
  do
    if [[ "$n_swx" == "$m_swx" ]]
    then
      [[ "$DEBUG" == "true" ]] && message "    (found in both, dropping)"
      march_s_has_out="$(echo $march_s_has_out | sed "s/$m_swx//")"
      native_s_has_out="$(echo $native_s_has_out | sed "s/$n_swx//")"
    fi
  done
done
# convert what is left in $march_s_has_out into "no-" switches
march_s_has_out_final=""
for m_swx in $march_s_has_out
do
  march_s_has_out_final="$march_s_has_out_final $(echo $m_swx | sed 's/-m/-mno-/')"
done
message "${BYon}Here are your suggested CFLAGS:${Boff}"
message "${BGon}  -march=${march}${march_s_has_out_final} ${native_s_has_out}${Boff}"

if [ "$old_dir" != "$user_home_dir" ]
then
  message "Cleaning up and changing back from [${user_home_dir}] to [${old_dir}]..."
  rm march.*; rm native.*
  cd ${old_dir}
else
  message "Cleaning up..."
  rm -v native.* 2>/dev/null; rm -v march.* 2>/dev/null
fi
message "---[ done ]---"


cheers
Back to top
View user's profile Send private message
rufnut
Apprentice
Apprentice


Joined: 16 May 2005
Posts: 205

PostPosted: Mon Jan 23, 2017 11:23 pm    Post subject: Reply with quote

cool, thanks.

I only had "-march=ivybridge"

not:
Quote:
* Here are your suggested CFLAGS:
* -march=ivybridge -mno-aes -mno-rdrnd


Quote:
model name : Intel(R) Core(TM) i3-3120M CPU @ 2.50GHz



:D
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43425
Location: 56N 3W

PostPosted: Mon Jan 23, 2017 11:34 pm    Post subject: Reply with quote

rufnut,

There is no need to set -mno
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
rufnut
Apprentice
Apprentice


Joined: 16 May 2005
Posts: 205

PostPosted: Mon Jan 23, 2017 11:43 pm    Post subject: Reply with quote

NeddySeagoon wrote:
rufnut,

There is no need to set -mno


I am not so sure?

It is suggested and I can see this is the difference between "-march=ivybridge" and "-march=native"

I am using Distcc so a specific "-march=" is needed.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43425
Location: 56N 3W

PostPosted: Tue Jan 24, 2017 12:10 am    Post subject: Reply with quote

rufnut,

-mno is the default state.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
rufnut
Apprentice
Apprentice


Joined: 16 May 2005
Posts: 205

PostPosted: Tue Jan 24, 2017 12:27 am    Post subject: Reply with quote

this is interesting :-)

Quote:
gcc -march=ivybridge -Q --help=target

The following options are target specific:
-m128bit-long-double [disabled]
-m16 [disabled]
-m32 [disabled]
-m3dnow [disabled]
-m3dnowa [disabled]
-m64 [enabled]
-m80387 [enabled]
-m8bit-idiv [disabled]
-m96bit-long-double [enabled]
-mabi= sysv
-mabm [disabled]
-maccumulate-outgoing-args [disabled]
-maddress-mode= short
-madx [disabled]
-maes [disabled]
.........................
-mrdrnd [disabled]
...............


yes, you are right.

So why does it suggest it ?
Back to top
View user's profile Send private message
brendlefly62
Tux's lil' helper
Tux's lil' helper


Joined: 19 Dec 2009
Posts: 93

PostPosted: Tue Jan 24, 2017 2:22 am    Post subject: Reply with quote

So, Neddy - below is the example that the wiki uses to explain the procedure ( which is what the script tries to automate). I gather the -mno switches set in the final recommendation in the example are also not necessary? (will that be true in all cases? -- not all switches are disabled by default) Also, in my own Athlon example further below, is the sahf recommendation also not necessary? (its default is also disabled, but native enables it)

Quote:
Output empty? You found your -march=, use it. In other cases:

20,25c20,23
< # -maccumulate-outgoing-args -maes -malign-stringops -mavx
< # -mavx256-split-unaligned-load -mavx256-split-unaligned-store -mcx16
< # -mf16c -mfancy-math-387 -mfp-ret-in-387 -mfsgsbase -mfxsr -mglibc
< # -mieee-fp -mlong-double-80 -mmmx -mpclmul -mpopcnt -mpush-args -mrdrnd
< # -mred-zone -msahf -msse -msse2 -msse3 -msse4 -msse4.1 -msse4.2 -mssse3
< # -mtls-direct-seg-refs -mxsave -mxsaveopt
---
> # -maccumulate-outgoing-args -malign-stringops -mcx16 -mfancy-math-387
> # -mfp-ret-in-387 -mfsgsbase -mfxsr -mglibc -mieee-fp -mlong-double-80
> # -mmmx -mpclmul -mpopcnt -mpush-args -mred-zone -msahf -msse -msse2 -msse3
> # -msse4 -msse4.1 -msse4.2 -mssse3 -mtls-direct-seg-refs
Now guess, which switches -march=core-avx-i enable and -march=native not: -maes -mavx and some more. Now we build march.cc again with this two switches disabled:

gcc -fverbose-asm -march=core-avx-i -mno-aes -mno-avx march.cc -S
sed -i 1,/options\ enabled/d march.s
diff march.s native.s

< # -mmmx -mpclmul -mpopcnt -mpush-args -mrdrnd -mred-zone -msahf -msse
< # -msse2 -msse3 -msse4 -msse4.1 -msse4.2 -mssse3 -mtls-direct-seg-refs
---
> # -mmmx -mpclmul -mpopcnt -mpush-args -mred-zone -msahf -msse -msse2 -msse3
> # -msse4 -msse4.1 -msse4.2 -mssse3 -mtls-direct-seg-refs
One switch we need to disable too: -mrdrnd

Finally set CFLAGS:

FILE /etc/portage/make.conf
CFLAGS="-march=ivybridge -mno-avx -mno-aes -mno-rdrnd -O2 -pipe"


Here is what results on my old Athlon --
Code:
$ uname -a
Linux oromis 4.4.39-gentoo #1 Sat Dec 31 21:15:02 EST 2016 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux
...
* I will now try to do that automatically...
*   Using march.s switches:   -mieee-fp -mlong-double-80 -mmmx -mno-sse4 -mprfchw -mpush-args
  -mred-zone -msse -msse2 -msse3 -mtls-direct-seg-refs
*   Using native.s switches:   -mieee-fp -mlong-double-80 -mmmx -mno-sse4 -mpush-args -mred-zone -msahf
  -msse -msse2 -msse3 -mtls-direct-seg-refs
* Here are your suggested CFLAGS:
*   -march=k8-sse3 -mno-prfchw -msahf
Back to top
View user's profile Send private message
Kajan
Guru
Guru


Joined: 23 Dec 2004
Posts: 321
Location: Warsaw, Poland

PostPosted: Tue Jan 24, 2017 8:30 am    Post subject: Reply with quote

Hmm interesting I will check it on my Haswelll as well.
_________________
Powered By Gentoo Linux !!!
Back to top
View user's profile Send private message
Roman_Gruber
Advocate
Advocate


Joined: 03 Oct 2006
Posts: 3806
Location: Austro Bavaria

PostPosted: Tue Jan 24, 2017 10:01 am    Post subject: Reply with quote

why not use march native? works afaik for gcc 4.9.x or above flawless.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43425
Location: 56N 3W

PostPosted: Tue Jan 24, 2017 10:09 am    Post subject: Reply with quote

Roman_Gruber,

It won't work with distcc.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Leio
Developer
Developer


Joined: 27 Feb 2003
Posts: 487
Location: Estonia

PostPosted: Tue Jan 24, 2017 1:30 pm    Post subject: Reply with quote

Safe CFLAGS should be like -O2 with -march=native for most x86 and x86_64 machines. Most of that page seems like historical stuff for when -march=native didn't exist yet.
If I need distcc, I just check what -march=native would do with

Code:
gcc -march=native -E -v - </dev/null 2>&1 | grep cc1


and then putting whatever is between -march and -mtune in my CFLAGS (-march and -mtune included), and re-doing that after major gcc upgrades.

-mno flags are not necessarily default off when -march is involved. Then they can be used to disable it despite -march enabling them.
_________________
GNOME team lead; GStreamer; MIPS/ARM64
Back to top
View user's profile Send private message
archenroot
Apprentice
Apprentice


Joined: 13 Dec 2011
Posts: 203
Location: Lake Macha, Czech republic

PostPosted: Fri Apr 14, 2017 7:46 am    Post subject: Reply with quote

@brendlefly62 - thanks for the script, the origin has been removed during time. I found some differences on my Skylake laptop processor as well, that is why I generated manual set of CFLAGS, anyway required as I go with distcc, thanks again.

There is one error reported on my system:
Code:
---[ ares diff march.s native.s ]./cpuflags_native_vs_real_check: řádek 77:  : příkaz nenalezen
---------------------------------------------------------------------------------------------------------------------------
./cpuflags_native_vs_real_check: řádek 78:  : příkaz nenalezen


příkaz nenalezen stands for command not found

It refers to following lines in script:
Code:
echo -en "$msg" && \
echo -n $(repeat "-" $(( $(termwidth) - $(( $msg_len + ${#Boff} )) )) ) && echo -e ${Boff} && \


I am on gcc 6.3.0 and with Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz the result is:
Code:
* Here are your suggested CFLAGS:
* -march=skylake -mabm -mrtm


Whereas on 5.4.0:
Code:
* Here are your suggested CFLAGS:
*   -march=broadwell -mabm -mavx256-split-unaligned-load -mavx256-split-unaligned-store -mclflushopt -mrtm -mxsavec  -mxsaves


And finally on 4.9.4:
Code:
* Here are your suggested CFLAGS:
*   -march=broadwell -mabm -mavx256-split-unaligned-load -mavx256-split-unaligned-store -mrt


Visible improvement with gcc 6 support of Skylake in this case.
_________________
Emperor wants to control outer space Yoda wants to explore inner space that's the fundamental difference between good and bad sides of the Force
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum