Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved] libgcrypt AES crashes with gcc <4.8 on some CPUs
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
mmogilvi
n00b
n00b


Joined: 13 May 2011
Posts: 40

PostPosted: Mon Oct 24, 2016 4:23 am    Post subject: [solved] libgcrypt AES crashes with gcc <4.8 on some CPUs Reply with quote

Old GCC (in my case, primarily 4.6 [pre C++11]) is not supported by modern gentoo, so I'm not filing a bug, but the variables affecting this were a bit hard to track down, so I thought I would document it somewhere.

Code:
$ echo password > pass
$ echo Some Text | gpg --cipher-algo AES -c --batch --passphrase-fd 3 --no-tty 3< pass > something.gpg

gpg: signal Segmentation fault caught ... exiting


There are several conditions relevant to repeating this bug:
  • "--cipher-algo AES" is only needed with gpg 2.0, when the default is CAST5. gpg 2.1 changes the default to AES. I first encountered the problem after gpg 2.1.x was stabilized, although I think the bug is probably older, and in libgcrypt.
  • Only repeatable on machines that do NOT have the "aes" flag in /proc/cpuinfo, but (unconfirmed) DO have "ssse3". (I only tried amd64 [on intel]. I'm not sure if 64 bit is required.)
  • gcc 4.6.4 and 4.7.4 are broken (but only subject to above), but gcc 4.8.5 always works.
  • Kernel version doesn't seem to have any effect. (Tried 3.10.7 and 4.4.26.)

Some possible workarounds include:
  • Use make.env to compile libgcrypt with gcc 4.8 or newer. It has a clean C ABI and typically not many reverse dependencies, so this should be safe to upgrade narrowly.
  • Update configurations and/or scripts to continue to prefer the old CAST5 default.
  • Mask gpg 2.1 and continue using 2.0, with it's CAST5 default.

It probably is NOT advisable to downgrade libgcrypt, due to recent vulnerabilities, including CVE-2016-6316.

Call stack when built with (export FEATURES=nostrip ; export CFLAGS="-march=core2 -msse4 -msse4.1 -msse4.2 -mcx16 -msahf -O2 -pipe -ggdb -fno-stack-protector" ; export LDFLAGS="-ggdb" ; emerge -1av libgcrypt gnupg):
Code:
#0  got_fatal_signal (sig=11) at signal.c:101
#1  <signal handler called>
#2  _gcry_aes_ssse3_decrypt (ctx=0x6e8240,
    dst=0x7ffff76e60e5 <_aes_schedule_core+725> "f\017\357\330f\017\070",
    src=<optimized out>)
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/cipher/rijndael-ssse3-amd64.c:430
#3  0x00007ffff76e2f12 in selftest_basic_128 ()
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/cipher/rijndael.c:1532
#4  0x00007ffff76e294f in selftest ()
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/cipher/rijndael.c:1681
#5  do_setkey (ctx=0x7ffff7fea950,
    key=0x7ffff7fea67c "\274\221\350\321\353]q\241\365\a*\306\071\251\241\017", keylen=16)
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/cipher/rijndael.c:285
#6  0x00007ffff76c0df7 in cipher_setkey (keylen=<optimized out>,
    key=<optimized out>, c=0x7ffff7fea6d0)
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/cipher/cipher.c:633
#7  _gcry_cipher_setkey (hd=<optimized out>, key=<optimized out>,
    keylen=<optimized out>)
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/cipher/cipher.c:1064
#8  0x00007ffff76b5b3e in gcry_cipher_setkey (hd=0x7ffff7fea6d0,
    key=0x7ffff7fea67c, keylen=16)
    at /var/tmp/portage/dev-libs/libgcrypt-1.7.3/work/libgcrypt-1.7.3/src/visibility.c:724
#9  0x000000000044c5b7 in write_header (a=0x6e3b90, cfx=0x7fffffffd890)
    at cipher.c:97
#10 cipher_filter (opaque=0x7fffffffd890, control=<optimized out>, a=0x6e3b90,
    buf=0x6e3c20 "", ret_len=<optimized out>) at cipher.c:127
#11 0x000000000048f9b7 in filter_flush (a=0x6e1af0) at iobuf.c:1969
#12 0x00000000004914b5 in iobuf_push_filter2 (a=0x6e1af0,
    f=0x41db30 <compress_filter>, ov=0x7fffffffd8d0, rel_ov=0) at iobuf.c:1552
#13 0x000000000044ce98 in encrypt_simple (filename=<optimized out>, mode=1,
    use_seskey=0) at encrypt.c:361
#14 0x000000000040cf92 in main (argc=0, argv=0x7fffffffe038) at gpg.c:3883


I haven't tried to identify exactly what gcc 4.6/4.7 does with the inline assembly in rijndael-ssse3-amd64.c (presumably incorrectly) that 4.8 does differently or better.

Broken /proc/cpuinfo:
Code:
processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Core(TM) i7 CPU         975  @ 3.33GHz
stepping        : 5
microcode       : 0x11
cpu MHz         : 1600.000
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 7
initial apicid  : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bugs            :
bogomips        : 6652.49
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:


Another broken /proc/cpuinfo:
Code:
processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 30
model name      : Intel(R) Xeon(R) CPU           X3470  @ 2.93GHz
stepping        : 5
microcode       : 0x3
cpu MHz         : 1197.000
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 7
initial apicid  : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 5854.74
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:


AN ALWAYS WORKING /proc/cpuinfo:
Code:
processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
stepping        : 4
microcode       : 0x416
cpu MHz         : 1215.218
cache size      : 10240 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 7
initial apicid  : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 7385.70
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum