Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[Solved]System shutdown hangs after "remounting / read only"
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
Tractor Girl
Apprentice
Apprentice


Joined: 16 May 2013
Posts: 159

PostPosted: Thu May 29, 2014 10:37 pm    Post subject: [Solved]System shutdown hangs after "remounting / read Reply with quote

My freshly installed gentoo hardened (3.13.10) hangs during shutdown procedure on
Code:
remounting remaining filesystems read-only ...
     remounting / read only ...

This situation occurs only if grsecurity is compiled into kernel. On kernel without grsecurity, everything seems to be ok.
I don't know where to find any detailed logs cause system logger is already stopped when this happens.

My kernel config

Code:
Portage 2.2.8-r1 (hardened/linux/amd64, gcc-4.7.3, glibc-2.17, 3.11.4-dell x86_64)
=================================================================
System uname: Linux-3.11.4-dell-x86_64-Intel-R-_Core-TM-_i5-2430M_CPU_@_2.40GHz-with-gentoo-2.2
KiB Mem:     8071408 total,   6730088 free
KiB Swap:    5857276 total,   5857276 free
Timestamp of tree: Tue, 27 May 2014 16:30:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
app-shells/bash:          4.2_p45
dev-lang/python:          2.7.6, 3.3.3
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.13.4
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.13 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="pl_PL.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
USE="acl amd64 berkdb bindist bzip2 cli cracklib crypt cxx dri gdbm hardened iconv justify mmx modules multilib ncurses nls nptl openmp pam pax_kernel pcre readline session sse sse2 ssl tcpd unicode urandom xtpax zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="synaptics evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="pl en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="intel radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON

Code:
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5)
00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation HM67 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Whistler [Radeon HD 6630M/6650M/6750M/7670M/7690M] (rev ff)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
09:00.0 Network controller: Intel Corporation Centrino Wireless-N 1030 [Rainbow Peak] (rev 34)
0b:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)


Last edited by Tractor Girl on Sun Jun 01, 2014 3:57 pm; edited 1 time in total
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13867

PostPosted: Fri May 30, 2014 1:25 am    Post subject: Reply with quote

GRsecurity is a large and complex patch set. Some of its features are always enabled, but others are enabled by Kconfig. If you configure out all optional features, leaving only the GRsecurity functionality that is not configurable, does the problem occur? If yes, you can add back Kconfig options until you find the one that controls the failure.
Back to top
View user's profile Send private message
Tractor Girl
Apprentice
Apprentice


Joined: 16 May 2013
Posts: 159

PostPosted: Sat May 31, 2014 6:30 pm    Post subject: Reply with quote

After countless recompilations I've finally found what is causing the problem. It is:
Code:
CONFIG_PAX_SIZE_OVERFLOW

which is located under:
Code:
Grsecurity -> PaX -> Miscellaneous Hardening Features

Here's some description what it should do:
Quote:
Prevent various integer overflows in function size parameters: When enabled, the kernel will recompute expressions passed as function arguments with double precision and, if an overflow occurs, the event is logged and the process killed.


I have no idea why it hangs shutdown (remounting / ?) process. It seem to be hardware related as on two other machines I have no such problems.
It would be nice to have any log of that problem but it happens right before poweroff moment and system logger is disabled at that time.
Is the any way I can get such logs? After all in the above description it is said that "...if an overflow occurs, the event is logged..." - Logged but where?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13867

PostPosted: Sat May 31, 2014 9:09 pm    Post subject: Reply with quote

I think that feature also detects underflow. Perhaps you have somehow triggered a situation where a reference count related to / underflows when it is remounted read-only. I think you need more help than can be easily provided here. Please file a bug with the information found so far.
Back to top
View user's profile Send private message
Tractor Girl
Apprentice
Apprentice


Joined: 16 May 2013
Posts: 159

PostPosted: Sat May 31, 2014 11:25 pm    Post subject: Reply with quote

Hu wrote:
Please file a bug with the information found so far.

Ok I will.

But maybe I was inexact saying it hangs ON "remount / read only". It looks like it hangs AFTER it. So maybe this doesn't concern remounting but something which is done after it?
Here's the complete output of my system's halt procedure (please notice that there's [OK] next to "remounting / read only". So it looks like this step was done correctly):

Quote:
The system is going down for halt NOW!
INIT: Switching to runlevel: 0
INIT: Sending processes the TERM signal
Stopping local ... [OK]
Saving random seed ... [OK]
sysklogd -> stop klogd ... [OK]
sysklogd -> stop syslogd ... [OK]
Deactivating additional swap space ... [OK]
Unmounting network filesystems ... [OK]
Unmounting loop devices ...
Unmounting filesystems ...
Deactivating swap devices ... [OK]
Stopping udev ... [OK]
Setting hardware clock using using the system clock [UTC] ...
[ 195.940266] grsec: time set by /sbin/hwclock[hwclock:4449] uid/euid: 0/0 gid/egid: 0/0, parent /lib64/rc/sh/runscript.sh[runscript.sh:4448] uid/euid: 0/0 gig/egid: 0/0 [OK]
Terminating remaining processes ... [OK]
Killing remaining processes ... [OK]
Saving dependency cache ... [OK]
remounting remaining filesystms read-only ...
Remounting / read only ... [OK]


My question is: what is the next step after remounting? On my other machines system just poweroffs, but there's some additional line, which unfortunately disappears to quickly to read it.

Also I don't know if this is anyhow related, but in kernel log I found PaX report about size overflow (so it was generated by this CONFIG_PAX_SIZE_OVERFLOW thing)
Code:
Jun  1 00:12:17 gentoo kernel: [   15.388688] PAX: size overflow detected in function ttm_page_pool_free drivers/gpu/drm/ttm/ttm_page_alloc.c:301 cicus.339_237 max, count: 3

It happens at every boot, and has something to do with radeon driver.

Here's the complete kernel log (from start to shutdown):
http://bpaste.net/show/325604/
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Jun 01, 2014 9:28 am    Post subject: Reply with quote

Hi Tractor Girl,


the following might be related:

http://marc.info/?l=linux-kernel&m=140150530622103&w=2 ([PATCH 4/5] gpu/drm/ttm: Fix possible stack overflow by recursive shrinker calls.)


for more search results:

http://marc.info/?l=linux-kernel&w=2&r=1&s=ttm+overflow&q=b
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Tractor Girl
Apprentice
Apprentice


Joined: 16 May 2013
Posts: 159

PostPosted: Sun Jun 01, 2014 3:56 pm    Post subject: Reply with quote

Thank you for the hint kernelOfTruth.

It turned out to be damn open source radeon driver fault.
Without radeon module compiled problem goes away. Blacklisting radeon module works too. Another way is to install binary blob: radeon-ucode.
This is most disgusting but unfortunately I need it just to disable this goddamn ati gpu (my laptop has both intel and ati). Otherwise it just heating the laptop and spins the fan.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Jun 01, 2014 6:59 pm    Post subject: Reply with quote

disabling the card in Bios,

disabling via vga_switcheroo

or other ways don't work ?

argh :? - it's stupid how the manufacturers assume that only Windows will be used and hide control options from the users
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum