Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
umount freezes my system
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
zark
n00b
n00b


Joined: 27 Mar 2004
Posts: 74
Location: Belgium

PostPosted: Fri Jul 27, 2018 7:49 am    Post subject: umount freezes my system Reply with quote

Okay, it's been a long night trying to find the source of the problem.


But basically my system won't boot anymore, it hangs on init.d/bootmisc
I've added some echo in there, and it's crashing during the clean_run function at the "umount" command.

And indeed, if i log with single user mode, i mount a usb drive to /mnt/gentoo and then try to umount , it just hangs there.

I'm stuck there, i don't know how to look further at what could go wrong.
i tryed an strace and am getting stuck after a call to umount2()


please help

edit: switching back to my previous kernel helps.
But it's essentially the same kernel, i just enabled SMT and enabled > 8 cpu (switched to a 12threaded xeon).
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7071

PostPosted: Fri Jul 27, 2018 11:15 am    Post subject: Reply with quote

Quote:
switched to a 12threaded xeon

That could be the problem, improper migration may lead to "bad" code for older cpu use with the newer one.
Tried rebuild util-linux?

What you could do is simply use a livecd, mount and umount your usb disk with it and see what happen.
Back to top
View user's profile Send private message
zark
n00b
n00b


Joined: 27 Mar 2004
Posts: 74
Location: Belgium

PostPosted: Fri Jul 27, 2018 11:20 am    Post subject: Reply with quote

Hello.

I switched from a Xeon E5504 to a X5670 (yay baby)

i did rebuild util-linux and baselayout.

No issue with a liveCD.


It's really and clearly a kernel issue.

Basically same kernel 4.9.6-r1

I switched on "Support for big SMP systems with more than 8 CPUs"
that way i could change "Maximum number of CPUs" from 8 to 16
and enabled the SMT (Hyperthreading) scheduler support.

Now obviously when switching those on, some other setting must have changed somewhere ....
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7071

PostPosted: Fri Jul 27, 2018 11:33 am    Post subject: Reply with quote

if your changes were so few, you could diff your .config and see what kernel has change to agree with your changes.
Back to top
View user's profile Send private message
zark
n00b
n00b


Joined: 27 Mar 2004
Posts: 74
Location: Belgium

PostPosted: Fri Jul 27, 2018 1:32 pm    Post subject: Reply with quote

mmh, apart from the switch to NUMA and the change from FLATMEM to DISCONTIGMEM ... not much that would crash umount

Oh yes, and i forgot to mention, that the kernel booted properly the first time.

i managed to run lscpu and see all my 12 glorious cores present.
Also ran some compilations etc... then i rebooted to make sure machine was stable, and it got stuck. (And i didn't think of a kernel issue because .. well it just booted with it)

I'm running 32bit kernel.

Code:
buffy /usr/src/linux # diff .config_current .config
388c388
< # CONFIG_X86_BIGSMP is not set
---
> CONFIG_X86_BIGSMP=y
445,446c445,446
< CONFIG_NR_CPUS=8
< # CONFIG_SCHED_SMT is not set
---
> CONFIG_NR_CPUS=16
> CONFIG_SCHED_SMT=y
490c490,496
< CONFIG_ARCH_FLATMEM_ENABLE=y
---
> CONFIG_NUMA=y
> # CONFIG_NUMA_EMU is not set
> CONFIG_NODES_SHIFT=3
> CONFIG_ARCH_HAVE_MEMORY_PRESENT=y
> CONFIG_NEED_NODE_MEMMAP_SIZE=y
> CONFIG_ARCH_DISCONTIGMEM_ENABLE=y
> CONFIG_ARCH_DISCONTIGMEM_DEFAULT=y
495c501,502
< CONFIG_FLATMEM_MANUAL=y
---
> # CONFIG_FLATMEM_MANUAL is not set
> CONFIG_DISCONTIGMEM_MANUAL=y
497c504
< CONFIG_FLATMEM=y
---
> CONFIG_DISCONTIGMEM=y
498a506,507
> CONFIG_NEED_MULTIPLE_NODES=y
> CONFIG_HAVE_MEMORY_PRESENT=y
506a516
> CONFIG_MIGRATION=y
560a571
> CONFIG_USE_PERCPU_NUMA_NODE_ID=y
588a600
> # CONFIG_ACPI_NUMA is not set
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7071

PostPosted: Fri Jul 27, 2018 1:59 pm    Post subject: Reply with quote

Quote:
Symbol: ARCH_DISCONTIGMEM_ENABLE [=n] │
│ Type : boolean │
│ Defined at arch/x86/Kconfig:1562 │
│ Depends on: NUMA [=n] && X86_32 [=y] │


Look at the depends, it mean it could be enable if NUMA is off and X86_32 is on.
so you can't get NUMA and this enable but your kernel do that.

Did you manually edit the .config? oh no, you're so evil :)
Back to top
View user's profile Send private message
zark
n00b
n00b


Joined: 27 Mar 2004
Posts: 74
Location: Belgium

PostPosted: Fri Jul 27, 2018 2:50 pm    Post subject: Reply with quote

no no, used menuconfig :(

i'll have to do some research on kernel configs for multi-core systems.

Hope i can stick to x86 .. switching to 64 is a hassle :(
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5760

PostPosted: Fri Jul 27, 2018 3:30 pm    Post subject: Reply with quote

There's really no reason to use an i686 kernel on an x86-64 CPU, especially one with more cores than 32-bit was designed for, and doing so is probably the source of your problem. Just set CONFIG_X86_64=y and rebuild.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13830

PostPosted: Sat Jul 28, 2018 12:04 am    Post subject: Reply with quote

Is the hang contingent on whether the filesystem was mounted read-only? Does it hang for any unmount or only USB? If only USB, does it fail for every USB mass storage device or only specific ones? Why do you say that this hangs the system, rather than hanging the umount? Does the kernel panic on the umount?
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 668

PostPosted: Sat Jul 28, 2018 8:26 am    Post subject: Reply with quote

zark wrote:
...
Basically same kernel 4.9.6-r1
...
I think current stable kernel is sys-kernel/gentoo-sources-4.14.52; before that it was sys-kernel/gentoo-sources-4.9.95, so maybe you're missing some relevant patches/upgrades.
_________________
Greybeard
Back to top
View user's profile Send private message
zark
n00b
n00b


Joined: 27 Mar 2004
Posts: 74
Location: Belgium

PostPosted: Sun Jul 29, 2018 8:50 am    Post subject: Reply with quote

Ant P. wrote:
There's really no reason to use an i686 kernel on an x86-64 CPU, especially one with more cores than 32-bit was designed for, and doing so is probably the source of your problem. Just set CONFIG_X86_64=y and rebuild.


Other than switching a live system to x64 is not as trivial as booting with an x64 kernel unfortunately.
But you are probably right , i should see how i can migrate with minimal downtime.
Back to top
View user's profile Send private message
zark
n00b
n00b


Joined: 27 Mar 2004
Posts: 74
Location: Belgium

PostPosted: Sun Jul 29, 2018 8:53 am    Post subject: Reply with quote

Hu wrote:
Is the hang contingent on whether the filesystem was mounted read-only? Does it hang for any unmount or only USB? If only USB, does it fail for every USB mass storage device or only specific ones? Why do you say that this hangs the system, rather than hanging the umount? Does the kernel panic on the umount?


Yeah i first noticed the freeze when trying to boot normally with "bootmisc" freezeing after unmounting a tmpfs.
I confirmed it by booting in Single User Mode and unmounting anything.

But as @Goverp pointed out, kernel is pretty old (and buggy AF). I've built a newer kernel and will try again before trying to switch to x64

but the odd thing is that kernel booted once without issue, i even compiled a dozen packages before saying to myself i'll do a reboot to make sure everything boot smoothly before leaving the datacenter... and then i get those umount freezes everytime.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7071

PostPosted: Sun Jul 29, 2018 11:27 am    Post subject: Reply with quote

you could still build a 64bits kernel and run 32bits userlands with it as your cpu is 64bits capable (the memory handling is done by the kernel without PAE need so, but the tools use 32bits limit as per CHOST=i686... set in your make.conf).
or build again a 32bits one, but avoid going down strange options (on my kernel 4.14.53 the NUMA do disable ARCH_DISCONTIGMEM_ENABLE as you see, so at least get a newer one where they have fix such issues.
or switch to 64bits userlands (need migration)

Quote:
the odd thing is that kernel booted once without issue

The odd thing in oddity is that they are odd :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum