Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
/sys and /proc get unmounted during boot
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
oleid
n00b
n00b


Joined: 05 Dec 2012
Posts: 10

PostPosted: Tue Feb 03, 2015 8:18 pm    Post subject: /sys and /proc get unmounted during boot Reply with quote

Dear *,

I'm trying to boot gentoo/systemd via NFS as rootfs, when I stumble across the following problem: It seems, as if at some point /proc, /sys and /dev get unmounted.

This is best observable when booting to a emergency shell (using the keyword emergency at the kernel command line) and repeatedly executing stuff like:

mount
ls /sys
ls /proc

in the very rescue shell. At some point proc and sys are empty and thus mount doesn't show any mountpoints ( /etc/mtab -> ../proc/self/mounts ).

I'm not sure, if this is in *any* way related to systemd. It could also be some kernel problem, I really don't know.


The system is mostly amd64, the kernel being a notable exception (gentoo-sources-3.18.5).
I used the default config file from ArchLinux (plus the additional stuff I get asked due to make oldconfig) and thus I used the same kernel version which they are using.

Debugging ( I boot with: debug initcall_debug udev.log_priority=8 ignore_loglevel) sheds no light onto this situation. There is no suspicious output before the problem and no additional output after the unmount.

So my question:

Did *anybody* notice an unmounting of /proc and /sys on your systems? What might cause such an unmount?



Edit:

I should add, that I installed a numbercrunching cluster, whose nodes boot via NFS as rootfs, using gentoo and systemd about a year ago. So it's definitively possible. But it uses and older kernel and an older systemd version. I might try the older systemd version later on, I just realized it's still in the tree. At the moment, I'm copying the installation from the file server to one of the clients to test local boot.

Edit2:

Okay, booting the copy of the image from a HD works fine.

Edit3:

systemd-215-r3 (the same version as on the working cluster nodes mentioned in the first edit) doesn't do the trick. Strange world.

Best,
Olaf
Back to top
View user's profile Send private message
DONAHUE
Watchman
Watchman


Joined: 09 Dec 2006
Posts: 7557
Location: Goose Creek SC

PostPosted: Wed Feb 04, 2015 12:19 am    Post subject: Reply with quote

Do you specify a "init=/usr/lib64/systemd/systemd" eqivalent appropriate for your NFS (after the kernel has loaded) in the kernel cmdline built into the kernel or the bootloader? kernel config has?
Code:
grep -i systemd /usr/src/linux/.config
Quote:
CONFIG_GENTOO_LINUX_INIT_SYSTEMD=y
Supposedly just provided to autoselect systemd support but might affect functionality if unselected.?.
_________________
Defund the FCC.
Back to top
View user's profile Send private message
oleid
n00b
n00b


Joined: 05 Dec 2012
Posts: 10

PostPosted: Wed Feb 04, 2015 10:42 am    Post subject: Reply with quote

DONAHUE wrote:
Do you specify a "init=/usr/lib64/systemd/systemd" eqivalent appropriate for your NFS (after the kernel has loaded) in the kernel cmdline built into the kernel or the bootloader? kernel config has?
Code:
grep -i systemd /usr/src/linux/.config
Quote:
CONFIG_GENTOO_LINUX_INIT_SYSTEMD=y
Supposedly just provided to autoselect systemd support but might affect functionality if unselected.?.


Yes to both. The system boots just fine when not using NFS as rootfs, but e.g. btrfs. Even when mounting btrfs ro.

The initramfs is quite simple. I use archlinux's mkinitcpio which did the job well for me for the other server (in contrast to dracut or genkernel). The code paths are nearly the same for NFS and local mount.

I brought the topic to systemd-devel mailing list[1]. They suggested to strace all the running stuff. I'll do that this afternoon.


[1] http://lists.freedesktop.org/archives/systemd-devel/2015-February/027927.html
Back to top
View user's profile Send private message
LubosD
Apprentice
Apprentice


Joined: 30 Jun 2005
Posts: 210
Location: Czech Republic

PostPosted: Thu Mar 19, 2015 5:52 pm    Post subject: Reply with quote

I am seeing an identical issue when using nfsroot and OpenRC. I know for *sure* that procfs is mounted at boot time. Minutes later, while using the system, it suddenly disappears (unmounts itself) and applications start complaining.
Back to top
View user's profile Send private message
oleid
n00b
n00b


Joined: 05 Dec 2012
Posts: 10

PostPosted: Thu Mar 19, 2015 6:17 pm    Post subject: Reply with quote

What kernel are you using? For me, the problem appeared on gentoo-sources-3.18, going back to 3.17 worked around the problem.

Oh, by the way, I could roule out systemd as source of the problem, as PID1 and children didn't yield any syscalls when the unmount disappeared. There is more information on what I tested in the email thread.
Back to top
View user's profile Send private message
LubosD
Apprentice
Apprentice


Joined: 30 Jun 2005
Posts: 210
Location: Czech Republic

PostPosted: Thu Mar 19, 2015 7:09 pm    Post subject: Reply with quote

oleid wrote:
What kernel are you using? For me, the problem appeared on gentoo-sources-3.18, going back to 3.17 worked around the problem.

I'm currently running 3.19.1. I was previously running 3.3.5 and it worked just fine.
Back to top
View user's profile Send private message
LubosD
Apprentice
Apprentice


Joined: 30 Jun 2005
Posts: 210
Location: Czech Republic

PostPosted: Wed Sep 09, 2015 8:40 am    Post subject: Reply with quote

I have now filed a kernel bug: https://bugzilla.kernel.org/show_bug.cgi?id=104081

I'll also try to find the problematic commit.
Back to top
View user's profile Send private message
LubosD
Apprentice
Apprentice


Joined: 30 Jun 2005
Posts: 210
Location: Czech Republic

PostPosted: Wed Sep 09, 2015 8:53 am    Post subject: Reply with quote

It seems that this commit fixes the problem:

Code:

commit 6c441c254eea2354d686be7f5544bcd79fb6a61f
Author: Trond Myklebust <trond.myklebust@primarydata.com>
Date:   Sun Feb 22 16:35:36 2015 -0500

    NFS: Don't invalidate a submounted dentry in nfs_prime_dcache()
   
    If we're traversing a directory which contains a submounted filesystem,
    or one that has a referral, the NFS server that is processing the READDIR
    request will often return information for the underlying (mounted-on)
    directory. It may, or may not, also return filehandle information.
   
    If this happens, and the lookup in nfs_prime_dcache() returns the
    dentry for the submounted directory, the filehandle comparison will
    fail, and we call d_invalidate(). Post-commit 8ed936b5671bf
    ("vfs: Lazily remove mounts on unlinked files and directories."), this
    means the entire subtree is unmounted.
   
    The following minimal patch addresses this problem by punting on
    the invalidation if there is a submount.
   
    Kudos to Neil Brown <neilb@suse.de> for having tracked down this
    issue (see link).
   
    Reported-by: Nix <nix@esperi.org.uk>
    Link: http://lkml.kernel.org/r/87iofju9ht.fsf@spindle.srvr.nix
    Cc: stable@vger.kernel.org # 3.18+
    Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum