Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
nfsd: sent only ... shutting down socket
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
musv
Advocate
Advocate


Joined: 01 Dec 2002
Posts: 3254
Location: de

PostPosted: Mon Sep 18, 2017 1:32 pm    Post subject: nfsd: sent only ... shutting down socket Reply with quote

Found this thread today. Since the last kernel update I'm suffering this problem too. But the circumstances are quite different.

fredbear5150 wrote:
It does seem to be during times of "reasonably" heavy loads

Mostly yes. But the last freeze occurred without any noticeable load.

journalctl -ar:
Sep 18 15:16:53 nas systemd-logind[174]: System is powering down.
Sep 18 15:16:53 nas systemd-logind[174]: Powering Off...
Sep 18 15:16:53 nas systemd-logind[174]: Power key pressed.
Sep 18 15:16:19 nas kernel: sysrq: SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memor>
Sep 18 15:16:19 nas kernel: sysrq: SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memor>
Sep 18 15:16:14 nas kernel: sysrq: SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memor>
Sep 18 15:15:35 nas kernel: rpc-srv/tcp: nfsd: sent only 108 when sending 32876 bytes - shutting down socket


At least the powerbutton still works to bring the system down in a clean way.

Now the difference:
uname -a:
Linux nas 4.12.8-gentoo #3 SMP Fri Aug 25 11:28:33 CEST 2017 armv7l ARMv7 Processor rev 1 (v7l) Marvell Armada 380/385 (Device Tree) GNU/Linux


It's a Zyxel NSA326. So, surely it's not related to Intel or AMD IOMMU.

Any ideas? With the previous kernel, this freezes never occurred. So I think, the problem is related to a special kernel option or the changed device tree.


EDIT: Added "this thread" link and split from that topic. Due to thread age and differences, a new thread is more useful. Plus, having 0 replies in this thread will likely get some additional attention. --pjp


Last edited by musv on Wed Sep 27, 2017 7:50 pm; edited 1 time in total
Back to top
View user's profile Send private message
musv
Advocate
Advocate


Joined: 01 Dec 2002
Posts: 3254
Location: de

PostPosted: Mon Sep 18, 2017 4:27 pm    Post subject: Re: System freeze since the last kernel update Reply with quote

I'm a step forward:

musv wrote:
journalctl -ar:
Sep 18 15:15:35 nas kernel: rpc-srv/tcp: nfsd: sent only 108 when sending 32876 bytes - shutting down socket


This seems to be the source of the problems. Found some older bugs from Redhat. But they did use nfs-utils-1.x and something else from Ubuntu.

Strange.
Back to top
View user's profile Send private message
musv
Advocate
Advocate


Joined: 01 Dec 2002
Posts: 3254
Location: de

PostPosted: Wed Sep 27, 2017 4:50 pm    Post subject: Reply with quote

Some more investigation. Got again:

Code:
Sep 27 17:35:37 nas kernel: rpc-srv/tcp: nfsd: sent only 36488 when sending 65644 bytes - shutting down socket


This time the network kept alive. So I could restart the nfs-server. Redhat has an entry to this topic: https://access.redhat.com/solutions/28104. They recommend to update to nfs-utils-1.0.9 or higher. Ok regarding to the fact, Enterprise Linux is always 5 years behind, this solution isn't really useful:
eix -e nfs-utils:

[I] net-fs/nfs-utils
     Verfügbare Versionen:   1.3.1-r5^t 1.3.4-r1^t (~)2.1.1-r1^t **2.1.2_rc3^t **2.1.2_rc5^t {caps ipv6 kerberos +libmount nfsdcld +nfsidmap +nfsv4 nfsv41 selinux tcpd +uuid}
     Installierte Versionen: 2.1.1-r1^t(12:58:52 25.05.2017)(caps libmount nfsidmap nfsv4 nfsv41 tcpd uuid -ipv6 -kerberos -nfsdcld -selinux)

At least version 2.1.1 was already installed in May of this year. And the problems started only after the last kernel upgrade from 4.10.10 to 4.12.8 and continue with 4.13.2.

The next proposal is to adjust the read buffer values, which are set by the system to:
Code:
cat /proc/sys/net/core/rmem_default
163840
cat /proc/sys/net/core/rmem_max
163840

I don't have any idea, if 160 kb is to low or to high.

My kernel config regarding NFS:
zgrep -i nfs /proc/config.gz:

CONFIG_KERNFS=y
CONFIG_NFS_FS=y
# CONFIG_NFS_V2 is not set
# CONFIG_NFS_V3 is not set
CONFIG_NFS_V4=y
# CONFIG_NFS_SWAP is not set
CONFIG_NFS_V4_1=y
CONFIG_NFS_V4_2=y
CONFIG_PNFS_FILE_LAYOUT=y
CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org"
# CONFIG_NFS_V4_1_MIGRATION is not set
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
# CONFIG_NFSD_BLOCKLAYOUT is not set
# CONFIG_NFSD_SCSILAYOUT is not set
# CONFIG_NFSD_FLEXFILELAYOUT is not set
# CONFIG_NFSD_FAULT_INJECTION is not set
CONFIG_NFS_COMMON=y
Back to top
View user's profile Send private message
musv
Advocate
Advocate


Joined: 01 Dec 2002
Posts: 3254
Location: de

PostPosted: Wed Sep 27, 2017 8:00 pm    Post subject: Reply with quote

Update:

It seems, as I've at least identified the problem part. Due to the fact, that this error only occurs on my Zyxel NAS it has something to do with this special configuration.

I changed the device tree file (dts) to the previous version and so far there are no error messages anymore. I'll investigate this some time further.

works:
cat /usr/src/linux/arch/arm/boot/dts/armada-380-zyxel-nas326.dts:


      internal-regs {
         ethernet@70000 {
            status = "okay";
            phy-mode = "rgmii";
                mac-address = [ 1a 07 a8 2c 04 76 ];
            fixed-link {
               speed = <1000>;
               full-duplex;
            };
         };



doesn't work:
cat /usr/src/linux/arch/arm/boot/dts/armada-380-zyxel-nas326.dts:


      internal-regs {
         ethernet@70000 {
            pinctrl-0 = <&ge0_rgmii_pins>;
            pinctrl-names = "default";
            status = "okay";
            phy = <&phy1>;
            phy-mode = "rgmii-id";
                mac-address = [ 1a 07 a8 2c 04 76 ];
         };


But I guess, this is to specific to describe this problem in this forum. We've hacked this device in this thread. So it's better to continue there.
Back to top
View user's profile Send private message
musv
Advocate
Advocate


Joined: 01 Dec 2002
Posts: 3254
Location: de

PostPosted: Tue Oct 31, 2017 2:41 pm    Post subject: Reply with quote

Wrote a bug-report to the developers of the mvneta driver:

https://www.spinics.net/lists/netdev/msg461420.html
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum