Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
TWEAK:Trim unused exported kernel symbols IF extern modules
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Sat Jan 11, 2020 3:05 pm    Post subject: TWEAK:Trim unused exported kernel symbols IF extern modules Reply with quote

IMHO interesting post for those with kernel external modules whishing to shrink kernel.
Likely not for the faint-hearted.
Thks 4 ur attention.


Last edited by CaptainBlood on Tue Mar 31, 2020 9:04 am; edited 6 times in total
Back to top
View user's profile Send private message
Ionen
Guru
Guru


Joined: 06 Dec 2018
Posts: 538

PostPosted: Sat Jan 11, 2020 4:54 pm    Post subject: Reply with quote

Interesting, it's something I've thought about before (it's an option I wanted to enable given my minimalist kernel), but then I'm (also) stuck with using nvidia proprietary.

Doesn't sound like it'd be hard to integrate in my build scripts and automate, may set it up later. Thanks for bringing it up :)

Formerly I did want to just build in nvidia (was the only modules I had at all, then I could disable module support), but lately I'm happy with them being modules because it's easier to unload them for GPU passthrough (on that note, if I do the trim, I guess I'll be getting even more warnings from portage which _really_ wants me to enable CONFIG_ZONE_DMA for nvidia drivers, but no! I'm using IOMMU, no thanks 8)).
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Sat Jan 11, 2020 5:25 pm    Post subject: Reply with quote

Cdemu stack kernel integration issued the concern here.

No Nvidia, so nothing to share in this regard.
i965 instead. Maybe not so out of scope....
Haven't figured that out, yet.

Thks 4 ur attention, interest & support.
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Fri Mar 27, 2020 5:52 pm    Post subject: Reply with quote

Code:
-rw-r--r--  1 root root 7987760 mars  27 20:11 vmlinuz-5.4.28-gento
-rw-r--r--  1 root root 8110640 mars  27 15:14 vmlinuz-5.4.28-gentoo.old
Code:
lsmod
Module                  Size  Used by
snd_aloop              28672  3
vhba                   20480  0 # <=external module
pcspkr                 16384  0
Thks 4 ur attention.

Last edited by CaptainBlood on Sat Mar 28, 2020 11:51 pm; edited 2 times in total
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Sat Mar 28, 2020 9:16 pm    Post subject: Reply with quote

Code:
-rw-r--r-- 1 root root 7352368 mars  29 00:12 /boot/vmlinuz-4.19.97-gentoo-classic
-rw-r--r-- 1 root root 7467056 mars  29 00:11 /boot/vmlinuz-4.19.97-gentoo-classic.old
Code:
lsmod
Module                  Size  Used by
vhba                   20480  0 # <=external module
snd_aloop              24576  3
pcspkr                 16384  0
Thks 4 ur attention.
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Mon Mar 30, 2020 12:43 am    Post subject: Reply with quote

A couple of scripts & files for those willing to try an 'easier way':
Calling script:
Code:
ls /usr/local/sbin/kermak-extern-ksym -la
-rwxr----- 1 root root 2776 mars  30 01:03 /usr/local/sbin/kermak-extern-ksym
Code:
#!/bin/bash
#parameters
#@1 mandatory kernel version
#@2 mandatory config version

if [[ -z $2 && ! -d /usr/src/linux-$1 ]] || [[ ! -z $2 && ! -d /usr/src/linux-$1-$2 ]]; then
    echo syntax: $0 \'kernel version\', mandatory, [\'config version\'], optional
    echo \'kernel version\', mandatory e.g. 5.5.13 or 5.5.13-gentoo for sys-kernel/gentoo-sources package class user
    echo "[\'config version\'], optional e.g. classic as here kernel is configured in /usr/src/linux-5.5.13-gentoo-classic/]"
    exit
    #    echo $1 $2 #debug
else
    krn_pth=$1
    if [[ ! -z $2 ]]; then
        krn_pth+=-$2
    fi
    dst_dir=/usr/src/linux-$krn_pth
    if [[ 1 -ne $( grep -c "# CONFIG_TRIM_UNUSED_KSYMS is not set" $dst_dir/.config ) ]]; then
        echo Kernel already configured for unused ksyms trimming, cancelling...
        unset dst_dir
    else
        cd $dst_dir
        unset dst_dir

        path_to_kernel_unused_ksyms_trim_patch=/home/luc/src/git/kernel-trim-external-modules/
        kernel_unused_ksyms_trim_patch_filename=/support-trim-unused-ksyms-for-external-modules.patch

        krn_vrs_nmb=$( echo $1 | sed -En "s/([[:digit:]].[[:digit:]]*.[[:digit:]]*)(.*)/\1/p" )
       
        krn_vrs_nmb_suffix=.x
        full_path_to_kernel_unused_ksyms_trim_patch=$path_to_kernel_unused_ksyms_trim_patch$krn_vrs_nmb$kernel_unused_ksyms_trim_patch_filename
        while [[ $krn_vrs_nmb =~ \. ]] && [[ ! -f  $full_path_to_kernel_unused_ksyms_trim_patch ]]
        do
            krn_vrs_nmb=${krn_vrs_nmb%.*}
            full_path_to_kernel_unused_ksyms_trim_patch=$path_to_kernel_unused_ksyms_trim_patch$krn_vrs_nmb$krn_vrs_nmb_suffix$kernel_unused_ksyms_trim_patch_filename
        done
        unset path_to_kernel_unused_ksyms_trim_patch
        unset kernel_unused_ksyms_trim_patch_filename
        unset krn_vrs_nmb
        unset krn_vrs_nmb_suffix
       
        if [[ ! -f $full_path_to_kernel_unused_ksyms_trim_patch ]]; then
            echo patch file $full_path_to_kernel_unused_ksyms_trim_patch not found, cancelling...
        else
            extern-ksym-2 $krn_pth > include/generated/autoksyms.h.extern
            unset krn_pth
            cat include/generated/autoksyms.h.extern

            patch --dry-run -p1 < $full_path_to_kernel_unused_ksyms_trim_patch
            if  [[ 0 -ne $? ]]; then
                echo patch testing failed, cancelling...
            else
                patch -p1 < $full_path_to_kernel_unused_ksyms_trim_patch
               
                # manual setting
                #make nconfig # enable CONFIG_TRIM_UNUSED_KSYMS
                #make menuconfig
               
                # automatic setting
                sed -i 's/# CONFIG_TRIM_UNUSED_KSYMS is not set/CONFIG_TRIM_UNUSED_KSYMS=y/g' .config
                make
                make install
                sed -i 's/CONFIG_TRIM_UNUSED_KSYMS=y/# CONFIG_TRIM_UNUSED_KSYMS is not set/g' .config
                patch -p1 -R < $full_path_to_kernel_unused_ksyms_trim_patch
            fi
            rm include/generated/autoksyms.h.extern
        fi
        unset full_path_to_kernel_unused_ksyms_trim_patch
    fi
fi
Called script, renamed because of slight change with original, check 1st post.
Code:
ls /usr/local/sbin/extern-ksym-2 -la
-rwxr----- 1 root root 1331 mars   1 11:43 /usr/local/sbin/extern-ksym-2
Code:
cat /usr/local/sbin/extern-ksym-2
#!/bin/bash
#parameter: $1
list_required_ksyms() {
    case ${1##*.} in
        ko)
            cmd="nm $1"
            ;;
        gz)
            gunzip -c $1>a.out
            cmd="nm"
            ;;
        xz)
            unxz -c $1>a.out
            cmd="nm"
            ;;
        *)
            exit
            ;;
    esac
    eval $cmd |
        sed -n 's/^ \+U //p' |
        sed -ns -e '{s/ /\n/g;/^$/!p;}' |
        sort -u |
        while read sym; do
            if [[ -n "$CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX" ]]; then
                sym="${sym#_}"
            fi
            echo "#define __KSYM_${sym} 1"
        done
}

if [[ linux-$1 != $(basename $PWD) ]]; then # test current directory matches kernel version[-config directory name']
    echo "syntax: $0 'kernel version'[-'config version']"
else
    # calling matching kernel script
    . ./include/config/auto.conf
   
    # Find all external modules
    mod_dir_list=$(find /lib/modules/$1 -mindepth 1 -maxdepth 1 -type d  \! -name 'kernel' | xargs)
    for mod_dir in $mod_dir_list; do
        for ko in ${mod_dir}/*.ko*; do
            list_required_ksyms ${ko}
        done
    done | sort -u
    [[ -f ./a.out ]] && rm ./a.out
fi
patch tree:
Code:
tree /home/user/src/git/kernel-trim-external-modules/
/home/user/src/git/kernel-trim-external-modules/
├── 4.14.x
│   └── support-trim-unused-ksyms-for-external-modules.patch
├── 4.19.x
│   └── support-trim-unused-ksyms-for-external-modules.patch
└── 5.x
    └── support-trim-unused-ksyms-for-external-modules.patch
Code:
cat /home/user/src/git/kernel-trim-external-modules/4.14.x/support-trim-unused-ksyms-for-external-modules.patch
--- ./scripts/adjust_autoksyms.sh.ori   2020-03-16 07:48:55.196492298 +0300
+++ ./scripts/adjust_autoksyms.sh   2020-03-29 01:50:56.150936670 +0300
@@ -22,6 +22,7 @@
 
 cur_ksyms_file="include/generated/autoksyms.h"
 new_ksyms_file="include/generated/autoksyms.h.tmpnew"
+ext_ksyms_file="include/generated/autoksyms.h.extern"
 
 info() {
    if [ "$quiet" != "silent_" ]; then
@@ -66,7 +67,11 @@
       sym="${sym#_}"
    fi
    echo "#define __KSYM_${sym} 1"
-done >> "$new_ksyms_file"
+done >> "$new_ksyms_file".tmp
+
+# Merge external and new ksym files
+sort -u "$new_ksyms_file".tmp "$ext_ksyms_file" >> "$new_ksyms_file"
+rm -f "$new_ksyms_file".tmp
 
 # Special case for modversions (see modpost.c)
 if [ -n "$CONFIG_MODVERSIONS" ]; then
@@ -99,7 +104,8 @@
    # Replace the old list with tne new one
    old=$(grep -c "^#define __KSYM_" "$cur_ksyms_file" || true)
    new=$(grep -c "^#define __KSYM_" "$new_ksyms_file" || true)
-   info "KSYMS" "symbols: before=$old, after=$new, changed=$changed"
+    ext=$(grep -c "^#define __KSYM_" "$ext_ksyms_file" || true)
+    info "KSYMS" "symbols: before=$old, after=$new, external=$ext, changed=$changed"
    info "UPD" "$cur_ksyms_file"
    mv -f "$new_ksyms_file" "$cur_ksyms_file"
    # Then trigger a rebuild of affected source files
Code:
cat /home/user/src/git/kernel-trim-external-modules/4.19.x/support-trim-unused-ksyms-for-external-modules.patch
--- ./scripts/adjust_autoksyms.sh.ori   2018-10-22 09:37:37.000000000 +0300
+++ ./scripts/adjust_autoksyms.sh   2020-03-28 23:09:34.933620716 +0300
@@ -22,6 +22,7 @@
 
 cur_ksyms_file="include/generated/autoksyms.h"
 new_ksyms_file="include/generated/autoksyms.h.tmpnew"
+ext_ksyms_file="include/generated/autoksyms.h.extern"
 
 info() {
    if [ "$quiet" != "silent_" ]; then
@@ -62,7 +63,11 @@
 done | sort -u |
 while read sym; do
    echo "#define __KSYM_${sym} 1"
-done >> "$new_ksyms_file"
+done >> "$new_ksyms_file".tmp
+
+# Merge external and new ksym files
+sort -u "$new_ksyms_file".tmp "$ext_ksyms_file" >> "$new_ksyms_file"
+rm -f "$new_ksyms_file".tmp
 
 # Special case for modversions (see modpost.c)
 if [ -n "$CONFIG_MODVERSIONS" ]; then
@@ -95,7 +100,8 @@
    # Replace the old list with tne new one
    old=$(grep -c "^#define __KSYM_" "$cur_ksyms_file" || true)
    new=$(grep -c "^#define __KSYM_" "$new_ksyms_file" || true)
-   info "KSYMS" "symbols: before=$old, after=$new, changed=$changed"
+    ext=$(grep -c "^#define __KSYM_" "$ext_ksyms_file" || true)
+    info "KSYMS" "symbols: before=$old, after=$new, external=$ext, changed=$changed"
    info "UPD" "$cur_ksyms_file"
    mv -f "$new_ksyms_file" "$cur_ksyms_file"
    # Then trigger a rebuild of affected source files
Code:
cat /home/user/src/git/kernel-trim-external-modules/5.x/support-trim-unused-ksyms-for-external-modules.patch
--- ./scripts/adjust_autoksyms.sh.ori   2019-11-25 03:32:01.000000000 +0300
+++ ./scripts/adjust_autoksyms.sh   2020-03-27 19:54:21.572733145 +0300
@@ -19,6 +19,7 @@
 
 cur_ksyms_file="include/generated/autoksyms.h"
 new_ksyms_file="include/generated/autoksyms.h.tmpnew"
+ext_ksyms_file="include/generated/autoksyms.h.extern"
 
 info() {
    if [ "$quiet" != "silent_" ]; then
@@ -49,7 +50,11 @@
 sed 's/ko$/mod/' modules.order |
 xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |
 sort -u |
-sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"
+sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file".tmp
+
+# Merge external and new ksym files
+sort -u "$new_ksyms_file".tmp "$ext_ksyms_file" >> "$new_ksyms_file"
+rm -f "$new_ksyms_file".tmp
 
 # Special case for modversions (see modpost.c)
 if [ -n "$CONFIG_MODVERSIONS" ]; then
@@ -82,7 +87,8 @@
    # Replace the old list with tne new one
    old=$(grep -c "^#define __KSYM_" "$cur_ksyms_file" || true)
    new=$(grep -c "^#define __KSYM_" "$new_ksyms_file" || true)
-   info "KSYMS" "symbols: before=$old, after=$new, changed=$changed"
+   ext=$(grep -c "^#define __KSYM_" "$ext_ksyms_file" || true)
+    info "KSYMS" "symbols: before=$old, after=$new, external=$ext, changed=$changed"
    info "UPD" "$cur_ksyms_file"
    mv -f "$new_ksyms_file" "$cur_ksyms_file"
    # Then trigger a rebuild of affected source files
All three patches have been tested, with xzipped modules here.

Hoping it's good enough for some of you to try & succeed.

Thks 4 ur attention, interest and support


Last edited by CaptainBlood on Wed Apr 01, 2020 1:03 pm; edited 2 times in total
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Mon Mar 30, 2020 2:23 am    Post subject: Reply with quote

Fixed & tested with single parameter.
Thks 4 ur attention, interest & support
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Mon Mar 30, 2020 10:24 pm    Post subject: Reply with quote

Fine with 5.6.0 here, without any change.
Code:
uname -r
5.6.0-gentoo-classic
Code:
ls /boot/vmlinuz-5.6.0-gentoo-classic* -ll
-rw-r--r-- 1 root root 8153136 mars  31 01:43 /boot/vmlinuz-5.6.0-gentoo-classic
-rw-r--r-- 1 root root 8300592 mars  31 01:12 /boot/vmlinuz-5.6.0-gentoo-classic.old
Code:
lsmod
Module                  Size  Used by
snd_aloop              32768  2
vhba                   20480  0 # <= external module
pcspkr                 16384  0
Thks 4 ur attention, interest & support.
Back to top
View user's profile Send private message
CaptainBlood
Veteran
Veteran


Joined: 24 Jan 2010
Posts: 1442

PostPosted: Wed Apr 01, 2020 1:05 pm    Post subject: Reply with quote

Dry-run patch test & cancellation added in calling script, i.e. updated above.
NOK and OK cases verified.
Thks 4 ur attention, interest & support.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum