Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Linking everything with jemalloc at build time
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
s_j_newbury
n00b
n00b


Joined: 05 Apr 2007
Posts: 55
Location: UK

PostPosted: Wed Dec 21, 2016 3:22 pm    Post subject: Linking everything with jemalloc at build time Reply with quote

I don't know if anybody else has tried this, but I've been emerging my systems lately with jemalloc linked in unconditionally, not using any LD_PRELOAD tricks.

I've achieved this through LIBS+="-Wl,-no-as-needed -ljemalloc -Wl,-as-needed" or LDFLAGS as needed. The reason for the no-as-needed/as-needed is otherwise the Gentoo system-wide default of "as-needed" results in jemalloc being dropped since glibc meets the symbol resolution requirements. Some packages had to be patched not to complain about using non-library flags in LIBS, this is a little contraversial since it's generally accepted that LIBS should only contain libraries, and LDFLAGS link flags, but the implementation of as-needed is per library, that is it applies to all following libraries on the link command line unless subsequently toggled, the only of doing this is to allow as-needed in LIBS (or add the lib to LDFLAGS).

The only real problem package is www-client/chromium which segfaults if you try this, so don't! Solution there would be to add jemalloc as a malloc option in the Chromium codebase since it already makes use of the malloc hooks internally.

World emerged this way results in a system with much better memory utililisation/performance characteristics than with the default glibc malloc. Well, that's what the Kool Aid tastes like anyway! ;-)
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Dec 26, 2016 6:41 pm    Post subject: Reply with quote

Interesting approach !

I'm running chromium with explicitly disabled tcmalloc and ld_preload jemalloc,

so how's the performance or latency impact when running it on a system-wide basis ?
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Gatak
Apprentice
Apprentice


Joined: 04 Jan 2004
Posts: 168

PostPosted: Tue Nov 26, 2019 11:40 pm    Post subject: Reply with quote

Really nice! I've been using the LD_PRELOAD trick on things like Apache, php, etc with good effect. I'll try your method now instead :)

I haven't seen the use of "LIBS". What is the difference against using LDFLAGS?
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6354

PostPosted: Wed Nov 27, 2019 5:19 am    Post subject: Reply with quote

Gatak wrote:
I haven't seen the use of "LIBS". What is the difference against using LDFLAGS?

LDFLAGS and LIBS both get picked up automagically by GNU Makefile default rules, and anything that tries to be compatible. The LDFLAGS string is inserted before the source files and LIBS comes after them.

--as-needed and -l flags are actually position-sensitive, which is why this hack works.
Back to top
View user's profile Send private message
Gatak
Apprentice
Apprentice


Joined: 04 Jan 2004
Posts: 168

PostPosted: Wed Nov 27, 2019 11:58 am    Post subject: Reply with quote

I see. Thanks. Anyone that can share some experience in using jemalloc systemwide? For now I have enabled it during compile of specific apps. Was thinking to use package.env in the future instead of force linking to jemalloc on all applications. Do all applications use malloc anyway?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum