Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Traffic shaping with Wondershaper
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Sat Nov 13, 2004 4:40 pm    Post subject: Traffic shaping with Wondershaper Reply with quote

"Howdy"...

I thought about writing a guide to Wondershaper, for those who do not know how to use/install it.

---

What is Wondershaper?

Wondershaper is a script for bandwidth control for Linux. If you've ever downloaded lots of "backed up" games and similar, while uploding other stuff to a friend via xDSL, you've probably noticed that the upstream chokes the downstream. What Wondershaper does is to fix so that the connection is almost maximized in both up and downstream.

Though notice: I have not written the original script, I've only written the rc script and this guide. For information about Wondershaper, see http://lartc.org/wondershaper/ ...

How does the script work?

What this script does is to prioritize incomming ACK packages while throwing packages that comes in too fast. The result is a bit slower upstream but a way faster downstream. This is achieved by something called QoS (Quality of Service). The reason why the upstream chokes the downstream is written in the Wondershaper documentation for those who are interrested.

What's needed?

This guide is first and foremost written for Gentoo Linux, so I recommend you to have Gentoo, but it works just fine with other distributions. Although other distributions handles rc scripts differently, so I recommend you to consult your distribution's forum and/or documentation for further information about rc script.

Besides that you need Linux (see above), you also need a Linux 2.4.x or 2.6.x kernel with QoS and iptables support. An Internet connection (*doh*) and the knowledge of how to compile your own kernel. You also need to know your own connection's theoretical up and downstream.

Step 1:

Configure the kernel so that it has support for QoS (if you don't already have it)...

Code:

$ su -l
Type your root password

$ cd /usr/src/linux
$ make menuconfig


For Linux 2.6.x:

Device Drivers -> Networking support -> Networking options -> QoS and/or fair queueing

Code:

[*] QoS and/or fair queueing
      Packet scheduler clock source (Timer interrupt)  --->
<M>   CBQ packet scheduler
<M>   HTB packet scheduler
<M>   HFSC packet scheduler
<M>   The simplest PRIO pseudoscheduler
<M>   RED queue
<M>   SFQ queue
<M>   TEQL queue
<M>   TBF queue
<M>   GRED queue
<M>   Diffserv field marker
<M>   Network emulator
<M>   Ingress Qdisc
[*]   QoS support
[*]     Rate estimator
[*]   Packet classifier API
<M>     TC index classifier
<M>     Routing table based classifier
<M> Firewall based classifier
<M> U32 classifier
[ ]   U32 classifier performance counters
[ ] classify input device (slows things u32/fw)
<M> Special RSVP classifier
<M> Special RSVP classifier for IPv6
[ ] Packet ACTION
[*] Traffic policing (needed for in/egress)


Exit the configurating program and make sure that you save your setting.

Code:

$ mount /boot/
(If you run the boot "partition" on a separate partition)
$ make && make install && make modules && make modules_install && modules-update
$ umount /boot/
$ exit


Step 2:

Install iproute2:

Code:

$ sudo emerge iproute2
(Requires you to have sudo and be in the group wheel, if not, do as written below)

$ su -l
(Type your root password)
$ emerge iproute2
$ exit


Step 3:

Download Wondershaper, unpack and configure it:

Code:

$ cd ~/
$ wget http://lartc.org/wondershaper/wondershaper-1.1a.tar.gz
$ tar xvfz wondershaper-1.1a.tar.gz
$ rm wondershaper-1.1a.tar.gz
$ cd wondershaper-1.1a/
$ nano -w wshaper.htb


Edit the following:

Code:

DOWNLINK=
UPLINK=
DEV=


On my computer and connection (8/1 Mbit), it looks like this:

Code:

DOWNLINK=8196
UPLINK=896
DEV=eth0


You might have to lower the values somewhat to maximize the connection speed. You'll have to experiment with the values later on to find those who suit your connection the best.

When you've configured everything, it's time to edit some more. Comment the following:

Code:

echo Please read the documentation in 'README' first
exit


So that it looks like this:

Code:

#echo Please read the documentation in 'README' first
#exit


Save the document (ctrl+o) and exit (ctrl+x).

Step 4:

Write the rc script:

Code:

$ sudo nano -w /etc/init.d/wondershaper


Write the following into the file:

Code:

#!/sbin/runscript
#By Fredrik Blom

depend() {
        need net iptables
}

start() {
        if [ -f /etc/wshaper.htb ]
        then
                ebegin "Setting up Wondershaper"
                /etc/wshaper.htb
        else
                eerror "Unable to start Wondershaper"
        fi
        eend $?
}

stop() {
        if [ -f /etc/wshaper.htb ]
        then
                ebegin "Closing down Wondershaper"
                /etc/wshaper.htb stop
        else
                eerror "Unable to stop (locate) Wondershaper"
        fi
        eend $?
}

restart() {
        svc_stop
        svc_start
        eend $?
}


Save the document (ctrl+o) and exit (ctrl+x), then make it executable:

Code:

$ chmod 0744 /etc/init.d/wondershaper


Step 5:

"Install" the Wondershaper script:

Code:

$ sudo cp wshaper.htb /etc/
$ sudo chown root:root /etc/wshaper.htb
$ sudo chmod 0744 /etc/wshaper.htb


Time to start the script:

Code:

$ sudo /etc/init.d/wondershaper start
$ sudo /etc/init.d/iptables save


To make the script start each time the computer starts:

Code:

$ sudo /sbin/rc-update add wondershaper default


---

"That's all folks!"... :)

Fredrik Blom, 2004

This document is under the "Creative Commons - Attribution / Share Alike" license. ( http://creativecommons.org/licenses/by-sa/2.0/ )
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)


Last edited by Sachankara on Fri Dec 24, 2004 11:17 am; edited 4 times in total
Back to top
View user's profile Send private message
Vanquirius
Retired Dev
Retired Dev


Joined: 14 Jun 2002
Posts: 1297
Location: Ethereal plains

PostPosted: Sat Nov 13, 2004 9:00 pm    Post subject: Reply with quote

In addition to step 4, making the script executable:

Code:
chmod +x /etc/init.d/wondershaper

_________________
Hello.
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Sat Nov 13, 2004 11:23 pm    Post subject: Reply with quote

Vanquirius wrote:
In addition to step 4, making the script executable:

Code:
chmod +x /etc/init.d/wondershaper
Missed it, thanks... :)
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
DocterD
Tux's lil' helper
Tux's lil' helper


Joined: 15 May 2004
Posts: 129

PostPosted: Sun Nov 14, 2004 1:24 am    Post subject: Reply with quote

what do i have to do, to run it with ip-up and down?
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Sun Nov 14, 2004 9:51 am    Post subject: Reply with quote

DocterD wrote:
what do i have to do, to run it with ip-up and down?
How do you mean? Please tell me more... :)

Edit: Ah, now I understand... I'll see what I can do, but unfortunatly I'm still quite new to Linux so I might not be able to figure it out... :/

Edit 2: Perhaps you can just just simply paste the text from ip-down.local just after
Code:
tc qdisc del dev $DEV root    2> /dev/null > /dev/null
tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null
in wshaper.htb, and paste ip-up.local anywhere after
Code:
if [ "$1" = "stop" ]
then
        exit
fi
in the same file?

It isn't the nicest way to do it, but it might work...
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
krazyj
n00b
n00b


Joined: 02 Oct 2004
Posts: 5

PostPosted: Wed Dec 15, 2004 9:57 am    Post subject: Reply with quote

I'm going to start by saying "I MIGHT NOT KNOW WHAT I"M SAYING (still newb)" that said

My init.d start script did not work so I changed it to look like this... would this be right?

Code:


#!/sbin/runscript
#By Fredrik Blom

depend() {
        need net iptables
}

start() {
        if [ -f /etc/wshaper.htb ]
        then
                ebegin "Setting up Wondershaper"
                /etc/wshaper.htb
        else
                eerror "Unable to start Wondershaper"
        fi
        eend $?
}

stop() {
        if [ -f /etc/wshaper.htb ]
        then
                ebegin "Closing down Wondershaper"
                /etc/wshaper.htb stop
        else
                eerror "Unable to stop (locate) Wondershaper"
        fi
        eend $?
}
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Wed Dec 15, 2004 3:36 pm    Post subject: Reply with quote

krazyj wrote:
I'm going to start by saying "I MIGHT NOT KNOW WHAT I"M SAYING (still newb)" that said

My init.d start script did not work so I changed it to look like this... would this be right?

Code:
...
Well, does it work when you added eend? If that's the case, then I'll add it to the guide...

(I didn't include eend since it worked just fine for me and my friends without it...)
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
skunk
l33t
l33t


Joined: 28 May 2003
Posts: 640
Location: granada, spain

PostPosted: Wed Dec 15, 2004 8:41 pm    Post subject: Reply with quote

if you're using shorewall like i do, you can copy the wondershaper script to /etc/shorewall/tcstart, so it will be executed when shorewall starts and you won't need the additional init script...
Back to top
View user's profile Send private message
krazyj
n00b
n00b


Joined: 02 Oct 2004
Posts: 5

PostPosted: Thu Dec 16, 2004 6:17 am    Post subject: Reply with quote

Yep I had to add it for mine to work.



On another note thanks for the small tutorial, it helped
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Thu Dec 16, 2004 9:52 am    Post subject: Reply with quote

krazyj wrote:
Yep I had to add it for mine to work.

On another note thanks for the small tutorial, it helped
No problem... Glad it helped you, and thanks for the "changes". I added them to the "guide"... :)
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Fri Dec 24, 2004 11:18 am    Post subject: Reply with quote

Update: Added a restart function to the wondershaper.sh script... :)
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Sun Jan 02, 2005 12:11 pm    Post subject: Reply with quote

I found that wondershaper does not work correctly without using the modifications detailed here :
http://www.my-opensource.org/howto/qostrafficshaping-shorewall-wondershaper-howto.html

This uses shorewall but I expect you could use iptables directly also.
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Mon Jan 03, 2005 4:31 pm    Post subject: Reply with quote

darklegion wrote:
I found that wondershaper does not work correctly without using the modifications detailed here :
http://www.my-opensource.org/howto/qostrafficshaping-shorewall-wondershaper-howto.html

This uses shorewall but I expect you could use iptables directly also.
What where the problems with my instructions that worked better with those other instructions? Improvements are always welcome... :)
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
Illissius
Guru
Guru


Joined: 31 Jul 2004
Posts: 395
Location: Hungary

PostPosted: Sat Jan 29, 2005 5:54 pm    Post subject: Reply with quote

Is there any way to make it either exempt local addresses from the filtering, or ideally set a different bandwidth threshold for them? I have my music dir shared over smb, and for a while was stumped why music on the clients was skipping like mad -- turned out it was wondershaper limiting the connection to the speed of my 128kbkps internet upload :/.
_________________
Work is punishment for failing to procrastinate effectively.
last.fm
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Sat Jan 29, 2005 9:52 pm    Post subject: Reply with quote

Illissius wrote:
Is there any way to make it either exempt local addresses from the filtering, or ideally set a different bandwidth threshold for them? I have my music dir shared over smb, and for a while was stumped why music on the clients was skipping like mad -- turned out it was wondershaper limiting the connection to the speed of my 128kbkps internet upload :/.
Sounds like you actually need "per-port" speed limiting or something similar. Since Wondershaper is very limited in its functionality, you'll probably have to make some rather big changes to it to make it work the way you want it.

The easiest way is simply to install another NIC and use that one for the internal net instead... Not the best solution, I know... :/
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
thebigslide
l33t
l33t


Joined: 23 Dec 2004
Posts: 790
Location: under a car or on top of a keyboard

PostPosted: Mon Jan 31, 2005 11:40 pm    Post subject: Reply with quote

Try this. Might take some touchups. Only use port numbers, not names. If you pull apart the second for loop, you can setup the other interfaces separately. The script flushes and starts from scratch everytime it's run, so there's no start, stop, restart commands. Just put a reference to it in /etc/conf.d/local.start or some such. If it doesn't work for you, let me know and I'll fix it (tonight).


Code:
#!/bin/sh
# You must have IPROUTE2 installed for this to do anything.
#NFS is trickey unless you lock down portmapper because the ports will be picked dynamically otherwise

TC="/sbin/tc"
INTERFACES="eth0 eth1"
INTERACTIVEPORTS="59 80 443"  #  Don't put anything high load in here
BULKPORTS="666 47 56"   #Put any P2P in here
         #$#$#$#Anything coming from osuosl.org will be sent to the bulk traffic queue since that is where I get all my source from :D.
MODULES=""   #put any modules in this line you'd like loaded at runtime

VNCPORT="1026"
NFSPORTS="400:4000"

#########################################################################
#Don't really mess with anything below unless you know what you're doing#
#########################################################################

for i in $MODULES;
do
  echo "Inserting module $i"
    modprobe $i
done

for IF in $INTERFACES;
do     
        #Flush...
$TC qdisc del dev $IF root
$TC qdisc add dev $IF root handle 1: htb default 10
done

for IF in $INTERFACES;
do     

########SETUP QUEUES########
   TCADD="$TC class add dev $IF parent"
   $TCADD 1: classid 1:1 htb rate 80mbit burst 1m   #<-- Limit link speed
   $TCADD 1:1 classid 1:10 htb rate 1mbit burst 32k prio 5 #<-- Default bucket for unclassified traffic
   $TCADD 1:1 classid 1:20 htb rate 2kbit burst 1k prio 1  #<-- ICMP ONLY for LOW latency
   $TCADD 1:1 classid 1:30 htb rate 400kbit burst 6k prio 2 #<--Interactive Traffic
   $TCADD 1:1 classid 1:40 htb rate 1600kbit ceil 2000kbit burst 64k prio 3 #<-- Bulk Traffic (P2P and 'superfluous' things)
   $TCADD 1:1 classid 1:50 htb rate 50mbit burst 576k prio 4
       
   $TCADD 1:50 classid 50:10 htb rate 10mbit burst 64k prio 1
   $TCADD 1:50 classid 50:20 htb rate 40mbit burst 512k prio 2


########ADD FINAL QUEUES########
   TCQDISC="$TC qdisc add dev $IF parent"
        $TCQDISC 1:10 handle 10: sfq perturb 10
        $TCQDISC 1:20 handle 20: sfq perturb 10
        $TCQDISC 1:30 handle 30: sfq perturb 10
        $TCQDISC 1:40 handle 40: sfq perturb 10
        $TCQDISC 50:10 handle 510: sfq perturb 10
        $TCQDISC 50:20 handle 520: sfq perturb 10
        $TCQDISC 50:30 handle 530: sfq perturb 10

########FILTERS########
   FILTER="$TC filter add dev $IF protocol ip"
        #Reroute icmp
        $FILTER parent 1: u32 match ip protocol 1 0xff flowid 1:20

   #special rules
   #for osuosl.org
           $FILTER parent 1: u32 match ip src 128.193.0.0/24 flowid 1:40
   #for VNC
           $FILTER parent 1: u32 match ip sport $VNCPORT 0xffff flowid 1:50
           $FILTER parent 1: u32 match ip dport $VNCPORT 0xffff flowid 1:50
           $FILTER parent 50: u32 match ip sport $VNCPORT 0xffff flowid 50:10
           $FILTER parent 50: u32 match ip dport $VNCPORT 0xffff flowid 50:10

        #Reroute interactive traffic
        for PORT in $INTERACTIVEPORTS;
        do
                $FILTER parent 1: u32 match ip sport $PORT 0xffff flowid 1:30
                $FILTER parent 1: u32 match ip dport $PORT 0xffff flowid 1:30

        done

        for PORT in $BULKPORTS;
        do
                $FILTER parent 1: u32 match ip sport $PORT 0xffff flowid 1:40
                $FILTER parent 1: u32 match ip sport $PORT 0xffff flowid 1:40
        done

                 #for NFS
           $FILTER parent 1: u32 match ip sport $NFSPORTS 0xffff flowid 1:50
           $FILTER parent 1: u32 match ip dport $NFSPORTS 0xffff flowid 1:50
           $FILTER parent 50: u32 match ip sport $NFSPORTS 0xffff flowid 50:20
           $FILTER parent 50: u32 match ip dport $NFSPORTS 0xffff flowid 50:20

done
Back to top
View user's profile Send private message
zbindere
Guru
Guru


Joined: 27 May 2004
Posts: 356
Location: Switzerland

PostPosted: Tue Feb 01, 2005 12:50 pm    Post subject: Reply with quote

how can I test if everything works (when noone is using my upload)?
Code:
ifptables -L

gives nothing.

if I run:
Code:
tc -d qdisc


I get

Code:
qdisc htb 1: dev eth0 r2q 10 default 20 direct_packets_stat 0 ver 3.17
qdisc sfq 10: dev eth0 parent 1:10
qdisc sfq 20: dev eth0 parent 1:20
qdisc sfq 30: dev eth0 parent 1:30
qdisc ingress ffff: dev eth0 ----------------
Back to top
View user's profile Send private message
thebigslide
l33t
l33t


Joined: 23 Dec 2004
Posts: 790
Location: under a car or on top of a keyboard

PostPosted: Tue Feb 01, 2005 5:20 pm    Post subject: Reply with quote

Code:
tc -s qdisc show dev eth0
Back to top
View user's profile Send private message
zbindere
Guru
Guru


Joined: 27 May 2004
Posts: 356
Location: Switzerland

PostPosted: Wed Feb 02, 2005 8:07 am    Post subject: Reply with quote

ok, this gives:
Code:

qdisc htb 1: r2q 10 default 20 direct_packets_stat 0
 Sent 688037189 bytes 533808 pkt (dropped 0, overlimits 767528 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 10: parent 1:10
 Sent 5194427 bytes 57488 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 20: parent 1:20
 Sent 682842762 bytes 476320 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 30: parent 1:30
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
qdisc ingress ffff: ----------------
 Sent 26987080 bytes 385991 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0

I assume that it works correctly?
Back to top
View user's profile Send private message
thebigslide
l33t
l33t


Joined: 23 Dec 2004
Posts: 790
Location: under a car or on top of a keyboard

PostPosted: Wed Feb 02, 2005 3:28 pm    Post subject: Reply with quote

Looks good :)

queue :30 has not been used, but that might just be the particular traffic you were experiencing. at the time. There were a lot of overlimits on the :1 queue which would lead me to believe the WAN IF upstream is being limited a little too much.
Back to top
View user's profile Send private message
JohnerH
Apprentice
Apprentice


Joined: 01 Mar 2004
Posts: 197
Location: Manchester,United Kingdom

PostPosted: Tue Feb 22, 2005 10:58 am    Post subject: Reply with quote

Does this work with wireless? I can get it to work on a normal cabled connection (net.eth0) but when I do it for the wireless it produces a kernel panic...

Any ideias??

J
_________________
------------------------------------------------------
I think I had a thought once but it Died of Lonelyness...
Back to top
View user's profile Send private message
AlterEgo
Veteran
Veteran


Joined: 25 Apr 2002
Posts: 1619

PostPosted: Tue Feb 22, 2005 7:51 pm    Post subject: Reply with quote

thebigslide wrote:
Try this.


How do you set the available upload/download values in your script? I mean the kilobit-values my ISP is supposed to give me.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum