Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Console bittorrent and bittorrent (psuedo) deamon HOWTO
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
chunderbunny
Veteran
Veteran


Joined: 31 May 2004
Posts: 1281
Location: 51°24'27" N, 0°57'15" W

PostPosted: Tue Sep 07, 2004 4:14 pm    Post subject: Console bittorrent and bittorrent (psuedo) deamon HOWTO Reply with quote

Someone asked about this elsewhere on the forum, and I realised that there wasn't a good howto anywhere detailing the console based bittorrent clients, so I thought I would write one!

Most of you already know how to use the gui bittorrent client (and indeed have moved on to other, improved clients like azureus and shadows experimental client) but fewer people know about the console based clients that come with standard bittorrent package. The console client is best used in combination with screen, since screen allows you to run programs in the background. If you are administering a computer over an SSH connection than screen is especially useful since it doesn't quit when the SSH connection is killed and the screen session can be resumed from any computer that connects via SSH. If you don't already have screen installed then
Code:
emerge -av screen

to install it.

If you already have bittorrent installed then the console client is also already installed and you needn't do anything. If you need to install bittorrent, then setting the -X USE flag not install the dependencies of the gui client (which includes xorg and wxpython among other things)
Code:
echo "net-p2p/bittorrent -X" >> /etc/portage/package.use
emerge -av bittorrent


The basic bittorrent console client is btdownloadcurses.py, or btdownloadheadless.py if you prefer not to have the curses library in memory. There are two ways of using this client, either download the torrent file manually and set the client to use the local file, or you can get the client to download and execute the torrent file all in one step.
The choice is up to you, personally I find it easier and more useful to download the torrent files using lynx or wget, since then it's very easy to resume a download if the need arises. You can also browse for torrents using lynx on most websites. Once you have your torrent file, save it in your download directory and start a screen session.
Code:
screen -S bittorrent

Next, execute the download client
Code:
btdownloadcurses.py /path/to/file.torrent

And voila! The torrent should start downloading. Note that the file will be downloaded to the directory in which the client was started by default. You can change this by using the option --saveas
Code:
btdownloadcurses.py /path/to/file.torrent --saveas /save/directory/filename

Note that you have to specify the save path AND the filename when using the --saveas option, if this option is not used then the filename is chosen from the data in the .torrent file.
If you want the client to fetch the torrent file for you then you have to specify the url of the torrent file using the --url option
Code:
btdownloadcurses.py --url http://www.torrentfiles.com/some_torrentfile.torrent

Remember that you can't use wildcards in urls and you have to specify 'http://' or 'ftp://' or it won't work. If the file name has a space in it, then replace the space with '%20', similarly, a left bracket '(' should be replaced with '%28' and a right hand bracket ')' with '%29'.

Now detach the screen by pressing crtl + a, followed by d, the download will continue in the background, even if you log out or close your SSH session. You can monitor the progress of your downloads by issuing the command
Code:
screen -r bittorrent

And you can detach it using ctrl+a then d, as before. Simple huh? If you want to stop the download, reattach the screen and then press 'q'. Be patient, it may take a few seconds for the client to quit if it's a large file.

If you want to download multiple files simultaneously you can either set up a seperate screen session for each file, or you can use the btlaunchmanycurses.py client (or btlaunchmany.py if you're not using the curses interface.) Again, in order to run in the background btlaunchmanycurses needs a screen session
Code:
screen -S bittorrent

In this case you don't specify an actual torrent file to begin a download, rather a directory in which the torrent files are located. All torrent files in this directory wil be activated when the client is started.
Code:
btlaunchmanycurses.py /path/to/torrents/

Now all you have to do is put any torrents in the /path/to/torrents/ directory and btlaunchmancurses will automatically start downloading the files. When you remove the torrent file the download stops. Again, by default the files will be downloaded to the directory in which the client was started. Remember that when a file has finished downloading the bittorrent client will continue to upload the file.

Some options you might want to append to the btdownloadcurses.py or btlaunchmanycurses.py command:
Code:
--max_uploads <arg>
          the maximum number of uploads to allow at once. (defaults to 7)
--display_interval <arg>
          time between updates of displayed information (defaults to 0.5)
--max_upload_rate <arg>
          maximum kB/s to upload at, 0 means no limit (defaults to 0)

There are more options, run btdownloadheadless.py with no arguments to get a full list. Note that in the case of --max_upload_rate, the upload on each individual transfer is capped, rather than the total upload of all the transfers.


Last edited by chunderbunny on Thu Mar 17, 2005 7:00 pm; edited 1 time in total
Back to top
View user's profile Send private message
czo
Tux's lil' helper
Tux's lil' helper


Joined: 01 Sep 2004
Posts: 107
Location: Denmark

PostPosted: Tue Sep 07, 2004 4:41 pm    Post subject: Reply with quote

Hmmmmm i like the way you rock :)!!!!!!
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Wed Sep 08, 2004 1:28 pm    Post subject: Reply with quote

If you really want a smart bittorrent multi downloader, here's what i created:
http://www.mooktakim.com/code/show.php?id=3
It works really well.

All you do is run btdir ./TorrentFile and it will download all the torrents from TorrentFile.
Oh and you better change the default save folder inside the script.
It download one torrent after another. Once a torrent has completed download it exits and starts the next torrent download.

I also run this inside screen session.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
Doxer
n00b
n00b


Joined: 12 Nov 2003
Posts: 43
Location: Bremen-Nord, Germany

PostPosted: Sun Sep 12, 2004 10:37 pm    Post subject: Reply with quote

I think it doesn't work with torrent-files with spaces in it.
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Sun Sep 12, 2004 11:11 pm    Post subject: Reply with quote

Doxer wrote:
I think it doesn't work with torrent-files with spaces in it.


Yeh. i never liked spaces in file names. I always use a script to replace spaces with a _.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
Doxer
n00b
n00b


Joined: 12 Nov 2003
Posts: 43
Location: Bremen-Nord, Germany

PostPosted: Mon Sep 13, 2004 6:08 am    Post subject: Reply with quote

ok ;)

Is it possibel to change your script to work with spaces. I don't have the knowledege to change it.
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Mon Sep 13, 2004 10:42 am    Post subject: Reply with quote

I'm afraid not.
The problem arrises here:
Code:
files=$(cd $dir && ls *.torrent)

This basicly gets the output from ls and splits it up using spaces and puts it inside files.
Therefore if you have spaces in the name of one file. It will think its 2 files.

I'm afraid i haven't figured out how to sort this out.

However, it might have been easier to use Python or something like that.
Bash is bash :D
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
Twink
Apprentice
Apprentice


Joined: 06 Dec 2002
Posts: 178
Location: New Zealand

PostPosted: Mon Sep 13, 2004 11:24 am    Post subject: Reply with quote

Not the most elegant fix but here is one (perhaps someone else can figure out a better one)

Code:

dir -1 *.torrent | while true
do
  read file
  [[ "$file" == "" ]] && break

  echo $file

done


note that is a "dir -1" (one) not L or I
Back to top
View user's profile Send private message
Doxer
n00b
n00b


Joined: 12 Nov 2003
Posts: 43
Location: Bremen-Nord, Germany

PostPosted: Mon Sep 13, 2004 11:46 am    Post subject: Reply with quote

First: Thx for your responce.
Second: I solved it:

Here ist the new btdir.sh

Code:
#!/bin/bash

#########################################################
#  Md Mooktakim Ahmed   mma@mooktakim.com               #
#  http://www.mooktakim.com                             #
#                                                       #
# This is a script i use to start multiple bittorrent   #
# downloads. Basicly i've edited the         #
# btdownloadcurses.py file so that it exits after a   #
# complete download. Then this script runs many    #
# downloads one after the other, so that multiple    #
# downloads is possible, without any intervention.   #
# and i must say it works very well. It keeps a log and   #
# renames the torrent files into .OLD so that it does   #
# not get redownloaded. There is also some nice    #
# colourful outputs, just to complete the whole script   #
#                                                       #
# As always, this script is GPL Licensed.               #
#                                                       #
#########################################################

TMP="btdir.log"
DATE=`date +%Y-%m-%d`
TIME=`date +%r`
##ADDED
IFS="
"
##ADDED end
die()
{
   echo -e $*
   exit 1
}
echo "$DATE - $TIME -- ### New download started ###" >> $TMP
dir=$*

test -d $dir || die "$dir doesn't exist\nUsage:\n\tbtdir <dir>"
ls $dir/*.torrent >/dev/null 2>&1 || die "No torrents available in $dir"

files=$(cd $dir && ls *.torrent)

for I in $files
do
   if `test -e $dir/$I`
   then
      echo "$I exists"
   else
      echo "-- Torrents doesn't exist" >> $TMP
      echo "-- $dir/$I does not exist" >> $TMP
      echo "-- EXITING" >> $TMP
      die "Torrent $dir/$I doesn't exist"
   fi
done
echo "-- Torrents exist" >> $TMP
echo "-- Download will now be started..." >> $TMP
echo "*** All torrents exist."
echo "*** Download will now be started..."
echo "***"" Downloading from $dir ""***"""

sleep 5

for I in $files
do
   cd $dir || die "ERROR: Cannot change to directory [1;33$dir"
   echo "$TIME -- $dir/$I --- Started" >> $TMP
##MODIFIED
   btdownloadcurses.py --minport 6900 --max_upload_rate 3 --maxport 6999 --responsefile "$I" || die "ERROR: bittorrent exited with an error for [1;33$I"
##MODIFIED end
   echo "$TIME -- $dir/$I --- DONE!" >> $TMP
   echo "***"" $I ""***"""
   mv $I $I.OLD
done

echo "***"" #### ALL ### DONE ### ""***"""
echo "$TIME -- *** #### ALL ### DONE ### ***" >> $TMP


The important thing is the IFS. I've changed it in the beginning of the script, so the bash doesn't use a space for splitting.
_________________
read U
DoXer
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Mon Sep 13, 2004 12:46 pm    Post subject: Reply with quote

Doxer wrote:
First: Thx for your responce.
Second: I solved it:

Here ist the new btdir.sh

Code:
#!/bin/bash

#########################################################
#  Md Mooktakim Ahmed   mma@mooktakim.com               #
#  http://www.mooktakim.com                             #
#                                                       #
# This is a script i use to start multiple bittorrent   #
# downloads. Basicly i've edited the         #
# btdownloadcurses.py file so that it exits after a   #
# complete download. Then this script runs many    #
# downloads one after the other, so that multiple    #
# downloads is possible, without any intervention.   #
# and i must say it works very well. It keeps a log and   #
# renames the torrent files into .OLD so that it does   #
# not get redownloaded. There is also some nice    #
# colourful outputs, just to complete the whole script   #
#                                                       #
# As always, this script is GPL Licensed.               #
#                                                       #
#########################################################

TMP="btdir.log"
DATE=`date +%Y-%m-%d`
TIME=`date +%r`
##ADDED
IFS="
"
##ADDED end
die()
{
   echo -e $*
   exit 1
}
echo "$DATE - $TIME -- ### New download started ###" >> $TMP
dir=$*

test -d $dir || die "$dir doesn't exist\nUsage:\n\tbtdir <dir>"
ls $dir/*.torrent >/dev/null 2>&1 || die "No torrents available in $dir"

files=$(cd $dir && ls *.torrent)

for I in $files
do
   if `test -e $dir/$I`
   then
      echo "$I exists"
   else
      echo "-- Torrents doesn't exist" >> $TMP
      echo "-- $dir/$I does not exist" >> $TMP
      echo "-- EXITING" >> $TMP
      die "Torrent $dir/$I doesn't exist"
   fi
done
echo "-- Torrents exist" >> $TMP
echo "-- Download will now be started..." >> $TMP
echo "*** All torrents exist."
echo "*** Download will now be started..."
echo "***"" Downloading from $dir ""***"""

sleep 5

for I in $files
do
   cd $dir || die "ERROR: Cannot change to directory [1;33$dir"
   echo "$TIME -- $dir/$I --- Started" >> $TMP
##MODIFIED
   btdownloadcurses.py --minport 6900 --max_upload_rate 3 --maxport 6999 --responsefile "$I" || die "ERROR: bittorrent exited with an error for [1;33$I"
##MODIFIED end
   echo "$TIME -- $dir/$I --- DONE!" >> $TMP
   echo "***"" $I ""***"""
   mv $I $I.OLD
done

echo "***"" #### ALL ### DONE ### ""***"""
echo "$TIME -- *** #### ALL ### DONE ### ***" >> $TMP


The important thing is the IFS. I've changed it in the beginning of the script, so the bash doesn't use a space for splitting.


Wow :D:D
absolutely fantastic.
I never heard about this IFS before.
Thanks alot.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Mon Sep 13, 2004 12:54 pm    Post subject: Reply with quote

I've now updated the script in my website.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
wjholden
l33t
l33t


Joined: 01 Mar 2004
Posts: 826
Location: Augusta, GA

PostPosted: Thu Sep 23, 2004 4:15 pm    Post subject: Reply with quote

I'd like to run a bittorrent of the x86 Univeral Gentoo Install CD 24/7. I'm a student at North Carolina State University and we have an OC3 line that's been capped pretty severely, but it can still max out the 10 MBps cables in my dorm pretty easily and I'm really not worried about using TONS of bandwidth. If they say anything then I have a perfectly legitamite reason to use so much bandwidth and I read the Terms Of Service and it only says that you can't do things that actively damage the network, so uploading a legal torrent should be fine. I'd like to seed the LiveCD in order to contribute something to the Gentoo project.
However, I really don't feel like opening a window each time I logout or restart my PC (which isnt' very often), so how can I make a console-based bittorrent client run as a Cron job to begin as soon as I boot up?
Back to top
View user's profile Send private message
chunderbunny
Veteran
Veteran


Joined: 31 May 2004
Posts: 1281
Location: 51°24'27" N, 0°57'15" W

PostPosted: Thu Sep 23, 2004 4:51 pm    Post subject: Reply with quote

Well it depends on your crom deamon I think. In vixie cron you can set a cron job like this :
Code:
@reboot screen launchmanycurses.py /path/to/torrents


The '@reboot' is a special string which is used to execute a command once, at startup. Screen will automatically launch any program you specify and screen will exit if the program exits.
Back to top
View user's profile Send private message
Wi1d
Apprentice
Apprentice


Joined: 15 Mar 2004
Posts: 282
Location: USA, Iowa

PostPosted: Thu Sep 23, 2004 6:13 pm    Post subject: Reply with quote

Great ideas and topic. My hd curses you all. :)
Back to top
View user's profile Send private message
wjholden
l33t
l33t


Joined: 01 Mar 2004
Posts: 826
Location: Augusta, GA

PostPosted: Thu Sep 23, 2004 8:39 pm    Post subject: Reply with quote

And that's going to run in the background, right?

Are there any cool tricks that I use to grab it from the background and see how much it's uploaded? Better yet, is there a piece of software or command to monitor all of my combined bandwidth useage?
Back to top
View user's profile Send private message
chunderbunny
Veteran
Veteran


Joined: 31 May 2004
Posts: 1281
Location: 51°24'27" N, 0°57'15" W

PostPosted: Thu Sep 23, 2004 9:05 pm    Post subject: Reply with quote

Actually, having just tried it it doesn't work. For some reason Vixie cron isn't executing @reboot commands. But yeah, if you could find a way of executing 'screen btlaunchmanycurses.py /path/to/torrents' at startup (perhaps using /etc/conf.d/local.startup.. but then it runs as root which isn't good) then it would run bittorrent in the background. If you read the how to carefull you will also note that you can use 'screen -r' to reattach the screen session and monitor the bittorrent transfer.
Back to top
View user's profile Send private message
apyh
n00b
n00b


Joined: 17 Mar 2003
Posts: 67
Location: Texas

PostPosted: Thu Sep 23, 2004 10:07 pm    Post subject: Reply with quote

http://sourceforge.net/projects/btmanager/
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Fri Sep 24, 2004 9:08 am    Post subject: Reply with quote

put it in /etc/conf.d/local.start (using something like "screen btclient" That should start a screen session with the bt command running, when you boot.
Then you can check the session whenever you want.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
chunderbunny
Veteran
Veteran


Joined: 31 May 2004
Posts: 1281
Location: 51°24'27" N, 0°57'15" W

PostPosted: Fri Sep 24, 2004 9:32 am    Post subject: Reply with quote

If you put it in /etc/conf.d/local.start the screen and bt session will as root, which isn't exactly ideal. Is there a way to have it executed as a certain user?
Back to top
View user's profile Send private message
elias`
n00b
n00b


Joined: 29 Jul 2004
Posts: 4
Location: Finland

PostPosted: Fri Sep 24, 2004 1:30 pm    Post subject: Reply with quote

chunderbunny wrote:
If you put it in /etc/conf.d/local.start the screen and bt session will as root, which isn't exactly ideal. Is there a way to have it executed as a certain user?

You could just put in there
Code:
su user -c "command"
Back to top
View user's profile Send private message
wjholden
l33t
l33t


Joined: 01 Mar 2004
Posts: 826
Location: Augusta, GA

PostPosted: Sat Sep 25, 2004 7:30 pm    Post subject: Reply with quote

That's awesome. Thanks, guys. I'll be seeding the athlon-xp packages CD and the x86 Universal Install CD once I get everything configured here. Good thing I work for the campus ISP...lol.
Back to top
View user's profile Send private message
pharaoh
Apprentice
Apprentice


Joined: 20 Nov 2003
Posts: 211
Location: Pennsylvania

PostPosted: Fri Oct 22, 2004 3:00 pm    Post subject: Reply with quote

Ever since I read this thread I've been using btlaunchmanycurses and I love it :D
I'm curious though - is there any way to change the --max_upload_rate after it's already running? I like to leave my upload maxed over night, but once I'm using my internet I need to cut it down a little so I can load pages without waiting 5 minutes. Any ideas?
_________________
Intel Core i7 860 Processor 2.80 GHz 8 MB LGA1156
Gigabyte GA-P55-USB3 rev2.0
EVGA GeForce GTX 960 02G-P4-2966-KR 2GB
8GB DDR3 1600 CL8
Back to top
View user's profile Send private message
wjholden
l33t
l33t


Joined: 01 Mar 2004
Posts: 826
Location: Augusta, GA

PostPosted: Fri Oct 22, 2004 9:31 pm    Post subject: Reply with quote

I don't have an answer to your question.

I'm just wondering what bttrack is when you run rc-update show. Anyone???
Back to top
View user's profile Send private message
chunderbunny
Veteran
Veteran


Joined: 31 May 2004
Posts: 1281
Location: 51°24'27" N, 0°57'15" W

PostPosted: Fri Oct 22, 2004 10:10 pm    Post subject: Reply with quote

That's the tracker service. Every torrent file has an associated tracker server that coordinates the swapping of chunks between peers. If you want to run your own tracker then you start bttrack.
Back to top
View user's profile Send private message
Deranger
Veteran
Veteran


Joined: 26 Aug 2004
Posts: 1215

PostPosted: Sun Nov 07, 2004 6:02 pm    Post subject: Reply with quote

Nice howto. I'm Direct Connect user, but because there's no decent DC client on GNU/Linux yet, I tried BT out and I like it very much ;)

Especially many instances on screens ;)

Thanks!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum