Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO:Download Cache for your LAN-Http-Replicator (ver 3.0)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 6, 7, 8 ... 22, 23, 24  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Tue Oct 05, 2004 10:27 pm    Post subject: Re: cpu usage Reply with quote

fritz wrote:

think this is rather high for just downloading, am i the only one with this problem or is this lack of effiency in http-replicator?


I have an even slower box and I don't see cpu that high while maxing out a 100Mbs lan. cpu usage is dependent on many factors though. http-replicator is usually waiting on your disk or network. If I had to guess I'd say that you need to check your network config - especially your network card module or driver. I've heard of some drivers that have two versions that work but with high cpu usage on one. Could also be as simple as you don't have dma on your disks.

Start checking by transferring some data on your lan with ftp or http and watching the cpu usage.
Back to top
View user's profile Send private message
fritz
n00b
n00b


Joined: 15 Aug 2004
Posts: 7

PostPosted: Thu Oct 07, 2004 6:33 pm    Post subject: Reply with quote

indeed transfering a large file through nfs also takes 10-15% cpu usage, which also seems rather high. dma is on though, and i'm using the sis900 driver, any issues with that one?

hmm it's an onboard-nic, maybe that got something to do with it?

cheers
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Thu Oct 07, 2004 7:38 pm    Post subject: Reply with quote

fritz wrote:
i'm using the sis900 driver, any issues with that one?


A quick google shows a few possibilities that may or may not apply to your exact motherboard/kernel combination.

Most point towards 2.6.X problems with that chipset while 2.4.X works better. The problem seems to be sharing pci irq's which shows up in high demand usage of both the disk and the network driver. Try copying a large file to another disk or to /dev/null and see it the cpu is still high.

Some fixes I've seen recommend making sure to disable the Plug and Play support in the motherboard BIOS.

Check you dmesg for errors also.

Sorry I couldn't give you an easy fix.....
Back to top
View user's profile Send private message
fritz
n00b
n00b


Joined: 15 Aug 2004
Posts: 7

PostPosted: Thu Oct 07, 2004 9:48 pm    Post subject: Reply with quote

using 2.6.8-gentoo, no errors in dmesg and the sis900 gets it own irq. what's weird is cat'ing a file to /dev/null takes 15% usage, using dd 30% 8O (prob not related to my problem, just weird). is this too high because i get similar results on another computer?

appreciate the help and the fast reply :)
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Fri Oct 08, 2004 5:43 am    Post subject: Reply with quote

Those results would seem to say your level of cpu usage is normal to your box and http-replicator is very efficient, using the same cpu as cat'ing to /dev/null.....
Back to top
View user's profile Send private message
fritz
n00b
n00b


Joined: 15 Aug 2004
Posts: 7

PostPosted: Fri Oct 08, 2004 4:03 pm    Post subject: Reply with quote

flybynite wrote:
Those results would seem to say your level of cpu usage is normal to your box and http-replicator is very efficient, using the same cpu as cat'ing to /dev/null.....


ooooooh :oops: indeed, see your point

guess i'll just accept it takes 25% of my cpu then, i'll try to play a bit with hdparm
Back to top
View user's profile Send private message
piyo
n00b
n00b


Joined: 20 Mar 2004
Posts: 12

PostPosted: Sat Oct 09, 2004 12:37 pm    Post subject: Reply with quote

The instructions say nothing about what to clean periodically in the cache directory.

Here I delete anything that hasn't been referenced in the last two weeks.

Code:
colinux root # cat /etc/cron.daily/http-replicator
#!/bin/sh
/bin/nice /usr/bin/find /var/cache/http-replicator -type f -ctime +14 | \
    /usr/bin/xargs --no-run-if-empty /usr/bin/rm -f
/bin/nice /usr/bin/repcacheman

---
piyo
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Sat Oct 09, 2004 7:31 pm    Post subject: Reply with quote

piyo wrote:
The instructions say nothing about what to clean periodically in the cache directory.


Thanks for the script piyo!

I've talked about this before in the thread. The truth is it is much harder than it looks. That's the reason gentoo doesn't have a default way of cleaning distfiles either, many people need different things.

Most scripts in this board that clean distfiles will work with http-replicators cache. You can find scripts to clean the cache based on any number of criteria.

Your script is nicely done. I like the way you nice'd the tasks and you remembered to still run repcacheman.

The one problem I see with your script is that many gentooers have followed the Gentoo Install Handbook's recommendation of using `noatime` in fstab for performance reasons.
Quote:

Now, to improve performance, most users would want to add the noatime option as mountoption, which results in a faster system since access times aren't registered (you don't need those generally anyway):

From http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=1&chap=8#doc_chap1


With noatime as a mount option, your script would simply delete files older than two weeks regardless of how recently they were served to clients.......
Back to top
View user's profile Send private message
piyo
n00b
n00b


Joined: 20 Mar 2004
Posts: 12

PostPosted: Sun Oct 10, 2004 1:20 am    Post subject: Turning on atime? Reply with quote

flybynite wrote:
The one problem I see with your script is that many gentooers have followed the Gentoo Install Handbook's recommendation of using `noatime` in fstab for performance reasons.
...
With noatime as a mount option, your script would simply delete files older than two weeks regardless of how recently they were served to clients.......


Whoa, thanks for the correction! I was wondering why the script wasn't really working. I'm one of those gentooers! I'll be sure to delete noatime from my /etc/fstab because I really want atime. How else can you get the access time and changed time unless you modify your script to output those values?

---
piyo
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Sun Oct 10, 2004 8:12 am    Post subject: Re: Turning on atime? Reply with quote

piyo wrote:

Whoa, thanks for the correction! I was wondering why the script wasn't really working. I'm one of those gentooers! I'll be sure to delete noatime from my /etc/fstab because I really want atime.



I would try and leave noatime. Without it, every read becomes a disk write which invalidates the disk cache...

piyo wrote:

How else can you get the access time and changed time unless you modify your script to output those values?




Http-replicator saves a log of every request received from clients. The log entries were designed for easy parsing to generate stats on the cache usage and efficiency. Those stats would show when the last time a file was requested by a client.

The stats generation should be in the next version of http-replicator. Until then, you could try and parse the log yourself. This is an actual log entry from my box:

Code:

10 Oct 2004 02:53:37 INFO: HttpClient 75 received request for http://gentoo.osuosl.org/distfiles/samba-3.0.7.tar.gz


There are different levels of logging with various levels of info logged. See the config file....
Back to top
View user's profile Send private message
piyo
n00b
n00b


Joined: 20 Mar 2004
Posts: 12

PostPosted: Sun Oct 10, 2004 12:52 pm    Post subject: Re: Turning on atime. Reply with quote

flybynite wrote:
I would try and leave noatime. Without it, every read becomes a disk write which invalidates the disk cache...


So you prefer speediness over correctness? Premature optimization? :wink:

Admittedly it's probably true that most people don't use last accessed time in Linux and turning it on for a whole drive when you only want to cover the cache on probably hinders performance. However in my normal computer experience I don't feel the difference, in my Linux or Windows sessions.

Let the system bring you features and take advantage of them. That's what the operating system is for.
---
piyo
Back to top
View user's profile Send private message
Bob Shroom
n00b
n00b


Joined: 14 Oct 2003
Posts: 14
Location: Nuremberg, Germany

PostPosted: Mon Oct 11, 2004 9:46 am    Post subject: Reply with quote

hi guys,

i am having a problem with one of my clients, who is supposed to use the replicator proxy... but for some strange reasons doesn't want to do so.

i've checked my settings in /etc/make.conf now for several times, even started with an entirely new /etc/make.conf.orig... and i can't get the damn client to use the http-proxy. i'm about to give up on this.

i think my server is setup correct, as another client in my LAN does connect to it and also uses it just fine.

the /etc/replicator.conf on the server:
Code:

server root # cat /etc/http-replicator.conf
#   ************README-Gentoo Http-Replicator *******************
#   The defaults in Http-Replicator have been changed to work with the
#   default Gentoo install and shouldn't have to be changed.  The only
#   changes required to activate Http-Replicator are in /etc/make.conf
#   on the clients and the server itself.
#
#   Find the Default fetch command section in /etc/make.conf.  If you are
#   already using one of the alternate fetch commands, apply the changes
#   to your section.  Http-Replicator does not support "continued" downloads.
#
#   Make the following changes:
#
# 1. Add http_proxy="http://YourProxyHere.com:8080" Line
#       replacing YourProxyHere.com with your proxy hostname or IP address.
# 2. Uncomment (remove the leading '#') from RESUMECOMMAND
# 3. Remove the -c from the RESUMECOMMAND and replace -P \${DISTDIR} with -O \${DISTDIR}/\${FILE}
#
#   It will look like this when complete:
#
#   Default fetch command (5 tries, passive ftp for firewall compatibility)
#   http_proxy="http://YourMirrorHere.com:8080"
#   #FETCHCOMMAND="$PROXY /usr/bin/wget -t 5 --passive-ftp \${URI} -P \${DISTDIR}"
#   RESUMECOMMAND="$PROXY /usr/bin/wget -t 5 --passive-ftp \${URI} -O \${DISTDIR}/\${FILE}"
#
#   execute:
#   /etc/init.d/http-replicator start
#   to run http-replicator.
#   execute:
#   rc-update add http-replicator default
#   to make http-replicator start at boot
#   execute:
#   /usr/bin/repcacheman
#   frequently (/etc/cron.daily) to delete
#   dup files and add new files to the cache
#
#   ************END README-Gentoo Http-Replicator *******************

#  This is the configuration file for the replicator proxy server.
#  Settings from this file will apply to the server in daemon mode and also to the cache cleaner script, if used.
#
#  The proxy server will act on port [PORT] of the localhost.
#  The default value of 8080 is a common value for an http proxy server.
#  If you have another http proxy running it is likely that you should should change this port.

PORT = 12000

#  Replicator's behaviour can be altered through a number of server flags.
#  They are by default disabled; [FLAGS] is a list of flags that should be enabled.
#  The possible flags are the same as on the command line:
#  * static: never check for modifications
#  * flat: save files in a single directory
#  * debug: crash on exceptions

FLAGS = ['static','flat','debug']

#  For security reasons the hosts for which access to the proxy is granted should be specified in the [IP] list.
#  A '?' can be used as wildcard for a single digit and a '*' for a multiple digits.
#  For example '10.0.?.*' grants access from 10.0.1.25 but not from 10.0.15.25.

IP = ['127.0.0.1','192.168.4.10','192.168.4.20','192.168.4.30']

#  The proxy server can be monitored via telnet on port [TELNET].
#  This is disabled by entering a zero value.
#  Otherwise make sure the port is available or replicator will not start.

TELNET = 0

#  The process user id is set to [USER].
#  The daemon must be started as root because no other user can change into another.
#  Not even [USER] can change into itself!

USER = 'portage'

#  All cached files ar saved in directory [DIR].
#  The [USER] should of course have write permission in this directory.
#  Where in this directory the files are actually put depends on if the server is in flat mode.
#  By default the entire directory structure is copied.

DIR = '/var/cache/http-replicator'

#  The process id of the running process is saved in [PID].
#  As this is done before changing into [USER], write permission for [USER] for this file is not needed.

PID = '/var/run/http-replicator.pid'

#  All messages on stdout and stderr are in daemon mode written to the [LOG].
#  Just as for [PID], write permission for [LOG] is not necessary.

LOG = '/var/log/http-replicator'

#  When the proxy server is used to maintain a package cache a cron script can delete the oldest packages.
#  The value of [KEEP] sets the maximum number of versions of each package to be kept.
#  For example a value of one will delete all versions but the most recent.
#  The script is disabled by setting this value to zero.

#  Not implemented in Gentoo yet!
KEEP = 2


scenario:
192.168.4.10 = server
192.168.4.20 = client (BAD)
192.168.4.30 = client (GOOD)

the good(=working) client obviously uses the http_proxy-setting in /etc/make.conf
Code:

good client root # emerge -v netcat
>>> Downloading http://gentoo.oregonstate.edu/distfiles/nc-v6-20000918.patch.gz
--10:25:55--  http://gentoo.oregonstate.edu/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
Verbindungsaufbau zu 192.168.4.10:12000... verbunden.
Proxy Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 8,740

100%[=============================================================================================================================================>] 8,740         --.--K/s

10:25:55 (16.31 MB/s) - »/usr/portage/distfiles/nc-v6-20000918.patch.gz« gespeichert [8740/8740]

>>> Downloading http://gentoo.oregonstate.edu/distfiles/nc110.tgz
--10:25:55--  http://gentoo.oregonstate.edu/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
Verbindungsaufbau zu 192.168.4.10:12000... verbunden.
Proxy Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 75,267

100%[=============================================================================================================================================>] 75,267        --.--K/s

10:25:55 (9.55 MB/s) - »/usr/portage/distfiles/nc110.tgz« gespeichert [75267/75267]


eventho it's working on this client, i've noticed that it wants to use the official mirrors, what i cannot understand, cause i've set the GENTOO_MIRRORS in /etc/make.conf to just use only http://-Mirrors here in germany.
Code:

good client root # cat /etc/make.conf | grep GENTOO_MIRRORS
# Portage uses GENTOO_MIRRORS to specify mirrors to use for source retrieval.
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/"


on the bad(=non-working client) it looks like this:

/etc/make.conf of the bad client:
Code:

GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http:
//ftp.uni-erlangen.de/pub/mirrors/gentoo http://mirrors.sec.informatik.tu-darmst
adt.de/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/"
# Default fetch command (5 tries, passive ftp for firewall compatibility)
http_proxy="http://192.168.4.10:12000"
#FETCHCOMMAND="/usr/bin/wget -t 5  --passive-ftp \${URI} -P \${DISTDIR}"
RESUMECOMMAND=" /usr/bin/wget -t 5 --passive-ftp  \${URI} -O \${DISTDIR}/\${FILE}"



but when i try to emerge something, i get this:
Code:

bad client ~ # emerge netcat
>>> Downloading http://gentoo.oregonstate.edu/distfiles/nc-v6-20000918.patch.gz
--11:14:37--  http://gentoo.oregonstate.edu/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc-v6-20000918.patch.gz
--11:14:37--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
--11:14:37--  http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc-v6-20000918.patch.gz
--11:14:37--  http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
--11:14:37--  http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading ftp://sith.mimuw.edu.pl/pub/users/baggins/IPv6/nc-v6-20000918.patch.gz
--11:14:37--  ftp://sith.mimuw.edu.pl/pub/users/baggins/IPv6/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
Auflösen des Hostnamen »sith.mimuw.edu.pl«.... 193.0.96.4
Verbindungsaufbau zu sith.mimuw.edu.pl[193.0.96.4]:21... verbunden.
Anmelden als anonymous ... Angemeldet!
==> SYST ... fertig.    ==> PWD ... fertig.
==> TYPE I ... fertig.  ==> CWD /pub/users/baggins/IPv6 ... fertig.
==> PASV ... fertig.    ==> RETR nc-v6-20000918.patch.gz ... fertig.
Länge: 8,740 (unmaßgeblich)

100%[=============================================================================================================================================>] 8,740         --.--K/s             

11:14:38 (99.17 KB/s) - »/usr/portage/distfiles/nc-v6-20000918.patch.gz« gespeichert [8740]

>>> Downloading http://gentoo.oregonstate.edu/distfiles/nc110.tgz
--11:14:38--  http://gentoo.oregonstate.edu/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
--11:14:38--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc110.tgz
--11:14:38--  http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc110.tgz
--11:14:38--  http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc110.tgz
--11:14:38--  http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://www.atstake.com/research/tools/network_utilities/nc110.tgz
--11:14:38--  http://www.atstake.com/research/tools/network_utilities/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
!!! Couldn't download nc110.tgz. Aborting.

it doesn't seem to recognize the http_proxy at all in the above.

i saw, that somebody before me had a similar problem, but that was on the server and not on the client. and unlike him... when i comment out my the http_proxy line in /etc/make.conf portage seems to be able to download files:
Code:

bad client ~ # emerge netcat
>>> Downloading http://gentoo.oregonstate.edu/distfiles/nc-v6-20000918.patch.gz
--11:21:29--  http://gentoo.oregonstate.edu/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
Auflösen des Hostnamen »gentoo.oregonstate.edu«.... 140.211.166.134
Verbindungsaufbau zu gentoo.oregonstate.edu[140.211.166.134]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 301 Moved Permanently
Platz: http://gentoo.osuosl.org/distfiles/nc-v6-20000918.patch.gz[folge]
--11:21:30--  http://gentoo.osuosl.org/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
Auflösen des Hostnamen »gentoo.osuosl.org«.... 140.211.166.134
Verbindungsaufbau zu gentoo.osuosl.org[140.211.166.134]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 8,740 [application/x-gzip]

100%[=============================================================================================================================================>] 8,740         43.10K/s             

11:21:30 (42.96 KB/s) - »/usr/portage/distfiles/nc-v6-20000918.patch.gz« gespeichert [8740/8740]

>>> Downloading http://gentoo.oregonstate.edu/distfiles/nc110.tgz
--11:21:30--  http://gentoo.oregonstate.edu/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
Auflösen des Hostnamen »gentoo.oregonstate.edu«.... 140.211.166.134
Verbindungsaufbau zu gentoo.oregonstate.edu[140.211.166.134]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 301 Moved Permanently
Platz: http://gentoo.osuosl.org/distfiles/nc110.tgz[folge]
--11:21:31--  http://gentoo.osuosl.org/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
Auflösen des Hostnamen »gentoo.osuosl.org«.... 140.211.166.134
Verbindungsaufbau zu gentoo.osuosl.org[140.211.166.134]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 75,267 [application/x-tar]

100%[=============================================================================================================================================>] 75,267        97.18K/s             

11:21:32 (96.90 KB/s) - »/usr/portage/distfiles/nc110.tgz« gespeichert [75267/75267]

also here from the "official" mirrors first, before it finds my GENTOO_MIRRORS in /etc/make.conf

my /etc/portage/mirrors looks like this: (on server and both clients)
Code:

# Http-Replicator Override for FTP and RESTRICT="nomirror packages
local http://gentoo.oregonstate.edu


so, anybody got an idea why this is not working for me on the one client?
is there any setting that overrides my GENTOO_MIRRORS in /etc/make.conf ?
and why doesn't portage recognize the http_proxy line in /etc/make.conf ?

the only difference between the two clients is the architecture and the keywords. the working client is a PPC (stable) and the non-working is a X86 (unstable)... but all that shouldnt make a difference, no?

thanks in advance for any tip pointing me into the right direction. ;)

.bob
_________________
[planet-earth] root # rm -rf /
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Mon Oct 11, 2004 6:42 pm    Post subject: Reply with quote

Bob Shroom wrote:

i think my server is setup correct, as another client in my LAN does connect to it and also uses it just fine.


Ok, your server is working....

Bob Shroom wrote:

my /etc/portage/mirrors looks like this: (on server and both clients)
Code:

# Http-Replicator Override for FTP and RESTRICT="nomirror packages
local http://gentoo.oregonstate.edu



is there any setting that overrides my GENTOO_MIRRORS in /etc/make.conf ?


Yes, the local mirror in /etc/portage/mirrors is always checked first. Change it from gentoo.oregonstate.edu to your preferred german mirror.

Bob Shroom wrote:

and why doesn't portage recognize the http_proxy line in /etc/make.conf ?


Let's check a couple of things.

1. In your server config you only allow specific IP's - are you sure the IP of the bad box didn't change through dhcpd and is now being denied?

You may be using static IP's, I don't know.... But, unless you have a good reason - leave the IP set to the range of your lan probably '192.168.4.*' Security is good but can be frustrating when you have a mistake in your config....

If your bad client is being denied or not even reaching your server due to routing issues it will show in /var/log/http-replicator log on the server. Try posting a couple of log entries showing the good and the bad box trying to connect.

2. You can test that your http_proxy is set several ways. Try these tests on the bad client...

Code:

source /etc/make.conf
echo $http_proxy


Does this print "http://192.168.4.10:12000" ?

Code:

http_proxy=http://192.168.4.10:12000 wget http://gentoo.oregonstate.edu/distfiles/nc110.tgz


Does wget contact http-replicator?

How about this one:
Code:

http_proxy=http://192.168.4.10:12000 emerge netcat


Do you still get the same errors?

If this solves the problem, the http_proxy isn't set in the bad client make.conf - probably an extra " or ' messing up the settings. The extra " or ' could be several lines above the line with your http_proxy setting...

I know we can get http-replicator working for you!!!
Back to top
View user's profile Send private message
Bob Shroom
n00b
n00b


Joined: 14 Oct 2003
Posts: 14
Location: Nuremberg, Germany

PostPosted: Tue Oct 12, 2004 9:07 am    Post subject: Reply with quote

hi, thanx for your reply

let's see what we got here:

Quote:
Yes, the local mirror in /etc/portage/mirrors is always checked first. Change it from gentoo.oregonstate.edu to your preferred german mirror.


changed it to my fav german mirror
Code:

# Http-Replicator Override for FTP and RESTRICT="nomirror packages
local http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/


Quote:
1. In your server config you only allow specific IP's - are you sure the IP of the bad box didn't change through dhcpd and is now being denied?


i dont use DHCP ... only static IPs
Code:

bad client ~ # ifconfig eth0
eth0      Protokoll:Ethernet  Hardware Adresse 00:50:04:45:BE:19 
          inet Adresse:192.168.4.20  Bcast:192.168.4.255  Maske:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:11 Basisadresse:0xb400


Quote:
You may be using static IP's, I don't know.... But, unless you have a good reason - leave the IP set to the range of your lan probably '192.168.4.*' Security is good but can be frustrating when you have a mistake in your config....

i understand... eventhough i am sure this is not the problem, i modified the server.conf:
Code:

server root # cat /etc/http-replicator.conf | grep 192
IP = ['127.0.0.1','192.168.4.*']


Quote:
If your bad client is being denied or not even reaching your server due to routing issues it will show in /var/log/http-replicator log on the server. Try posting a couple of log entries showing the good and the bad box trying to connect.

well... the bad client definitely doesn't reach the proxy. this shouldn't be a routing issue tho. when i try to emerge something it goes straight for the server in /etc/portage/mirror and then the GENTOO_MIRRORS in /etc/make.conf . it doesn't even try to connect to the proxy. i see that in the emerge output and on my switch-LEDs.
i was tailing /var/log/http-replicator while i emerged something from the working client. unfortunately nothing was logged.
i was also greping thru /var/log/http-replicator.old and found out that only the server itself shows up in the logfile, when i emerge packages. (server=192.168.4.10 uses proxy AND shows in logfile , good client=192.168.4.30 uses proxy but doesn't show up in logfile , bad client=192.168.4.20 doesn't use proxy and of course doesn't show up in its logfile)

Quote:
2. You can test that your http_proxy is set several ways. Try these tests on the bad client...

Code:

source /etc/make.conf
echo $http_proxy


Does this print "http://192.168.4.10:12000" ?


yes it does.
Code:

bad client ~ # source /etc/make.conf
bad client ~ # echo $http_proxy
http://192.168.4.10:12000


Quote:


Code:

http_proxy=http://192.168.4.10:12000 wget http://gentoo.oregonstate.edu/distfiles/nc110.tgz


Does wget contact http-replicator?


no... it aborts.(=Abgebrochen.)
this is all i get:
Code:

bad client ~ # http_proxy=http://192.168.4.10:12000 wget http://gentoo.oregonstate.edu/distfiles/nc110.tgz
--10:42:52--  http://gentoo.oregonstate.edu/distfiles/nc110.tgz
           => `nc110.tgz'
Abgebrochen


Quote:
How about this one:
Code:

http_proxy=http://192.168.4.10:12000 emerge netcat


Do you still get the same errors?


yes... unfortunately that didn't do the trick: :(
Code:

bad client ~ # http_proxy=http://192.168.4.10:12000 emerge netcat
Calculating dependencies ...done!
>>> emerge (1 of 1) net-analyzer/netcat-110-r6 to /
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror//distfiles/nc110.tgz
--10:52:10--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
--10:52:10--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc110.tgz
--10:52:10--  http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc110.tgz
--10:52:10--  http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc110.tgz
--10:52:10--  http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://www.atstake.com/research/tools/network_utilities/nc110.tgz
--10:52:10--  http://www.atstake.com/research/tools/network_utilities/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
!!! Couldn't download nc110.tgz. Aborting.


i suppose my portage cannot be messed up, cause when i comment out the http_proxy line in /etc/make.conf it downloads the file and merges it.

Quote:
I know we can get http-replicator working for you!!!

yeah, that would be sweet. :)
i followed your rsync howto and it worked like a charm. replicator worked like a charm too until i started setting up the bad client.

greets,

.bob
_________________
[planet-earth] root # rm -rf /
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Wed Oct 13, 2004 6:58 pm    Post subject: Reply with quote

Bob Shroom wrote:

well... the bad client definitely doesn't reach the proxy. this shouldn't be a routing issue tho. ...

...it doesn't even try to connect to the proxy. i see that in the emerge output and on my switch-LEDs.


I believe this is telling you that it is a routing problem!

1. We know http-replicator works because another box connects and gets excellent download speeds.

2. We know that wget itself can't connect to http-replicator from the bad box. The wget test we tried bypasses any config errors or portage problems and should have worked independant of any other programs/setting/configs etc.

3. We know wget itself works because it can connect to a working server on the internet and complete a download. It just can't connect to a working server on your lan.

This only leaves routing and firewall issues!!!

The first issue is replicator itself has a security feature that limits IP's that can connect to it. You changed the list to include your whole lan. I didn't specifically mention you should restart http-replicator after you changed the config. We both don't think was the problem, but, If you didn't do the restart do it and check the previous tests just to make sure....


I've looked again at your error's and they stop at the same place - name/IP resolution...

Here is what I get when I try the same test you did:
Code:

 $ http_proxy=http://192.168.4.10:12000 wget http://gentoo.oregonstate.edu/distfiles/nc110.tgz
--12:33:48--  http://gentoo.oregonstate.edu/distfiles/nc110.tgz
           => `nc110.tgz'
Connecting to 192.168.4.10:12000...



Compare this to your error:
Code:

bad client ~ # http_proxy=http://192.168.4.10:12000 wget http://gentoo.oregonstate.edu/distfiles/nc110.tgz
--10:42:52--  http://gentoo.oregonstate.edu/distfiles/nc110.tgz
           => `nc110.tgz'
Abgebrochen



Your proxy IP isn't even on my network and won't work of course. But notice where wget stopped on my box compared with your box.

On my box wget shows that it parsed the request and then did a name/ip resolution lookup and then tried to connect to that destination and then just quit. Your box seems to have failed at the name/IP lookup and never tried to connect to the destination. Thats why you never saw the lights on your router because wget never tried to connect.

The question is why did wget fail at that step? Since we used IP's there is no name resolution going on. It just needs to lookup the destination in the routing table.

Let's check the network.

What do you get when you try to ping the proxy?
Code:

ping -c 3 192.168.4.10


Show the route of the good box and the bad box - you must be root to do this...

Code:

route


I assume you have a normal lan? All the boxes are plugged into a plain router?
Back to top
View user's profile Send private message
Bob Shroom
n00b
n00b


Joined: 14 Oct 2003
Posts: 14
Location: Nuremberg, Germany

PostPosted: Fri Oct 15, 2004 4:01 pm    Post subject: Reply with quote

Quote:
I didn't specifically mention you should restart http-replicator after you changed the config. We both don't think was the problem, but, If you didn't do the restart do it and check the previous tests just to make sure....


ok, did that...just to make sure...
Code:
server root # /etc/init.d/http-replicator stop 
 * Stopping Http-Replicator...                                            [ ok ]
server root # /etc/init.d/http-replicator start
 * Starting Http-Replicator...                                            [ ok ]
server root #


...but that didn't do the trick.
Code:

bad client ~ # emerge netcat
Calculating dependencies ...done!
>>> emerge (1 of 1) net-analyzer/netcat-110-r6 to /
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror//distfiles/nc-v6-20000918.patch.gz
--17:30:18--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc-v6-20000918.patch.gz
--17:30:18--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
--17:30:18--  http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc-v6-20000918.patch.gz
--17:30:18--  http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
--17:30:18--  http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
>>> Downloading ftp://sith.mimuw.edu.pl/pub/users/baggins/IPv6/nc-v6-20000918.patch.gz
--17:30:18--  ftp://sith.mimuw.edu.pl/pub/users/baggins/IPv6/nc-v6-20000918.patch.gz
           => `/usr/portage/distfiles/nc-v6-20000918.patch.gz'
Auflösen des Hostnamen »sith.mimuw.edu.pl«.... 193.0.96.4
Verbindungsaufbau zu sith.mimuw.edu.pl[193.0.96.4]:21... verbunden.
Anmelden als anonymous ... Angemeldet!
==> SYST ... fertig.    ==> PWD ... fertig.
==> TYPE I ... fertig.  ==> CWD /pub/users/baggins/IPv6 ... fertig.
==> PASV ... fertig.    ==> RETR nc-v6-20000918.patch.gz ... fertig.
Länge: 8,740 (unmaßgeblich)

100%[====================================>] 8,740         --.--K/s             

17:30:21 (80.55 KB/s) - »/usr/portage/distfiles/nc-v6-20000918.patch.gz« gespeichert [8740]

>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror//distfiles/nc110.tgz
--17:30:21--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
--17:30:21--  http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc110.tgz
--17:30:21--  http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc110.tgz
--17:30:21--  http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc110.tgz
--17:30:21--  http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
>>> Downloading http://www.atstake.com/research/tools/network_utilities/nc110.tgz
--17:30:21--  http://www.atstake.com/research/tools/network_utilities/nc110.tgz
           => `/usr/portage/distfiles/nc110.tgz'
!!! Couldn't download nc110.tgz. Aborting.
bad client ~ #


Quote:
I've looked again at your error's and they stop at the same place - name/IP resolution...


yes, i've noticed that too. on my good box the output matches yours...connecting to the proxy and all that...but the bad box doesn't give a damn about the proxy for some strange reason... :?

Quote:
What do you get when you try to ping the proxy?
Code:
ping -c 3 192.168.4.10


looks all normal to me...
Code:
bad client ~ # ping -c 3 192.168.4.10
PING 192.168.4.10 (192.168.4.10) 56(84) bytes of data.
64 bytes from 192.168.4.10: icmp_seq=1 ttl=64 time=0.199 ms
64 bytes from 192.168.4.10: icmp_seq=2 ttl=64 time=0.182 ms
64 bytes from 192.168.4.10: icmp_seq=3 ttl=64 time=0.188 ms

--- 192.168.4.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.182/0.189/0.199/0.017 ms


Quote:
Show the route of the good box and the bad box - you must be root to do this...
Code:
route



the good box:
Code:
good client root # route
Kernel IP Routentabelle
Ziel            Router          Genmask         Flags Metric Ref    Use Iface
192.168.4.0     *               255.255.255.0   U     0      0        0 eth0
loopback        localhost       255.0.0.0       UG    0      0        0 lo
default         doobcop         0.0.0.0         UG    0      0        0 eth0


the bad box:
Code:
bad client ~ # route
Kernel IP Routentabelle
Ziel            Router          Genmask         Flags Metric Ref    Use Iface
192.168.4.0     *               255.255.255.0   U     0      0        0 eth0
loopback        localhost       255.0.0.0       UG    0      0        0 lo
default         doobcop         0.0.0.0         UG    0      0        0 eth0

they both look the same. one thing confusing me a little bit is that IP '192.168.4.0'....i ain't got a device on my network with that IP!? but if that would be my problem, it wouldn't work on both boxen, no?

Quote:
I assume you have a normal lan? All the boxes are plugged into a plain router?


yes, all boxen are connected to the same switch (unmanaged...runs several years without a problem... didn't unplug or even touch the cables lately)
no firewall or similar between them two boxen.
_________________
[planet-earth] root # rm -rf /
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Sat Oct 16, 2004 3:01 am    Post subject: Reply with quote

Haven't found it yet, but let's move this off list. PM me the following.

on the server and the bad box:

Code:

iptables -L


and on the bad box:
Code:

cat ~/.wgetrc
cat .wgetrc
cat /etc/wget/wgetrc


and check the router config. Just to make sure you don't have any firewall/forwarding setup in there.
Back to top
View user's profile Send private message
Bob Shroom
n00b
n00b


Joined: 14 Oct 2003
Posts: 14
Location: Nuremberg, Germany

PostPosted: Sat Oct 16, 2004 7:32 am    Post subject: Reply with quote

Quote:
PM me the following.


alright, check your inbox.
_________________
[planet-earth] root # rm -rf /
Back to top
View user's profile Send private message
cdunham
Apprentice
Apprentice


Joined: 06 Jun 2003
Posts: 211
Location: Rhode Island

PostPosted: Sun Oct 17, 2004 3:06 am    Post subject: repcacheman problem Reply with quote

I have been seeing this, both with 1.3 and 1.5. I'm guessing I have a borked file or something:
Code:
# repcacheman


Replicator's cache directory: /var/cache/http-replicator/
Portage's DISTDIR: /usr/portage/distfiles/

Comparing directories....
Done!

Deleting duplicate file(s) in /usr/portage/distfiles/
http-replicator-2.1_rc3.tar.gz
Done!

New files in DISTDIR:
libsoup-2.2.1.tar.bz2
.locks
evolution-data-server-1.0.2.tar.bz2
gal-2.2.3.tar.bz2
evolution-2.0.2.tar.bz2

Checking authenticity and integrity of new files...
Searching for ebuilds's ....
Done!

Found 16331 ebuilds.

Extracting the checksums....
Missing digest: dev-perl/Net-SNMP-4.1.2
Done!

Verifying checksum's....
/usr/portage/distfiles/libsoup-2.2.1.tar.bz2
Traceback (most recent call last):
  File "/usr/bin/repcacheman", line 204, in ?
    if t[0]:
KeyError: 0


Anyone else seeing this? A known problem? Everything seems to be working great, but this error keeps happening...

Thanks!
_________________
This post more meaningful in a scalar context.
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Sun Oct 17, 2004 4:27 am    Post subject: Re: repcacheman problem Reply with quote

cdunham wrote:
I have been seeing this, both with 1.3 and 1.5. I'm guessing I have a borked file or something:

New files in DISTDIR:
libsoup-2.2.1.tar.bz2
.locks
evolution-data-server-1.0.2.tar.bz2
gal-2.2.3.tar.bz2
evolution-2.0.2.tar.bz2




This is a quick guess because of time, but I don't know of any reason why (.) hidden files should be in distfiles dir.

Delete the dot file and try again.

Code:

rm /usr/portage/distfiles/.locks



Have you been sharing the distfile dir or running some other program that would leave a ".locks" file there? That certainly isn't a package portage would download!

If this is the fix I'll add some code in the next version of repcacheman to allow for this, but I don't know how that file got there.....
Back to top
View user's profile Send private message
cdunham
Apprentice
Apprentice


Joined: 06 Jun 2003
Posts: 211
Location: Rhode Island

PostPosted: Sun Oct 17, 2004 7:11 am    Post subject: Reply with quote

Actually, it's a directory, and seems to be on every Gentoo machine I looked at.

In any case, removing it didn't help. I'm not big on python. What is t[0] supposed to have at that point, and what does the error message actually mean?

Thanks for taking a look!
_________________
This post more meaningful in a scalar context.
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Sun Oct 17, 2004 10:36 pm    Post subject: Reply with quote

cdunham wrote:
Actually, it's a directory, and seems to be on every Gentoo machine I looked at.

In any case, removing it didn't help. I'm not big on python. What is t[0] supposed to have at that point, and what does the error message actually mean?

Thanks for taking a look!


First, the .locks dir isn't on any gentoo machine I have. I still suspect that you are currently or have in the past shared the distfile dir using samba or nfs. I'm curious because it may be something I need to account for in later versions of repcacheman.

But it doesn't matter in this case, the problem seems to be the previous file. Repcacheman skips dir's. I couldn't tell .locks was a dir from your post :-)

I believe the problem is in portage. Could be their error or mine I don't know yet. Sometimes dev's forget something in the portage tree and later correct it. An example is that someone probably forgot to check in the missing Net-SNMP MD5 shown in your repcacheman output:

Code:

Extracting the checksums....
Missing digest: dev-perl/Net-SNMP-4.1.2
Done!


When I run repcacheman I don't see that missing md5 so someone added it after you last sync'd!


Repcacheman uses portage functions to get the md5 of the new files. t[0] should be that md5sum.

Please show me the following:

Code:

cat  /usr/portage/net-libs/libsoup/files/digest-libsoup-2.2.1
ls -l /usr/portage/distfiles/libsoup-2.2.1.tar.bz2
md5sum /usr/portage/distfiles/libsoup-2.2.1.tar.bz2



Then :
Code:

emerge sync
cat  /usr/portage/net-libs/libsoup/files/digest-libsoup-2.2.1


The idea here is to check the md5 listed in portage and the actual md5 of the problem file, sync and see if it changes.


Then run repcacheman again to see if you still get the same error.

If you do, and the md5's do match, then:
Code:

mv /usr/portage/distfiles/libsoup-2.2.1.tar.bz2  /var/cache/http-replicator/


This will fail if /usr and /var are on different partitions, but I wanted to know if they were :-) Just copy the file to replicator's cache, delete the /distifle copy and try repcacheman again.
Back to top
View user's profile Send private message
cdunham
Apprentice
Apprentice


Joined: 06 Jun 2003
Posts: 211
Location: Rhode Island

PostPosted: Mon Oct 18, 2004 2:25 am    Post subject: Reply with quote

flybynite wrote:
Please show me the following:

Code:

cat  /usr/portage/net-libs/libsoup/files/digest-libsoup-2.2.1
ls -l /usr/portage/distfiles/libsoup-2.2.1.tar.bz2
md5sum /usr/portage/distfiles/libsoup-2.2.1.tar.bz2


Code:
% cat  /usr/portage/net-libs/libsoup/files/digest-libsoup-2.2.1
MD5 8132b0bce469affed688c4863702aa41 libsoup-2.2.1.tar.bz2 403907
% ls -l /usr/portage/distfiles/libsoup-2.2.1.tar.bz2
-rw-rw-r--  1 root portage 403907 Oct 16 17:15 /usr/portage/distfiles/libsoup-2.2.1.tar.bz2
% md5sum /usr/portage/distfiles/libsoup-2.2.1.tar.bz2
8132b0bce469affed688c4863702aa41  /usr/portage/distfiles/libsoup-2.2.1.tar.bz2

Quote:


Then :
Code:

emerge sync
cat  /usr/portage/net-libs/libsoup/files/digest-libsoup-2.2.1


Code:
% cat  /usr/portage/net-libs/libsoup/files/digest-libsoup-2.2.1
MD5 8132b0bce469affed688c4863702aa41 libsoup-2.2.1.tar.bz2 403907

Quote:
Then run repcacheman again to see if you still get the same error.

Yup.
Quote:
If you do, and the md5's do match, then:
Code:

mv /usr/portage/distfiles/libsoup-2.2.1.tar.bz2  /var/cache/http-replicator/


This will fail if /usr and /var are on different partitions, but I wanted to know if they were :-) Just copy the file to replicator's cache, delete the /distifle copy and try repcacheman again.

They are on the same partition, so mv worked, but now it seems to have moved on:
Code:
# repcacheman


Replicator's cache directory: /var/cache/http-replicator/
Portage's DISTDIR: /usr/portage/distfiles/

Comparing directories....
Done!

New files in DISTDIR:
.locks
evolution-data-server-1.0.2.tar.bz2
evolution-2.0.2.tar.bz2

Checking authenticity and integrity of new files...
Searching for ebuilds's ....
Done!

Found 16359 ebuilds.

Extracting the checksums....
Missing digest: dev-perl/Net-SNMP-4.1.2
Done!

Verifying checksum's....

WARNING .locks is not in portage!!!

/usr/portage/distfiles/evolution-data-server-1.0.2.tar.bz2
Traceback (most recent call last):
  File "/usr/bin/repcacheman", line 204, in ?
    if t[0]:
KeyError: 0

Note that .locks has returned. This machine has never run samba or nfs. It seems to be coming from emerge --sync, but I'll work more on verifying that.

Hope this helps!
_________________
This post more meaningful in a scalar context.
Back to top
View user's profile Send private message
cdunham
Apprentice
Apprentice


Joined: 06 Jun 2003
Posts: 211
Location: Rhode Island

PostPosted: Mon Oct 18, 2004 6:36 am    Post subject: Reply with quote

Just a little more information. I tracked down the problem with:
Code:
Extracting the checksums....
Missing digest: dev-perl/Net-SNMP-4.1.2
Done!

which was a redundant (and un-Manifested) ebuild in PORTAGE_OVERLAY. removing it cleared up this message, but not the ultimate problem.

However, changing t[0] to t['MD5'] did seem to fix things...
_________________
This post more meaningful in a scalar context.
Back to top
View user's profile Send private message
Bob Shroom
n00b
n00b


Joined: 14 Oct 2003
Posts: 14
Location: Nuremberg, Germany

PostPosted: Mon Oct 18, 2004 9:08 am    Post subject: Reply with quote

@flybynite:

just wanted to let you know, that re-emerging wget did the trick for me.
now the bad box connects to the proxy as it should.

thanks for your help and time... and keep up the good work. :)

.bob
_________________
[planet-earth] root # rm -rf /
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 6, 7, 8 ... 22, 23, 24  Next
Page 7 of 24

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum