Sylpheed-Claws + Bayesian Spamfilter

Lowspirit · Last edited by Lowspirit on Tue Dec 17, 2002 3:45 am; edited 1 time in total

I've always wanted a EASY to use bayesian spamfilter added to my Sylpheed, and this is so far the best solution I've found, peek at this url for files and description :

:arrow:

http://web.purplefrog.com/~thoth/spam/

It works by moving all spam that hit your inbox to "Spam" based on all the spam in your "Oldspam", false positives are close to none after you have trained the filter with a mere 100+ spam mails or so.

It's really easy, I just periodically check in my "Spam" and see if any non-spam are there, and then I move all real spam in there to my "Oldspam" and run my spamupdate.sh script to train my filter with all the new spam thus making it even better at detecting.

Works great for me.

Vancouverite · Posted: Mon Dec 16, 2002 1:40 am Post subject:

Great post! Thanks. I'm going to set this up right away.

kerframil · l33t Joined: 19 Apr 2002 Posts: 712

Brilliant! Thanks.

crencom · n00b Joined: 29 Oct 2002 Posts: 25

Dumb Questions,

The instructions say to download the 3 scripts and put them in your $PATH. What is meant by $PATH? Does this mean to make a separate folder for these files? Where do I save them ?

Also, are the commands in step 4 each run separate in the bash shell or are the commands to be saved as an update script as its own file?

Thx,
Scott

zhware · Retired Dev Joined: 19 Jun 2002 Posts: 23

Just a little note - the post is related to sylpheed-claws. In the usual sylpheed only filters, based
on header fields are possible.

About $PATH - this is list with directories, from where you can start some program/script without specifing the path to it:
so if you have a bin/ directory in your home directory - /home/user/bin and you add this
directory to your PATH environment variable (see .bash_profile file in your home dir) you
can start scripts just with there names, without every time to write path to them.
Like nospam.sh -param1 -param2 in oposite to /home/usr/bin/nospam.sh -param1

How to start commands - it's up to you. I made a script with name nospam.sh, put it in my bin/
directory and i'm running it. this save me typing all commands every time. But you can think different

DNH · n00b Joined: 16 Apr 2002 Posts: 60 Location: Ann Arbor, MI

Is there a simple way to get this to work with an imap email account? I can manually filter all of my messages at once, but I'd like to be able to have them filtered as they are checked with sylpheed-claws. Otherwise, is there another option to get my imap mail to show up in the MH folders rather than its own IMAP folder tree? Thanks!
_________________
The answer

H-Pi · Posted: Wed Jan 08, 2003 1:01 pm Post subject:

edit: forget this, I forgot an '>' in the script :oops:

kerframil · l33t Joined: 19 Apr 2002 Posts: 712

Check the permissions on the referenced file. Stuff that goes into /tmp is supposed to inherit permissions and ownership of the process that put it there. You'd get that error if the open command in the script was denied access.

Really, it might be better to be putting this stuff somewhere else, in a directory that belongs explictly to your user account.

H-Pi · Posted: Wed Jan 08, 2003 7:00 pm Post subject:

<edit>

sessionID · Posted: Wed Jan 08, 2003 7:15 pm Post subject:

[Another quite good solution is POPFile - http://popfile.sourceforge.net/, it's a POP3 proxy.]

NineMinuteNap · n00b Joined: 22 Jan 2003 Posts: 9 Location: Vancouver, WA

Does this work with .maildir directories as well as MH folders?
_________________
Tomorrow I was nothing.
Yesterday I'll be.

oktay · n00b Joined: 21 Jul 2002 Posts: 5

My article about this was recently published on oreillynet.com

http://linux.oreillynet.com/pub/a/linux/synd/2003/01/30/bogofilter.html

oktay

giant · Tux's lil' helper Joined: 01 Aug 2002 Posts: 107

very nice article oktay ;-)

Be sure to read the comments / update at the end of the article.

Set this up last night in a fever haze and I got it running without any problems.

One question though:

I haven't installed the cron job yet because I have several folders in my Mail Dir. Manual I can add them. What I need - I am a bash shell skript noobie - is that I can add an array of dirs to check for good messages.

And a possibility to recursively run through the folders then ... like check all Messages as good in the Folder Mailinglists - but Mailinglist got 4 subfolders and so on ...

Anyways - thanks again for the nice article.

elmie · Posted: Sun May 04, 2003 4:10 pm Post subject:

THis is great, thanks.. I love it.. work like a charm!!

Tyris · Posted: Sun May 04, 2003 6:56 pm Post subject:

While possibly easier for Windows users right now, there's also SpamBayes. It works well also, and it adds an "Unsure" folder so that with training, you should only have to watch the Unsure folder and tell it whether those emails are good or bad. It's Python based, but the team seems to be focusing on Outlook integration. However, there are some scripts (and the source code) available for Unix users, so I'm sure someone could come up with an integrated solution for Linux/Gentoo. (Note: I have noticed how much people on this board get annoyed when n00bs like myself make "requests", so I'm just making sure that you realize I'm only saying "someone could do this", not "give me this now!")

burmashave · Tux's lil' helper Joined: 01 Dec 2002 Posts: 82

ghetto · Guru Joined: 10 Jul 2002 Posts: 369 Location: BC, Canada

Does anyone know if this works for evolution as well? or is it only a slypheed thing?
_________________
Blizzard you suck.

burmashave · Tux's lil' helper Joined: 01 Dec 2002 Posts: 82

I have a feeling that the Bayesian filters on this page won't work with Evolution. It seems to me that you cannot create a filter in Evolution that executes a binary or script as the *condition* of the filter. (You can create a filter action that does this.) Otherwise, I think you could cobble the scripts to make one of these filters work (Most notably, the mail locations are different for Evolution.)

If you can figure out how to execute a binary as the condition of a filter or cobble ~/Evolution/filters.xml to do that, I think you could get it to work in Evolution.

My bad, it seems it can be done. See below (edit)

sessionID · Posted: Tue May 27, 2003 6:41 pm Post subject:

crimson · Posted: Thu May 29, 2003 6:03 pm Post subject:

How do I set up MH style inboxes? right now I use /var/spool/mail/... and I'm using Postfix. Is MH equivilant to mbox?

ghetto · Guru Joined: 10 Jul 2002 Posts: 369 Location: BC, Canada

nikai · Posted: Wed Sep 03, 2003 12:37 am Post subject: Re: bogofilter & sylpheed claws

TGL · Posted: Wed Sep 03, 2003 10:47 am Post subject: Re: bogofilter & sylpheed claws

nikai · Posted: Wed Sep 03, 2003 11:18 am Post subject: Re: bogofilter & sylpheed claws

ronmon · Posted: Mon Sep 15, 2003 2:53 pm Post subject:

TGL, did you read Oktay's article? The 'Marrying bogofilter to Sylpheed' section explains the mechanics. Since bogofilter is in portage, this is incredibly easy. Yes you need sylpheed-claws but that is in portage too. I am using it and it works. Essentially what happens is that anything identified as spam gets moved to the "spam" folder and is displayed as unread. Spam that is missed shows up in the inbox along with "ham" (i.e. good email). Whatever spam that slips through needs to simply be dragged into the spam folder and any false positives that go into spam get dragged to you ham folder. Training takes care of the rest.

I like the second training script the best, modified for my file system layout. It's called bogolearn.sh and lives in ~/bin. Since I use fcron, I just put a link to it in /etc/cron.daily/.

BTW, great article Oktay, but wtf with the n00b status on the forums? Shouldn't an O'Reilly contributor get some kind of honorary upgrade?