View previous topic :: View next topic |
Author |
Message |
Lowspirit Apprentice


Joined: 31 Jul 2002 Posts: 258 Location: Northern Sweden
|
Posted: Sun Dec 15, 2002 4:32 pm Post subject: Sylpheed-Claws + Bayesian Spamfilter |
|
|
I've always wanted a EASY to use bayesian spamfilter added to my Sylpheed, and this is so far the best solution I've found, peek at this url for files and description :
http://web.purplefrog.com/~thoth/spam/
It works by moving all spam that hit your inbox to "Spam" based on all the spam in your "Oldspam", false positives are close to none after you have trained the filter with a mere 100+ spam mails or so.
It's really easy, I just periodically check in my "Spam" and see if any non-spam are there, and then I move all real spam in there to my "Oldspam" and run my spamupdate.sh script to train my filter with all the new spam thus making it even better at detecting.
Works great for me.
Last edited by Lowspirit on Tue Dec 17, 2002 3:45 am; edited 1 time in total |
|
Back to top |
|
 |
Vancouverite Apprentice


Joined: 28 Sep 2002 Posts: 162 Location: Vancouver, Canada
|
Posted: Mon Dec 16, 2002 1:40 am Post subject: |
|
|
Great post! Thanks. I'm going to set this up right away. |
|
Back to top |
|
 |
kerframil l33t


Joined: 19 Apr 2002 Posts: 712
|
Posted: Mon Dec 16, 2002 12:20 pm Post subject: |
|
|
Brilliant! Thanks. |
|
Back to top |
|
 |
crencom n00b

Joined: 29 Oct 2002 Posts: 25
|
Posted: Mon Dec 16, 2002 9:19 pm Post subject: |
|
|
Dumb Questions,
The instructions say to download the 3 scripts and put them in your $PATH. What is meant by $PATH? Does this mean to make a separate folder for these files? Where do I save them ?
Also, are the commands in step 4 each run separate in the bash shell or are the commands to be saved as an update script as its own file?
Thx,
Scott |
|
Back to top |
|
 |
zhware Retired Dev


Joined: 19 Jun 2002 Posts: 23
|
Posted: Tue Dec 17, 2002 2:58 am Post subject: just a note |
|
|
Just a little note - the post is related to sylpheed-claws. In the usual sylpheed only filters, based
on header fields are possible.
About $PATH - this is list with directories, from where you can start some program/script without specifing the path to it:
so if you have a bin/ directory in your home directory - /home/user/bin and you add this
directory to your PATH environment variable (see .bash_profile file in your home dir) you
can start scripts just with there names, without every time to write path to them.
Like nospam.sh -param1 -param2 in oposite to /home/usr/bin/nospam.sh -param1
How to start commands - it's up to you. I made a script with name nospam.sh, put it in my bin/
directory and i'm running it. this save me typing all commands every time. But you can think different  |
|
Back to top |
|
 |
DNH n00b


Joined: 16 Apr 2002 Posts: 60 Location: Ann Arbor, MI
|
Posted: Wed Jan 01, 2003 11:08 pm Post subject: |
|
|
Is there a simple way to get this to work with an imap email account? I can manually filter all of my messages at once, but I'd like to be able to have them filtered as they are checked with sylpheed-claws. Otherwise, is there another option to get my imap mail to show up in the MH folders rather than its own IMAP folder tree? Thanks! _________________ The answer |
|
Back to top |
|
 |
H-Pi Apprentice


Joined: 26 Jun 2002 Posts: 175 Location: Delft (NL)
|
Posted: Wed Jan 08, 2003 1:01 pm Post subject: |
|
|
edit: forget this, I forgot an '>' in the script 
Last edited by H-Pi on Sat Jan 11, 2003 1:31 pm; edited 1 time in total |
|
Back to top |
|
 |
kerframil l33t


Joined: 19 Apr 2002 Posts: 712
|
Posted: Wed Jan 08, 2003 4:57 pm Post subject: |
|
|
Check the permissions on the referenced file. Stuff that goes into /tmp is supposed to inherit permissions and ownership of the process that put it there. You'd get that error if the open command in the script was denied access.
Really, it might be better to be putting this stuff somewhere else, in a directory that belongs explictly to your user account. |
|
Back to top |
|
 |
H-Pi Apprentice


Joined: 26 Jun 2002 Posts: 175 Location: Delft (NL)
|
Posted: Wed Jan 08, 2003 7:00 pm Post subject: |
|
|
<edit>
Last edited by H-Pi on Sat Jan 11, 2003 1:32 pm; edited 1 time in total |
|
Back to top |
|
 |
sessionID Apprentice


Joined: 11 Nov 2002 Posts: 266 Location: hungary
|
|
Back to top |
|
 |
NineMinuteNap n00b

Joined: 22 Jan 2003 Posts: 9 Location: Vancouver, WA
|
Posted: Sun Feb 23, 2003 3:14 pm Post subject: |
|
|
Does this work with .maildir directories as well as MH folders? _________________ Tomorrow I was nothing.
Yesterday I'll be. |
|
Back to top |
|
 |
oktay n00b

Joined: 21 Jul 2002 Posts: 5
|
|
Back to top |
|
 |
giant Tux's lil' helper


Joined: 01 Aug 2002 Posts: 107
|
Posted: Fri Mar 14, 2003 8:27 am Post subject: |
|
|
very nice article oktay
Be sure to read the comments / update at the end of the article.
Set this up last night in a fever haze and I got it running without any problems.
One question though:
I haven't installed the cron job yet because I have several folders in my Mail Dir. Manual I can add them. What I need - I am a bash shell skript noobie - is that I can add an array of dirs to check for good messages.
And a possibility to recursively run through the folders then ... like check all Messages as good in the Folder Mailinglists - but Mailinglist got 4 subfolders and so on ...
Anyways - thanks again for the nice article. |
|
Back to top |
|
 |
elmie Tux's lil' helper


Joined: 12 Sep 2002 Posts: 123 Location: Brisbane - Australia
|
Posted: Sun May 04, 2003 4:10 pm Post subject: |
|
|
THis is great, thanks.. I love it.. work like a charm!!  |
|
Back to top |
|
 |
Tyris n00b


Joined: 04 May 2003 Posts: 44 Location: Cincinnati, OH
|
Posted: Sun May 04, 2003 6:56 pm Post subject: |
|
|
While possibly easier for Windows users right now, there's also SpamBayes. It works well also, and it adds an "Unsure" folder so that with training, you should only have to watch the Unsure folder and tell it whether those emails are good or bad. It's Python based, but the team seems to be focusing on Outlook integration. However, there are some scripts (and the source code) available for Unix users, so I'm sure someone could come up with an integrated solution for Linux/Gentoo. (Note: I have noticed how much people on this board get annoyed when n00bs like myself make "requests", so I'm just making sure that you realize I'm only saying "someone could do this", not "give me this now!") |
|
Back to top |
|
 |
burmashave Tux's lil' helper


Joined: 01 Dec 2002 Posts: 82
|
Posted: Tue May 27, 2003 3:51 am Post subject: Re: bogofilter & sylpheed claws |
|
|
Superb article. I got it up and running, and it's working. Note that bogofilter is in portage. |
|
Back to top |
|
 |
ghetto Guru


Joined: 10 Jul 2002 Posts: 369 Location: BC, Canada
|
Posted: Tue May 27, 2003 6:56 am Post subject: |
|
|
Does anyone know if this works for evolution as well? or is it only a slypheed thing? _________________ Blizzard you suck. |
|
Back to top |
|
 |
burmashave Tux's lil' helper


Joined: 01 Dec 2002 Posts: 82
|
Posted: Tue May 27, 2003 5:50 pm Post subject: Bogofilter and Evolution |
|
|
I have a feeling that the Bayesian filters on this page won't work with Evolution. It seems to me that you cannot create a filter in Evolution that executes a binary or script as the *condition* of the filter. (You can create a filter action that does this.) Otherwise, I think you could cobble the scripts to make one of these filters work (Most notably, the mail locations are different for Evolution.)
If you can figure out how to execute a binary as the condition of a filter or cobble ~/Evolution/filters.xml to do that, I think you could get it to work in Evolution.
My bad, it seems it can be done. See below (edit)
Last edited by burmashave on Tue May 27, 2003 6:57 pm; edited 1 time in total |
|
Back to top |
|
 |
sessionID Apprentice


Joined: 11 Nov 2002 Posts: 266 Location: hungary
|
|
Back to top |
|
 |
crimson Guru


Joined: 27 Apr 2002 Posts: 430 Location: Cedar Rapids, IA
|
Posted: Thu May 29, 2003 6:03 pm Post subject: |
|
|
How do I set up MH style inboxes? right now I use /var/spool/mail/... and I'm using Postfix. Is MH equivilant to mbox? |
|
Back to top |
|
 |
ghetto Guru


Joined: 10 Jul 2002 Posts: 369 Location: BC, Canada
|
Posted: Sun Jun 01, 2003 10:27 pm Post subject: |
|
|
sessionID wrote: | ghetto wrote: | Does anyone know if this works for evolution as well? or is it only a slypheed thing? |
You'll find a HOWTO in the latest Gentoo newsletter.
(http://www.gentoo.org/news/en/gwn/gwn.xml) |
However that seems to be a pretty poor tip since it doesnt even explain how (or even mention the need) to teach bogofilter what is spam and what is not. Enless Im mistaken bogofilter cant automatically guess what spam is, you need to tell it that email of a certain kind is spam and then it starts to filter for that sort of email based on keywords.. I dont see how this GWN tip could even work. Not trying to be offense, maybe I just dont understand.
This is a bit of a better guide imho however even it seem to get a few things incorrent since it doesnt mention switching to MH format instead of mbox. (Im asuming thats a requirement)
http://www.ime.usp.br/~rsilva/bogo-and-evo/
However both guides are poorly worded in places. Once I figure this out maybe Im going to write a new guide. Both guides fail to specify if they are refering to vfolder or regular folders. The GWN tip also fails to mention what the filter action should be for the second filter. _________________ Blizzard you suck. |
|
Back to top |
|
 |
nikai Apprentice


Joined: 02 Oct 2002 Posts: 270 Location: Kitzbühel, Austria
|
Posted: Wed Sep 03, 2003 12:37 am Post subject: Re: bogofilter & sylpheed claws |
|
|
Very nice article, I'm now using bogofilter + sylpheed-claws as well.
I found it convenient to define "actions" in sylpheed-claws for training bogofilter:
Code: | bogo spam:
cat %f | bogofilter -s -v
bogo ham:
cat %f | bogofilter -n -v |
But until now, I never needed "bogo ham", only "bogo spam" a couple of times
With shortcut, of course. I don't use "strg-s" for saving messages, so now I'm using it to train bogofilter for spam  |
|
Back to top |
|
 |
TGL Bodhisattva

Joined: 02 Jun 2002 Posts: 1978 Location: Rennes, France
|
Posted: Wed Sep 03, 2003 10:47 am Post subject: Re: bogofilter & sylpheed claws |
|
|
nikai wrote: | I found it convenient to define "actions" in sylpheed-claws for training bogofilter |
That's a good idea. Do you know if it is possible to create actions that would also move the mails to the right folder? I mean:
- "bogo spam" would:
1) train bogofilter with the spam
2) move it to the spam folder
- "bogo ham" would:
1) train bogofilter with the false spam
2) move it back to inbox
I'm not very familiar with actions, is it possible to do such mix with shell command + sylpheed commands? |
|
Back to top |
|
 |
nikai Apprentice


Joined: 02 Oct 2002 Posts: 270 Location: Kitzbühel, Austria
|
Posted: Wed Sep 03, 2003 11:18 am Post subject: Re: bogofilter & sylpheed claws |
|
|
TGL wrote: | Do you know if it is possible to create actions that would also move the mails to the right folder? |
That would be nice, but I haven't found a way to do so.
Not that it's a big problem for me, with "strg-s strg-d" I can bogo-train a mail and move it to trash. |
|
Back to top |
|
 |
ronmon Veteran


Joined: 15 Apr 2002 Posts: 1043 Location: Key West, FL
|
Posted: Mon Sep 15, 2003 2:53 pm Post subject: |
|
|
TGL, did you read Oktay's article? The 'Marrying bogofilter to Sylpheed' section explains the mechanics. Since bogofilter is in portage, this is incredibly easy. Yes you need sylpheed-claws but that is in portage too. I am using it and it works. Essentially what happens is that anything identified as spam gets moved to the "spam" folder and is displayed as unread. Spam that is missed shows up in the inbox along with "ham" (i.e. good email). Whatever spam that slips through needs to simply be dragged into the spam folder and any false positives that go into spam get dragged to you ham folder. Training takes care of the rest.
I like the second training script the best, modified for my file system layout. It's called bogolearn.sh and lives in ~/bin. Since I use fcron, I just put a link to it in /etc/cron.daily/.
BTW, great article Oktay, but wtf with the n00b status on the forums? Shouldn't an O'Reilly contributor get some kind of honorary upgrade? |
|
Back to top |
|
 |
|