Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Sylpheed-Claws + Bayesian Spamfilter
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Lowspirit
Apprentice
Apprentice


Joined: 31 Jul 2002
Posts: 258
Location: Northern Sweden

PostPosted: Sun Dec 15, 2002 4:32 pm    Post subject: Sylpheed-Claws + Bayesian Spamfilter Reply with quote

I've always wanted a EASY to use bayesian spamfilter added to my Sylpheed, and this is so far the best solution I've found, peek at this url for files and description :

:arrow: http://web.purplefrog.com/~thoth/spam/

It works by moving all spam that hit your inbox to "Spam" based on all the spam in your "Oldspam", false positives are close to none after you have trained the filter with a mere 100+ spam mails or so.

It's really easy, I just periodically check in my "Spam" and see if any non-spam are there, and then I move all real spam in there to my "Oldspam" and run my spamupdate.sh script to train my filter with all the new spam thus making it even better at detecting.

Works great for me.


Last edited by Lowspirit on Tue Dec 17, 2002 3:45 am; edited 1 time in total
Back to top
View user's profile Send private message
Vancouverite
Apprentice
Apprentice


Joined: 28 Sep 2002
Posts: 162
Location: Vancouver, Canada

PostPosted: Mon Dec 16, 2002 1:40 am    Post subject: Reply with quote

Great post! Thanks. I'm going to set this up right away.
Back to top
View user's profile Send private message
kerframil
l33t
l33t


Joined: 19 Apr 2002
Posts: 712

PostPosted: Mon Dec 16, 2002 12:20 pm    Post subject: Reply with quote

Brilliant! Thanks.
Back to top
View user's profile Send private message
crencom
n00b
n00b


Joined: 29 Oct 2002
Posts: 25

PostPosted: Mon Dec 16, 2002 9:19 pm    Post subject: Reply with quote

Dumb Questions,

The instructions say to download the 3 scripts and put them in your $PATH. What is meant by $PATH? Does this mean to make a separate folder for these files? Where do I save them ?

Also, are the commands in step 4 each run separate in the bash shell or are the commands to be saved as an update script as its own file?

Thx,
Scott
Back to top
View user's profile Send private message
zhware
Retired Dev
Retired Dev


Joined: 19 Jun 2002
Posts: 23

PostPosted: Tue Dec 17, 2002 2:58 am    Post subject: just a note Reply with quote

Just a little note - the post is related to sylpheed-claws. In the usual sylpheed only filters, based
on header fields are possible.

About $PATH - this is list with directories, from where you can start some program/script without specifing the path to it:
so if you have a bin/ directory in your home directory - /home/user/bin and you add this
directory to your PATH environment variable (see .bash_profile file in your home dir) you
can start scripts just with there names, without every time to write path to them.
Like nospam.sh -param1 -param2 in oposite to /home/usr/bin/nospam.sh -param1

How to start commands - it's up to you. I made a script with name nospam.sh, put it in my bin/
directory and i'm running it. this save me typing all commands every time. But you can think different ;)
Back to top
View user's profile Send private message
DNH
n00b
n00b


Joined: 16 Apr 2002
Posts: 60
Location: Ann Arbor, MI

PostPosted: Wed Jan 01, 2003 11:08 pm    Post subject: Reply with quote

Is there a simple way to get this to work with an imap email account? I can manually filter all of my messages at once, but I'd like to be able to have them filtered as they are checked with sylpheed-claws. Otherwise, is there another option to get my imap mail to show up in the MH folders rather than its own IMAP folder tree? Thanks!
_________________
The answer
Back to top
View user's profile Send private message
H-Pi
Apprentice
Apprentice


Joined: 26 Jun 2002
Posts: 175
Location: Delft (NL)

PostPosted: Wed Jan 08, 2003 1:01 pm    Post subject: Reply with quote

edit: forget this, I forgot an '>' in the script :oops:

Last edited by H-Pi on Sat Jan 11, 2003 1:31 pm; edited 1 time in total
Back to top
View user's profile Send private message
kerframil
l33t
l33t


Joined: 19 Apr 2002
Posts: 712

PostPosted: Wed Jan 08, 2003 4:57 pm    Post subject: Reply with quote

Check the permissions on the referenced file. Stuff that goes into /tmp is supposed to inherit permissions and ownership of the process that put it there. You'd get that error if the open command in the script was denied access.

Really, it might be better to be putting this stuff somewhere else, in a directory that belongs explictly to your user account.
Back to top
View user's profile Send private message
H-Pi
Apprentice
Apprentice


Joined: 26 Jun 2002
Posts: 175
Location: Delft (NL)

PostPosted: Wed Jan 08, 2003 7:00 pm    Post subject: Reply with quote

<edit>

Last edited by H-Pi on Sat Jan 11, 2003 1:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
sessionID
Apprentice
Apprentice


Joined: 11 Nov 2002
Posts: 266
Location: hungary

PostPosted: Wed Jan 08, 2003 7:15 pm    Post subject: Reply with quote

[Another quite good solution is POPFile - http://popfile.sourceforge.net/, it's a POP3 proxy.]
Back to top
View user's profile Send private message
NineMinuteNap
n00b
n00b


Joined: 22 Jan 2003
Posts: 9
Location: Vancouver, WA

PostPosted: Sun Feb 23, 2003 3:14 pm    Post subject: Reply with quote

Does this work with .maildir directories as well as MH folders?
_________________
Tomorrow I was nothing.
Yesterday I'll be.
Back to top
View user's profile Send private message
oktay
n00b
n00b


Joined: 21 Jul 2002
Posts: 5

PostPosted: Mon Feb 24, 2003 6:48 pm    Post subject: bogofilter & sylpheed claws Reply with quote

My article about this was recently published on oreillynet.com

http://linux.oreillynet.com/pub/a/linux/synd/2003/01/30/bogofilter.html


oktay
Back to top
View user's profile Send private message
giant
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2002
Posts: 107

PostPosted: Fri Mar 14, 2003 8:27 am    Post subject: Reply with quote

very nice article oktay ;-)

Be sure to read the comments / update at the end of the article.

Set this up last night in a fever haze and I got it running without any problems.

One question though:

I haven't installed the cron job yet because I have several folders in my Mail Dir. Manual I can add them. What I need - I am a bash shell skript noobie - is that I can add an array of dirs to check for good messages.

And a possibility to recursively run through the folders then ... like check all Messages as good in the Folder Mailinglists - but Mailinglist got 4 subfolders and so on ...

Anyways - thanks again for the nice article.
Back to top
View user's profile Send private message
elmie
Tux's lil' helper
Tux's lil' helper


Joined: 12 Sep 2002
Posts: 123
Location: Brisbane - Australia

PostPosted: Sun May 04, 2003 4:10 pm    Post subject: Reply with quote

THis is great, thanks.. I love it.. work like a charm!! :)
Back to top
View user's profile Send private message
Tyris
n00b
n00b


Joined: 04 May 2003
Posts: 44
Location: Cincinnati, OH

PostPosted: Sun May 04, 2003 6:56 pm    Post subject: Reply with quote

While possibly easier for Windows users right now, there's also SpamBayes. It works well also, and it adds an "Unsure" folder so that with training, you should only have to watch the Unsure folder and tell it whether those emails are good or bad. It's Python based, but the team seems to be focusing on Outlook integration. However, there are some scripts (and the source code) available for Unix users, so I'm sure someone could come up with an integrated solution for Linux/Gentoo. (Note: I have noticed how much people on this board get annoyed when n00bs like myself make "requests", so I'm just making sure that you realize I'm only saying "someone could do this", not "give me this now!")
Back to top
View user's profile Send private message
burmashave
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2002
Posts: 82

PostPosted: Tue May 27, 2003 3:51 am    Post subject: Re: bogofilter & sylpheed claws Reply with quote

oktay wrote:
My article about this was recently published on oreillynet.com

http://linux.oreillynet.com/pub/a/linux/synd/2003/01/30/bogofilter.html

oktay


Superb article. I got it up and running, and it's working. Note that bogofilter is in portage.
Back to top
View user's profile Send private message
ghetto
Guru
Guru


Joined: 10 Jul 2002
Posts: 369
Location: BC, Canada

PostPosted: Tue May 27, 2003 6:56 am    Post subject: Reply with quote

Does anyone know if this works for evolution as well? or is it only a slypheed thing?
_________________
Blizzard you suck.
Back to top
View user's profile Send private message
burmashave
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2002
Posts: 82

PostPosted: Tue May 27, 2003 5:50 pm    Post subject: Bogofilter and Evolution Reply with quote

I have a feeling that the Bayesian filters on this page won't work with Evolution. It seems to me that you cannot create a filter in Evolution that executes a binary or script as the *condition* of the filter. (You can create a filter action that does this.) Otherwise, I think you could cobble the scripts to make one of these filters work (Most notably, the mail locations are different for Evolution.)

If you can figure out how to execute a binary as the condition of a filter or cobble ~/Evolution/filters.xml to do that, I think you could get it to work in Evolution.

My bad, it seems it can be done. See below (edit)


Last edited by burmashave on Tue May 27, 2003 6:57 pm; edited 1 time in total
Back to top
View user's profile Send private message
sessionID
Apprentice
Apprentice


Joined: 11 Nov 2002
Posts: 266
Location: hungary

PostPosted: Tue May 27, 2003 6:41 pm    Post subject: Reply with quote

ghetto wrote:
Does anyone know if this works for evolution as well? or is it only a slypheed thing?


You'll find a HOWTO in the latest Gentoo newsletter.
(http://www.gentoo.org/news/en/gwn/gwn.xml)
_________________
((( WeFunk )))
Back to top
View user's profile Send private message
crimson
Guru
Guru


Joined: 27 Apr 2002
Posts: 430
Location: Cedar Rapids, IA

PostPosted: Thu May 29, 2003 6:03 pm    Post subject: Reply with quote

How do I set up MH style inboxes? right now I use /var/spool/mail/... and I'm using Postfix. Is MH equivilant to mbox?
Back to top
View user's profile Send private message
ghetto
Guru
Guru


Joined: 10 Jul 2002
Posts: 369
Location: BC, Canada

PostPosted: Sun Jun 01, 2003 10:27 pm    Post subject: Reply with quote

sessionID wrote:
ghetto wrote:
Does anyone know if this works for evolution as well? or is it only a slypheed thing?


You'll find a HOWTO in the latest Gentoo newsletter.
(http://www.gentoo.org/news/en/gwn/gwn.xml)


However that seems to be a pretty poor tip since it doesnt even explain how (or even mention the need) to teach bogofilter what is spam and what is not. Enless Im mistaken bogofilter cant automatically guess what spam is, you need to tell it that email of a certain kind is spam and then it starts to filter for that sort of email based on keywords.. I dont see how this GWN tip could even work. Not trying to be offense, maybe I just dont understand.

This is a bit of a better guide imho however even it seem to get a few things incorrent since it doesnt mention switching to MH format instead of mbox. (Im asuming thats a requirement)

http://www.ime.usp.br/~rsilva/bogo-and-evo/

However both guides are poorly worded in places. Once I figure this out maybe Im going to write a new guide. Both guides fail to specify if they are refering to vfolder or regular folders. The GWN tip also fails to mention what the filter action should be for the second filter.
_________________
Blizzard you suck.
Back to top
View user's profile Send private message
nikai
Apprentice
Apprentice


Joined: 02 Oct 2002
Posts: 270
Location: Kitzbühel, Austria

PostPosted: Wed Sep 03, 2003 12:37 am    Post subject: Re: bogofilter & sylpheed claws Reply with quote

oktay wrote:
My article about this was recently published on oreillynet.com

http://linux.oreillynet.com/pub/a/linux/synd/2003/01/30/bogofilter.html


Very nice article, I'm now using bogofilter + sylpheed-claws as well.
I found it convenient to define "actions" in sylpheed-claws for training bogofilter:
Code:
bogo spam:
cat %f | bogofilter -s -v
bogo ham:
cat %f | bogofilter -n -v

But until now, I never needed "bogo ham", only "bogo spam" a couple of times :)
With shortcut, of course. I don't use "strg-s" for saving messages, so now I'm using it to train bogofilter for spam :lol:
Back to top
View user's profile Send private message
TGL
Bodhisattva
Bodhisattva


Joined: 02 Jun 2002
Posts: 1978
Location: Rennes, France

PostPosted: Wed Sep 03, 2003 10:47 am    Post subject: Re: bogofilter & sylpheed claws Reply with quote

nikai wrote:
I found it convenient to define "actions" in sylpheed-claws for training bogofilter

That's a good idea. Do you know if it is possible to create actions that would also move the mails to the right folder? I mean:
- "bogo spam" would:
1) train bogofilter with the spam
2) move it to the spam folder
- "bogo ham" would:
1) train bogofilter with the false spam
2) move it back to inbox

I'm not very familiar with actions, is it possible to do such mix with shell command + sylpheed commands?
Back to top
View user's profile Send private message
nikai
Apprentice
Apprentice


Joined: 02 Oct 2002
Posts: 270
Location: Kitzbühel, Austria

PostPosted: Wed Sep 03, 2003 11:18 am    Post subject: Re: bogofilter & sylpheed claws Reply with quote

TGL wrote:
Do you know if it is possible to create actions that would also move the mails to the right folder?

That would be nice, but I haven't found a way to do so.
Not that it's a big problem for me, with "strg-s strg-d" I can bogo-train a mail and move it to trash.
Back to top
View user's profile Send private message
ronmon
Veteran
Veteran


Joined: 15 Apr 2002
Posts: 1043
Location: Key West, FL

PostPosted: Mon Sep 15, 2003 2:53 pm    Post subject: Reply with quote

TGL, did you read Oktay's article? The 'Marrying bogofilter to Sylpheed' section explains the mechanics. Since bogofilter is in portage, this is incredibly easy. Yes you need sylpheed-claws but that is in portage too. I am using it and it works. Essentially what happens is that anything identified as spam gets moved to the "spam" folder and is displayed as unread. Spam that is missed shows up in the inbox along with "ham" (i.e. good email). Whatever spam that slips through needs to simply be dragged into the spam folder and any false positives that go into spam get dragged to you ham folder. Training takes care of the rest.

I like the second training script the best, modified for my file system layout. It's called bogolearn.sh and lives in ~/bin. Since I use fcron, I just put a link to it in /etc/cron.daily/.

BTW, great article Oktay, but wtf with the n00b status on the forums? Shouldn't an O'Reilly contributor get some kind of honorary upgrade?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum