Automated community moderation for reducing spam

Published Mon, Aug 30, 2010 by Austin Govella. Updated Mon, Aug 30, 2010.

Some ways to improve online community moderation, and I wanted to get some feedback. But before we jump, three assumptions:

First, the community has a reputation system. This can be explicit as a reputation score visible in the interface. Or, implicit as a score you manage behind the scenes.

Second, you can assign members to levels. Specifically, I'm thinking of a "new user" who is defined as any member with a reputation score below 10 (where ten is an easy to reach number, but just beyond casual one- or two-time use).

Third, there are classes of users: free, watched, moderated, and banned.

(A lot to assume, but I'm not trying to theorize. I'm defining specific things in a specific system.)

(The names aren't important. They're just handles for conversation.)

Training wheels - New users (reputation below 10) are automatically "watched". They can post, but their posts are automatically emailed to moderators so they can be managed. New members can join and participate, but moderators can catch spammers very early.

Time out - If a new user (reputation below 10) posts more than three comments in one day, switch them from "watched" to "moderated". They can post, but posts are held, and they're are told they're moderated. Posts are emailed to moderators for approval. This stops massive spamming on weekends when mods aren't paying attention.

Ministry of Truth - Content by banned members is immediately hidden. If there is child content by valid members, it's hidden, too. If you have threaded conversations and you ban member A's initial post, then responses by member's B, C, and D are also hidden. This removes flames, trolls, and whining from your communities history.

Speed trap - If any member posts content with more than two links (or some appropriate threshold), graduate them one "class": from free to watched, from watched to moderated, or from moderated to banned. This lets the system catch spammers if they manage to hack the accounts of trusted community members.

Big kid's table - Don't count when new users (reputation below 10) boost another user's reputation. This prevents spammers from creating fake accounts and boosting a shill's reputation, so they move out of watched status. (New users should still be allowed to accrue their own reputation through behavior on the site.)

These would all be in addition to your normal tools for marking spam and abuse, and this assumes you have a system with "moderator-useful touch points"; especially good notification emails. The emails should include links to move members up or down classes (free, watched, moderated, banned), as well as to approve, hide, or delete content. And a way to message the member. Maybe the new user is really excited and posted ten good comments with fifteen useful links in each.

Any feedback on how to vet new users, limit abuse, and automate some of the moderation would be awesome.