Community Operations, and How to Deal with Spam and Bad Content

Sites like TGC grow because of good content. But many UGC companies we have talked with, including community platforms, constantly suffer because of malicious content. Dealing with that requires continuous analysis and significant technical investment. In addition to algorithms and heuristic methods, companies like Google and Facebook hire full-time staff to filter out criminal and malicious content, and the work is extremely difficult.

People who post spam often create throwaway accounts, which are comparatively easy to find. Cases involving stolen user accounts are much harder. Most UGC sites give users the ability to flag spam content, which makes review easier. But relying on users to identify bad content is not enough. On Reddit, for example, many posts flagged as bad content were actually marked that way by spammers trying to promote their own content. That forced Reddit to build systems for evaluating the quality of each user's spam reports.

At Reddit, automated filters and administrators together found most spam. In 2011, roughly half of user-submitted content was spam, though it came from a much smaller share of users. The team studied successful bad actors, identified how they operated, and used those patterns to build detection models.

Later, Reddit even devised an ad-revenue model that made use of spammers' tendencies. Since spammers wanted to force their links into visibility, the platform realized it might as well charge money to place those links openly as sponsored content.

The point is simple. As your site becomes more popular, you will spend considerable time and money fighting spam. From the beginning, you should start deciding what counts as good content, what counts as bad content, and which users are good at identifying that difference. Since content quality is a leading indicator of user satisfaction, you need to watch for declines in quality and respond quickly before the community weakens.