Information Resources

Usenet News services at Berkeley

Chris van den Berg

Usenet News is a distributed worldwide discussion system, with different "newsgroups" that are categorized hierarchically based upon subject. Articles (or messages) are "posted" to these newsgroups by people connecting to a Usenet News server, which in turn broadcasts the message to other interconnected Usenet News servers.

The initial purpose of Usenet News was to provide a central repository for accessing discussions on a given topic--in contrast to mailing lists, which send a copy of a message to every person subscribed to the mailing list. People interested in a particular discussion topic can, with the appropriate news reader software on their own computer, connect to a Usenet News server and participate in the different newsgroups available there. Articles belonging to a particular newsgroup are propagated among Usenet sites, creating a network of sites which share common forums.

The Usenet News server for the Berkeley campus is Agate (agate.berkeley.edu). Over the last few years, as the level of Internet usage in general has increased, so has Usenet traffic. There are now as many as 30,000 different Usenet newsgroups covering a wide variety of topics, languages, and specializations. Technical improvements in networks and news servers now guarantee that the vast majority of all Usenet articles arrive at major redistribution points within seconds of their having been posted, making Usenet virtually a "real-time" forum. These same changes have also been accompanied by some difficult obstacles for Usenet.

Foremost among these problems are the level of network traffic generated by Usenet services (a Usenet News server receiving all newsgroups now takes in an estimated 18-20 GB of data per day, the volume having doubled every six months for the last 3 years), and the drastic increase in the amount of "spam" (unsolicited commercial or bulk messages posted frequently or to an excessive number of newsgroups). It is estimated that spam articles, and "cancel" messages (messages requesting that an article be removed from Usenet circulation) now account for the majority of Usenet articles.

Spam filtering

One natural response to the increase in Usenet traffic is to filter incoming articles for spam. This has become common practice at most large Usenet sites over the last two years, and CNS has had spam filters in place on Agate since October 1997. Until Agate's article redistribution was offloaded to a dedicated server (newsfeed.berkeley.edu) this last July, it typically filtered 250,000 to 300,000 spam articles daily, while accepting the same number of articles. Now that news article redistribution ("news peering") has been offloaded to a dedicated news transit host, which also provides a front line for spam filtering, Agate only has to filter about 50,000 articles daily. (More information on newsfeed.berkeley.edu and other recent hardware and software upgrades will be the topic of a future BC&C article). Spam filtering resulted in better server performance, less network traffic, a reduction in server load for peering with other Usenet sites, and, most visible to users, fewer spam articles in newsgroups.

For more information on the details of the spam filter policy in use on Agate, refer to Usenet Spam Filter Policy [Draft] (http://www.net.berkeley.edu/dcns/usenet/spam-policy.html).

The ucb.* and ucb.class.* hierarchies

One of the newsgroup hierarchies available on Agate is the ucb.* hierarchy, which is dedicated to issues specific to the Berkeley campus. There are currently almost 400 different newsgroups within the ucb.* hierarchy, the most widely used being in the ucb.class.* hierarchy.

These "class" newsgroups groups are specifically for the discussion of class-related topics. Increasingly, the class newsgroups have become a critical tool for instruction outside of the classroom. In many cases, homework or other assignments can be distributed via a newsgroup, and the class newsgroups provide an easily accessible forum for students to exchange ideas and questions with instructors as well as other students.

For more information on how to request a newsgroup in the ucb.* or ucb.class.* hierarchies, please see Creating a Newsgroup at UC Berkeley (http://www.net.berkeley.edu/dcns/usenet/ucbcreate.html).

Access to Usenet News service and distribution of the ucb.* newsgroups

Access to Agate for reading news and posting news articles is restricted by IP address to UC Berkeley students, faculty, and staff. Also, because the ucb.* hierarchy is intended for postings from members of the campus community, Agate does not accept articles in the ucb.* hierarchy from other news servers with which it peers.

Many other Usenet sites do carry the ucb.* newsgroups, and receive the articles posted to those newsgroups. In effect, this means that the ucb.* hierarchy is "read-only" outside of Berkeley. There are a few exceptions to this read-only policy, for example the ucb.alumni newsgroup, which is a group for topics related to Berkeley alumni, most of whom do not have access to Agate.

Another exception to the read-only status of the ucb.* newsgroups are the ucb.* email aliases available on Agate, which have been created to forward an email message into a newsgroup posting. The email aliases are of the same format as the newsgroup, with the dots replaced by dashes, and "@agate.berkeley.edu" suffixed. An example would be the newsgroup ucb.class.cs61a, the class newsgroup for Computer Science 61A. Email sent to

ucb-class-cs61a@agate.berkeley.edu

would be forwarded into the ucb.class.cs61a newsgroup.

If you have comments or questions about this article, please send mail to Chris van den Berg, chrisvdb@ack.berkeley.edu. More information about Usenet services, both on the Berkeley campus and in general, may be found at Usenet Services at UC Berkeley (http://www.net.berkeley.edu/dcns/usenet/).

IST | UC Berkeley ]

Berkeley Computing & Communications, Volume 8, Number 5 (November-December 1998)
Copyright 1998, The Regents of the University of California