Misplaced Pages

Network News Transfer Protocol

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.

The Network News Transfer Protocol ( NNTP ) is an application protocol used for transporting Usenet news articles ( netnews ) between news servers , and for reading/posting articles by the end user client applications. Brian Kantor of the University of California, San Diego , and Phil Lapsley of the University of California, Berkeley , wrote RFC   977 , the specification for the Network News Transfer Protocol, in March 1986. Other contributors included Stan O. Barber from the Baylor College of Medicine and Erik Fair of Apple Computer .

#610389

55-557: Usenet was originally designed based on the UUCP network, with most article transfers taking place over direct point-to-point telephone links between news servers, which were powerful time-sharing systems . Readers and posters logged into these computers reading the articles directly from the local disk. As local area networks and Internet participation proliferated, it became desirable to allow newsreaders to be run on personal computers connected to local networks. The resulting protocol

110-524: A 16-bit checksum of the payload, not including the header. The next byte is the data type and finally, the last byte is the XOR of the header, allowing it to be checked separately from the payload. The control byte consists of three bit-fields in the format TTXXXYYY. TT is the packet type, 0 for control packets (which also requires K=9 to be valid), 1 for alternate data (not used in UUCP), 2 for data, and 3 indicates

165-500: A few phone calls to the administrators of prospective neighbor systems. Neighbor systems were often close enough to avoid all but the most basic charges for telephone calls. uux is remote command execution over UUCP. The uux command is used to execute a command on a remote system , or to execute a command on the local system using files from remote systems. The command is run by the uucico daemon, which handles remote execution requests as simply another kind of file to batch-send to

220-529: A form such as …!bigsite!foovax!barbox!user . This directs people to route their mail to machine bigsite (presumably a well-known and well-connected machine accessible to everybody) and from there through the machine foovax to the account of user user on barbox . Publishing a full path would be pointless, because it would be different, depending on where the sender was. (e.g. Ann at one site may have to send via path gway!tcol!canty!uoh!bigsite!foovax!barbox!user , whereas from somewhere else, Bill has to send via

275-562: A news server with Transport Layer Security (TLS), TCP port 563 is often used. This is sometimes referred to as NNTPS . Alternatively, a plain-text connection over port 119 may be changed to use TLS via the STARTTLS command. In October 2006, the IETF released RFC   3977 , which updates NNTP and codifies many of the additions made over the years since RFC 977. At the same time, the IETF also released RFC   4642 , which specifies

330-517: A number of new link layer protocols. These newer connections also reduced the need for UUCP at all, as newer application protocols developed to take advantage of the new networks. Today, UUCP is rarely used over dial-up links, but is occasionally used over TCP/IP . The number of systems involved, as of early 2006, ran between 1500 and 2000 sites across 60 enterprises. UUCP's longevity can be attributed to its low cost, extensive logging, native failover to dialup, and persistent queue management. UUCP

385-476: A series of file requests. There are four types: After sending an H command, the calling system sends a final packet \20OOOOOO\0 (control-P, six ohs, null-terminator) and the called system responds with \20OOOOOO\0 (control-P, seven ohs, null-terminator). Some systems will simply hang up on the successful reception of the H command and not bother with the final handshake. Within the suite of protocols in UUCP,

440-404: A short packet that re-defines the meaning of K. In a data packet, XXX is the packet number for this packet from 0 to 7, and YYY is the last that was received correctly. This provides up to 8 packets in a window. In a control packet, XXX indicates the command and YYY is used for various parameters. For instance, transfers are started by sending a short control packet with TT=0 (control), XXX=7 and YYY

495-609: A single set of files describing all connections in the network. These files were then published monthly in a newsgroup dedicated to this purpose. The UUCP map files could then be used by software such as "pathalias" to compute the best route path from one machine to another for mail, and to supply this route automatically. The UUCP maps also listed contact information for the sites, and so gave sites seeking to join UUCPNET an easy way to find prospective neighbors. Many UUCP hosts, particularly those at universities, were also connected to

550-746: A system using 4k packets and 7 packet windows (4096x7) would offer performance matching or beating the best file-transfer protocols like ZMODEM . In practice, many implementations only supported a single setting of 64x3. As a result, the g-protocol has an undeserved reputation for poor performance. Confusion over the packet and window sizes led to the G-protocol, differing only in that it always used 4096x3. Taylor UUCP did not support G, but did support any valid requested window or packet size, so remote systems starting G would work fine with Taylor's g, while two Taylor systems could negotiate even faster connections. Telebit modems used protocol spoofing to improve

605-461: Is normally started by having a user log into the target system and then running the UUCP program. In most cases, this is automated by logging into a known user account used for transfers, whose account's shell has been set to uucico . Thus, for automated transfers, another machine simply has to open a modem connection to the called machine and log into the known account. When uucico runs, it will expect to receive commands from another UUCP program on

SECTION 10

#1732772173611

660-409: Is one of the programs in the suite; it provides a user interface for requesting file copy operations. The UUCP suite also includes uux (user interface for remote command execution), uucico (the communication program that performs the file transfers), uustat (reports statistics on recent activity), uuxqt (execute commands sent from remote machines), and uuname (reports the UUCP name of

715-565: Is sometimes still referred to as "NNRP". UUCP Early research and development: Merging the networks and creating the Internet: Commercialization, privatization, broader access leads to the modern Internet: Examples of Internet services: UUCP ( Unix-to-Unix Copy ) is a suite of computer programs and protocols allowing remote execution of commands and transfer of files , email and netnews between computers . A command named uucp

770-629: Is the chat file format, largely inherited by the Expect software package. UUCP was in use over special-purpose high cost links (e.g. marine satellite links) long after its disappearance elsewhere, and still remains in legacy use. In addition to legacy use, in 2021 new and innovative UUCP uses are growing, especially for telecommunications in the HF band, for example, for communities in the Amazon rainforest for email exchange and other uses. A patch to Ian's UUCP

825-557: The Internet in its early years, and e-mail gateways between Internet SMTP -based mail and UUCP mail were developed. A user at a system with UUCP connections could thereby exchange mail with Internet users, and the Internet links could be used to bypass large portions of the slow UUCP network. A "UUCP zone" was defined within the Internet domain namespace to facilitate these interfaces. With this infrastructure in place, UUCP's strength

880-496: The Usenet network, though not for routing; they are used to record, in the header of a message, the nodes through which that message has passed, rather than to direct where it will go next. "Bang path" is also used as an expression for any explicitly specified routing path between network hosts. That usage is not necessarily limited to UUCP, IP routing, email messaging, or Usenet. The concept of delay-tolerant networking protocols

935-461: The " @ notation ", even by sites still using UUCP. A UUCP-only site can register a DNS domain name, and have the DNS server that handles that domain provide MX records that cause Internet mail to that site to be delivered to a UUCP host on the Internet that can then deliver the mail to the UUCP site. UUCPNET was the name for the totality of the network of computers connected through UUCP. This network

990-480: The Chairman of EUnet GB, worked with EUnet Europe to form a pan-European Eunet in 1994. In April 1998 the company together with nearly all of the national European business partners of EUnet was sold to Qwest Communications International, which in turn later merged EUnet into the ill-fated joint-venture KPNQwest . In 2000, it was estimated that KPNQwest was carrying more than 50% of European IP traffic. Some of

1045-628: The DNS; .uucp works where it needs to ; some hosts punt mail out of SMTP queue into uucp queues on gateway machines if a .uucp address is recognized on an incoming SMTP connection. Usenet traffic was originally transmitted over the UUCP protocol using bang paths. These are still in use within Usenet message format Path header lines. They now have only an informational purpose, and are not used for routing, although they can be used to ensure that loops do not occur. In general, like other older e-mail address formats , bang paths have now been superseded by

1100-467: The ISPs operating under the name EUnet today can be traced back to the original EUnet. Most national EUnet affiliates or subsidiaries predated other commercial Internet offerings in the respective countries by many years. Until the early 1990s, nearly every European country had a telecommunications monopoly with an incumbent national PTT . Commercial and non-commercial provision of telecommunications services

1155-405: The UUCP stack. The packet format consisted of a 6-byte header and then between zero and 4096 bytes in the payload. The packet starts with a single \020 (control-P). This is followed by a single byte, known as "K", containing a value of 1 to 8 indicating a packet size from 32 to 4096 bytes, or a 9 indicating a control packet. Many systems only supported K=2, meaning 64 bytes. The next two bytes were

SECTION 20

#1732772173611

1210-406: The adjacent machine name, an exclamation mark (often pronounced bang ), followed by the user name on the adjacent machine. For example, the address barbox!user would refer to user user on adjacent machine barbox . Mail could furthermore be routed through the network, traversing any number of intermediate nodes before arriving at its destination. Initially, this had to be done by specifying

1265-645: The adoption of TCP/IP in Europe beginning in 1988. A separate company, EUnet GB, was founded in 1993 in the United Kingdom, which also played a role in the early commercial Internet in the UK . The roots of EUnet, originally an abbreviation for European UNIX Network, go back to 1982 under the auspices of the EUUG (European UNIX Users Group), later EurOpen, and the first international UUCP connections. FNET

1320-432: The caller's machine and begin a session. The session has three distinct stages: On starting, uucico will respond by sending an identification string, \20Shere= hostname \0 , where \20 is the control-P character, and \0 is a trailing null. The caller's UUCP responds with \20S callername options \0 , where options is a string containing zero or more Unix-like option switches. These can include packet and window sizes,

1375-404: The complete path, with a list of intermediate host names separated by bangs. For example, if machine barbox is not connected to the local machine, but it is known that barbox is connected to machine foovax which does communicate with the local machine, the appropriate address to send mail to would be foovax!barbox!user . User barbox!user would generally publish their UUCP email address in

1430-642: The computers' modems to dial out to other computers, establishing temporary, point-to-point links between them. Each system in a UUCP network has a list of neighbor systems, with phone numbers, login names and passwords, etc. When work (file transfer or command execution requests) is queued for a neighbor system, the uucico program typically calls that system to process the work. The uucico program can also poll its neighbors periodically to check for work queued on their side; this permits neighbors without dial-out capability to participate. Over time, dial-up links were replaced by Internet connections, and UUCP added

1485-492: The entire file is sent as a single long string followed by a whole-file checksum. The similar x-protocol appears to have seen little or no use. d-protocol was similar to x, but intended for use on Datakit networks that connected many of Bell Labs offices. t-protocol originated in the BSD versions of UUCP and like some similar ones, is designed to run over 8-bit error-free TCP/IP links. It has no error correction at all, and

1540-560: The first companies to sell Internet access to the general public. EUnet provided local service through a respective national EUnet business partner in many European countries. In 1990 the Soviet IP-based network RELCOM , mostly operated by DEMOS -powered computers, was connected to the EUnet. EUnet GB Ltd was founded in the United Kingdom by a group of academics as a commercial Internet Service Provider (ISP) in 1993. Jim Omand,

1595-466: The local system). Some versions of the suite include uuencode / uudecode (convert 8-bit binary files to 7-bit text format and vice versa). Although UUCP was originally developed on Unix in the 1970s and 1980s, and is most closely associated with Unix-like systems, UUCP implementations exist for several non-Unix-like operating systems, including DOS , OS/2 , OpenVMS (for VAX hardware only), AmigaOS , classic Mac OS , and even CP/M . UUCP

1650-480: The maximum supported file size, debugging options, and others. Depending on the setup of the two systems, the call may end here. For instance, when the caller responds with their system name, the called system may optionally hang up if it does not recognize the caller, sending the RYou are unknown to me\0 response string and then disconnecting. If the two systems successfully handshake, the caller will now begin to send

1705-440: The network beyond the interconnected university systems. FSUUCP formed the basis for many bulletin board system (BBS) packages such as Galacticomm's Major BBS and Mustang Software 's Wildcat! BBS to connect to the UUCP network and exchange email and Usenet traffic. As an example, UFGATE (John Galvin, Garry Paxinos, Tim Pozar) was a package that provided a gateway between networks running Fidonet and UUCP protocols. FSUUCP

Network News Transfer Protocol - Misplaced Pages Continue

1760-473: The number of packets in a window, then sending another packet with XXX=6 and YYY as the packet length (encoded as it would be in K) and then a third packet that is identical to the first but XXX=5. g-protocol uses a simple sliding window system to deal with potentially long latencies between endpoints. The protocol allows packets to size from 64 to 4096 8-bit bytes, and windows that include 1 to 7 packets. In theory,

1815-582: The path pdp10!router22!bigsite!foovax!barbox!user ). Many users would suggest multiple routes from various large well-known sites, providing even better and perhaps faster connection service from the mail sender. An email address of this form was known as a bang path . Bang paths of eight to ten machines (or hops ) were not uncommon in 1981, and late-night dial-up UUCP links could cause week-long transmission times. Bang paths were often selected by both transmission time and reliability, as messages would often get lost. Some hosts went so far as to try to " rewrite "

1870-427: The path, sending mail via "faster" routes—this practice tended to be frowned upon. The "pseudo-domain" ending .uucp was sometimes used to designate a hostname as being reachable by UUCP networking, although this was never formally registered in the domain name system (DNS) as a top-level domain . The uucp community administered itself and did not mesh well with the administration methods and regulations governing

1925-402: The performance of g-protocol transfers by noticing end-of-packet markers being sent to the remote system and immediately sending an ACK back to the local host, pretending that the remote system had already received the packet and decoded it correctly. This triggered the software stack to send the next packet, so rapidly that the transfer became almost continuous. The data between the two modems

1980-414: The protocol consists simply of breaking up command and file data into 512 or 1024-byte packets to easily fit within typical TCP frames. e-protocol ("e" for Ethernet) was developed by Clem Cole at MASSCOMP and was widely released by Brian Redman in the later HoneyDanBer versions. It was developed and released before the t-protocol, but the t-protocol was more commonly used because the BSD version of UUCP

2035-510: The remote system whenever a next-hop node is available. The remote system will then execute the requested command and return the result, when the original system is available. Both of these transfers may be indirect, via multi-hop paths, with arbitrary windows of availability. Even when executing a command on an always-available neighbor, uux is not instant. UUCP usage began to die out with the rise of Internet service providers offering inexpensive SLIP and PPP services. The UUCP Mapping Project

2090-436: The speed of a 2400 bit/s modem. They were widely used on UUCP hosts as they could quickly pay for themselves in reduced long-distance charges. UUCP implementations also include other transfer protocols for use over certain links. f-protocol is designed to run over 7-bit error-corrected links. This was originally intended for use on X.25 links, which were popular for a time in the 1980s. It does not packetize data, instead,

2145-497: The underlying g-protocol is responsible for transferring information in an error-free form. The protocol originated as a general-purpose system for packet delivery, and thus offers a number of features that are not used by the UUCP package as a whole. These include a secondary channel that can send command data interspersed with a file transfer, and the ability to renegotiate the packet and window sizes during transmission. These extra features may not be available in some implementations of

2200-534: The use of Transport Layer Security (TLS) via NNTP over STARTTLS . During an abortive attempt to update the NNTP standard in the early 1990s, a specialized form of NNTP intended specifically for use by clients, NNRP, was proposed. This protocol was never completed or fully implemented, but the name persisted in InterNetNews 's (INN) nnrpd program. As a result, the subset of standard NNTP commands useful to clients

2255-437: Was rewritten by AT&T researchers Peter Honeyman, David A. Nowitz, and Brian E. Redman around 1983. The rewrite is referred to as HDB or HoneyDanBer uucp, which was later enhanced, bug fixed, and repackaged as BNU UUCP ("Basic Network Utilities"). Each of these versions was distributed as proprietary software, which inspired Ian Lance Taylor to write a new free software version from scratch in 1991. Taylor UUCP

Network News Transfer Protocol - Misplaced Pages Continue

2310-560: Was NNTP, which resembled the Simple Mail Transfer Protocol (SMTP) but was tailored for exchanging newsgroup articles. A newsreader, also known as a news client, is a software application that reads articles on Usenet, either directly from the news server's disks or via the NNTP. The well-known TCP port 119 is reserved for NNTP. Well-known TCP port 433 ( NNSP ) may be used when doing a bulk transfer of articles from one server to another. When clients connect to

2365-400: Was a volunteer, largely successful effort to build a map of the connections between machines that were open mail relays and establish a managed namespace. Each system administrator would submit, by e-mail, a list of the systems to which theirs would connect, along with a ranking for each such connection. These submitted map entries were processed by an automatic program that combined them into

2420-579: Was contributed to UUCP Debian Linux package to adapt for the HERMES (High-Frequency Emergency and Rural Multimedia Exchange System) project, which provides UUCP HF connectivity. In the mid 2000s, UUCP over TCP/IP (often encrypted, using the SSH protocol ) was proposed for use when a computer does not have any fixed IP addresses but is still willing to run a standard mail transfer agent (MTA) like Sendmail or Postfix . Bang-like paths are still in use within

2475-441: Was error-corrected using a proprietary protocol based on MNP that ran over Telebit's half-duplex connections much better than g-protocol would normally, because in the common 64x3 case the remote system would be sending a constant stream of ACK s that would overflow the low-speed return channel. Combined with the modem's naturally higher data rates, they greatly improved overall throughput and generally performed about seven times

2530-551: Was formally shut down in late 2000. The UUCP protocol has now mostly been replaced by the Internet TCP/IP based protocols SMTP for mail and NNTP for Usenet news. In July 2012, Dutch Internet provider XS4ALL closed down its UUCP service, claiming it was "probably one of the last providers in the world that still offered it"; it had only 13 users at that time (prior to its shut-down it had refused requests from new users for several years). One surviving feature of UUCP

2585-557: Was originally written at AT&T Bell Laboratories by Mike Lesk . By 1978 it was in use on 82 UNIX machines inside the Bell system, primarily for software distribution. It was released in 1979 as part of Version 7 Unix . The first UUCP emails from the U.S. arrived in the United Kingdom in 1979 and email between the UK, the Netherlands and Denmark started in 1980, becoming a regular service via EUnet in 1982. The original UUCP

2640-516: Was prohibited or, at least, took place in a legal "grey zone". During the same period, as part of an industrial political strategy to stop US domination of future network technology, the European Community embarked on efforts to promote OSI protocols, founding for example RARE and associated national "research" network operators ( DFN , SURFnet , SWITCH to name a few). During this period, EUnet played an important and decisive role in

2695-799: Was released under the GNU General Public License . Taylor UUCP addressed security holes which allowed some of the original network worms to remotely execute unexpected shell commands. Taylor UUCP also incorporated features of all previous versions of UUCP, allowing it to communicate with any other version and even use similar config file formats from other versions. UUCP was also implemented for non- UNIX operating systems, most-notably DOS systems. Packages such as UUSLAVE/GNUUCP ( John Gilmore , Garry Paxinos, Tim Pozar), UUPC/extended (Drew Derbyshire of Kendra Electronic Wonderworks) and FSUUCP (Christopher Ambler of IODesign), brought early Internet connectivity to personal computers, expanding

2750-400: Was revisited in the early 2000s. Similar techniques as those used by UUCP can apply to other networks that experience delay or significant disruption. EUnet EUnet was a very loose collaboration of individual European UNIX sites in the 1980s that evolved into the fully commercial entity EUnet International Ltd in 1996. It was sold to Qwest in 1998. EUnet played a decisive role in

2805-407: Was that it permitted a site to gain Internet e-mail and Usenet connectivity with only a dial-up modem link to another cooperating computer. This was at a time when true Internet access required a leased data line providing a connection to an Internet Point of Presence , both of which were expensive and difficult to arrange. By contrast, a link to the UUCP network could usually be established with

SECTION 50

#1732772173611

2860-503: Was the French branch of EUnet. Once there was a central European backbone node that was separate from the expensive telecom network, TCP/IP was adopted in place of store and forward. This enabled EUnet to connect with NSFNET in the US and with CERN’s TCP/IP connections. On January 1, 1990 EUnet began selling Internet access to non-academic customers in the Netherlands, making them one of

2915-404: Was the dominant implementation. The e-protocol differs from the t-protocol only in that commands are not packetized and are instead sent as normal strings, while files are padded to the nearest 20 bytes. The uucp and uuxqt capabilities could be used to send email between machines, with suitable mail user interfaces and delivery agent programs. A simple UUCP mail address was formed from

2970-467: Was the only other implementation of Taylor's enhanced 'i' protocol, a significant improvement over the standard 'g' protocol used by most UUCP implementations. Before the widespread availability of Internet access , computers were only connected by smaller local area networks within a company or organization. They were also often equipped with modems so they could be used remotely from character-mode terminals via dial-up telephone lines . UUCP used

3025-422: Was very informal, maintained in a spirit of mutual cooperation between systems owned by thousands of private companies, universities, and so on. Often, particularly in the private sector, UUCP links were established without official approval from the companies' upper management. The UUCP network was constantly changing as new systems and dial-up links were added, others were removed, etc. The UUCP Mapping Project

#610389