Misplaced Pages

#288711

47-542: First appearing in Version 7 Unix , sed is one of the early Unix commands built for command line processing of data files. It evolved as the natural successor to the popular grep command. The original motivation was an analogue of grep (g/re/p) for substitution, hence "g/re/s". Foreseeing that further special-purpose programs for each command would also arise, such as g/re/d, McMahon wrote a general-purpose line-oriented stream editor, which became sed. The syntax for sed, notably

94-410: A one-liner program originally was textual input to the command line of an operating system shell that performed some function in just one line of input. In the present day, a one-liner can be Certain dynamic languages for scripting , such as AWK , sed , and Perl , have traditionally been adept at expressing one-liners. Shell interpreters such as Unix shells or Windows PowerShell allow for

141-615: A PDP-11/70 running Unix Version 7. Many new features were introduced in Version 7. The Portable C Compiler (pcc) was provided along with the earlier, PDP-11-specific, C compiler by Ritchie . These first appeared in the Research Unix lineage in Version 7, although early versions of some of them had already been picked up by PWB/UNIX . A feature that did not survive long was a second way (besides pipes) to do inter-process communication : multiplexed files. A process could create

188-413: A few symbols. In the 1970s, one-liners became associated with the rise of the home computer and BASIC . Computer magazines published type-in programs in many dialects of BASIC. Some magazines devoted regular columns solely to impressive short and one-line programs. The word One-liner also has two references in the index of the book The AWK Programming Language (the book is often referred to by

235-404: A function that can be used to return a list of primes up to the value of the first parameter: It can be used on the command line, like this: to print out a comma-separated list of primes in the range 2 - number. The following Haskell program is a one-liner: it sorts its input lines ASCIIbetically . An even shorter version: Usable on the command line like: The following Racket program

282-439: A line in a way more complicated than a regex extracting and template replacement, though arbitrarily complicated transforms are in principle possible by using the hold buffer. Conversely, for simpler operations, specialized Unix utilities such as grep (print lines matching a pattern), head (print the first part of a file), tail (print the last part of a file), and tr (translate or delete characters) are often preferable. For

329-426: A script file such as subst.sed , and then use the -f option to run the commands (such as s/x/y/g ) from the file: Any number of commands may be placed into the script file, and using a script file also avoids problems with shell escaping or substitutions. Such a script file may be made directly executable from the command line by prepending it with a " shebang line" containing the sed command and assigning

376-449: A space. A separate special buffer, the hold space , may be used by a few sed commands to hold and accumulate text between cycles. sed's command language has only two variables (the "hold space" and the "pattern space") and GOTO -like branching functionality; nevertheless, the language is Turing-complete , and esoteric sed scripts exist for games such as sokoban , arkanoid , chess , and tetris . A main loop executes for each line of

423-535: A special type of file with the mpx system call; other processes could then open this file to get a "channel", denoted by a file descriptor , which could be used to communicate with the process that created the multiplexed file. Mpx files were considered experimental, not enabled in the default kernel, and disappeared from later versions, which offered sockets (BSD) or CB UNIX 's IPC facilities (System V) instead (although mpx files were still present in 4.1BSD ). One-liner program In computer programming ,

470-434: A string that corresponds to a list of three elements: Strings with unbalanced quotes or braces, or non-space characters directly following closing braces, cannot be parsed as lists directly. You can explicitly split them to make a list. The "constructor" for lists is of course called list. It's recommended to use when elements come from variable or command substitution (braces won't do that). As Tcl commands are lists anyway,

517-446: A web server with CGI. To start the web server enter: Tcl (Tool Command Language) is a dynamic programming/scripting language based on concepts of Lisp, C, and Unix shells. It can be used interactively, or by running scripts (programs) which can use a package system for structuring. Many strings are also well-formed lists. Every simple word is a list of length one, and elements of longer lists are separated by whitespace. For instance,

SECTION 10

#1732765861289

564-510: Is equivalent to the above Haskell example: and this can be used on the command line as follows: Performing one-liners directly on the Unix command line can be accomplished by using Python 's -cmd flag (-c for short), and typically requires the import of one or more modules. Statements are separated using ";" instead of newlines. For example, to print the last field of unix long listing: Several open-source scripts have been developed to facilitate

611-431: Is possible to write terse one-liner programs . For example, the sed program given by: will print the first 10 lines of input, then stop. The following example shows a typical, and the most common, use of sed: substitution. This usage was indeed the original motivation for sed: In some versions of sed, the expression must be preceded by -e to indicate that an expression follows. The s stands for substitute, while

658-432: The g stands for global, which means that all matching occurrences in the line would be replaced. The regular expression (i.e. pattern) to be searched is placed after the first delimiting symbol (slash here) and the replacement follows the second symbol. Slash ( / ) is the conventional symbol, originating in the character for "search" in ed, but any other could be used to make syntax more readable if it does not occur in

705-424: The { instruction starts a subsequence of commands (up to the matching } ); in most cases, it will be conditioned by an address pattern. Under Unix, sed is often used as a filter in a pipeline : That is, a program such as "generateData" generates data, and then sed makes the small change of replacing x with y . For example: It is often useful to put several sed commands, one command per line, into

752-416: The command line ( -e option) or read from a separate file ( -f option). Commands in the sed script may take an optional address, in terms of line numbers or regular expressions . The address determines when the command is run. For example, 2d would only run the d (delete) command on the second input line (printing all lines but the second), while /^ /d would delete all lines beginning with

799-419: The pattern space . Each line read starts a cycle . To the pattern space, sed applies one or more operations which have been specified via a sed script . sed implements a programming language with about 25 commands that specify the operations on the text. For each input line, after running the script, sed ordinarily outputs the pattern space (the line as modified by the script) and begins the cycle again with

846-532: The GNU Project wrote a new version of sed based on the new GNU regular expression library. The current minised contains some extensions to BSD sed but is not as feature-rich as GNU sed. Its advantage is that it is very fast and uses little memory. It is used on embedded systems and is the version of sed provided with Minix . sed is a line-oriented text processing utility: it reads text, line by line, from an input stream or file, into an internal buffer called

893-550: The abbreviation TAPL ). It explains the programming language AWK , which is part of the Unix operating system . The authors explain the birth of the one-liner paradigm with their daily work on early Unix machines: The 1977 version had only a few built-in variables and predefined functions. It was designed for writing short programs […] Our model was that an invocation would be one or two lines long, typed in and used immediately. Defaults were chosen to match this style […] We, being

940-459: The authors, knew how the language was supposed to be used, and so we only wrote one-liners. Notice that this original definition of a one-liner implies immediate execution of the program without any compilation. So, in a strict sense, only source code for interpreted languages qualifies as a one-liner . But this strict understanding of a one-liner was broadened in 1985 when the IOCCC introduced

987-487: The better-known demo one-liners is colloquially known as 10PRINT , written for the Commodore 64 : The following example is a C program (a winning entry in the "Best one-liner" category of the IOCCC ). This one-liner program is a glob pattern matcher. It understands the glob characters * , meaning zero or more characters, and ? , meaning exactly one character, just like most Unix shells . Run it with two args,

SECTION 20

#1732765861289

1034-452: The category of Best One Liner for C , which is a compiled language. One-liners are also used to show off the differential expressive power of programming languages . Frequently, one-liners are used to demonstrate programming ability. Contests are often held to see who can create the most exceptional one-liner. A single line of BASIC can typically hold up to 255 characters, and one liners ranged from simple games to graphical demos. One of

1081-403: The construction of Python one-liners. Scripts such as pyp or Pyline import commonly used modules and provide more human-readable variables in an attempt to make Python functionality more accessible on the command line. Here is a redo of the above example (printing the last field of a unix long listing): The Python CGIHTTPServer module for example is also an executable library that performs as

1128-458: The construction of powerful one-liners. The use of the phrase one-liner has been widened to also include program-source for any language that does something useful in one line. The concept of a one-liner program has been known since the 1960s with the release of the APL programming language. With its terse syntax and powerful mathematical operators, APL allowed useful programs to be represented in

1175-736: The edition of the user's manual with which they were accompanied. Released in 1979, the Seventh Edition was preceded by Sixth Edition , which was the first version licensed to commercial users. Development of the Research Unix line continued with the Eighth Edition , which incorporated development from 4.1BSD , through the Tenth Edition, after which the Bell Labs researchers concentrated on developing Plan 9 . V7

1222-453: The executable permission to the file. For example, a file subst.sed can be created with contents: The file may then be made executable by the current user with the chmod command: The file may then be executed directly from the command line: The -i option, introduced in GNU sed, allows in-place editing of files (actually, a temporary output file is created in the background, and then

1269-499: The following text: The sed script below will turn the text above into the following text. Note that the script affects only the input lines that start with a space: The script is: This is explained as: This can be expressed on a single line via semicolons: While simple and limited, sed is sufficiently powerful for a large number of purposes. For more sophisticated processing, more powerful languages such as AWK or Perl are used instead. These are particularly used if transforming

1316-426: The following uses the d command to filter out lines that only contain spaces, or only contain the end of line character: This example uses some of the following regular expression metacharacters (sed supports the full range of regular expressions): Complex sed constructs are possible, allowing it to serve as a simple, but highly specialized, programming language . Flow of control, for example, can be managed by

1363-422: The input stream, evaluating the sed script on each line of the input. Lines of a sed script are each a pattern-action pair, indicating what pattern to match and which action to perform, which can be recast as a conditional statement . Because the main loop, working variables (pattern space and hold space), input and output streams, and default actions (copy line to pattern space, print pattern space) are implicit, it

1410-464: The kernel for the PDP-11 line of computers including significantly improved hardware error recovery and many additional device drivers. UEG evolved into the group that later developed Ultrix . Due to its power yet elegant simplicity, many old-time Unix users remember V7 as the pinnacle of Unix development and have dubbed it "the last true Unix", an improvement over all preceding and following Unices. At

1457-467: The matching and substitution operators. GNU sed added several new features, including in-place editing of files. Super-sed is an extended version of sed that includes regular expressions compatible with Perl . Another variant of sed is minised , originally reverse-engineered from 4.1BSD sed by Eric S. Raymond and currently maintained by René Rebe . minised was used by the GNU Project until

sed - Misplaced Pages Continue

1504-473: The next line. Other end-of-script behaviors are available through sed options and script commands, e.g. d to delete the pattern space, q to quit, N to add the next line to the pattern space immediately, and so on. Thus a sed script corresponds to the body of a loop that iterates through the lines of a stream, where the loop itself and the loop variable (the current line number) are implicit and maintained by sed. The sed script can either be specified on

1551-583: The operating system version. More recent systems have seen incredible growth in the number of supported system calls. Linux 5.15.0 has 449 system calls and FreeBSD 8.0 has over 450. In 2002, Caldera International released V7 as FOSS under a permissive BSD-like software license . Bootable images for V7 can still be downloaded today, and can be run on modern hosts using PDP-11 emulators such as SIMH . An x86 port has been developed by Nordier & Associates. Paul Allen maintained several publicly accessible historic computer systems, including

1598-409: The original file is replaced by the temporary file). For example: This "Hello, world!" script is in a file (e.g., script.txt) and invoked with sed -f script.txt inputFileName , where "inputFileName" is the input text file. The script changes "inputFileName" line #1 to "Hello, world!" and then quits, printing the result before sed exits. Any input lines past line #1 are not read, and not printed. So

1645-415: The pattern or replacement; this is useful to avoid " leaning toothpick syndrome ". The substitution command, which originates in search-and-replace in ed, implements simple parsing and templating . The regexp provides both pattern matching and saving text via sub-expressions, while the replacement can be either literal text, or a format string containing the characters & for "entire match" or

1692-427: The sole output is "Hello, world!". The example emphasizes many key characteristics of sed: Below follow various sed scripts; these can be executed by passing as an argument to sed, or put in a separate file and executed via -f or by making the script itself executable. To replace any instance of a certain word in a file with "REDACTED", such as an IRC password, and save the result: To delete any line containing

1739-487: The special escape sequences \1 through \9 for the n th saved sub-expression. For example, sed -r "s/(cat|dog)s?/\1s/g" replaces all occurrences of "cat" or "dog" with "cats" or "dogs", without duplicating an existing "s": (cat|dog) is the 1st (and only) saved sub-expression in the regexp, and \1 in the format string substitutes this into the output. Besides substitution, other forms of simple processing are possible, using some 25 sed commands. For example,

1786-581: The specific tasks they are designed to carry out, such specialized utilities are usually simpler, clearer, and faster than a more general solution such as sed. The ed/sed commands and syntax continue to be used in descendent programs, such as the text editors vi and vim . An analog to ed/sed is sam /ssam, where sam is the Plan 9 editor, and ssam is a stream interface to it, yielding functionality similar to sed. Version 7 Unix Version 7 Unix , also called Seventh Edition Unix , Version 7 or just V7 ,

1833-406: The string and the glob pattern. The exit status is 0 (shell true) when the pattern matches, 1 otherwise. The glob pattern must match the whole string, so you may want to use * at the beginning and end of the pattern if you are looking for something in the middle. Examples: The book The AWK Programming Language contains 20 examples of one-liners at the end of the book's first chapter. Here are

1880-573: The time of its release, though, its greatly extended feature set came at the expense of a decrease in performance compared to V6, which was to be corrected largely by the user community. The number of system calls in Version 7 was only around 50, while later Unix and Unix-like systems continued to add many more: Version 7 of the Research UNIX System provided about 50 system calls, 4.4BSD provided about 110, and SVR4 had around 120. The exact number of system calls varies depending on

1927-524: The use of / for pattern matching , and s/// for substitution, originated with ed , the precursor to sed, which was in common use at the time, and the regular expression syntax has influenced other languages, notably ECMAScript and Perl . Later, the more powerful language AWK developed, and these functioned as cousins, allowing powerful text processing to be done by shell scripts . sed and AWK are often cited as progenitors and inspiration for Perl, and influenced Perl's syntax and semantics, notably in

sed - Misplaced Pages Continue

1974-423: The use of a label (a colon followed by a string) and the branch instruction b , as well as the conditional branch t . An instruction b followed by a valid label name will move processing to the command following that label. The t instruction will only do so if there was a successful substitution since the previous t (or the start of the program, in case of the first t encountered). Additionally,

2021-493: The very first of them: Here are examples in J : Here are examples in the Perl programming language : Many one-liners are practical. For example, the following Perl one-liner will reverse all the bytes in a file: While most Perl one-liners are imperative, Perl's support for anonymous functions, closures, map, filter (grep) and fold (List::Util::reduce) allows the creation of 'functional' one-liners. This one-liner creates

2068-439: The word "yourword" (the address is '/yourword/'): To delete all instances of the word "yourword": To delete two words from a file simultaneously: To express the previous example on one line, such as when entering at the command line, one may join two commands via the semicolon: In the next example, sed, which usually only works on one line, removes newlines from sentences where the second line starts with one space. Consider

2115-423: Was an important early release of the Unix operating system . V7, released in 1979, was the last Bell Laboratories release to see widespread distribution before the commercialization of Unix by AT&T Corporation in the early 1980s. V7 was originally developed for Digital Equipment Corporation 's PDP-11 minicomputers and was later ported to other platforms. Unix versions from Bell Labs were designated by

2162-707: Was derived from V7 and Onyx Systems soon produced a Zilog Z8000 computer running V7. The VAX port of V7, called UNIX/32V , was the direct ancestor of the popular 4BSD family of Unix systems. The group at the University of Wollongong that had ported V6 to the Interdata 7/32 ported V7 to that machine as well. Interdata sold the port as Edition VII, making it the first commercial UNIX offering. DEC distributed their own PDP-11 version of V7, called V7M (for modified). V7M, developed by DEC's original Unix Engineering Group (UEG), contained many enhancements to

2209-515: Was the first readily portable version of Unix. As this was the era of minicomputers , with their many architectural variations, and also the beginning of the market for 16-bit microprocessors, many ports were completed within the first few years of its release. The first Sun workstations (then based on the Motorola 68000 ) ran a V7 port by UniSoft ; the first version of Xenix for the Intel 8086

#288711