Misplaced Pages

#235764

50-413: AEGIS is distinctive mainly for being designed for the networked computer, as distinct from its competitors, which are essentially standalone systems with added network features. The prime examples of this are the file system , which is fully integrated across machines, as opposed to Unix which draws a distinction between file systems on the host system and on others, and the user administration system, which

100-471: A GUI named DM (Display Manager). DM is integrated with the operating system's own window manager known as wmgr (Window Manager). The DM contains two built-in functions, a text editor and a transcript, which is a kind of virtual terminal . Additional functions can be added by user programs. One of the unique features of the DM is "universal editing". All text in any of the built-in windows can be edited using

150-501: A multitasking OS , by launching all elements at the same time as processes, and automatically servicing the data read requests by each process with the data written by the upstream process. This can be called a multiprocessed pipeline. In this way, the scheduler will naturally switch the CPU among the processes so as to minimize its idle time. In other common models, elements are implemented as lightweight threads or as coroutines to reduce

200-464: A record-oriented filesystem and connected I/O devices operate in record mode rather than stream mode. As a consequence, data in CMS Pipelines is handled in record mode. For text files, a record holds one line of text. In general, CMS Pipelines does not buffer the data but passes records of data in a lock-step fashion from one program to the next. This ensures a deterministic flow of data through

250-424: A device, device type, directory prefix, file path separator, or file type. File systems typically support organizing files into directories , also called folders , which segregate files into groups. This may be implemented by associating the file name with an index in a table of contents or an inode in a Unix-like file system. Directory structures may be flat (i.e. linear), or allow hierarchies by allowing

300-672: A directory to contain directories, called subdirectories. The first file system to support arbitrary hierarchies of directories was used in the Multics operating system. The native file systems of Unix-like systems also support arbitrary directory hierarchies, as do, Apple 's Hierarchical File System and its successor HFS+ in classic Mac OS , the FAT file system in MS-DOS 2.0 and later versions of MS-DOS and in Microsoft Windows ,

350-400: A file system can be managed by the user via various utility programs. Pipeline (software) In software engineering , a pipeline consists of a chain of processing elements ( processes , threads , coroutines , functions , etc. ), arranged so that the output of each element is the input of the next. The concept is analogous to a physical pipeline . Usually some amount of buffering

400-443: A file system creates a file, it allocates space for data. Some file systems permit or require specifying an initial space allocation and subsequent incremental allocations as the file grows. To delete a file, the file system records that the file's space is free; available to use for another file. A local file system manages storage space to provide a level of reliability and efficiency. Generally, it allocates storage device space in

450-422: A file system implementation. The physical file system layer provides relatively low-level access to a storage device (e.g. disk). It reads and writes data blocks , provides buffering and other memory management and controls placement of blocks in specific locations on the storage medium. This layer uses device drivers or channel I/O to drive the storage device. A file name , or filename , identifies

500-417: A file to consuming applications and in some cases users. A file name is unique so that an application can refer to exactly one file for a particular name. If the file system supports directories, then generally file name uniqueness is enforced within the context of each directory. In other words, a storage can contain multiple files with the same name, but not in the same directory. Most file systems restrict

550-706: A forked file system on the Macintosh, and Microsoft supports streams in NTFS. Some file systems maintain multiple past revisions of a file under a single file name; the file name by itself retrieves the most recent version, while prior saved version can be accessed using a special naming convention such as "filename;4" or "filename(-4)" to access the version four saves ago. See comparison of file systems#Metadata for details on which file systems support which kinds of metadata. A local file system tracks which areas of storage belong to which file and which are not being used. When

SECTION 10

#1732791327236

600-477: A granular manner, usually multiple physical units (i.e. bytes ). For example, in Apple DOS of the early 1980s, 256-byte sectors on 140 kilobyte floppy disk used a track/sector map . The granular nature results in unused space, sometimes called slack space , for each file except for those that have the rare size that is a multiple of the granular allocation. For a 512-byte allocation, the average unused space

650-407: A lightweight dispatcher to concurrently execute instances of more than 200 built-in programs that implement typical UNIX utilities and interface to devices and operating system services. In addition to the built-in programs, CMS Pipelines defines a framework to allow user-written REXX programs with input and output streams that can be used in the pipeline. Data on IBM mainframes typically resides in

700-618: A network of interconnected pipelines. Beside byte stream-based pipelines, there are also object pipelines. In an object pipeline, processing elements output objects instead of text. PowerShell includes an internal object pipeline that transfers .NET objects between functions within the PowerShell runtime. Channels , found in the Limbo programming language , are other examples of this metaphor. Graphical environments such as RISC OS and ROX Desktop also use pipelines. Rather than providing

750-552: A pipe. Pipes and filters can be viewed as a form of functional programming , using byte streams as data objects. More specifically, they can be seen as a particular form of monad for I/O . The concept of pipeline is also central to the Cocoon web development framework or to any XProc (the W3C Standards) implementations, where it allows a source stream to be modified before eventual display. This pattern encourages

800-647: A pipeline is analogous to function composition . Narrowly speaking, a pipeline is linear and one-directional, though sometimes the term is applied to more general flows. For example, a primarily one-directional pipeline may have some communication in the other direction, known as a return channel or backchannel, as in the lexer hack , or a pipeline may be fully bi-directional. Flows with one-directional trees and directed acyclic graph topologies behave similarly to linear pipelines. The lack of cycles in such flows makes them simple, and thus they may be loosely referred to as "pipelines". Pipelines are often implemented in

850-400: A save dialog box containing a file manager to let the user specify where a program should write data, RISC OS and ROX provide a save dialog box containing an icon (and a field to specify the name). The destination is specified by dragging and dropping the icon. The user can drop the icon anywhere an already-saved file could be dropped, including onto icons of other programs. If the icon

900-467: A similar capability, but with some drawbacks, principally the fact that core administrative tasks still require AEGIS commands. Also, the SR9 permissions system is not fully compatible with Unix behaviour. Domain/OS provides new administrative commands and a more complex permissions system which can be configured to behave properly under any of the three environments. Domain/OS also provides an improved version of

950-431: A stream of bytes . Typically, to read file data, a program provides a memory buffer and the file system retrieves data from the medium and then writes the data to the buffer. A write involves the program providing a buffer of bytes that the file system reads and then stores to the medium. Some file systems, or layers on top of a file system, allow a program to define a record so that a program can read and write data as

1000-585: A structure; not an unorganized sequence of bytes. If a fixed length record definition is used, then locating the n record can be calculated mathematically, which is relatively fast compared to parsing the data for record separators. An identification for each record, also known as a key, allows a program to read, write and update records without regard to their location in storage. Such storage requires managing blocks of media, usually separating key blocks and data blocks. Efficient algorithms can be developed with pyramid structures for locating records. Typically,

1050-432: A user's use of storage space. A file system typically ensures that stored data remains consistent in both normal operations as well as exceptional situations like: Recovery from exceptional situations may include updating metadata, directory entries and handling data that was buffered but not written to storage media. A file system might record events to allow analysis of issues such as: Many file systems access data as

SECTION 20

#1732791327236

1100-405: Is 256 bytes. For 64 KB clusters, the average unused space is 32 KB. Generally, the allocation unit size is set when the storage is configured. Choosing a relatively small size compared to the files stored, results in excessive access overhead. Choosing a relatively large size results in excessive unused space. Choosing an allocation size based on the average size of files expected to be in

1150-448: Is desired that the source process complete its task as soon as possible. E.g., if the source process consists of a command which reads an audio track from a CD and the destination process consists of a command which compresses the waveform audio data to a format like MP3 . In this case, buffering the entire track in a pipe buffer would allow the CD drive to spin down more quickly, and enable

1200-457: Is dropped onto a program's icon, it is loaded and the contents that would otherwise have been saved are passed in on the new program's standard input stream. For instance, a user browsing the world-wide web might come across a .gz compressed image which they want to edit and re-upload. Using GUI pipelines, they could drag the link to their de-archiving program, drag the icon representing the extracted contents to their image editor , edit it, open

1250-602: Is fundamentally network-based. So basic is this orientation that even a standalone Apollo machine cannot be configured without a network card . Domain/OS implements functionality derived from both System V and early BSD Unix systems. It improves on AEGIS by providing a core OS upon which the user can install any or all of three environments : AEGIS, System V Unix, and BSD Unix. This was done in order to provide greater compatibility with Unix; AEGIS version SR9, which immediately preceded Domain/OS (itself numbered SR10) has an optional product called Domain/IX available, which provides

1300-534: Is invisible to the end user and the system still works correctly. However this can degrade performance on some storage hardware that work better with contiguous blocks such as hard disk drives . Other hardware such as solid-state drives are not affected by fragmentation. A file system often supports access control of data that it manages. The intent of access control is often to prevent certain users from reading or modifying certain files. Access control can also restrict access by program in order to ensure that data

1350-510: Is modified in a controlled way. Examples include passwords stored in the metadata of the file or elsewhere and file permissions in the form of permission bits, access control lists , or capabilities . The need for file system utilities to be able to access the data at the media level to reorganize the structures and provide efficient backup usually means that these are only effective for polite users but are not effective against intruders. Methods for encrypting file data are sometimes included in

1400-439: Is provided between consecutive elements. The information that flows in these pipelines is often a stream of records , bytes , or bits , and the elements of a pipeline may be called filters . This is also called the pipe(s) and filters design pattern which is monolithic . Its advantages are simplicity and low cost while its disadvantages are lack of elasticity , fault tolerance and scalability . Connecting elements into

1450-551: Is similar to Unix , in that it has a command line interpreter which understands pipes , redirection , scripting , etc., and invokes other commands as separate programs, but the actual commands themselves are designed to be easier to remember and use than their Unix equivalents, and wildcards are expected to be expanded by individual commands rather than by the command line interpreter itself. The user may embed environment variables in symbolic links , which, for example, allows switching between different versions of Unix by setting

1500-550: Is stored inside the allocation group itself. Additional attributes can be associated on file systems, such as NTFS , XFS , ext2 , ext3 , some versions of UFS , and HFS+ , using extended file attributes . Some file systems provide for user defined attributes such as the author of the document, the character encoding of a document or the size of an image. Some file systems allow for different data collections to be associated with one file name. These separate collections may be referred to as streams or forks . Apple has long used

1550-463: Is supported by the Unix kernel, but few programs use it as it makes for complicated syntax and blocking modes, although some shells do support it via arbitrary file descriptor assignment). Traditional application programs on IBM mainframe operating systems have no standard input and output streams to allow redirection or piping. Instead of spawning processes with external programs, CMS Pipelines features

Domain/OS - Misplaced Pages Continue

1600-680: The NTFS file system in the Windows NT family of operating systems, and the ODS-2 (On-Disk Structure-2) and higher levels of the Files-11 file system in OpenVMS . In addition to data, the file content, a file system also manages associated metadata which may include but is not limited to: A file system stores associated metadata separate from the content of the file. Most file systems store

1650-599: The X Window System , complete with VUE (HP's predecessor to CDE ), but performance tends to be poor. User upgrading from AEGIS SR9 to Domain/OS SR10 was slowed by the fact that many users saw no requirement, by increased disk space requirements, by new and more complex administration tools, by SR10's poorer performance, and by the buggy nature of SR10.0, although later versions are more reliable. However, later HP/Apollo machines (the DN10000, DN2500 and 4xx series workstations) can only run SR10. Unlike many operating systems of

1700-459: The OS overhead often involved with processes. Depending on the OS, threads may be scheduled directly by the OS or by a thread manager. Coroutines are always scheduled by a coroutine manager of some form. Read and write requests are usually blocking operations. This means that the execution of the source process, upon writing, is suspended until all data can be written to the destination process. Likewise,

1750-502: The SYSTYPE environment variable accordingly; symbolic links then point to the appropriate versions of the files. Domain/OS incorporates several ideas from Multics , including single-level store and dynamic linking . File system In computing , a file system or filesystem (often abbreviated to FS or fs ) governs file organization and access. A local file system is a capability of an operating system that services

1800-793: The applications running on the same computer . A distributed file system is a protocol that provides file access between networked computers. A file system provides a data storage service that allows applications to share mass storage . Without a file system, applications could access the storage in incompatible ways that lead to resource contention , data corruption and data loss . There are many file system designs and implementations – with various structure and features and various resulting characteristics such as speed, flexibility, security, size and more. Files systems have been developed for many types of storage devices , including hard disk drives (HDDs), solid-state drives (SSDs), magnetic tapes and optical discs . A portion of

1850-421: The computer main memory can be set up as a RAM disk that serves as a storage device for a file system. File systems such as tmpfs can store files in virtual memory . A virtual file system provides access to files that are either computed on request, called virtual files (see procfs and sysfs ), or are mapping into another, backing storage. From c.  1900 and before the advent of computers

1900-519: The concepts. The logical file system layer provides relatively high-level access via an application programming interface (API) for file operations including open, close, read and write – delegating operations to lower layers. This layer manages open file table entries and per-process file descriptors. It provides file access, directory operations, security and protection. The virtual file system , an optional layer, supports multiple concurrent instances of physical file systems, each of which called

1950-655: The day, which were written in C or assembly language , many Domain/OS components are written in Pascal . Compilers for users are available for C , C++ , Pascal , and Fortran . All of the distributed administration features of Domain/OS are built around a remote procedure call system called NCS RPC. Though RPC was later end-of-lifed with the operating system, HP contributed RPC to the Open Software Foundation , which incorporated its Interface Definition Language (IDL) into their DCE product, from which

2000-468: The execution of the destination process, upon reading, is suspended until at least some of the requested data can be obtained from the source process. This cannot lead to a deadlock , where both processes would wait indefinitely for each other to respond, since at least one of the processes will soon have its request serviced by the operating system, and continue to run. For performance, most operating systems implementing pipes use pipe buffers , which allow

2050-416: The file system. This is very effective since there is no need for file system utilities to know the encryption seed to effectively manage the data. The risks of relying on encryption include the fact that an attacker can copy the data and use brute force to decrypt the data. Additionally, losing the seed means losing the data. Some operating systems allow a system administrator to enable disk quotas to limit

Domain/OS - Misplaced Pages Continue

2100-500: The length of a file name. Some file systems match file names as case sensitive and others as case insensitive. For example, the names MYFILE and myfile match the same file for case insensitive, but different files for case sensitive. Most modern file systems allow a file name to contain a wide range of characters from the Unicode character set. Some restrict characters such as those used to indicate special attributes such as

2150-595: The names of all the files in one directory in one place—the directory table for that directory—which is often stored like any other file. Many file systems put only some of the metadata for a file in the directory table, and the rest of the metadata for that file in a completely separate structure, such as the inode . Most file systems also store metadata not associated with any one particular file. Such metadata includes information about unused regions— free space bitmap , block availability map —and information about bad sectors . Often such information about an allocation group

2200-426: The same editing language. This includes the history displayed in a transcript window, although that text is read-only. In addition, the history is unbounded. It starts from the birth of the process to which it is attached, and older history is never deleted. Each transcript is attached to a mini-input window where the process input can be edited using the same editing language used elsewhere. The AEGIS command interface

2250-421: The same technology was later used for CORBA . One of the original developers went to work for Microsoft , where he developed MSRPC as a fairly compatible clone which today forms a central component of Windows systems. Traces of the history can be seen in protocol names such as nc acn_http. AEGIS is similar to other workstations of the time, in that it uses a high-resolution graphics screen and mouse to provide

2300-488: The save as dialog, and drag its icon to their uploading software. Conceptually, this method could be used with a conventional save dialog box, but this would require the user's programs to have an obvious and easily accessible location in the filesystem. As this is often not the case, GUI pipelines are rare. The name "pipeline" comes from a rough analogy with physical plumbing in that a pipeline usually allows information to flow in only one direction, like water often flows in

2350-414: The source process to provide more data than the destination process is currently able or willing to receive. Under most Unixes and Unix-like operating systems, a special command is also available, typically called "buffer", that implements a pipe buffer of potentially much larger and configurable size. This command can be useful if the destination process is significantly slower than the source process, but it

2400-429: The storage tends to minimize unusable space. As a file system creates, modifies and deletes files, the underlying storage representation may become fragmented . Files and the unused space between files will occupy allocation blocks that are not contiguous. A file becomes fragmented if space needed to store its content cannot be allocated in contiguous blocks. Free space becomes fragmented when files are deleted. This

2450-442: The terms file system , filing system and system for filing were used to describe methods of organizing, storing and retrieving paper documents. By 1961, the term file system was being applied to computerized filing alongside the original meaning. By 1964, it was in general use. A local file system's architecture can be described as layers of abstraction even though a particular file system design may not actually separate

2500-594: The user to remove the CD from the drive before the encoding process has finished. Such a buffer command can be implemented using system calls for reading and writing data. Wasteful busy waiting can be avoided by using facilities such as poll or select or multithreading . Some notable examples of pipeline software systems include: CMS Pipelines is a port of the pipeline idea to VM/CMS and z/OS systems. It supports much more complex pipeline structures than Unix shells, with steps taking multiple input streams and producing multiple output streams. (Such functionality

#235764