Misplaced Pages

#765234

60-449: PostGIS is implemented as a PostgreSQL external extension . The PostGIS implementation is based on "light-weight" geometries and indexes optimized to reduce disk and memory footprint . Using light-weight geometries helps servers increase the amount of data migrated up from physical disk storage into RAM, improving query performance substantially. PostGIS is registered as "implements the specified standard" for "Simple Features for SQL" by

120-421: A VMM, guest OS or guest application stack needs three. The difference between paging and segmentation systems is not only about memory division; segmentation is visible to user processes, as part of memory model semantics. Hence, instead of memory that looks like a single large space, it is structured into multiple spaces. This difference has important consequences; a segment is not a page with variable length or

180-426: A combination of hardware and software, maps memory addresses used by a program, called virtual addresses , into physical addresses in computer memory . Main storage , as seen by a process or task, appears as a contiguous address space or collection of contiguous segments . The operating system manages virtual address spaces and the assignment of real memory to virtual memory. Address translation hardware in

240-402: A constrained resource – so much so that developers typically sacrificed efficiency (processing speeds) just to make program footprints small enough to fit into the available RAM. For example, Sun Microsystems brought out a version of its Java Virtual Machine (JVM) for such constrained devices; it goes by the name of KVM . The KVM works on platforms where memory is in kilobytes as opposed to

300-425: A file (or a segment from a multi-segment file) is mapped into a segment in the address space, so files are always mapped at a segment boundary. A file's linkage section can contain pointers for which an attempt to load the pointer into a register or make an indirect reference through it causes a trap. The unresolved pointer contains an indication of the name of the segment to which the pointer refers and an offset within

360-416: A large free block from which further segments may be allocated. Since there is a single master descriptor for each segment the new block address only needs to be updated in a single descriptor, since all copies refer to the master descriptor. Paging is not free from fragmentation — the fragmentation is internal to pages ( internal fragmentation ). If a requested block is smaller than a page, then some space in

420-642: A means to improve performance, rather than to solve the problems involved in multi-programming. The first true virtual memory system was that implemented at the University of Manchester to create a one-level storage system as part of the Atlas Computer . It used a paging mechanism to map the virtual addresses available to the programmer onto the real memory that consisted of 16,384 words of primary core memory with an additional 98,304 words of secondary drum memory . The addition of virtual memory into

480-504: A presentation on virtual memory in the Atlas I computer. Paul King took the ideas back to Burroughs and it was determined that virtual memory should be designed into the core of the B5000. . Burroughs Corporation released the B5000 in 1964 as the first commercial computer with virtual memory. IBM developed the concept of hypervisors in their CP-40 and CP-67 , and in 1972 provided it for

540-433: A problem called “ thrashing ” can occur, in which the computer spends an unsuitably large amount of time transferring pages to and from a backing store, hence slowing down useful work. A task's working set is the minimum set of pages that should be in memory in order for it to make useful progress. Thrashing occurs when there is insufficient memory available to store the working sets of all active programs. Adding real memory

600-482: A process or between processes. Descriptors are central to the working of virtual memory in MCP systems. Descriptors contain not only the address of a segment, but the segment length and status in virtual memory indicated by the ‘p-bit’ or ‘presence bit’ which indicates if the address is to a segment in main memory or to a secondary-storage block. When a non-resident segment (p-bit is off) is accessed, an interrupt occurs to load

660-412: A shared memory space, ability to share memory used by libraries between processes, increased security due to memory isolation, and being able to conceptually use more memory than might be physically available, using the technique of paging or segmentation. Virtual memory makes application programming easier by hiding fragmentation of physical memory; by delegating to the kernel the burden of managing

SECTION 10

#1732782732766

720-475: A simple way to lengthen the address space. Segmentation that can provide a single-level memory model in which there is no differentiation between process memory and file system consists of only a list of segments (files) mapped into the process's potential address space. This is not the same as the mechanisms provided by calls such as mmap and Win32 's MapViewOfFile, because inter-file pointers do not work when mapping files into semi-arbitrary places. In Multics,

780-466: A stable "1.0" version followed on April 19, 2005. In 2006 the OGC registered PostGIS as "implement[ing] the specified standard" for "Simple Features for SQL". Many software products can use PostGIS as a database backend, including: Memory footprint Memory footprint refers to the amount of main memory that a program uses or references while running. The word footprint generally refers to

840-546: Is exactly the situation in computers with cache memory, one of the earliest commercial examples of which was the IBM System/360 Model 85. In the Model 85 all addresses were real addresses referring to the main core store. A semiconductor cache store, invisible to the user, held the contents of parts of the main store in use by the currently executing program. This is exactly analogous to Güntsch's system, designed as

900-582: Is loaded indirectly when a Java application launches. In addition, on most operating systems, disk files opened by an application too are read into the application's address space, thereby, contributing to its footprint. During the 1990s, computer memory became cheaper and programs with larger memory footprints became commonplace. This trend has been mostly due to the widespread use of computer software, from large enterprise-wide applications that consume vast amounts of memory (such as databases ), to memory intensive multimedia authoring and editing software. To tackle

960-471: Is the simplest response, but improving application design, scheduling, and memory usage can help. Another solution is to reduce the number of active tasks on the system. This reduces demand on real memory by swapping out the entire working set of one or more processes. A system thrashing is often a result of a sudden spike in page demand from a small number of running programs. Swap-token is a lightweight and dynamic thrashing protection mechanism. The basic idea

1020-420: Is to set a token in the system, which is randomly given to a process that has page faults when thrashing happens. The process that has the token is given a privilege to allocate more physical memory pages to build its working set, which is expected to quickly finish its execution and to release the memory pages to other processes. A time stamp is used to handover the token one by one. The first version of swap-token

1080-516: The Intel 80386 and later IA-32 processors, the segments reside in a 32-bit linear, paged address space. Segments can be moved in and out of that space; pages there can "page" in and out of main memory, providing two levels of virtual memory; few if any operating systems do so, instead using only paging. Early non-hardware-assisted x86 virtualization solutions combined paging and segmentation because x86 paging offers only two protection domains whereas

1140-755: The S/370 as Virtual Machine Facility/370. IBM introduced the Start Interpretive Execution ( SIE ) instruction as part of 370-XA on the 3081, and VM/XA versions of VM to exploit it. Before virtual memory could be implemented in mainstream operating systems, many problems had to be addressed. Dynamic address translation required expensive and difficult-to-build specialized hardware; initial implementations slowed down access to memory slightly. There were worries that new system-wide algorithms utilizing secondary storage would be less effective than previously used application-specific algorithms. By 1969,

1200-585: The SDS 940 , used page registers instead of page tables in memory for address translation. This part of the operating system creates and manages page tables and lists of free page frames. In order to ensure that there will be enough free page frames to quickly resolve page faults, the system may periodically steal allocated page frames, using a page replacement algorithm , e.g., a Least recently used (LRU) algorithm. Stolen page frames that have been modified are written back to auxiliary storage before they are added to

1260-505: The Technische Universität Berlin in 1956 in his doctoral thesis, Logical Design of a Digital Computer with Multiple Asynchronous Rotating Drums and Automatic High Speed Memory Operation does not stand up to careful scrutiny. The computer proposed by Güntsch (but never built) had an address space of 10 words which mapped exactly onto the 10 words of the drums, i.e. the addresses were real addresses and there

SECTION 20

#1732782732766

1320-468: The megabytes (or even gigabytes ) of memory available on a regular home PC or more modern mobile phones and tablets. Virtual memory In computing , virtual memory , or virtual storage , is a memory management technique that provides an "idealized abstraction of the storage resources that are actually available on a given machine" which "creates the illusion to users of a very large (main) memory". The computer's operating system , using

1380-489: The memory hierarchy (eliminating the need for the program to handle overlays explicitly); and, when each process is run in its own dedicated address space, by obviating the need to relocate program code or to access memory with relative addressing . Memory virtualization can be considered a generalization of the concept of virtual memory. Virtual memory is an integral part of a modern computer architecture ; implementations usually require hardware support, typically in

1440-494: The 1960s include: During the 1960s and early '70s, computer memory was very expensive. The introduction of virtual memory provided an ability for software systems with large memory demands to run on computers with less real memory. The savings from this provided a strong incentive to switch to virtual memory for all systems. The additional capability of providing virtual address spaces added another level of security and reliability, thus making virtual memory even more attractive to

1500-447: The Atlas also eliminated a looming programming problem: planning and scheduling data transfers between main and secondary memory and recompiling programs for each change of size of main memory. The first Atlas was commissioned in 1962 but working prototypes of paging had been developed by 1959. As early as 1958, Robert S. Barton , working at Shell Research, suggested that main storage should be allocated automatically rather than have

1560-525: The CPU, often referred to as a memory management unit (MMU), automatically translates virtual addresses to physical addresses. Software within the operating system may extend these capabilities, utilizing, e.g., disk storage , to provide a virtual address space that can exceed the capacity of real memory and thus reference more memory than is physically present in the computer. The primary benefits of virtual memory include freeing applications from having to manage

1620-476: The OGC. PostGIS has not been certified as compliant by the OGC. May of 2001, at that point I was running a small consulting company here in Victoria, working for the provincial government, mostly working in the geospatial field. Because all of our contracts came from the provincial government, we were very tied to the cycle of when they would release and sign contracts. And the government changed that year, so all

1680-468: The OS has a special facility for "fast fixing" these short-term fixed data buffers (fixing which is performed without resorting to a time-consuming Supervisor Call instruction ). Multics used the term "wired". OpenVMS and Windows refer to pages temporarily made nonpageable (as for I/O buffers) as "locked", and simply "nonpageable" for those that are never pageable. The Single UNIX Specification also uses

1740-429: The OS writes those pages and segments currently in real memory to swap files. In a swap-in, the OS reads back the data from the swap files but does not automatically read back pages that had been paged out at the time of the swap out operation. IBM's MVS , from OS/VS2 Release 2 through z/OS , provides for marking an address space as unswappable; doing so does not pin any pages in the address space. This can be done for

1800-423: The civil servants were very risk averse in terms of signing new contract contracts. They didn't know whether what their budget situation was going to be that year. So we ended up having like three months where we got almost no direct revenue. We, you know, didn't all go on vacation for three months. We were kind of young, excited about the field. So we thought, you know, let's see if we can build something to store

1860-454: The current process, etc., that the program ever needs while executing and will be loaded at least once during the entire run. Larger programs have larger memory footprints. An application's memory footprint is roughly proportionate to the number and sizes of shared libraries or classes it loads, whereas static libraries, executable programs and static data areas contribute to a fixed (constant) portion. Programs themselves often do not contribute

PostGIS - Misplaced Pages Continue

1920-571: The debate over virtual memory for commercial computers was over; an IBM research team led by David Sayre showed that their virtual memory overlay system consistently worked better than the best manually controlled systems. Throughout the 1970s, the IBM 370 series running their virtual-storage based operating systems provided a means for business users to migrate multiple older systems into fewer, more powerful, mainframes that had improved price/performance. The first minicomputer to introduce virtual memory

1980-727: The details: In most cases, there will be an update to the page table, possibly followed by purging the Translation Lookaside Buffer (TLB), and the system restarts the instruction that causes the exception. If the free page frame queue is empty then the paging supervisor must free a page frame using the same page replacement algorithm for page stealing. Operating systems have memory areas that are pinned (never swapped to secondary storage). Other terms used are locked , fixed , or wired pages. For example, interrupt mechanisms rely on an array of pointers to their handlers, such as I/O completion and page fault . If

2040-597: The duration of a job by entering the name of an eligible main program in the Program Properties Table with an unswappable flag. In addition, privileged code can temporarily make an address space unswappable using a SYSEVENT Supervisor Call instruction (SVC); certain changes in the address space properties require that the OS swap it out and then swap it back in, using SYSEVENT TRANSWAP. Swapping does not necessarily require memory management hardware, if, for example, multiple jobs are swapped in and out of

2100-603: The ever increasing memory needs, virtual memory systems were introduced that divide the available memory into equally sized portions and loads them from " pages " stored on the hard-disk on an as-and-when required basis. This approach to support programs with huge-memory-footprints has been quite successful. Most modern operating systems including Microsoft Windows , Apple's macOS , and all versions of Linux and Unix provide virtual memory systems. Traditionally, low-memory-footprint programs were of importance to running applications on embedded systems where memory would often be

2160-597: The extent of physical dimensions that an object occupies, giving a sense of its size. In computing, the memory footprint of a software application indicates its runtime memory requirements, while the program executes. This includes all sorts of active memory regions like code segment containing (mostly) program instructions (and occasionally constants), data segment (both initialized and uninitialized), heap memory , call stack , plus memory required to hold any additional data structures, such as symbol tables , debugging data structures , open files, shared libraries mapped to

2220-580: The first place, to relieve programmers of such memory considerations. In multi-processing systems, optimal operation of the system depends on the mix of independent processes at any time. Hybrid schemes of segmentation and paging may be used. The Intel 80286 supports a similar segmentation scheme as an option, but it is rarely used. Segmentation and paging can be used together by dividing each segment into pages; systems with this memory structure, such as Multics and IBM System/38 , are usually paging-predominant, segmentation providing memory protection. In

2280-500: The form of a memory management unit built into the CPU . While not necessary, emulators and virtual machines can employ hardware support to increase performance of their virtual memory implementations. Older operating systems, such as those for the mainframes of the 1960s, and those for personal computers of the early to mid-1980s (e.g., DOS ), generally have no virtual memory functionality, though notable exceptions for mainframes of

2340-449: The free queue. On some systems the paging supervisor is also responsible for managing translation registers that are not automatically loaded from page tables. Typically, a page fault that cannot be resolved results in an abnormal termination of the application. However, some systems allow the application to have exception handlers for such errors. The paging supervisor may handle a page fault exception in several different ways, depending on

2400-441: The largest portions to their own memory footprints; rather, structures introduced by the run-time environment take up most of the memory. For example, a C++ compiler inserts vtables , type info objects and many temporary and anonymous objects that are active during a program's execution. In a Java program, the memory footprint is predominantly made up of the runtime environment in the form of Java virtual machine (JVM) itself that

2460-777: The marketplace. Most modern operating systems that support virtual memory also run each process in its own dedicated address space . Each program thus appears to have sole access to the virtual memory. However, some older operating systems (such as OS/VS1 and OS/VS2 SVS ) and even modern ones (such as IBM i ) are single address space operating systems that run all processes in a single address space composed of virtualized memory. Embedded systems and other special-purpose computer systems that require very fast and/or very consistent response times may opt not to use virtual memory due to decreased determinism ; virtual memory systems trigger unpredictable traps that may produce unwanted and unpredictable delays in response to input, especially if

PostGIS - Misplaced Pages Continue

2520-625: The page fault exception to chain with other exceptions without double fault . However, loading segment descriptors was an expensive operation, causing operating system designers to rely strictly on paging rather than a combination of paging and segmentation. Nearly all current implementations of virtual memory divide a virtual address space into pages , blocks of contiguous virtual memory addresses. Pages on contemporary systems are usually at least 4 kilobytes in size; systems with large virtual address ranges or amounts of real memory generally use larger page sizes. Page tables are used to translate

2580-509: The page is stored. When a reference is made to a page by the hardware, if the page table entry for the page indicates that it is not currently in real memory, the hardware raises a page fault exception , invoking the paging supervisor component of the operating system . Systems can have, e.g., one page table for the whole system, separate page tables for each address space or process, separate page tables for each segment; similarly, systems can have, e.g., no segment table, one segment table for

2640-419: The page will be wasted. If a block requires larger than a page, a small area in another page is required resulting in large wasted space. The fragmentation thus becomes a problem passed to programmers who may well distort their program to match certain page sizes. With segmentation, the fragmentation is external to segments ( external fragmentation ) and thus a system problem, which was the aim of virtual memory in

2700-500: The pages containing these pointers or the code that they invoke were pageable, interrupt-handling would become far more complex and time-consuming, particularly in the case of page fault interruptions. Hence, some part of the page table structures is not pageable. Some pages may be pinned for short periods of time, others may be pinned for long periods of time, and still others may need to be permanently pinned. For example: In IBM's operating systems for System/370 and successor systems,

2760-608: The physical view of a computer, although pages themselves are an artificial division in memory. The designers of the B5000 would have found the artificial size of pages to be Procrustean in nature, a story they would later use for the exact data sizes in the B1000 . In the Burroughs and Unisys systems, each memory segment is described by a master descriptor which is a single absolute descriptor which may be referenced by other relative (copy) descriptors, effecting sharing either within

2820-474: The priority: the higher the number of swap-out pages of a process, the longer the time stamp for it will be. Some systems, such as the Burroughs B5500, and the current Unisys MCP systems use segmentation instead of paging, dividing virtual address spaces into variable-length segments. Using segmentation matches the allocated memory blocks to the logical needs and requests of the programs, rather than

2880-606: The programmer being concerned with overlays from secondary memory, in effect virtual memory. By 1960 Barton was lead architect on the Burroughs B5000 project. From 1959 to 1961, W.R. Lonergan was manager of the Burroughs Product Planning Group which included Barton, Donald Knuth as consultant, and Paul King. In May 1960, UCLA ran a two-week seminar ‘Using and Exploiting Giant Computers’ to which Paul King and two others were sent. Stan Gill gave

2940-466: The segment from secondary storage at the given address, or if the address itself is 0 then allocate a new block. In the latter case, the length field in the descriptor is used to allocate a segment of that length. A further problem to thrashing in using a segmented scheme is checkerboarding, where all free segments become too small to satisfy requests for new segments. The solution is to perform memory compaction to pack all used segments together and create

3000-618: The segment; the handler for the trap maps the segment into the address space, puts the segment number into the pointer, changes the tag field in the pointer so that it no longer causes a trap, and returns to the code where the trap occurred, re-executing the instruction that caused the trap. This eliminates the need for a linker completely and works when different processes map the same file into different places in their private address spaces. Some operating systems provide for swapping entire address spaces , in addition to whatever facilities they have for paging and segmentation. When this occurs,

3060-491: The spatial data in a database. And we had used Postgres for one of our data processing projects the year previous, so we were kind of familiar with it. And that experimental work in the spring of 2001 ended up being released at the end of May 2001 as PostGIS version 0.1. Refractions Research released the first version of PostGIS in 2001 under the GNU General Public License . After six release candidates ,

SECTION 50

#1732782732766

3120-613: The term "locked" in the specification for mlock () , as do the mlock () man pages on many Unix-like systems. In OS/VS1 and similar OSes, some parts of systems memory are managed in "virtual-real" mode, called "V=R". In this mode every virtual address corresponds to the same real address. This mode is used for interrupt mechanisms, for the paging supervisor and page tables in older systems, and for application programs using non-standard I/O management. For example, IBM's z/OS has 3 modes (virtual-virtual, virtual-real and virtual-fixed). When paging and page stealing are used,

3180-446: The term is "fixed", and such pages may be long-term fixed, or may be short-term fixed, or may be unfixed (i.e., pageable). System control structures are often long-term fixed (measured in wall-clock time, i.e., time measured in seconds, rather than time measured in fractions of one second) whereas I/O buffers are usually short-term fixed (usually measured in significantly less than wall-clock time, possibly for tens of milliseconds). Indeed,

3240-476: The trap requires that data be read into main memory from secondary memory. The hardware to translate virtual addresses to physical addresses typically requires a significant chip area to implement, and not all chips used in embedded systems include that hardware, which is another reason some of those systems do not use virtual memory. In the 1950s, all larger programs had to contain logic for managing primary and secondary storage, such as overlaying . Virtual memory

3300-424: The virtual addresses seen by the application into physical addresses used by the hardware to process instructions; such hardware that handles this specific translation is often known as the memory management unit . Each entry in the page table holds a flag indicating whether the corresponding page is in real memory or not. If it is in real memory, the page table entry will contain the real memory address at which

3360-567: The whole system, separate segment tables for each address space or process, separate segment tables for each region in a tree of region tables for each address space or process. If there is only one page table, different applications running at the same time use different parts of a single range of virtual addresses. If there are multiple page or segment tables, there are multiple virtual address spaces and concurrent applications with separate page tables redirect to different real addresses. Some earlier systems with smaller real memory sizes, such as

3420-456: Was implemented in Linux 2.6. The second version is called preempt swap-token and is also in Linux 2.6. In this updated swap-token implementation, a priority counter is set for each process to track the number of swap-out pages. The token is always given to the process with a high priority, which has a high number of swap-out pages. The length of the time stamp is not a constant but is determined by

3480-500: Was no form of indirect mapping, a key feature of virtual memory. What Güntsch did invent was a form of cache memory , since his high-speed memory was intended to contain a copy of some blocks of code or data taken from the drums. Indeed, he wrote (as quoted in translation ): "The programmer need not respect the existence of the primary memory (he need not even know that it exists), for there is only one sort of addresses ( sic ) by which one can program as if there were only one storage." This

3540-485: Was the Norwegian NORD-1 ; during the 1970s, other minicomputers implemented virtual memory, notably VAX models running VMS . Virtual memory was introduced to the x86 architecture with the protected mode of the Intel 80286 processor, but its segment swapping technique scaled poorly to larger segment sizes. The Intel 80386 introduced paging support underneath the existing segmentation layer, enabling

3600-489: Was therefore introduced not only to extend primary memory, but to make such an extension as easy as possible for programmers to use. To allow for multiprogramming and multitasking , many early systems divided memory between multiple programs without virtual memory, such as early models of the PDP-10 via registers . A claim that the concept of virtual memory was first developed by German physicist Fritz-Rudolf Güntsch at

#765234