Operating systems and language translators

Why a computer system needs an operating system

An operating system (OS) is system software that manages the hardware and the software of a computer system and provides the services that every program needs. Without one, each program would have to carry its own code for reading a disk, sending output to a screen, and deciding when it is allowed to use the processor.

The OS exists because the resources it shares out are limited. The processor can execute one fetch-execute cycle at a time and main memory is finite, so if two programs helped themselves to memory they would overwrite each other's data and bring the machine down. The OS allocates processor time and memory to each program, keeps the programs apart, and takes the resources back when a program ends.

It also hides the complexity of the hardware. A program that saves a file does not need to know whether the file is on a hard disk, a USB flash drive or a network server; it asks the OS, which knows the details of that device and its driver. This is why the same application can run on machines built from completely different components.

Finally, the OS provides the environment the user works in and the platform that application software is written against. It offers a user interface (a graphical interface of windows, icons and pointers, or a command line), it enforces security so that one user cannot open another's files, it reports and handles errors instead of letting a fault stop the machine, and it loads and runs the programs the user asks for.

• Manages resources: shares the processor, memory, storage and peripherals between the programs that want them, so that they do not conflict.
• Hides the hardware: application software calls the OS rather than the hardware, so it does not depend on the make and model of the components.
• Provides the user interface: a graphical interface or a command line, which is how the user actually controls the machine.
• Provides a platform: every application depends on the services of the OS, which is why software is written for Windows, macOS or Linux rather than for a processor.
• Manages data and security: files, folders, access rights and user accounts are all controlled by the OS.
• Handles errors: reports a failed read, an unavailable printer or a program that has crashed, and keeps the rest of the system running.

Memory management and process management

Memory management is the OS keeping track of main memory: which locations are free, which are in use, and which program is using them. When a program is loaded the OS allocates it a block large enough for its code and its data, and when the program ends it marks that memory free again so that the next program can have it.

The OS also has to protect memory. Each process is confined to the area it has been given, so a fault in one program cannot overwrite the code or data of another program, or of the OS itself. If that protection fails, one crashed program takes the whole system down with it.

Main memory is limited, and users expect to have several programs open at once. When there is not enough free memory the OS has to make room, for example by moving data that is not being used out to backing store and bringing it back when it is needed, and it has to decide how much memory each program may have so that one large program cannot starve the others.

A process is a program that is being executed. Process management is the OS deciding which process uses the processor, and for how long. It provides multitasking: each process gets a short turn on the processor in rotation, and the switching is fast enough that everything appears to be running at the same time.

It also gives some processes priority over others, so that the process reading input from the keyboard is not kept waiting behind a long calculation; it lets a process that is waiting for a device give up the processor instead of wasting it; and it closes a process that has finished or crashed and releases its memory and files.

• Memory management: records which memory is free, allocates a block to each program when it is loaded, frees it when the program ends, and stops one program writing into another's memory.
• Process management: decides which process runs next and for how long, switches between them to give multitasking, applies priorities, and terminates processes that have finished or crashed.
• A process is a program in execution - the program file on disk becomes a process when it is loaded into memory and being run.
• Process management is judged by what it achieves: several programs making progress, the processor never left idle waiting for a slow device, and no single program taking over the machine.

File management and hardware management

File management is how the OS stores, finds and protects data on secondary storage. It maintains the directory structure of folders and files, records where on the disk each file is stored, and provides the operations the user needs: create, open, save, copy, rename, move and delete.

It also keeps each file's metadata - the name, size, type, date created and date last modified - and the access rights that say who may read, write or delete it. On a shared machine those rights are what stop another user opening your work, and the OS checks them every time a file is opened.

Because secondary storage is finite, the OS tracks which blocks are free and which are in use and allocates space as a file grows. A file does not have to be stored in one piece: the OS records the list of blocks that belong to it, which is why a badly fragmented disk still works, just more slowly.

Hardware management means controlling the input, output and peripheral devices attached to the machine. Each device needs a device driver: software, supplied with the device, that tells the OS how to send it commands and how to read data back. The OS loads the driver, and a program can then print or read a sensor without knowing anything about the electronics.

The OS also has to cope with devices that are far slower than the processor. It uses a buffer, an area of memory that holds data while it is being transferred, so the processor can hand over a block of data and continue with something else instead of waiting. For printers it goes further and uses spooling: the documents are queued on disk and fed to the printer one at a time, which lets several users share one printer and carry on working the moment they press print.

Security management

Security management is the OS controlling who may use the computer and what they may do once they are in. It starts with user accounts: the OS holds a list of users, checks a username and password at logon, and attaches an identity to everything that user does afterwards.

Each account has a level of privilege. An administrator can install software, change settings and reach every file; a standard user can run programs and save work in their own area but cannot alter the system. File and folder permissions add a finer level: a file can be set so that one user may read it but not change it, another may change it, and a third cannot open it at all.

The OS also keeps processes apart, so a program run by one user cannot read the memory of another, and it logs activity - who logged on, when, and which files were opened - so that a suspicious event can be traced afterwards. It can enforce password rules, lock an unattended machine after a period of inactivity, and block programs from running from removable media.

What the OS cannot do is protect data from a user who has been given the right to see it, or stop someone with physical access to the machine removing the disk. That is why security management is backed up by encryption and by physical security, which are covered in the security section of the syllabus.

• User accounts and authentication: each user has a username and password, and the OS checks them before allowing access to the system.
• Access rights and permissions: each file, folder and device can be set to allow read, write or execute for particular users or groups.
• Privilege levels: an administrator account can change the system; a standard account cannot.
• Logging: records of logons and file access allow an event to be investigated after it has happened.
• Isolation: memory protection and process management stop one user's program reaching another user's data.
• Limits: the OS cannot protect data from someone who is entitled to see it, and it cannot protect a disk that has been physically removed.

The utility programs supplied with an operating system

Utility software is a set of small programs, usually supplied with the OS, that keep the computer healthy. Each one does a single housekeeping job and is started by the user or by a schedule. They are system software because they maintain the machine rather than do a task for the user, but the computer would still run without them.

A disk formatter prepares a storage medium so that it can be used: it sets up the file system and the structures the OS needs in order to store files, marks any bad sectors so that they are not used, and removes everything already on the disk. A virus checker scans files, memory and downloads for known malware, comparing what it finds against a database of signatures, and then quarantines, repairs or deletes anything infected; its signature list has to be updated regularly or it stops recognising new threats.

Defragmentation software deals with a magnetic hard disk whose files have become scattered. As files are created, resized and deleted, the blocks that belong to one file end up in different places, so the read/write head has to move to several locations to read it. Defragmentation rearranges the blocks so that each file occupies contiguous blocks and the free space is collected together, which means fewer head movements and faster reading. Disk contents analysis shows how much space each file and folder is using so that large or unneeded files can be found and removed, and disk repair checks the file system for errors, fixes them where it can, marks bad sectors, and may recover data from damaged files.

File compression reduces a file's size so that it takes less storage space and transfers faster over a network; a lossless method keeps all the original data, which is essential for a program or a document, while a lossy method discards detail and is only acceptable for images, sound and video. Back-up software makes copies of files to a separate medium, such as an external drive or cloud storage, so that work can be restored after accidental deletion, corruption, theft or a hardware failure; it can be scheduled to run automatically, and a full backup copies everything while an incremental backup copies only what has changed since the last one.

Program libraries and DLL files

A program library is a collection of pre-written, tested routines that a programmer can call from their own code instead of writing the code again. A library might provide routines for sorting, for handling text, for drawing a window or for reading a file; the programmer links the library into the program and calls a routine by name, passing it the data it needs.

The benefit to the developer is mostly time and risk. The routines have already been written and tested, often by specialists, so the developer does not spend weeks reinventing them and inherits fewer bugs. The library is usually efficient and deals with awkward cases such as unusual input, the code in the program itself is shorter and easier to read, and if the library is later corrected, every program that uses it gains the fix.

There are two ways to use a library. With a static library, the routine's code is copied into the finished program when it is linked, so the program carries its own copy and will run on its own. With a dynamic link library (DLL), the code stays in a separate file: the program records that it needs the DLL, and the OS loads the DLL into memory when the program runs and lets the program call into it.

The advantages of a DLL follow from that. The code is not copied into the executable, so the executable is smaller and takes less disk space; one copy of the DLL can be loaded into memory and shared by several programs running at once, so less memory is used overall; and the DLL can be replaced with a corrected or improved version without recompiling or redistributing the programs that use it. The drawback is dependence: if the DLL is missing, or is a different version from the one the program was built against, the program will not start, and a faulty DLL can affect every program that loads it.

The three translators: assembler, compiler and interpreter

A processor can execute only machine code: binary opcodes with binary operands. Anything a programmer writes has to be turned into machine code before it can run, and the software that does the turning is a translator.

An assembler translates an assembly language program into machine code. Assembly language uses mnemonics such as LDM, ADD and STO in place of binary opcodes, and labels in place of numeric addresses. The assembler converts each mnemonic into its opcode, works out the address that each label stands for and substitutes it, and produces machine code. Translation is close to one instruction for one instruction, so assembly language is specific to a processor family: a program written for one will not run on another.

A compiler translates a whole high-level language program into machine code, producing an executable file, before the program is run. It analyses the entire program, reports every syntax error it finds at the end of compilation, and produces no executable until the errors are corrected. Once compiled, the program runs on its own, without the compiler and without the source code, and it runs quickly because the translation has already been done.

An interpreter translates and executes a high-level language program one statement at a time, and does not produce a saved machine code file. It stops as soon as it reaches an error, reporting the statement where the error occurs, and it translates the program again every time it is run. That makes it slower at run time than a compiled program, but it gives immediate feedback and no compile stage, which is why it is used while a program is being written and tested.

Choosing between a compiler and an interpreter, and the Java compromise

A compiler's advantages all come from translating once, in advance. The finished program runs faster because no translation happens while it runs; it can be supplied without the source code, which protects the developer's work; every error is reported together after the whole program has been analysed, so they can all be fixed in one session; and the compiler can optimise the code as it translates. Its drawbacks are that nothing can be run until the whole program compiles, that every change means another compile, and that the executable is specific to one processor and operating system, so a version is needed for each platform.

An interpreter's advantages come from working line by line. There is no compile stage, so a change can be tried immediately; it stops at the exact statement where the error is and reports it there, which is far more useful while a program is being developed or learned; code can be run and tested a fragment at a time; and the same source file can run on any machine that has an interpreter. Its drawbacks are that it is slower at run time because the translation is repeated, that the interpreter must be installed on the user's machine, and that the source code is visible, so it cannot be protected or hidden.

Java is the standard example of a program that is partly compiled and partly interpreted. The Java compiler does not produce machine code for a particular processor; it produces bytecode, an intermediate code for a virtual machine. When the program is run, the Java Virtual Machine on that computer either interprets the bytecode or compiles the parts of it that are used most into machine code as it goes.

The result is a compromise that keeps some of both. Errors are caught by the compiler before anything runs, the source code is not distributed, and the same bytecode runs on any computer that has a JVM, so the program is portable without being recompiled for each platform. The cost is that a JVM must be installed and that execution is usually slower than a program compiled directly to machine code.

The integrated development environment

An integrated development environment (IDE) is a single application that brings together the tools a programmer needs: an editor for writing code, the translator for running it, a debugger, and usually a project manager for the files that make up the program. Working in one window means the programmer is not switching between separate programs and losing track of which file is which.

For coding, the IDE offers context-sensitive prompts. As the programmer types, the IDE offers only what is valid at that point: after typing the name of an object and a dot it lists that object's methods and properties; inside a procedure call it shows the parameters the routine expects. This reduces typing and prevents spelling mistakes in names. For initial error detection it applies dynamic syntax checks, checking the code as it is typed rather than only when it is compiled, so a missing bracket or a misspelt keyword is flagged on the line where it occurs before the program is ever run.

For presentation, prettyprint reformats the code automatically, indenting and spacing it so that loops, conditions and blocks can be seen at a glance, and expand and collapse code blocks lets the programmer fold away whole routines so that a long program can be navigated and its overall structure seen.

For debugging, single stepping executes one statement at a time so that the programmer can follow exactly what each line does, and a breakpoint pauses the program at a chosen line so that the state can be inspected at that moment without stepping through everything before it. At either point the variables, expressions and report window show the current value of each variable and of any expression the programmer has chosen to watch, which is how a wrong value is traced back to the statement that produced it.