Chapter 1: The Importance of Unix for Developers

Many computer users engage with various operating systems for their daily tasks. However, developers often gravitate towards Unix-based or Unix-like systems for software creation. These systems offer a developer-friendly environment, robust security features, portability, and high performance. The rise of the open-source .NET framework, particularly .NET Core, has even encouraged Windows developers to explore Unix-like systems.

There are multiple avenues to experience Unix-like environments. For instance, macOS provides a Unix-like experience for those who appreciate Apple hardware. Alternatively, if you admire the principles of the free software movement, you might opt for GNU/Linux distributions. Additionally, BSD-based systems such as FreeBSD and Solaris, which build upon the original Unix research, also offer a rich Unix experience.

Understanding the internals of operating systems can satisfy the curiosity surrounding computing systems. This knowledge can guide us in creating high-quality software, developing applications that are compatible with operating systems, engaging in low-level programming, and spearheading innovative software projects. In this discussion, I will outline Unix/Linux operating system concepts that can deepen your understanding of your preferred operating system.

Chapter 2: Unix and Raw Device I/O

Unix organizes its system files within the root directory, utilizing an inode structure to represent each file. Have you explored the contents of the /dev directory? Contrary to popular belief, this directory does not solely contain development files; rather, it is where device files reside. These device files offer a standardized interface for I/O devices, allowing userland programs to interact with hardware through conventional file handling system calls.

For example, you can directly capture raw mouse events by accessing the mice device file in Linux. By executing the following command and moving your mouse, you'll observe a stream of binary data in the console:

sudo cat /dev/input/mice

Popular libraries like the Python keyboard module directly read device events from /dev/input files. However, some programmers prefer using X display server tools (such as xdotool) and APIs to capture I/O device events instead of handling raw data. Notably, macOS does not utilize the /dev/input directory for mouse and keyboard events; instead, it provides input device events through framework APIs like Carbon and Cocoa, along with low-level access via IOKit and Quartz event taps.

Chapter 3: Special Device Files

Unix device files generally connect to physical hardware or virtual terminals (TTY or PTS), but there are three unique virtual devices: zero, null, and random. The well-known null device (/dev/null) produces no output and accepts any input, making it useful for discarding unwanted output streams. Meanwhile, the /dev/zero device accepts any input but continuously outputs zero-byte streams.

The random device generates random numbers based on hardware noise, producing more secure random numbers compared to other Pseudorandom number generators (PRNGs). In Linux, the random device gathers entropy from device driver states, interrupts, and internal timers.

Unix systems feature two random devices: /dev/random and /dev/urandom. The /dev/random device typically awaits sufficient kernel events to generate randomness, while /dev/urandom continually supplies random bits without waiting. On macOS, both random devices behave similarly. Familiarity with these devices can be beneficial when working with terminal interfaces, cryptographic systems, and OS-level utilities.

Chapter 4: Navigating POSIX Syscalls

The diversity of system calls across operating systems complicates native cross-platform software development. Imagine the effort required to adapt a Unix program for Windows without using Cygwin, which serves as a POSIX-compliant layer. In such cases, you would need to rewrite your program to accommodate Windows system calls. This challenge multiplies if you aim to port your software to a new operating system.

POSIX addresses this concern by establishing a clear standard for system calls, command-line interpreters, and core utilities. Most Unix-based and Unix-like systems adhere to the POSIX standard, allowing developers to create portable source code for native operating-system-level applications. For instance, the following C++ function can create a directory on both GNU/Linux and macOS:

#include <sys/stat.h>

void createDir(const std::string &path) {

mkdir(path.c_str(), 0700);

}

While POSIX does not impose strict guidelines for all user-space libraries, it does strive for portability regarding core system services and utilities. Understanding POSIX compliance is advantageous for developing portable native software and automation scripts.

Chapter 5: Distinguishing Unix-Based and Unix-Like Systems

Unix implementations can be classified into two categories: Unix-based and Unix-like. Unix-based systems, including BSD, Solaris, and macOS, derive their source code from the original Unix research project. In contrast, Unix-like systems, such as Linux and Minix, were developed independently based on Unix features and internal design principles.

Although popular Unix systems adhere to the POSIX standard for a portable user interface, they exhibit different architectural patterns. For instance, GNU/Linux distributions typically employ a monolithic kernel architecture centered around the Linux kernel project, while macOS utilizes a hybrid kernel model with the Darwin XNU kernel project.

Despite variations in system startup processes, the core components still align with Unix principles. All Unix implementations utilize the init daemon concept for initiating essential services. For example, GNU/Linux employs the systemd daemon (PID 1) for managing background services through the systemctl command, while macOS utilizes the launchd daemon with the launchctl command for similar service management.

An understanding of these concepts can aid in crafting error-free, portable native applications across various Unix-like and Unix-based systems. For example, I developed a portable C++ function to implement a URL opener compatible with many popular Unix implementations:

Chapter 6: User Interaction with Unix

Interacting with Unix operating systems can be accomplished through both graphical user interfaces (GUIs) and command-line interfaces (CLIs). Unix-like systems, particularly GNU/Linux, feature a loosely-coupled GUI layer, allowing users to operate solely via CLI through built-in TTYs while also installing various desktop environments like GNOME, KDE, and Xfce. Conversely, macOS typically provides a more integrated desktop environment called Aqua while still offering core OS components through the Darwin open-source project.

Unix syscalls can be employed to spawn processes from commands using C/C++, but how do these systems facilitate an efficient interface for process creation? Most Unix-like and Unix-based systems come equipped with command-line interpreters like Bash for executing commands from GUI terminals and TTYs.

Command-line interpreters feature user-friendly command languages that facilitate process spawning. For example, when you type ls in the terminal, Bash executes the /usr/bin/ls binary as a native process and relays the output back to the terminal through the connected PTS. You can verify the location of the ls binary with the command:

which ls # outputs /usr/bin/ls

However, the which command returns no output for built-in commands like history, cd, jobs, and alias, as they are not standalone Unix binaries but rather Bash built-ins. You can see which commands are built-in by running:

which history # no output

You can explore the Bash source code on GitHub to view all supported built-in commands.

Chapter 7: Conclusion

In this discussion, we explored several fundamental concepts related to Unix-based and Unix-like operating systems, including device files, special device files, syscalls, POSIX standards, GUI layers, and shell interpreters. Grasping these concepts demystifies the workings of Unix systems, empowering us to perform programming tasks with greater confidence and understanding. To further enhance your knowledge of Unix commands, consider exploring additional resources.

The first video, "Unix OS Class Lecture 2 - YouTube," provides an in-depth overview of Unix concepts essential for developers.

The second video, "Unix OS Class Lecture 8 - YouTube," explores advanced topics in Unix system programming.