How AI Was Built on Linux

There is a fact about artificial intelligence that rarely makes it into the popular coverage: essentially all of it runs on Linux. The models are trained on Linux clusters, served from Linux containers, and increasingly operated from a Linux command line. The frameworks that define them are open source projects, several of them governed by the same foundation that stewards the kernel. When a chatbot answers a question, the request lands on a machine whose process table, filesystem, and network stack look exactly like the ones described in the previous twenty chapters.

This is not a coincidence, and it is not merely a matter of cost. The reasons are historical, technical, and cultural, and they are worth understanding before you start using AI tools on your own machine in the chapters that follow.

The Machines That Train Models Run Linux

Start with the hardware. Training a large model is a supercomputing problem: thousands of accelerators, wired together with high-speed interconnects, running a single job for weeks. That is exactly the workload the high-performance computing community has been building for since the 1990s, and that community settled on Linux a long time ago. As Chapter 20 noted, every one of the world's 500 fastest supercomputers runs Linux.

The tooling reflects that heritage. Jobs are queued with a scheduler such as Slurm, which allocates nodes, enforces limits, and runs your workload when resources free up. Storage is a parallel filesystem mounted across every node. Nodes are provisioned from images and configured with the same tools a system administrator would use on any fleet. If you have understood processes, permissions, mounts, and systemd, you already understand the shape of a training cluster. It is a very large version of a familiar thing.

The commercial cloud inherited the same design. The GPU instances offered by every major provider are Linux virtual machines. The container images published for machine learning work are Linux images. A team training a model on rented hardware in 2026 is working in an environment that a Unix administrator from 1995 would find strange in scale but entirely legible in structure.

From AlexNet to CUDA

The modern era of deep learning has a conventional starting point: the 2012 ImageNet competition, where a network now known as AlexNet won by a margin large enough to change the field's direction. What made it possible was not a new idea so much as new hardware. Alex Krizhevsky trained the network on consumer NVIDIA graphics cards, using them as general-purpose parallel processors rather than as display adapters.

That trick depended on CUDA, NVIDIA's programming platform for general-purpose GPU computing, released in 2007. CUDA is proprietary software, and it is worth being clear-eyed about that: the single most important layer in the AI stack is closed. But CUDA was developed for Linux and Windows together, and the research community that adopted it worked almost entirely on Linux. The drivers, the profiling tools, the multi-GPU communication libraries, and the container integrations all matured on Linux first, because that is where the users were.

The practical consequence is that GPU support is one of the few areas where Linux is not merely adequate but clearly ahead. The commands are ones you already know how to reason about:

nvidia-smi

This prints the driver version, the CUDA version, the temperature and power draw of each card, and the processes currently holding GPU memory. It is top for accelerators, and like top it is usually the first thing you run when something is wrong.

NVIDIA's relationship with Linux has been famously uneasy. For years the driver was a proprietary blob that broke on kernel upgrades and irritated distribution maintainers, and Linus Torvalds gave the company a widely circulated one-finger review in 2012. That situation has improved: the nvidia-open kernel modules are now the recommended path for recent hardware, and AMD's ROCm stack is fully open. The friction has not vanished, but the trajectory is toward the open side.

The Software Stack Is Open Source Nearly All the Way Down

Above the driver, the stack is open source to an extent that is easy to overlook.

At the bottom sit the numerical libraries: BLAS and LAPACK, whose interfaces date to the 1970s and 1980s and whose optimised implementations still perform the matrix multiplications that dominate a training run. Above them sits NumPy, which gave Python a fast array type and made the language viable for scientific work at all. Above that sit the deep learning frameworks.

TensorFlow was released by Google in November 2015 under the Apache 2.0 licence. PyTorch followed from Facebook AI Research in 2016 and became the field's default. In September 2022 PyTorch's governance moved to a newly created PyTorch Foundation hosted by the Linux Foundation, with a board drawn from AMD, Amazon Web Services, Google Cloud, Meta, Microsoft Azure, and NVIDIA. The framework that trains most of the world's models is now stewarded by the same organisation that stewards the kernel it runs on.

Around the frameworks sits a wider ecosystem: Hugging Face Transformers for model implementations, tokenisers and dataset loaders, experiment trackers, evaluation harnesses. Nearly all of it is on GitHub under a permissive licence, installed with pip, and developed in the open.

Table 21.1: The layers of a typical training stack, and how open each one is

Notice the pattern. The proprietary parts are concentrated at the hardware boundary, exactly where proprietary parts have always concentrated in Linux. Everything a researcher actually reads, modifies, and publishes is open.

There is a deeper point here about why this happened. Machine learning research advances by reproduction: a paper is only useful if others can run it. Open source is not an ideological preference in that setting, it is a working requirement. The same argument Linus Torvalds made about kernel development, that many eyes and rapid iteration beat a closed process, turns out to describe the machine learning literature as well.

Containers Made Experiments Reproducible

Deep learning has a dependency problem of an unusual severity. A model implementation might require a specific PyTorch version, which requires a specific CUDA version, which requires a specific driver, on top of a particular Python and a long tail of libraries with their own version constraints. Reproducing a result from eighteen months ago by hand is genuinely difficult.

Containers, covered in Chapter 17, are the standard answer. An image pins the entire user-space stack, and the NVIDIA Container Toolkit exposes the host's GPUs inside it, so a container can use the accelerator while keeping its own CUDA libraries. The pattern is now universal:

docker run --gpus all --rm -it pytorch/pytorch:latest python -c \
  "import torch; print(torch.cuda.is_available())"

At cluster scale, Kubernetes schedules those containers across many machines, and the fact that Kubernetes was itself built at Google on Linux and donated to an open foundation is part of the same story. The AI industry did not invent its own operations layer. It adopted the one Linux already had.

Open Weights and the Second Open Source Movement

The most contested question in AI today is a licensing question, and it rhymes with arguments this book has already described.

Recall from Chapter 1 the distinction between free software and open source, and the long dispute over what counts as which. A similar dispute now surrounds model releases. Meta's Llama family, along with Mistral, Qwen, DeepSeek, and others, publish their trained parameters for download. You can run them on your own hardware, inspect them, and fine-tune them. That is a genuine and significant freedom, and Chapter 25 shows you how to use it.

But "open weights" is not the same as "open source". A released model typically ships the parameters without the training data, without the training code, and sometimes under a licence that restricts commercial use or field of application. You can run the artefact; you cannot rebuild it from source. By the standard the free software movement applied to compilers and kernels, that is closer to a freely redistributable binary than to open source.

The Open Source Initiative, which has maintained the Open Source Definition since 1998, published an Open Source AI Definition in October 2024 attempting to settle the question. It requires that a system usable under the definition provide enough information about training data for a skilled person to rebuild a substantially equivalent system. Several prominent open-weight models do not meet it. The argument has not concluded, and you should expect the terminology to remain slippery for some years yet.

The honest summary is this: the AI field inherited open source infrastructure almost completely, and has adopted open source norms for its tooling almost completely, but the models themselves sit in a new and unresolved category. When you read that a model is "open", check what was actually released.

Why This Matters for the Chapters Ahead

The last decade produced a class of tool this book has not yet covered: the agentic command-line assistant. Claude Code, OpenAI's Codex CLI, and Google's Gemini CLI all follow the same design. You describe a goal in prose; the tool reads files, runs shell commands, edits code, and reports back, looping until the task is done or it needs your input.

These are Unix programs in the traditional sense. They read from standard input and write to standard output, so they compose with pipes. They are configured through dotfiles in your home directory. They spawn child processes and are subject to the same permission model as any other program you run. Their most important safety features are built directly on kernel facilities described earlier in this book: user namespaces, seccomp filters, and Landlock, wrapped in tools such as bubblewrap.

That last point deserves emphasis, because it is the reason these tools belong in a Linux textbook at all rather than in a book about AI. An agent that can run arbitrary shell commands on your behalf is, from the operating system's point of view, simply a process. Everything you know about what a process can reach, who it runs as, and how to confine it applies unchanged. The chapters that follow lean on that repeatedly.

The next three chapters look at the major assistants in turn: Claude Code in Chapter 22, ChatGPT and Codex in Chapter 23, and Gemini in Chapter 24. Chapter 25 then steps away from the hosted services entirely and shows you how to run models on your own Linux machine, with no network connection and no account.