Frequently Asked Question

Do I need a GPU to learn about AI on Linux?

Not to learn, and not for most of what this part of the book covers. The command-line assistants in Chapters 22 to 24 run the model on someone else's hardware, so your machine only needs a network connection and a terminal.

A GPU matters when you want to run a model locally, as Chapter 25 describes. Even then it is a question of speed rather than possibility: a quantised model will run on CPU using system RAM at a few tokens per second, which is tedious for conversation but perfectly workable for a batch job. A model that fits in GPU memory runs perhaps ten times faster.

If you are buying hardware specifically for this, video memory is the constraint that matters, not compute throughput. A card with 16 GB of VRAM will run models that a faster card with 8 GB cannot.