Laptop Specs for Local AI and LLMs: Memory, GPU, NPU and Storage

Why memory size limits the model you can load

A large language model is mostly a very long list of numbers called parameters (or weights). To generate text, the software has to hold those weights in fast memory the whole time. If they do not fit in the memory the accelerator can reach, the model either refuses to load or spills into slower memory and runs far more slowly. That is why the first spec to check on a laptop for local LLM use is not the CPU name but how much memory the GPU (or the Apple chip) can use.

The size of the weights is simple arithmetic: number of parameters multiplied by the bytes used to store each one. The bytes per parameter depend on the precision format. NVIDIA's mixed precision documentation states that FP16 uses 16 bits compared to 32 bits for FP32, so FP32 takes 4 bytes per parameter and FP16 takes 2 bytes. BF16 is also a 16-bit format, so it also takes 2 bytes. The Hugging Face Transformers documentation for bitsandbytes says that quantizing a model to 8-bit halves memory use and that 4-bit quantization reduces memory use by 4x, which works out to 1 byte and half a byte per parameter.

The weights are not the whole story. The software also needs memory for the context (the conversation or document the model is reading), the runtime itself and, on a laptop, the operating system and your other apps. Treat the numbers below as a floor, and leave headroom above them.

GPU memory vs system memory vs Apple unified memory

On a Windows laptop with a discrete GPU there are two separate pools. System memory (RAM) belongs to the CPU. GPU memory (VRAM) sits on the graphics card and is what the GPU reads from at full speed. For local LLMs, VRAM is the number that matters most: a model that fits entirely in VRAM runs on the GPU, while one that does not has to split layers between VRAM and system RAM, and the part on the CPU side slows everything down. Laptop VRAM is fixed at purchase. NVIDIA lists the GeForce RTX 5060 Laptop GPU with 8GB of GDDR7, the RTX 5080 Laptop GPU with 16GB and the RTX 5090 Laptop GPU with 24GB.

System RAM still matters. It holds the model while it loads, it holds any layers that overflow VRAM, and CPU-only runtimes use it for everything. If you plan to run models larger than your VRAM, even slowly, more RAM is the only way to do it. Our guide to how much RAM a laptop needs covers the general case; for local AI, aim well above the everyday recommendation.

Apple silicon works differently. The CPU and GPU share one pool called unified memory, so the GPU can use most of the machine's memory for model weights. That is why a MacBook Pro can load models that no laptop GPU's VRAM can hold. Apple's tech specs for the 16-inch MacBook Pro list M5 Max configurations with 36GB, 48GB, 64GB or 128GB of unified memory and up to 614GB/s of memory bandwidth. Bandwidth matters because generating each token means reading through the weights, so faster memory generally means faster text generation. See the Apple M5 Max laptops guide for current configurations.

Why NVIDIA CUDA support matters

NVIDIA describes CUDA as its platform for accelerated computing and the software layer that lets applications use its GPUs, with support for frameworks such as PyTorch. In practice, most open-source AI tooling is built and tested on CUDA first. Quantization libraries are a good example: the bitsandbytes documentation lists NVIDIA GPUs through CUDA as a supported backend and gives minimum NVIDIA GPU generations for its 8-bit and 4-bit features.

That makes an NVIDIA GeForce RTX or RTX PRO GPU the lowest-friction choice if you want to run many different models, fine-tune small ones, or follow tutorials without workarounds. Other GPUs and Apple silicon run popular local LLM apps well, but less common tools may need extra setup or may not support them at all. For how the laptop GPU lineup is tiered, read laptop GPU tiers explained.

NPUs and Copilot+ PCs: small on-device models

A neural processing unit (NPU) is a low-power accelerator built into the processor. Microsoft's Copilot+ PC developer guide describes Copilot+ PCs as Windows 11 hardware with an NPU that can perform more than 40 trillion operations per second (TOPS), and lists chips such as Snapdragon X Elite, AMD Ryzen AI 300 series and Intel Core Ultra 200V series.

NPUs are designed for efficiency, not for loading the largest models. Microsoft's guide notes that many NPUs only support lower-bit integer math such as INT8, so models have to be converted (quantized) before they run there. Windows ships its own local language model (Phi Silica) for Copilot+ PCs that runs on the NPU, and apps reach it through the Windows App SDK. That is the sweet spot: small models for summarizing, rewriting and similar features with good battery life.

If your goal is to experiment with open models of your choice, an NPU is a bonus rather than the deciding spec. Memory size and a capable GPU still decide what you can load.

Storage for model files

Model files on disk are roughly the size of their weights at the stored precision, so the table above doubles as a download-size guide: a 70 billion parameter model at 4-bit is about 35GB before any extra files. Microsoft's own documentation for its Phi Silica language model on GPU warns that the download is several gigabytes. People who try local AI tend to keep several models and several precisions of the same model, and the total grows quickly.

Treat storage as part of the AI budget of the laptop. A fast NVMe SSD also shortens load times, since the whole file has to be read into memory each time a model starts. If you expect to keep a library of models, look at 2TB or larger drives; the 2TB storage laptops guide lists options.

What tier fits which model size

Using the weight arithmetic above and the published memory sizes, here is how the common tiers line up. These are fits for the weights plus some headroom, not speed figures.

Keep in mind that a model which only just fits leaves little room for long documents or long chats. When in doubt, step up one tier or drop to a smaller precision.

• 8GB VRAM (RTX 5060 Laptop GPU class): 4-bit models around 8 billion parameters fit with room to spare; 14 billion at 4-bit (about 7GB of weights) is tight.
• 16GB VRAM (RTX 5080 Laptop GPU class): 14 billion at 4-bit fits comfortably, and 8 billion at 8-bit fits. Browse 16GB graphics laptops.
• 24GB VRAM (RTX 5090 Laptop GPU class): 32 billion parameters at 4-bit (about 16GB of weights) fits with headroom for context. See 24GB graphics laptops and RTX 5090 laptops.
• 64GB of system RAM or unified memory: enough to hold 32 billion at 8-bit or 70 billion at 4-bit in memory, on the GPU only if it is Apple unified memory. See 64GB RAM laptops.
• 128GB unified memory (M5 Max): 70 billion parameters at 8-bit (about 70GB) fits with room left for the system. The 128GB RAM laptops guide includes Windows workstations too, where the extra RAM helps CPU offload rather than GPU speed.

What else to check before you buy

Cooling and power limits decide how long a laptop GPU can sustain its load; long generation runs are closer to a gaming session than to web browsing, so thin designs may slow down. Check that memory is enough for the operating system too, especially on a Mac where the same pool serves the system, apps and the model.

If AI is one of several heavy jobs, such as data analysis or notebooks alongside local models, the data science laptops guide balances RAM, GPU and storage for that mix. For a shortlist aimed at AI work in general, start with the laptops for AI guide.

What to pick for your use

Questions

How much VRAM do I need to run a local LLM?

Multiply the parameter count by the bytes per parameter for your precision, then add headroom. An 8 billion parameter model at 4-bit needs about 4GB for weights, so 8GB of VRAM runs it. A 32 billion parameter model at 4-bit needs about 16GB for weights, which points to a 24GB GPU.

Is a MacBook good for running local LLMs?

Yes, for loading large models. Apple silicon's unified memory lets the GPU use most of the system memory, and M5 Max configurations go up to 128GB according to Apple. The trade-off is that some AI tools are built for NVIDIA CUDA first.

Can I run an LLM on the NPU of a Copilot+ PC?

Small models, yes. Microsoft describes Copilot+ PCs as having an NPU capable of more than 40 TOPS and ships a local language model that runs on it. Models usually have to be quantized to formats such as INT8 first, and larger open models are better suited to a GPU.

Does system RAM help if my GPU has too little VRAM?

It lets a larger model load by keeping some layers in system RAM, but those layers run far more slowly than layers in VRAM. More RAM widens what you can run; more VRAM decides how fast it runs.

What does 4-bit or 8-bit quantization change?

It stores each weight with fewer bits. Compared with FP16 at 2 bytes per parameter, 8-bit uses 1 byte and 4-bit uses half a byte, so the same model needs a half or a quarter of the memory, usually with some loss in output quality.