Microsoft and NVIDIA say a new Windows laptop can run AI models with around 120 billion parameters on-device. That is a meaningful step for local AI, but the experience depends on which Surface Laptop Ultra you buy, how the model is compressed and how much context you expect it to handle.
Quick Take
- Yes, with conditions: A 120B-class model can run locally on RTX Spark hardware in a suitable compressed format. The 128 GB configuration gives a much more credible memory budget than the 24 GB base Surface Laptop Ultra.
- The hardware: RTX Spark joins an Arm-based NVIDIA Grace CPU, a Blackwell-generation RTX GPU and up to 128 GB of shared system memory.
- The measured result: One early Surface Laptop Ultra test reported roughly 34–35 output tokens per second for a 122B-class model on prompts up to about 9,000 tokens. That test does not establish one-million-token performance.
- The real question: A model fitting in memory is only the first step. Prompt processing, response latency, sustained speed, power and the agent software determine whether a local workflow is useful.
What Are Surface Laptop Ultra and NVIDIA RTX Spark?
Surface Laptop Ultra is Microsoft’s high-performance Windows laptop built around the NVIDIA RTX Spark platform. Microsoft opened preorders on 7 October 2026, with availability from 16 October. RTX Spark is the processor platform, not a separate GPU card plugged into the laptop: it combines a Grace CPU and Blackwell RTX graphics with memory shared across CPU and GPU. The Microsoft launch post and NVIDIA availability announcement describe the broader platform.
According to NVIDIA’s technical overview, the highest-end chip has a 20-core Arm CPU, a Blackwell iGPU and up to 128 GB of coherent LPDDR5 memory. NVIDIA lists 300 GB/s for the memory interface. Its announcement describes up to one petaflop of FP4 AI compute. These are platform specifications or peak figures, not a guarantee of response speed for every model and application.
The distinction from a typical laptop GPU matters: a 120B model is often too large for conventional notebook graphics memory. Shared memory allows the integrated GPU to work with a much larger pool. Windows, other applications and the inference runtime also need that pool, and Microsoft cautions that the GPU-addressable amount is less than the total 128 GB and varies by configuration and workload.
Why Does a 120B Model Need So Much Memory?
The “120B” label refers to approximately 120 billion model parameters. A simple way to estimate storage for weights alone is to multiply the parameter count by the number of bits stored for each parameter. That arithmetic explains why memory, rather than the marketing petaflop figure, is often the first gate.
| Illustrative weight precision | Raw weight estimate | What it implies for 128 GB |
|---|---|---|
| 16-bit | About 240 GB | Does not fit in a 128 GB system memory pool. |
| 8-bit | About 120 GB | Leaves too little room for Windows, metadata and inference. |
| 4-bit | About 60 GB | Creates room for a workable setup, subject to overhead and context. |
These are rough decimal calculations, not the actual download size of a named model. Real quantized formats store scale factors and other data. A running model also needs buffers and an attention cache for its context. Lower precision can change quality and compatibility; a model that supports a format in principle may still need a suitable kernel, driver and runtime on this specific device.
Some models use a mixture-of-experts architecture. A label such as 122B-A10B indicates many total parameters but a smaller active subset for each token. The active count can help explain compute demands; the weights for the whole model still need to be stored or accessed. A model’s parameter count alone does not tell you the RAM required or its user-visible speed.
Which Surface Laptop Ultra Can Actually Run the Big Model?
Microsoft’s US configuration page, checked for this article on 18 October 2026, lists a 24 GB starting configuration at $2,599.99 and a 128 GB, 20-core CPU / 6,144-core GPU configuration at $5,899.99. It also lists 32 GB, 48 GB and 64 GB options. Prices and available configurations vary by country and may change.
This difference changes the answer to the headline. The 24 GB model is still an RTX Spark laptop, but it cannot hold even the approximate 60 GB raw weights of a generic 120B model quantized to 4 bits in its system memory. The 64 GB model comes close to that theoretical number before adding Windows, GPU allocation, cache and runtime overhead. The 128 GB version gives a more realistic path for a suitably quantized 120B-class model with a practical context length.
Check the exact processor, memory and software build before buying a machine for this workload. A listing that says “RTX Spark” establishes the platform family, not identical memory capacity across all RTX Spark laptops. NVIDIA’s configuration guide spans devices from 24 GB to 128 GB.
What Have Real Tests Shown So Far?
There is early evidence beyond vendor capacity claims. Micro Center reports its own Surface Laptop Ultra measurements using an NVIDIA native Arm64 build of llama.cpp. For Qwen3.5 122B A10B, it recorded roughly 35 output tokens per second after generation began on short prompts and about 34 tokens per second on its approximately 9,000-token prompts. It measured time to first token at about 0.58 seconds on a short general prompt and 16.38 seconds on the longer one.
That is evidence of a 120B-class model generating locally at interactive speed in one tested setup. The report does not specify every model quantization and configuration detail needed to predict performance on another SKU. Its long prompt is approximately 9,000 tokens, so it also does not independently verify the million-token context claim that NVIDIA makes for selected RTX Spark model workflows.
Notice the difference between output rate and waiting for an answer. An agent that reads many files or calls several tools may spend substantial time processing context and loading data before it streams its first word. Thermal limits, background activity, battery operation and the number of simultaneous users can also change the experience. An advertised FP4 petaflop is a peak compute specification; it cannot be converted directly into tokens per second.
What Does “Local AI” Enable in Practice?
Run a compatible open-weight model against authorized source files, then test patches before any code leaves the development machine.
Search and summarize locally stored reports or recordings when the app and its connected tools are configured to stay on-device.
Use GPU memory for both generative features and larger 3D or video projects, while budgeting for competing workloads.
Prototype an assistant that can search files and propose actions, with narrowly scoped permissions and confirmation before changes.
Microsoft is adding local model support through Windows ML and experimental llama.cpp integration. A model runs locally only if the chosen application and its actual inference path use the local processor. Search, synchronization, updates, connected cloud models and tool calls may still reach outside services. Inspect the app’s settings and network behaviour before treating it as an offline workflow.
Three Limits to Check Before Calling This a Laptop AI Breakthrough
- Context costs memory and time: NVIDIA advertises a 120B model with up to a million tokens of context under its supported approach. A model’s advertised maximum context is not proof that all one-million-token requests fit comfortably, complete quickly or answer accurately on a particular laptop.
- Windows on Arm needs compatible tools: RTX Spark runs an Arm-based Windows system. NVIDIA’s porting guidance covers native Arm64 applications and emulation. Check your required inference engine, extensions, drivers and plugins; an x64 workflow should not be assumed to perform identically.
- Private inference is not agent security: A locally running assistant can still read the wrong files or follow malicious instructions embedded in documents. Scope file and network access, retain review logs and require approval before payment, deletion or publication. Microsoft describes Execution Containers as one way to enforce agent boundaries on Windows.
Should You Choose a 120B Model or a Smaller One?
A smaller model can be the better local tool if it answers your specific questions reliably, begins faster and leaves memory for the browser, editor and other work. A 120B-class model becomes compelling when its measurable gains on difficult coding, research or reasoning tasks justify the extra device cost and longer response times.
Try the same set of permitted tasks on a smaller local model, a 120B-class local model and a cloud model where policy allows. Record time to first token, complete-task time, memory use, power draw, errors, accepted outputs and cost per accepted task. Use the same documents and reviewer criteria. For work that changes frequently or needs tools across many systems, a hybrid setup may be more practical than forcing every step onto one laptop.
For European teams, on-device inference may reduce unnecessary transfers of confidential data, but it does not remove GDPR obligations for personal data. Check model licences and rights to source documents before deploying or publishing generated material. Assess EU AI Act duties for the specific application rather than assuming that keeping inference local determines its legal classification.
MaGeN-AI View
The Takeaway
The answer is yes, on the right machine with the right model format. RTX Spark and a high-memory Surface Laptop Ultra make 120B-class local inference credible, and an early 122B test shows interactive output in a specific setup. That achievement does not turn the 24 GB base laptop into a 120B workstation or guarantee a fast million-token agent. The next test is ordinary work: dependable answers, manageable latency and sensible cost over a full day on a portable device.
Frequently Asked Questions
Can Surface Laptop Ultra run a 120B-parameter AI model locally?
Yes, appropriately quantized models can run on suitable Surface Laptop Ultra configurations. The 128 GB version offers substantially more room than the 24 GB base model, and actual compatibility, context size and speed depend on the model and software.
Does the $2,599.99 Surface Laptop Ultra run 120B models?
That US starting price was for a 24 GB configuration in Microsoft’s October 2026 listing. It should not be treated as the price or memory capacity of the 128 GB configuration used for demanding 120B-class workloads.
How much memory does a 120B AI model need?
A rough raw-weights estimate is about 240 GB at 16 bits, 120 GB at 8 bits, or 60 GB at 4 bits. Real deployments need additional space for quantization metadata, runtime, operating system and attention cache.
Is one petaflop the speed of a 120B model on a laptop?
No. The advertised petaflop refers to peak FP4 AI compute under specific conditions. User-visible speed also depends on memory bandwidth, model architecture, prompt length, software and power limits.
Does running a local model guarantee private or offline AI?
No. Inference can happen on-device, while an app may still use cloud tools, sync files or send telemetry. Check the specific app, permissions, network calls and data-handling terms.
