Unpacking the Jetson Thor vs. Spark vs. Mac Mini Showdown

Hey homelabbers and AI enthusiasts!

If you’re trying to figure out the best hardware for running Llama 3.3 70B and other massive, 70-billion-parameter AI models locally, YouTube creator Alex Ziskind just dropped a masterclass video that you need to watch.

Instead of just reading spec sheets, Alex put three heavyweight machines head-to-head on his bench: the Nvidia Jetson Thor, the Nvidia DGX Spark, and the Apple Mac Mini M4 Pro.

I want to walk you through exactly what Alex discovered, step by step.

What immediately caught my attention was the massive price-to-performance question he posed right at the start.

Both the Jetson Thor and the DGX Spark feature a staggering 128GB of RAM, but the Thor is actually over $1,000 cheaper than the Spark. So, why wouldn’t you just save the money and buy the Thor?

Through a series of live terminal tests, power monitoring, and architectural deep dives, Alex unpacks the reality of running local AI. He emphasizes a point that shapes the whole benchmark: memory bandwidth is arguably the most important metric for machine learning—often more critical than raw compute or core count. Let’s break down the details of his incredible benchmarks.

Watch Alex’s Full Benchmark Video

Alex pits the Jetson Thor, DGX Spark, and Mac Mini M4 Pro against Ministral 3, Llama 3.3 70B, and OSS 12B—with live power monitoring and llama-bench verification.

BTW, all the credit for this post goes to Alex for his amazing video.

Quick Verdict: Which 128GB Unified Memory AI Hardware Should You Buy?

• Buy the Nvidia Jetson Thor if you need 128GB for 70B+ models on a budget and prioritize value over raw prefill speed.
• Buy the DGX Spark if you serve multiple users, run VLMs, or need the fastest prompt processing.
• Buy the Mac Mini M4 Pro if power efficiency and silence matter most, and you mainly run smaller models (8B–32B).

Abbreviations: t/s = tokens per second; TG = token generation; PP = prompt processing (prefill). All three systems share 273 GB/s memory bandwidth—the equalizer Alex deliberately chose so benchmark differences reflect compute, thermals, and software, not bandwidth inequality.

Tools & Models Reference

System Monitoring Tools

• nvtop: A terminal-based task monitor he uses to track GPU usage and performance on the Nvidia Thor and Spark.
• Activity Monitor: The native macOS tool he uses to track memory pressure and power usage on the Mac Mini.

Local AI Runners & Benchmarking

• Ollama: A streamlined tool he uses to quickly load and run the initial tests for the models.
• Llama.cpp: The underlying C++ port of the LLaMA model architecture used to run the quantized (GGUF) models. He specifically mentions compiling it with CUDA for the Nvidia machines to ensure they use the GPUs.
• llama-bench: A specific benchmarking tool included with Llama.cpp that he uses to get the precise, isolated numbers for prompt processing (prefill) and token generation.

The AI Models (The Test Subjects)

• Ministral 3: A new “sparse” (Mixture of Experts) model he tests first.
• Llama 3.3 70B: A massive, “dense” 70-billion parameter model—40GB+ of weights that must stream from memory for every token. It pushes the hardware to its limits and uses up almost all of the Mac’s memory.
• OSS 12B: Another sparse model he uses for his final test (which fails to load on the Mac due to memory constraints).

Step 1: Setting the Baseline — The 273 GB/s Equalizer

Before firing up any models, Alex explains exactly why he chose to pit these three specific ARM-based systems against each other. He emphasizes a point that many spec-sheet readers overlook: when it comes to machine learning and LLM inference, memory bandwidth is arguably the most important metric—often more critical than raw compute or core count.

Here’s why. During token generation, the model produces one word at a time in a sequential process. For every single token, the inference engine must read the entire set of model weights from memory. A 70B model has tens of billions of parameters; even quantized, that’s 40GB+ of data that must stream from RAM or VRAM to the compute units. If the memory bus can’t keep up, the GPU or NPU sits idle waiting for data—no amount of extra cores will help. Alex drives this home: memory bandwidth is the primary bottleneck for generative AI, and it directly determines how fast your model can “talk.”

The Mac Mini M4 Pro, the Nvidia Jetson Thor, and the Nvidia Spark all share roughly the exact same memory bandwidth: 273 GB per second.

By deliberately standardizing this one massive bottleneck, Alex ensures that any differences we see in the benchmarks are directly related to the underlying compute architecture, thermals, and software pipelines—not masked by bandwidth inequality.

Step 2: Idle Power Draw Check

If you run a 24/7 homelab, power consumption is a major factor. Alex kicks things off by opening up nvtop on the Nvidia machines and Activity Monitor on the Mac to check their idle power usage.

• Mac Mini M4 Pro: Sips power at an incredibly efficient 8 watts.
• NVIDIA Jetson Thor: Idles at a respectable 31 watts.
• NVIDIA Spark: Idles at about 44 watts.

Step 3: Testing a “Sparse” Model (Ministral 3)

Alex’s first real test uses a brand new model called Ministral 3, run through Ollama. He notes that this is a “sparse” model—specifically a Mixture of Experts (MoE) model—meaning it doesn’t use all of its available parameters at the same time during calculations.

This is where Alex teaches us the critical difference between Token Generation and Prompt Processing (Prefill).

Token Generation (Memory Bandwidth Bound)

Token generation is memory bandwidth bound: for every single token, the inference engine must read the entire set of model weights from memory.

If the memory bus can’t keep up, the GPU or NPU sits idle waiting for data—no amount of extra cores will help. Because all three machines share the same 273 GB/s bandwidth, they perform similarly.

In the first run, the Mac Mini won the round with 39 tokens per second (t/s), the Spark hit 34.97 t/s, and the Thor came in at 30.82 t/s. Memory bandwidth directly determines how fast your model can “talk.”

Prompt Processing / Prefill (Compute Bound)

Alex points out that prompt processing is “pure computation” and has nothing to do with memory bandwidth—unlike token generation, where the bus is the bottleneck.

Here, extra cores and faster computing do help.

This is where the Nvidia Blackwell architecture flexes its muscles. The Mac Mini struggled at just 353 tokens per second.

The Thor, which boasts 25% more cores than the older AGX Orin, tripled the Mac’s speed at 1,036 t/s. But the Spark completely obliterated the competition, processing the prompt at a massive 2,406 t/s.

During a second run of Ministral 3, the Spark hit an even more ridiculous 2,817 prompt eval tokens per second, drawing almost 90 watts on the Thor and nearly 141 watts on the Spark.

Step 4: Unpacking the 3nm vs. 4nm Architecture Divide

Why is the Spark so much faster at processing prompts than the Thor if they are both 128GB Blackwell systems?

Remember: prompt prefill is compute-bound, not bandwidth-bound—so the CPU-to-GPU pipeline matters. Alex drops some fantastic hardware knowledge here.

He explains that the Thor’s CPU is built on a 4-nanometer process, while the Spark utilizes a denser 3-nanometer process.

This smaller process size directly affects and improves the CPU-to-GPU communication pipeline, allowing the Spark to feed its Blackwell GPU significantly faster than the Thor.

Step 5: The Heavyweight Test — Llama 3.3 70B (Dense Model)

Next, Alex moves to a “dense” model, Llama 3.3 70B.

This is a chunky model taking up over 40GB on disk—tens of billions of parameters that must stream from RAM or VRAM to the compute units for every token.

Because it’s dense, it uses all of its parameters for every calculation. This test pushes the 64GB Mac Mini to its absolute physical limits, filling up its memory entirely.

Power Spikes and Acoustic Surprises

Under this extreme load, power consumption skyrocketed.

The Mac Mini hit about 80 watts, the Thor pushed near 100 watts, and the Spark guzzled up to 153 watts.

But the most surprising finding in Alex’s video is the acoustics.

Despite pulling the most power, the Spark is an extremely quiet machine.

Conversely, pushing the Mac Mini with a 70B model forced it to make a noticeable blowing sound that Alex notes he had never heard the machine make before.

The 70B Speed Results

Once again, token generation was neck-and-neck: Mac Mini at 5.43 t/s, Thor at 4.61 t/s, and Spark at 4.46 t/s.

Why?

All three share the same 273 GB/s memory bandwidth—the primary bottleneck for generative AI—so token gen speeds cluster together.

The prompt evaluation rate, by contrast, remained the Spark’s domain (compute-bound, not bandwidth-bound), hitting 283 t/s compared to the Thor’s 103 t/s and the Mac Mini’s sluggish 34 t/s.

Step 6: Verifying with Llama Bench & The Thor GPU Quirk

To verify his Ollama findings, Alex compiles llama-bench directly on the machines using CUDA for the Nvidia platforms.

Using the exact same Llama 3.3 70B GGUF Q4 model, he uncovers a bizarre hardware quirk.

While looking at the system monitors during prompt prefill, the Spark maintained a perfectly flat, steady line of 95% GPU usage.

The Thor, however, was violently bouncing up and down—dropping to 0% GPU usage and springing back up to 98% in rapid succession.

Alex explains that this constant loading and unloading indicates the Thor might not be as efficient at managing this specific processing pipeline as the Spark.

The Llama Bench results confirmed his earlier tests.

Token generation (TG) stays similar across all three—memory bandwidth is the equalizer Alex standardized.

Prompt processing (PP) reveals the compute gap: the Spark’s architecture feeds its GPU far faster than the Thor or Mac.

• Spark: 4.56 TG / 283 PP
• Thor: 4.4 TG / 130 PP
• Mac Mini: 5.48 TG / 42 PP

Step 7: The Final Test (OSS 12B) and Alex’s Verdict

For his final test, Alex attempts to run another sparse model, OSS 12B.

Unfortunately, the Mac Mini failed to even load the model due to memory constraints, beautifully illustrating why having 128GB of RAM on the Nvidia boxes is such a massive advantage.

The Spark handled it effortlessly, generating 52.77 t/s and prefilling 977 t/s, while the Thor managed 37.17 t/s and 464 t/s, respectively.

So, Which One Should You Buy?

Alex closes the video with a brilliant breakdown of who should buy what.

• Choose the Thor if you want 128GB for 70B+ models without breaking the bank—it’s the value king.
• Choose the Spark if you need maximum prefill speed for multi-user serving, RAG pipelines, or vision-language models.
• Choose the Mac Mini if you prioritize power efficiency, silence, and smaller models—but accept that 64GB caps you at 70B with minimal context headroom.

Conclusion

When it comes to local AI, RAM dictates what models you can run, but your compute architecture dictates how fast you can process complex context.

Alex’s benchmark drives home a key lesson: memory bandwidth is the primary bottleneck for token generation—it directly determines how fast your model can “talk.”

By equalizing bandwidth across all three systems, he ensured that differences in the results reflect compute, thermals, and software, not bandwidth inequality.

The Nvidia Jetson Thor vs DGX Spark debate ultimately comes down to budget vs. prefill performance.

The Mac Mini M4 Pro carves out its own niche as the quiet, efficient option for lighter workloads.

Ready to deploy?

Here are a few other posts to check out: run AI models on Proxmox VE, or check out our RAM and VRAM guide to size your homelab for optimal performance.

Frequently Asked Questions

1. Why is memory bandwidth so important for local LLMs?

During token generation, the model produces one word at a time—and for every single token, the inference engine must read the entire set of model weights from memory. A 70B model has tens of billions of parameters; even quantized, that’s 40GB+ of data streaming from RAM or VRAM. If the memory bus can’t keep up, the GPU sits idle—no amount of extra cores will help. Memory bandwidth is the primary bottleneck for generative AI and directly determines how fast your model can “talk.” Alex chose three systems with the same 273 GB/s so benchmark differences reflect compute and software, not bandwidth.

2. Is 64GB enough to run Llama 3.3 70B?

Yes, but barely. A Q4-quantized 70B model needs ~40–42GB. On a 64GB Mac Mini, that leaves little room for context expansion. For 70B with large context windows, 128GB is the safer choice.

3. Thor vs Spark — when should I choose each?

Choose the Thor for budget 128GB: best value if you mainly care about loading 70B models and token generation speed. Choose the Spark if prompt processing (prefill) speed matters—e.g., multi-user serving, RAG, or VLMs.

4. Why is the Mac Mini louder under 70B load?

Pushing 70B on a 64GB machine stresses the unified memory and GPU. The fan ramps up to dissipate heat. The DGX Spark, despite higher power draw, has a better thermal design and stays quieter.

5. What is the difference between sparse and dense models?

Dense models (e.g., Llama 3.3 70B) use all parameters for every calculation. Sparse/MoE models (e.g., Ministral 3) activate only a subset, making them faster but more compute-bound during prefill.