Tag: hardware

This post checks in on the status of Huawei GPUs nearly a year after initial hype about breaking NVIDIA's monopoly. The discussion reveals the reality of hardware alternatives in the AI acceleration space and provides ground truth on whether alternative GPU architectures have materialized for local AI workloads.

A detailed account of building extreme local hardware infrastructure to run GLM-5.2, escalating from a single 5090 to a multi-GPU setup with full PCIe 5.0 x16 across all slots. This post offers valuable insights into the practical challenges and cost escalation of running frontier-scale models locally.

Tensordyne announces breakthrough inference chip using logarithmic math hardware for dramatically improved efficiency: 17x better power efficiency and 13x higher throughput versus NVIDIA Blackwell. Claims efficient log-space computation as the key mathematical breakthrough.

Analysis of optimal budget hardware for running Qwen 3.6 models (27B and 35B-A3B) targeting 40+ tok/s. Compares RTX 3090 24GB, RTX 3080 20GB, and controversial Tesla V100 32GB options. Community consensus favors RTX 3090 for broader future compatibility.

A practical tip for running ~30B parameter models on consumer hardware: combining a modern 16GB card (like 5070Ti) with an older 6GB card (like RTX 2060) enables running larger models by splitting layers across GPUs. The key insight is that fitting everything in VRAM matters more than having matching GPUs, even if one card is significantly weaker.

Leaks describe OpenAI's wearable audio device: metal "eggstone" design worn behind the ear, powered by custom 2nm Samsung Exynos chip designed to command Siri and replace iPhone actions. Bill of materials closer to smartphone than earbuds. The Jony Ive collaboration has apparently prioritized this project.

Qwen team lead publicly states that Chinese AI companies are severely bottlenecked by inference compute rather than training compute. This suggests export controls on inference chips may be more impactful than training restrictions. The comment provides rare insight into how sanctions affect Chinese AI development.