Tag: self-hosted

Onyx is a self-hostable AI chat platform supporting any LLM, with built-in support for custom agents, knowledge source connections, and hybrid search/retrieval workflows. This is squarely in the intersection of self-hosted AI and RAG interests — a production-grade platform, not a toy demo.

Developer running Kimi K2.5 (600GB), MiniMax 2.5 (120GB), Qwen 3.5 (220GB), and GOT OSS 120B Heretic (60GB) across 3 Mac Studios with 512GB RAM each using EXO labs for distributed inference. This demonstrates that frontier-class models are now accessible for completely private, self-hosted deployment at reasonable hardware costs. Running 4 OpenClaws instances enables 24/7 coding, writing, and research workflows without cloud dependencies or rate limits.

A detailed account of building a $38K 6-GPU local AI workstation running three open models concurrently for internal business analytics and automation. Rare real-world documentation of what a serious on-premise AI infrastructure deployment looks like, including hardware specifics and lessons learned. With 94 comments, the thread drew genuine architectural discussion useful for anyone planning self-hosted AI at scale.

An impressive self-hosted GPU cluster featuring 12 RTX 5090s (1.5TB+ VRAM total) across 6 machines running Kubernetes with GPU scheduling. Built for AI/LLM inference, training, image/video generation, and self-hosted APIs—a glimpse into serious local AI infrastructure.

A detailed build log for a 4x AMD R9700 system (128GB VRAM) funded through a 50% digitalization subsidy in Germany. Built to run 120B+ models locally for data privacy, with comprehensive benchmarks and real-world performance data for local LLM deployment.

A practical guide to running Claude Code sessions remotely via phone using Tailscale (VPN) + tmux. The setup enables terminal access to home MacBook sessions from anywhere, demonstrating creative mobile workflows for AI-assisted development.

A sequel build featuring 4x R9700 GPUs (128GB VRAM total) optimized for local LLM deployment. The post includes detailed upgrade path from previous MI100 setup, performance benchmarks, and lessons learned—valuable for anyone planning serious local AI infrastructure.

A detailed perspective on the shift from cloud to local AI, citing rising subscription costs and over-tuning/censorship as primary motivations. After weeks testing Llama 3.3, Phi-4, and DeepSeek locally, the author argues 2026 marks the inflection point for local AI viability.

A unique mobile AI workstation in a Thermaltake Core W200 case featuring 10 GPUs (8× 3090 + 2× 5090 = 768GB VRAM), Threadripper Pro 3995WX, and 512GB DDR4. Built for extra-large MoE models and video generation at ~$17k total cost with full enclosure and portability.

A fun comparison post from someone with both maxed M3 Ultra (512GB) and ASUS GB10 in the same room, asking the community for 24-hour experiment ideas. The discussion explores practical use cases and benchmarks for high-end local AI hardware.

Practical discussion of GPU procurement in Shenzhen's electronics markets for local AI deployment, including modded cards and domestic alternatives. Provides insight into the global GPU market and alternative sourcing strategies.

Significant policy change affecting DIY server builders: Supermicro discontinuing standalone motherboard sales in favor of complete systems only. This constrains options for custom AI infrastructure builds and drives up costs for self-hosting enthusiasts.

Technical discovery enabling AMD integrated GPUs to access massive amounts of system RAM as unified memory on Linux, opening new possibilities for memory-bound AI workloads on consumer hardware. This demonstrates creative solutions for working around VRAM limitations.

Detailed account of challenges selling high-end server hardware on eBay, including buyer disputes and platform limitations. Important practical advice for the self-hosting community buying and selling equipment.