yabs.io

Yet Another Bookmarks Service

Viewing mzimmerm's Bookmarks


[https://docs.openwebui.com/] - - public:mzimmerm
ai, api, my, openai-api, software, webui - 6 | id:1572810 -

UI for chats - as advanced as any proffessional chat apps. Talks to model servers, http on the user chat end, openAI API on the model end.

[https://unsloth.ai/docs/basics/dynamic-3.0-ggufs] - - public:mzimmerm
good, local-ai, model-4gb, unsloth - 4 | id:1572800 -

Unsloth Dynamic V3.1 (Nov 2026) is our latest quantization algorithm for Qwen3.8-27B, DeepSeek-V4.1-Flash-1017 and Qwen3.8-Flash-Next, resulting in 10-15% increased accuracy and 50% less token usage vs other quants of the same size. We're working on an even better version of Dynamic quants which will be applied to Qwen4 and all future models.

[https://www.youtube.com/watch?v=qW2E2ay47dw] - - public:mzimmerm
code-model, good, MoE-model, strata, video - 5 | id:1572788 -

Also A) DeepSeekFlash-4.1-550B-FP16-1TB is better at coding than Qwen-3.8-27B-Q??-S??. B) If using Strata: 1) SSD NVMe PCIe 4.0 2) Faster CPU matters!! C) 17:10 DeepSeek: Typical coding agent step: send/reads ~ 3,000 new tokens (needs 25s) , writes ~ 750 (needs 150s). The rest of tokens, ~120,000 tokens, is already cached. D) NOTE Qwen is 5 times FASTER at the above E) on 2026-10-07, llama.cpp added their own GPU cache for MoE models.

[https://www.youtube.com/watch?v=bCeuhqP3V_o] - - public:mzimmerm
ai, full-stack, good, local, local-ai, on-device - 6 | id:1572773 -

Full-stack local AI, all on one harness/agent (uses Hermess agent) is defined as : 1) LLM with images 2) context 100K tokens 3) image generation 4) embedding - huge memory of AI 5) speech to text to understand speech 6) text to speech for answer as speech 7) video generation is NOT possible on 32GB

[https://forums.developer.nvidia.com/t/tutorial-build-llama-cpp-from-source-and-run-qwen3-235b/352604] - - public:mzimmerm
llama.cpp, nvidia-spark, tutorial - 3 | id:1572751 -

This is hopefully a simple tutorial on compiling llama.cpp on the DGX Spark, once compiled, it can be used to run GGML-based LLM models directly on the command line, served as an OpenAI compatible API, or accessed via a web browser (which is what we’ll be doing for this tutorial).

[https://www.reddit.com/r/LocalLLM/comments/1w58in6/whats_your_opinion_about_nvidia_dgx_spark/] - - public:mzimmerm
nvidia-spark - 1 | id:1572743 -

Max is slower than a DGX Spark for diffusion models. Those are compute bound and a Mac has like 60TFlops Fp16 while a Spark had 140. And it's x2 and x4 for Fp8 and Fp4 on the spark. For Mac it's unsure given that Fp32 and Fp16 go at the same speed.

With marked bookmarks
| (+) | |

Viewing 1 - 50, 50 links out of 2861 links, page: 1


Export:

JSONXMLRSS