Integrated GPU support · Issue #2637 · ollama/ollama
[https://github.com/ollama/ollama/issues/2637] - - public:mzimmerm
Latest (0.1.27) docker image with ROCm works for me on Ryzen 5600G with 8GB VRAM allocation. Prompt processing is 2x faster than with CPU. Generation runs at max speed even if CPU is busy running other processes. I am on Fedora 39. Container setup: HSA_OVERRIDE_GFX_VERSION=9.0.0