Skip to content
WordOllama

How to Enable GPU Support

NVIDIA

Supported Graphics Cards

  • NVIDIA GeForce RTX Series (20/30/40/50 series and above)
  • NVIDIA GeForce GTX 16 Series and above
  • NVIDIA Tesla Series
  • Recommended VRAM: 6GB or more
  • CUDA Capability: 7.0 or higher

Installing CUDA

  1. Visit the NVIDIA website to download CUDA Toolkit (https://developer.nvidia.com/cuda-downloads)
  2. Select the Windows operating system and corresponding version
  3. Download and install CUDA Toolkit (version 11.7 or higher recommended)
  4. After installation, open a command prompt and enter the following to verify:
Terminal window
nvidia-smi
  1. Restart Ollama to enable GPU support

AMD

Supported Graphics Cards

Officially supported:

  • AMD Radeon RX 9000 Series
  • AMD Radeon RX 7000 Series
  • AMD Radeon RX 6000 Series
  • AMD Instinct Series
  • Recommended VRAM: 6GB or more

Installing HIP

  1. Download and install the latest AMD driver
  2. Install HIP SDK (https://www.amd.com/en/developer/resources/rocm-hub/hip-sdk.html)
  3. Restart Ollama to enable GPU support

Workaround for Unofficially Supported AMD Graphics Cards

Some AMD cards (e.g., 500 Series, RDNA 5000 Series, 680M) lack official ROCm (HIP) support and cannot directly enable GPU functionality. Follow these steps to make them work:

Ollama-for-AMD: A Library for Unofficially Supported AMD Cards

  1. Visit the project page: https://github.com/likelovewant/ollama-for-amd
  2. Download the precompiled version or compile Ollama from source and install
  3. Download precompiled rocblas and library files, or compile them following the repository’s WIKI
  4. Replace rocblas.dll in C:\Program Files\AMD\ROCm\6.1\bin with the downloaded version, and overwrite the library folder in rocblas\library
  5. Restart Ollama

A Simpler Solution

  1. Use my tool: https://github.com/ByronLeeeee/Ollama-For-AMD-Installer
  2. Select your graphics card model, click “Check Latest Version” to automatically download and install the latest Ollama-for-AMD build, compatible rocblas, and library files, and complete the replacement.

Ollama-For-AMD-Installer

Notes

  1. If GPU support still fails (common in dual-GPU laptops), try forcing Ollama to use a specific GPU via environment variables.
  2. Set the system power plan to “High Performance” mode.
  3. Keep graphics drivers up to date.
  4. Monitor VRAM usage to avoid overflow.
  5. When using large models, close other GPU-intensive applications.