I used to think AMD’s ROCm was a nightmare to get working. I was wrong. I managed to get ROCm running in less than a day with minimal effort just by poking around on Google. Like NVIDIA, it is distribution-agnostic, even if it lacks maturity.

Then I met Intel’s AI stack. My definition of “nightmare” has been officially updated.

It took almost a week to get llama.cpp’s OpenVINO working on Gentoo. The software suffers from a severe case of “Ubuntunitis”—the delusional belief that Ubuntu is the only distribution in existence. This is the hallmark of companies with zero real Linux strategy. They treat Linux like Windows, showing a complete lack of willingness to learn how professional software development for the Linux actually works.

The experience is pure hell. Intel has scattered their AI software stack across a dozen different repositories. To make matters worse, every single component requires a perfect, fragile version match to function. I cannot fathom why they created such a mess. The fact that they have been working on this for nearly a decade and it still feels like beta-quality software is appalling.

I own the hardware, so I am going to make it work. However, due to the absolute chaos of version mismatches, these instructions may be out of date by the time you read them. Use this as a general guide, not a guarantee.

And yes, they provide OpenVINO via Docker. If you value your sanity, do that. This guide is essentially a cautionary tale for future package maintainers on other distributions to see just how dismal the native installation experience is.

1. Install the oneAPI Toolkit

First, download the Intel oneAPI toolkit. You must select the offline installer for Linux.

Run the offline installer with the following command. It is recommended to install all components except the Intel VTune Profiler.

sh ./intel-oneapi-toolkit-2026.1.0.192_offline.sh

2. Configure the Compiler (GCC 13)

The Intel stack is extremely picky about compiler versions. It currently refuses to compile with anything newer than GCC 13.

Install GCC 13 as a one-shot package:

emerge --oneshot sys-devel/gcc:13

Note for Heterogeneous CPUs: If you are using a CPU with both Performance (P) and Efficiency (E) cores, GCC 13 may fail to compile. To avoid this, use the following flags to target a generic x86-64 architecture:

USE="-ada -sanitize -fortran" CFLAGS="-march=x86-64" CXXFLAGS="-march=x86-64" emerge --oneshot sys-devel/gcc:13

Once installed, set GCC 13 as your system default using gcc-config. Find the index number for x86_64-pc-linux-gnu-13 and apply it:

gcc-config -l
# Example: if the index is 4
gcc-config 4
source /etc/profile

3. Set the Package Configuration Path

Gentoo does not include the local library path in the PKG_CONFIG_PATH by default. You must export it manually to ensure the build system finds the installed components:

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib64/pkgconfig/

4. Compile Level-Zero

Clone the specific version of the Level-Zero loader, configure it with CMake, and install it.

git clone --branch v1.27.0 --single-branch --depth 1 https://github.com/oneapi-src/level-zero.git
cd level-zero
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build --target install --parallel

5. Compile IGCS (Intel Graphics Compiler Shim)

Clone and build the IGCS component.

git clone --branch V0.9.5 --single-branch --depth 1 https://github.com/intel/igsc.git
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build --target install --parallel

6. Compile the Intel Graphics Compiler (IGC)

This step is complex because it requires a specific ecosystem of dependencies. You must clone several repositories with exact revisions to ensure compatibility: LLVM, OpenCL-Clang, SPIRV-LLVM, and several SPIRV tools/headers.

Once the repositories are cloned, build the IGC:

# [Massive git clone block for dependencies]
cmake -B build -S ../igc -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -DCMAKE_BUILD_TYPE=Release -Wno-dev
cmake --build build --target install --parallel 20

7. Compile GMMlib

Build the Intel Graphics Memory Management library.

git clone --branch intel-gmmlib-22.9.0 --single-branch --depth 1 https://github.com/intel/gmmlib.git
cd gmmlib
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build --target install --parallel

8. Compile OpenCV 4.13

OpenCV must be compiled with specific flags to support SYCL, TBB, and oneDNN.

git clone --revision=4.13.0 --single-branch --depth 1 https://github.com/opencv/opencv.git
cd opencv
cmake -B build -S . -DWITH_SYCL=ON -DWITH_TBB=ON -DWITH_ONEDNN=ON -DWITH_IPP=ON
cmake --build build --target install --parallel 20

9. Compile Compute Runtime (Neo)

Clone the Neo compute runtime. You will need to manually edit the CMake files to fix the directory paths for IGC and GMMlib.

Edit neo/cmake/find_igc.cmake and neo/cmake/find_gmmlib.cmake. At lines 23 and 24, modify the logic to handle the lib64 directory:

if(IS_DIRECTORY "${IGC_DIR}/lib64/pkgconfig/")
set(__tmp_LIBDIR "lib64")

Then build and install:

git clone --branch 26.05.37020.3 --single-branch --depth 1 --recurse-submodules https://github.com/intel/compute-runtime neo
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DNEO_SKIP_UNIT_TESTS=1 -DIGC_DIR="/usr/local/" -DGMM_DIR="/usr/local/"
cmake --build build --target install --parallel 20

10. Compile OpenVINO

Clone the OpenVINO repository. You must manually move the OpenCL headers into the system directory to prevent build errors.

git clone --branch 2026.2.1 --single-branch --depth 1 --recurse-submodules https://github.com/openvinotoolkit/openvino.git

# Manual header migration
cp /home/fbugno/bin/local_installs/openvino/thirdparty/ocl/clhpp_headers/include/CL/cl2.hpp /usr/include/CL/
cp /home/fbugno/bin/local_installs/openvino/thirdparty/ocl/clhpp_headers/include/CL/opencl.hpp /usr/include/CL/

# Configure and build
git config --global --add safe.directory /home/fbugno/bin/local_installs/intel_ai/openvino/thirdparty/level_zero/level-zero
cmake -B build -S . -DCMAKE_INSTALL_PREFIX=/opt/intel/openvino -DENABLE_INTEL_CPU=ON -DENABLE_INTEL_GPU=ON -DENABLE_INTEL_NPU=ON -DENABLE_WHEEL=OFF -DENABLE_PYTHON=OFF -DOpenCV_DIR=/usr/local/lib64/cmake/opencv4
cmake --build build --target install --parallel 20

11. Compile oneTBB

Build the Threading Building Blocks library with a custom install prefix.

git clone --depth 1 --branch v2022.3.0 https://github.com/oneapi-src/oneTBB
cd oneTBB
cmake -S . -B build -DCMAKE_INSTALL_PREFIX=/opt/intel/onetbb -DCMAKE_BUILD_TYPE=Release -DTBB_TEST=OFF -DTBB_STRICT=OFF
cmake --build build --target install --parallel 20
ldconfig

12. Compile the NPU Driver

To compile the NPU driver, you must first source the OpenVINO environment variables and update the PKG_CONFIG_PATH. Note that git-lfs is required for this step.

source /opt/intel/openvino/setupvars.sh
export PKG_CONFIG_PATH=/opt/intel/onetbb/lib64/pkgconfig:/usr/local/lib64/pkgconfig/:$PKG_CONFIG_PATH
git clone --branch v1.33.0 --single-branch --depth 1 --recurse-submodules https://github.com/intel/linux-npu-driver.git
cd linux-npu-driver
cmake -B build -S . -DENABLE_NPU_COMPILER_BUILD=ON -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -DCMAKE_BUILD_TYPE=Release -DENABLE_OPENVINO_PACKAGE=OFF -Wno-dev
cmake --build build --target install --parallel 20

Validation: You can verify the installation by running the NPU UMD tests:

cd validation
mkdir -p models/add_abc
curl -o models/add_abc/add_abc.xml https://raw.githubusercontent.com/openvinotoolkit/openvino/master/src/core/tests/models/ir/add_abc.xml
touch models/add_abc/add_abc.bin
npu-umd-test --config ./umd-test/configs/basic.yaml

13. Compile llama.cpp

You can compile llama.cpp using either the OpenVINO backend or the SYCL backend.

For OpenVINO support:

git clone --branch b9993 --single-branch --depth 1 https://github.com/ggml-org/llama.cpp.git
source /opt/intel/openvino/setupvars.sh
cmake -B build -S . -DCMAKE_INSTALL_PREFIX=/opt/intel/llama.cpp -DGGML_OPENVINO=ON -DCMAKE_BUILD_TYPE=Release -Wno-dev
cmake --build build --parallel 20

For SYCL support:

source /opt/intel/oneapi/setvars.sh
cmake -B build -S . -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_SYCL=ON -DGGML_SYCL_F16=ON -DCMAKE_BUILD_TYPE=Release -Wno-dev
cmake --build build --parallel 20

14. Compile OVMS (OpenVINO Model Server)

git clone https://github.com/openvinotoolkit/model_server.git
git checkout v2026.2.1

Summary of Dependencies

For future recipe support, ensure the following Gentoo packages are installed: sys-devel/gcc, dev-util/cmake, dev-util/ninja, dev-util/pkgconf, dev-vcs/git, dev-vcs/git-lfs, dev-util/patchelf, dev-cpp/tbb, dev-libs/pugixml, app-arch/snappy, dev-libs/rapidjson, dev-libs/flatbuffers, dev-libs/nlohmann_json, dev-libs/ocl-icd, dev-util/opencl-headers, media-libs/libva, dev-lang/python, dev-python/pip, dev-python/pybind11, dev-libs/libffi.

Testing Results

When running models on the GPU or NPU, use the following environment variables:

export GPU_ENABLE_LARGE_ALLOCATIONS=1
export GGML_OPENVINO_STATEFUL_EXECUTION=1
export GGML_OPENVINO_DEVICE=NPU

Performance/Stability Notes:

  • Most models (Qwen, GLM, Gemma, etc.): Failed.
  • DevQuasar/Goekdeniz-Guelmez_Josiefied-Qwen3-8B: Worked (11 tokens/sec) with a 25k context window and stateful execution.
  • saidutta69/Mistral-7B-Instruct-v0.3: Worked on NPU with context size below 10k.

Conclusion

The OpenVINO Model Server via Docker remains the superior choice for Intel CPUs. By a massive margin.