This project demonstrates how to run the Phi-4 Mini Instruct ONNX model locally using C# and the Microsoft.ML.OnnxRuntimeGenAI library. It is optimized for Intel Iris Xe GPUs but can be run on CPU as well.
- .NET 6.0 SDK or later
- Visual Studio 2022 or later (or any compatible C# IDE)
- Windows 10/11 (recommended for DirectML GPU acceleration)
- Git LFS (For downloading large model files)
- [Optional] Intel Iris Xe GPU for best performance
We use the INT4 ONNX version of Phi-4 for the best balance of speed and intelligence.
# Install Git LFS
git lfs install
# Clone the repository (approx 2.5GB)
git clone https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx
# Add the DirectML package for GPU acceleration
dotnet add package Microsoft.ML.OnnxRuntimeGenAI.DirectML --version 0.12.2After cloning, extract or locate the model files. The folder must contain:
genai_config.json.onnxmodel file(s)
-
Clone this repository:
git clone https://github.com/joshivilas/SLM-Phi-Local.git cd PhiSafeConsole -
Open the solution in Visual Studio.
-
Restore NuGet packages:
- Visual Studio will prompt you to restore packages on open, or right-click the solution and select
Restore NuGet Packages.
- Visual Studio will prompt you to restore packages on open, or right-click the solution and select
-
Set the model path:
- In
Program.cs, update themodelPathvariable to the folder where you extracted the ONNX model files. - Example:
string modelPath = @"C:\MyProjects\small-lang-proj\PhiLocal\Phi-4-mini-instruct-onnx\cpu_and_mobile\cpu-int4-rtn-block-32-acc-level-4";
- In
-
Build and run the project:
- Press
F5in Visual Studio or run:dotnet run
- Press
- When the console starts, type your prompt after
You >and press Enter. - The model will generate a response in real-time.
- Type
exitor press Enter on an empty line to quit.
- If you see
[FATAL ERROR], ensure:- The
modelPathis correct and points to the folder withgenai_config.jsonand the.onnxfile. - You have the correct INT4 ONNX model version.
- Your hardware supports DirectML (for GPU acceleration).
- The