README.md

Phi Cookbook: Hands-On Examples with Microsoft's Phi Models

Phi is a series of open source AI models developed by Microsoft.

Phi is currently the most powerful and cost-effective small language model (SLM), with very good benchmarks in multi-language, reasoning, text/chat generation,coding, images, audio and other scenarios.

You can deploy Phi to the cloud or to edge devices, and you can easily build generative AI applications with limited computing power.

Follow these steps to get started using these resource :

1. Fork the Repository: Click on the "Fork" button at the top-right corner of this page.
2. Clone the Repository: git clone https://github.com/microsoft/PhiCookBook.git

Table of Contents

• Introduction

  • Welcome to the Phi Family
  • Setting up your environment
  • Understanding Key Technologies
  • AI Safety for Phi Models
  • Phi Hardware Support
  • Phi Models & Availability across platforms
  • Using Guidance-ai and Phi
  • GitHub Marketplace Models
  • Azure AI Model Catalog

• Inference Phi in different environment

  • Hugging face
  • GitHub Models
  • Azure AI Foundry Model Catalog
  • Ollama
  • AI Toolkit VSCode (AITK)
  • NVIDIA NIM

• Inference Phi Family

  • Inference Phi in iOS
  • Inference Phi in Android
  • Inference Phi in Jetson
  • Inference Phi in AI PC
  • Inference Phi with Apple MLX Framework
  • Inference Phi in Local Server
  • Inference Phi in Remote Server using AI Toolkit
  • Inference Phi with Rust
  • Inference Phi--Vision in Local
  • Inference Phi with Kaito AKS, Azure Containers(official support)

• Quantifying Phi Family

  • Quantizing Phi-3.5 / 4 using llama.cpp
  • Quantizing Phi-3.5 / 4 using Generative AI extensions for onnxruntime
  • Quantizing Phi-3.5 / 4 using Intel OpenVINO
  • Quantizing Phi-3.5 / 4 using Apple MLX Framework

• Evaluation Phi

  • Response AI
  • Azure AI Foundry for Evaluation
  • Using Promptflow for Evaluation

• Phi application development samples

  • Text & Chat Applications

    • Phi-4 Samples 🆕
      • [📓] Chat With Phi-4-mini ONNX Model
    • Phi-3 / 3.5 Samples
      • Local Chatbot in the browser using Phi3, ONNX Runtime Web and WebGPU
      • OpenVino Chat
      • Multi Model - Interactive Phi-3-mini and OpenAI Whisper
      • MLFlow - Building a wrapper and using Phi-3 with MLFlow
      • Model Optimization - How to optimize Phi-3-min model for ONNX Runtime Web with Olive
      • WinUI3 App with Phi-3 mini-4k-instruct-onnx -WinUI3 Multi Model AI Powered Notes App Sample
      • Fine-tune and Integrate custom Phi-3 models with Prompt flow
      • Fine-tune and Integrate custom Phi-3 models with Prompt flow in Azure AI Foundry
      • Evaluate the Fine-tuned Phi-3 / Phi-3.5 Model in Azure AI Foundry Focusing on Microsoft's Responsible AI Principles
      • [📓] Phi-3.5-mini-instruct language prediction sample (Chinese/English)
      • Phi-3.5-Instruct WebGPU RAG Chatbot
      • Using Windows GPU to create Prompt flow solution with Phi-3.5-Instruct ONNX
      • Using Microsoft Phi-3.5 tflite to create Android app

  • Azure AI Inference SDK Code Based Samples

    • Phi-4 Samples 🆕
      • [📓] Generate project code using Phi-4-multimodal
    • Phi-3 / 3.5 Samples
      • Build your own Visual Studio Code GitHub Copilot Chat with Microsoft Phi-3 Family
      • Create your own Visual Studio Code Chat Copilot Agent with Phi-3.5 by GitHub Models

  • Advanced Reasoning Samples

    • Phi-4 Samples 🆕
      • [📓] Phi-4-mini Reasoning Samples

  • Vision Samples

    • Phi-4 Samples 🆕
      • [📓] Use Phi-4-multimodal to read images and generate code
    • Phi-3 / 3.5 Samples
      • [📓]Phi-3-vision-Image text to text
      • Phi-3-vision-ONNX
      • [📓]Phi-3-vision CLIP Embedding
      • DEMO: Phi-3 Recycling
      • Phi-3-vision - Visual language assistant - with Phi3-Vision and OpenVINO
      • Phi-3 Vision Nvidia NIM
      • Phi-3 Vision OpenVino
      • [📓]Phi-3.5 Vision multi-frame or multi-image sample

  • Audio Samples

    • Phi-4 Samples 🆕
      • [📓] Extracting audio transcripts using Phi-4-multimodal
      • [📓] Phi-4-multimodal Audio Sample
      • [📓] Phi-4-multimodal Speech Translation Sample

  • MOE Samples

    • Phi-3 / 3.5 Samples
      • [📓] Phi-3.5 Mixture of Experts Models (MoEs) Social Media Sample
      • [📓] Building a Retrieval-Augmented Generation (RAG) Pipeline with NVIDIA NIM Phi-3 MOE, Azure AI Search, and LlamaIndex

  • Function Calling Samples

    • Phi-4 Samples 🆕
      • [📓] Using Function Calling With Phi-4-mini

  • Multimodal Mixing Samples

    • Phi-4 Samples 🆕
      • [📓] Using Phi-4-multimodal as a Technology journalist

• Fine-tuning Phi Samples

  • Fine-tuning Scenarios
  • Fine-tuning vs RAG
  • Fine-tuning Let Phi-3 become an industry expert
  • Fine-tuning Phi-3 with AI Toolkit for VS Code
  • Fine-tuning Phi-3 with Azure Machine Learning Service
  • Fine-tuning Phi-3 with Lora
  • Fine-tuning Phi-3 with QLora
  • Fine-tuning Phi-3 with Azure AI Foundry
  • Fine-tuning Phi-3 with Azure ML CLI/SDK
  • Fine-tuning with Microsoft Olive
  • Fine-tuning with Microsoft Olive Hands-On Lab
  • Fine-tuning Phi-3-vision with Weights and Bias
  • Fine-tuning Phi-3 with Apple MLX Framework
  • Fine-tuning Phi-3-vision (official support)
  • Fine-Tuning Phi-3 with Kaito AKS , Azure Containers(official Support)
  • Fine-Tuning Phi-3 and 3.5 Vision

• Hands on Lab

  • Exploring cutting-edge models: LLMs, SLMs, local development and more
  • Unlocking NLP Potential: Fine-Tuning with Microsoft Olive

• Academic Research Papers and Publications

  • Textbooks Are All You Need II: phi-1.5 technical report
  • Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
  • Phi-4 Technical Report
  • Optimizing Small Language Models for In-Vehicle Function-Calling
  • (WhyPHI) Fine-Tuning PHI-3 for Multiple-Choice Question Answering: Methodology, Results, and Challenges

Using Phi Models

Phi on Azure AI Foundry

You can learn how to use Microsoft Phi and how to build E2E solutions in your different hardware devices. To experience Phi for yourself, start by playing with the models and customizing Phi for your scenarios using the Azure AI Foundry Azure AI Model Catalog you can learn more at Getting Started with Azure AI Foundry

Playground Each model has a dedicated playground to test the model Azure AI Playground.

Phi on GitHub Models

You can learn how to use Microsoft Phi and how to build E2E solutions in your different hardware devices. To experience Phi for yourself, start by playing with the model and customizing Phi for your scenarios using the GitHub Model Catalog you can learn more at Getting Started with GitHub Model Catalog

Playground Each model has a dedicated playground to test the model.

Phi on Hugging Face

You can also find the model on the Hugging Face

Playground Hugging Chat playground

Responsible AI

Microsoft is committed to helping our customers use our AI products responsibly, sharing our learnings, and building trust-based partnerships through tools like Transparency Notes and Impact Assessments. Many of these resources can be found at https://aka.ms/RAI. Microsoft’s approach to responsible AI is grounded in our AI principles of fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.

Large-scale natural language, image, and speech models - like the ones used in this sample - can potentially behave in ways that are unfair, unreliable, or offensive, in turn causing harms. Please consult the Azure OpenAI service Transparency note to be informed about risks and limitations.

The recommended approach to mitigating these risks is to include a safety system in your architecture that can detect and prevent harmful behavior. Azure AI Content Safety provides an independent layer of protection, able to detect harmful user-generated and AI-generated content in applications and services. Azure AI Content Safety includes text and image APIs that allow you to detect material that is harmful. Within Azure AI Foundry, the Content Safety service allows you to view, explore and try out sample code for detecting harmful content across different modalities. The following quickstart documentation guides you through making requests to the service.

Another aspect to take into account is the overall application performance. With multi-modal and multi-models applications, we consider performance to mean that the system performs as you and your users expect, including not generating harmful outputs. It's important to assess the performance of your overall application using Performance and Quality and Risk and Safety evaluators. You also have the ability to create and evaluate with custom evaluators.

You can evaluate your AI application in your development environment using the Azure AI Evaluation SDK. Given either a test dataset or a target, your generative AI application generations are quantitatively measured with built-in evaluators or custom evaluators of your choice. To get started with the azure ai evaluation sdk to evaluate your system, you can follow the quickstart guide. Once you execute an evaluation run, you can visualize the results in Azure AI Foundry.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.