sdk/jobs/automl-standalone-jobs/automl-image-classification-multiclass-batch-scoring/image-classification-multiclass-batch-scoring-non-mlflow-model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Batch Predictions for an Image Classification model trained using AutoML.\n",
    "\n",
    "**Requirements** - In order to benefit from this tutorial, you will need:\n",
    "- A basic understanding of Machine Learning\n",
    "- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
    "- A Compute Cluster. [Check this notebook to create a compute cluster](../../../resources/compute/compute.ipynb)\n",
    "- A python environment\n",
    "- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../../README.md) - check the getting started section\n",
    "\n",
    "**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
    "- Connect to your AML workspace from the Python SDK\n",
    "- Create an `AutoML Image Classification Multiclass Training Job` with the 'image_classification()' factory-function.\n",
    "- Train the model using AmlCompute by submitting/running the AutoML training job\n",
    "- Obtain the best model, register it and deploy it to a batch endpoint\n",
    "- Generate batch predictions using the batch endpoint\n",
    "\n",
    "**Please note**: For this notebook you can use an existing image classification model trained using AutoML for Images or use the simple model training we included below for convenience. For detailed instructions on how to train an image classification model with AutoML, please refer to the official [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2) and to the image classification multiclass [notebook](../automl-image-classification-multiclass-task-fridge-items/automl-image-classification-multiclass-task-fridge-items.ipynb).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Connect to Azure Machine Learning Workspace\n",
    "\n",
    "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the job will be run.\n",
    "\n",
    "## 1.1. Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "from azure.identity import DefaultAzureCredential\n",
    "from azure.ai.ml import MLClient\n",
    "\n",
    "from azure.ai.ml.constants import AssetTypes, InputOutputModes\n",
    "from azure.ai.ml import Input\n",
    "from azure.ai.ml.automl import ImageClassificationSearchSpace\n",
    "from azure.ai.ml.sweep import (\n",
    "    Choice,\n",
    "    Uniform,\n",
    "    BanditPolicy,\n",
    ")\n",
    "\n",
    "from azure.ai.ml import automl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2. Configure workspace details and get a handle to the workspace\n",
    "\n",
    "To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "credential = DefaultAzureCredential()\n",
    "ml_client = None\n",
    "try:\n",
    "    ml_client = MLClient.from_config(credential)\n",
    "except Exception as ex:\n",
    "    print(ex)\n",
    "    # Enter details of your AML workspace\n",
    "    subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "    resource_group = \"<RESOURCE_GROUP>\"\n",
    "    workspace = \"<AML_WORKSPACE_NAME>\"\n",
    "    ml_client = MLClient(credential, subscription_id, resource_group, workspace)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. MLTable with input Training Data\n",
    "\n",
    "In order to generate models for computer vision tasks with automated machine learning, you need to bring labeled image data as input for model training in the form of an MLTable. You can create an MLTable from labeled training data in JSONL format. If your labeled training data is in a different format (like, pascal VOC or COCO), you can use a conversion script to first convert it to JSONL, and then create an MLTable. Alternatively, you can use Azure Machine Learning's [data labeling tool](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-image-labeling-projects) to manually label images, and export the labeled data to use for training your AutoML model.\n",
    "\n",
    "In this notebook, we use a toy dataset called Fridge Objects, which consists of 134 images of 4 classes of beverage container {can, carton, milk bottle, water bottle} photos taken on different backgrounds.\n",
    "\n",
    "All images in this notebook are hosted in [this repository](https://github.com/microsoft/computervision-recipes) and are made available under the [MIT license](https://github.com/microsoft/computervision-recipes/blob/master/LICENSE).\n",
    "\n",
    "**NOTE:** In this PRIVATE PREVIEW we're defining the MLTable in a separate folder and .YAML file.\n",
    "In later versions, you'll be able to do it all in Python APIs.\n",
    "\n",
    "## 2.1. Download the Data\n",
    "We first download and unzip the data locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib\n",
    "from zipfile import ZipFile\n",
    "\n",
    "# download data\n",
    "download_url = \"https://cvbp-secondary.z19.web.core.windows.net/datasets/image_classification/fridgeObjects.zip\"\n",
    "data_file = \"fridgeObjects.zip\"\n",
    "urllib.request.urlretrieve(download_url, filename=data_file)\n",
    "\n",
    "# extract files\n",
    "with ZipFile(data_file, \"r\") as zip:\n",
    "    print(\"extracting files...\")\n",
    "    zip.extractall(path=\"./data\")\n",
    "    print(\"done\")\n",
    "\n",
    "# delete zip file\n",
    "os.remove(data_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a sample image from this dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "sample_image = \"./data/fridgeObjects/milk_bottle/99.jpg\"\n",
    "Image(filename=sample_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2. Upload the images to Datastore through an AML Data asset (URI Folder)\n",
    "\n",
    "In order to use the data for training in Azure ML, we upload it to our default Azure Blob Storage of our  Azure ML Workspace.\n",
    "\n",
    "[Check this notebook for AML data asset example](https://github.com/Azure/azureml-examples/blob/b674293eaec6c60cb26cb3e5672c100c41ab4769/sdk/assets/data/data.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uploading image files by creating a 'data asset URI FOLDER':\n",
    "\n",
    "from azure.ai.ml.entities import Data\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "\n",
    "my_data = Data(\n",
    "    path=\"./data/fridgeObjects\",\n",
    "    type=AssetTypes.URI_FOLDER,\n",
    "    description=\"Fridge-items images\",\n",
    "    name=\"fridge-items-images\",\n",
    ")\n",
    "\n",
    "uri_folder_data_asset = ml_client.data.create_or_update(my_data)\n",
    "\n",
    "print(uri_folder_data_asset)\n",
    "print(\"\")\n",
    "print(\"Path to folder in Blob Storage:\")\n",
    "print(uri_folder_data_asset.path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Convert the downloaded data to JSON metadata\n",
    "\n",
    "In this example, the fridge object dataset is stored in a directory. There are four different folders inside:\n",
    "\n",
    "/water_bottle\n",
    "/milk_bottle\n",
    "/carton\n",
    "/can\n",
    "\n",
    "This is the most common data format for multiclass image classification. Each folder title corresponds to the image label for the images contained inside.\n",
    "\n",
    "In order to use this data to create an AzureML MLTable, we first need to convert it to the required JSONL format. \n",
    "\n",
    "The following script is creating two .jsonl files (one for training and one for validation) in the parent folder of the dataset. The train / validation ratio corresponds to 20% of the data going into the validation file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "\n",
    "src_images = \"./data/fridgeObjects/\"\n",
    "\n",
    "# We'll copy each JSONL file within its related MLTable folder\n",
    "training_mltable_path = \"./data/training-mltable-folder/\"\n",
    "validation_mltable_path = \"./data/validation-mltable-folder/\"\n",
    "\n",
    "train_validation_ratio = 5\n",
    "\n",
    "# Path to the training and validation files\n",
    "train_annotations_file = os.path.join(training_mltable_path, \"train_annotations.jsonl\")\n",
    "validation_annotations_file = os.path.join(\n",
    "    validation_mltable_path, \"validation_annotations.jsonl\"\n",
    ")\n",
    "\n",
    "# Baseline of json line dictionary\n",
    "json_line_sample = {\n",
    "    \"image_url\": uri_folder_data_asset.path,\n",
    "    \"label\": \"\",\n",
    "}\n",
    "\n",
    "index = 0\n",
    "# Scan each sub directary and generate a jsonl line per image, distributed on train and valid JSONL files\n",
    "with open(train_annotations_file, \"w\") as train_f:\n",
    "    with open(validation_annotations_file, \"w\") as validation_f:\n",
    "        for className in os.listdir(src_images):\n",
    "            subDir = src_images + className\n",
    "            if not os.path.isdir(subDir):\n",
    "                continue\n",
    "            # Scan each sub directary\n",
    "            print(\"Parsing \" + subDir)\n",
    "            for image in os.listdir(subDir):\n",
    "                json_line = dict(json_line_sample)\n",
    "                json_line[\"image_url\"] += f\"{className}/{image}\"\n",
    "                json_line[\"label\"] = className\n",
    "\n",
    "                if index % train_validation_ratio == 0:\n",
    "                    # validation annotation\n",
    "                    validation_f.write(json.dumps(json_line) + \"\\n\")\n",
    "                else:\n",
    "                    # train annotation\n",
    "                    train_f.write(json.dumps(json_line) + \"\\n\")\n",
    "                index += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4. Create MLTable data input\n",
    "Create MLTable data input using the jsonl files created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Training MLTable defined locally, with local data to be uploaded\n",
    "my_training_data_input = Input(type=AssetTypes.MLTABLE, path=training_mltable_path)\n",
    "\n",
    "# Validation MLTable defined locally, with local data to be uploaded\n",
    "my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=validation_mltable_path)\n",
    "\n",
    "# WITH REMOTE PATH: If available already in the cloud/workspace-blob-store\n",
    "# my_training_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml://datastores/workspaceblobstore/paths/vision-classification/train\")\n",
    "# my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml://datastores/workspaceblobstore/paths/vision-classification/valid\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create data input from TabularDataset created using V1 sdk, specify the type as AssetTypes.MLTABLE, mode as InputOutputModes.DIRECT and path in the following format azureml:<tabulardataset_name>:<version>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "# Training MLTable with v1 TabularDataset\n",
    "my_training_data_input = Input(\n",
    "    type=AssetTypes.MLTABLE, path=\"azureml:fridgeObjectsTrainingDataset:1\",\n",
    "    mode=InputOutputModes.DIRECT\n",
    ")\n",
    "\n",
    "# Validation MLTable with v1 TabularDataset\n",
    "my_validation_data_input = Input(\n",
    "    type=AssetTypes.MLTABLE, path=\"azureml:fridgeObjectsValidationDataset:1\",\n",
    "    mode=InputOutputModes.DIRECT\n",
    ")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Compute target setup\n",
    "You will need to provide a [Compute Target](https://docs.microsoft.com/en-us/azure/machine-learning/concept-azure-machine-learning-v2?tabs=cli#computes) that will be used for your AutoML model training. AutoML models for image tasks require GPU SKUs such as the ones from the NC, NCv2, NCv3, ND, NDv2 and NCasT4 series. We recommend using the NCsv3-series (with v100 GPUs) for faster training. Using a compute target with a multi-GPU VM SKU will leverage the multiple GPUs to speed up training. Additionally, setting up a compute target with multiple nodes will allow for faster model training by leveraging parallelism, when tuning hyperparameters for your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import AmlCompute\n",
    "from azure.core.exceptions import ResourceNotFoundError\n",
    "\n",
    "compute_name = \"gpu-cluster\"\n",
    "\n",
    "try:\n",
    "    _ = ml_client.compute.get(compute_name)\n",
    "    print(\"Found existing compute target.\")\n",
    "except ResourceNotFoundError:\n",
    "    print(\"Creating a new compute target...\")\n",
    "    compute_config = AmlCompute(\n",
    "        name=compute_name,\n",
    "        type=\"amlcompute\",\n",
    "        size=\"Standard_NC6\",\n",
    "        idle_time_before_scale_down=120,\n",
    "        min_instances=0,\n",
    "        max_instances=4,\n",
    "    )\n",
    "    ml_client.begin_create_or_update(compute_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Configure and run the AutoML for Images Classification-MultiClass training job\n",
    "\n",
    "AutoML allows you to easily train models for Image Classification, Object Detection & Instance Segmentation on your image data. You can control the model algorithm to be used, specify hyperparameter values for your model as well as perform a sweep across the hyperparameter space to generate an optimal model.\n",
    "\n",
    "When using AutoML for image tasks, you need to specify the model algorithms using the model_name parameter. You can either specify a single model or choose to sweep over multiple models. Please refer to the [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#configure-model-algorithms-and-hyperparameters) for the list of supported model algorithms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1. Using default hyperparameter values for the specified algorithm\n",
    "Before doing a large sweep to search for the optimal models and hyperparameters, we recommend trying the default values for a given model to get a first baseline. Next, you can explore multiple hyperparameters for the same model before sweeping over multiple models and their parameters. This allows an iterative approach, as with multiple models and multiple hyperparameters for each (as we showcase in the next section), the search space grows exponentially, and  you need more iterations to find optimal configurations.\n",
    "\n",
    "Following functions are used to configure the AutoML image job:\n",
    "### image_classification() function parameters:\n",
    "The `image_classification()` factory function allows user to configure the training job.\n",
    "\n",
    "- `compute` - The compute on which the AutoML job will run. In this example we are using a compute called 'gpu-cluster' present in the workspace. You can replace it any other compute in the workspace.\n",
    "- `experiment_name` - The name of the experiment. An experiment is like a folder with multiple runs in Azure ML Workspace that should be related to the same logical machine learning experiment.\n",
    "- `name` - The name of the Job/Run. This is an optional property. If not specified, a random name will be generated.\n",
    "- `primary_metric` - The metric that AutoML will optimize for model selection.\n",
    "- `target_column_name` - The name of the column to target for predictions. It must always be specified. This parameter is applicable to 'training_data' and 'validation_data'.\n",
    "- `training_data` - The data to be used for training. It should contain both training feature columns and a target column. Optionally, this data can be split for segregating a validation or test dataset. \n",
    "You can use a registered MLTable in the workspace using the format '<mltable_name>:<version>' OR you can use a local file or folder as a MLTable. For e.g Input(mltable='my_mltable:1') OR Input(mltable=MLTable(local_path=\"./data\"))\n",
    "The parameter `training_data` must always be provided.\n",
    "\n",
    "\n",
    "### set_limits() function parameters:\n",
    "This is an optional configuration method to configure limits parameters such as timeouts.      \n",
    "    \n",
    "- `timeout_minutes` - Maximum amount of time in minutes that the whole AutoML job can take before the job terminates. If not specified, the default job's total timeout is 6 days (8,640 minutes).\n",
    "    \n",
    "### set_image_model() function parameters:\n",
    "This is an optional configuration method to configure fixed settings or parameters that don't change during the parameter space sweep. Some of the key parameters of this function are:\n",
    "\n",
    "- `model_name` - The name of the ML algorithm that we want to use in training job. Please refer to this [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-auto-train-image-models?tabs=CLI-v2#supported-model-algorithms) for supported model algorithm.\n",
    "- `number_of_epochs` - The number of training epochs. It must be positive integer (default value is 15).\n",
    "- `layers_to_freeze` - The number of layers to freeze in model for transfer learning. It must be a positive integer (default value is 0).\n",
    "- `early_stopping` - It enable early stopping logic during training, It must be boolean value (default is True).   \n",
    "- `optimizer` - Type of optimizer to use in training. It must be either sgd, adam, adamw (default is sgd).\n",
    "- `distributed` - It enable distributed training if compute target contain multiple GPUs. It must be boolean value (default is True).\n",
    "    \n",
    "If you wish to use the default hyperparameter values for a given algorithm (say `vitb16r224`), you can specify the job for your AutoML Image runs as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set up experiment name\n",
    "exp_name = \"dpv2-image-classification-experiment\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the AutoML job with the related factory-function.\n",
    "\n",
    "image_classification_job = automl.image_classification(\n",
    "    compute=compute_name,\n",
    "    # name=\"dpv2-image-classification-job-02\",\n",
    "    experiment_name=exp_name,\n",
    "    training_data=my_training_data_input,\n",
    "    validation_data=my_validation_data_input,\n",
    "    target_column_name=\"label\",\n",
    ")\n",
    "\n",
    "image_classification_job.set_limits(timeout_minutes=60)\n",
    "\n",
    "image_classification_job.set_image_model(model_name=\"vitb16r224\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting an AutoML job for Computer Vision tasks\n",
    "Once you've configured your job, you can submit it as a job in the workspace in order to train a vision model using your training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Submit the AutoML job\n",
    "returned_job = ml_client.jobs.create_or_update(image_classification_job)\n",
    "\n",
    "print(f\"Created job: {returned_job}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.stream(returned_job.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Retrieve the Best Trial (Best Model's trial/run)\n",
    "\n",
    "Use the MLFLowClient to access the results (such as Models, Artifacts, Metrics) of a previously completed AutoML Trial.\n",
    "\n",
    "### Initialize MLFlow Client\n",
    "The models and artifacts that are produced by AutoML can be accessed via the MLFlow interface. Initialize the MLFlow client here, and set the backend as Azure ML, via. the MLFlow Client.\n",
    "\n",
    "IMPORTANT, you need to have installed the latest MLFlow packages with:\n",
    "\n",
    "pip install azureml-mlflow\n",
    "\n",
    "pip install mlflow\n",
    "## 5.1 Obtain the tracking URI for MLFlow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlflow\n",
    "\n",
    "# Obtain the tracking URL from MLClient\n",
    "MLFLOW_TRACKING_URI = ml_client.workspaces.get(\n",
    "    name=ml_client.workspace_name\n",
    ").mlflow_tracking_uri\n",
    "\n",
    "print(MLFLOW_TRACKING_URI)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the MLFLOW TRACKING URI\n",
    "mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)\n",
    "print(\"\\nCurrent tracking uri: {}\".format(mlflow.get_tracking_uri()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlflow.tracking.client import MlflowClient\n",
    "\n",
    "# Initialize MLFlow client\n",
    "mlflow_client = MlflowClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2 Get the AutoML parent Job and find the best run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "job_name = returned_job.name\n",
    "\n",
    "# Get the parent run\n",
    "mlflow_parent_run = mlflow_client.get_run(job_name)\n",
    "\n",
    "# Print parent run tags. 'automl_best_child_run_id' tag should be there.\n",
    "print(mlflow_parent_run.data.tags)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the best model's child run\n",
    "best_child_run_id = mlflow_parent_run.data.tags[\"automl_best_child_run_id\"]\n",
    "print(\"Found best child run id: \", best_child_run_id)\n",
    "\n",
    "best_run = mlflow_client.get_run(best_child_run_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.3 Download the best model locally\n",
    "\n",
    "Access the results (such as Models, Artifacts, Metrics) of a previously completed AutoML Run and download them locally. We will need these artifacts when deploying the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create local folder\n",
    "local_dir = \"./artifact_downloads\"\n",
    "if not os.path.exists(local_dir):\n",
    "    os.mkdir(local_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download run's artifacts/outputs\n",
    "local_path = mlflow_client.download_artifacts(\n",
    "    best_run.info.run_id, \"outputs\", local_dir\n",
    ")\n",
    "print(\"Artifacts downloaded in: {}\".format(local_path))\n",
    "print(\"Artifacts: {}\".format(os.listdir(local_path)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Deploy the non-MLFlow model with batch endpoints and run batch scoring\n",
    "\n",
    "We will now deploy the non-MLFlow model to batch endpoint. Batch endpoint simply means, a REST endpoint which is capable of handing inputs in batch format.\n",
    "\n",
    "To create a batch deployment, you need all the following items:\n",
    "\n",
    "- **Model files**, or a registered model in your workspace referenced using `azureml:<model-name>:<model-version>`. In this notebook we will take the model file from the best_child_run.\n",
    "- **The code to score the model**. \n",
    "- **The environment** in which the model runs. It can be a Docker image with Conda dependencies, or an environment already registered in your workspace referenced using `azureml:<environment-name>:<environment-version>`.  In this notebook we will create a environment object using the environment definition downloaded from the best_child_run artifacts.\n",
    "- **The pre-created compute** referenced using `azureml:<compute-name>`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.1 Regsiter The Model in the workspace\n",
    "For deploying the model, we first need to register it with the existing workspace so that we can discover it during runtime."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import Model\n",
    "\n",
    "tags = dict()\n",
    "tags[\"accuracy\"] = best_run.data.metrics[\"accuracy\"]\n",
    "\n",
    "model_name = \"fridge-items-model\"\n",
    "model = Model(\n",
    "    path=f\"azureml://jobs/{best_run.info.run_id}/outputs/artifacts/outputs/model.pt\",\n",
    "    name=model_name,\n",
    "    description=\"fridge object classification model\",\n",
    "    tags=tags,\n",
    ")\n",
    "\n",
    "# for downloaded file\n",
    "# model = Model(path=\"artifact_downloads/outputs/model.pt\", name=model_name)\n",
    "\n",
    "registered_model = ml_client.models.create_or_update(model)\n",
    "registered_model.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.2 Configure environment\n",
    "It is recommended to use the same envionment for model deployment as the model training, therefore we are creating the Environment object using the conda environment file downloaded as artifacts. We also need to specify the base image, which in case of vision tasks, is `mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.1-cudnn8-ubuntu18.04`\n",
    "\n",
    "To read more about environments, please follow this [notebook](./../../../assets/environment/environment.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import Environment\n",
    "\n",
    "env = Environment(\n",
    "    name=\"automl-images-env\",\n",
    "    description=\"environment for automl images inference\",\n",
    "    image=\"mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.1-cudnn8-ubuntu18.04\",\n",
    "    conda_file=\"artifact_downloads/outputs/conda_env_v_1_0_0.yml\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.3 Get A Scoring Script\n",
    "\n",
    "To do the scoring, you need to create a batch scoring script batch_scoring.py, and write it to the scripts folder in current directory. The script takes a minibatch of input images, applies the classification model, and outputs the predictions to a results file.\n",
    "\n",
    "While creating the batch scoring script, refer to the scoring scripts generated under the outputs folder of the Automl training runs. This will help to identify the right model settings to be used in the batch scoring script init method while loading the model. Note: The batch scoring script we generate in the subsequent step is different from the scoring script generated by the training runs in the below screenshot. We refer to it just to identify the right model settings to be used in the batch scoring script.\n",
    "\n",
    "![scoring-script.png](./ui_outputs_updated.jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understanding the scoring script\n",
    "\n",
    "The scoring_script must contain two functions:\n",
    "\n",
    "- init(): Use this function to load the model into a global object. This function will be called once at the beginning of the process.\n",
    "- run(mini_batch): This function will be called for each mini_batch and do the actual scoring.\n",
    "    + `mini_batch`: The mini_batch value is a list of file paths.\n",
    "    + `result`: The run() method should return a pandas DataFrame or an array. Each returned output element indicates one successful run of an input element in the input mini_batch.\n",
    "    \n",
    "**Note** The scoring script used in this notebook is shown below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# View the batch scoring script. Use the model settings as appropriate for your model.\n",
    "with open(\"./scripts/batch_scoring.py\", \"r\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.4 Deploy the model to batch endpoint\n",
    "**Now, let's deploy the model with batch endpoints and run batch scoring.** \n",
    "\n",
    "It has three steps.\n",
    "- Create a batch endpoint\n",
    "- configure the endpoint\n",
    "- deploy the endpoint using MLClient.\n",
    "\n",
    "### 6.4.1 Create A Batch Endpoint\n",
    "\n",
    "**Please Note** that The name of the endpoint must be unique in the Azure region. For more information on the naming rules, see [managed endpoint limits](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-manage-quotas#azure-machine-learning-managed-online-endpoints). \n",
    "\n",
    "Optionally, you can add description, tags to your endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import (\n",
    "    BatchEndpoint,\n",
    "    BatchDeployment,\n",
    "    BatchRetrySettings,\n",
    ")\n",
    "from azure.ai.ml.constants import BatchDeploymentOutputAction\n",
    "\n",
    "import datetime\n",
    "\n",
    "batch_endpoint_name = \"batch-endpoint-\" + datetime.datetime.now().strftime(\"%Y%m%d%H%M\")\n",
    "\n",
    "# create a batch endpoint\n",
    "endpoint = BatchEndpoint(\n",
    "    name=batch_endpoint_name,\n",
    "    description=\"Sample batch endpoint for classifying fridge objects\",\n",
    "    tags={\"dataset\": \"fridge-objects\"},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the MLClient created earlier, we'll now create the Endpoint in the workspace. This command will start the endpoint creation and return a confirmation response while the endpoint creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.begin_create_or_update(endpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.2 Create batch compute\n",
    "\n",
    "Batch endpoint runs only on cloud computing resources. The cloud computing resource is a reusable virtual computer cluster. \n",
    "The following code creates an Azure Machine Learning compute cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import AmlCompute\n",
    "from azure.core.exceptions import ResourceNotFoundError\n",
    "\n",
    "compute_name = \"cpu-inference-cluster\"\n",
    "\n",
    "try:\n",
    "    _ = ml_client.compute.get(compute_name)\n",
    "    print(\"Found existing compute target.\")\n",
    "except ResourceNotFoundError:\n",
    "    print(\"Creating a new compute target...\")\n",
    "    compute_config = AmlCompute(\n",
    "        name=compute_name,\n",
    "        type=\"amlcompute\",\n",
    "        size=\"Standard_DS12_v2\",\n",
    "        idle_time_before_scale_down=120,\n",
    "        min_instances=0,\n",
    "        max_instances=5,\n",
    "    )\n",
    "    ml_client.begin_create_or_update(compute_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.3 Configure the deployment \n",
    "\n",
    "A deployment is a set of resources required for hosting the model that does the actual inferencing. We'll create a deployment for our endpoint using the BatchDeployment class.  This class allows user to configure the following key aspects.\n",
    "\n",
    "- **name**: Name of the deployment.\n",
    "- **endpoint_name**: Name of the endpoint to create the deployment under.\n",
    "- **model**: The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification.\n",
    "- **environment**: The environment to use for the deployment. This value can be either a reference to an existing versioned environment in the workspace or an inline environment specification. It should of type `azure.ai.ml.entities.environment` \n",
    "- **code_path**: Path to the source code directory for scoring the model\n",
    "- **scoring_script**: Relative path to the scoring file in the source code directory\n",
    "- **compute**: Name of the compute target to execute the batch scoring jobs on\n",
    "- **instance_count**: The number of nodes to use for each batch scoring job.\n",
    "- **max_concurrency_per_instance**: The maximum number of parallel scoring_script runs per instance.\n",
    "- **mini_batch_size**: The number of files the code_configuration.scoring_script can process in one run() call.\n",
    "- **retry_settings**: Retry settings for scoring each mini batch.\n",
    "- **max_retries**: The maximum number of retries for a failed or timed-out mini batch (default is 3)\n",
    "- **timeout**: The timeout in seconds for scoring a mini batch (default is 30)\n",
    "- **output_action**: Indicates how the output should be organized in the output file. Allowed values are append_row or summary_only. Default is append_row\n",
    "- **output_file_name**: Name of the batch scoring output file. Default is predictions.csv\n",
    "- **environment_variables**: Dictionary of environment variable name-value pairs to set for each batch scoring job.\n",
    "- **logging_level**: The log verbosity level. Allowed values are warning, info, debug. Default is info.\n",
    "\n",
    "\n",
    "For more details about the various arguments of the BatchDeployment method, please refer to the documentation [here](https://docs.microsoft.com/en-us/python/api/azure-ai-ml/azure.ai.ml.entities.batchdeployment?view=azure-python-preview)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a batch deployment\n",
    "deployment_name = \"classification-model\"\n",
    "deployment = BatchDeployment(\n",
    "    name=deployment_name,\n",
    "    description=\"sample batch deployment for fridge objects classification model\",\n",
    "    endpoint_name=batch_endpoint_name,\n",
    "    model=registered_model.id,\n",
    "    code_path=\"./scripts\",\n",
    "    scoring_script=\"batch_scoring.py\",\n",
    "    environment=env,\n",
    "    compute=compute_name,\n",
    "    instance_count=2,\n",
    "    max_concurrency_per_instance=2,\n",
    "    mini_batch_size=10,\n",
    "    environment_variables={\"batch_size\": 16},\n",
    "    # Specify inference batch_size, otherwise uses default value. (This is different from the mini_batch_size above)\n",
    "    # NOTE: Large batch sizes may result in OOM errors.\n",
    "    output_action=BatchDeploymentOutputAction.APPEND_ROW,\n",
    "    output_file_name=\"predictions.csv\",\n",
    "    retry_settings=BatchRetrySettings(max_retries=3, timeout=30),\n",
    "    logging_level=\"info\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4.4 Create the deployment\n",
    "\n",
    "Using the `MLClient` created earlier, we'll now create the deployment in the workspace. This command will start the deployment creation and return a confirmation response while the deployment creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.begin_create_or_update(deployment)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.5 Test the endpoint with sample data\n",
    "\n",
    "Using the MLClient created earlier, we'll get a handle to the endpoint. The endpoint can be invoked using the invoke command with the following parameters:\n",
    "\n",
    "- **endpoint_name** - Name of the endpoint\n",
    "- **input** - Dataset object holding the test dataset\n",
    "- **deployment_name** - Name of the specific deployment to test in an endpoint\n",
    "- **params_override** - Optionally, this parameter could be used to override the deployment configurations. \n",
    "\n",
    "For more details, please refer the documentation [here](https://docs.microsoft.com/en-us/python/api/azure-ai-ml/azure.ai.ml.operations.batchendpointoperations?view=azure-python-preview#azure-ai-ml-operations-batchendpointoperations-invoke)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "registered_data_asset = ml_client.data.get(name=\"fridge-items-images\", label=\"latest\")\n",
    "\n",
    "test_data = Input(type=AssetTypes.URI_FILE, path=registered_data_asset.id)\n",
    "\n",
    "# invoke the endpoint for batch scoring job\n",
    "job = ml_client.batch_endpoints.invoke(\n",
    "    endpoint_name=batch_endpoint_name,\n",
    "    input=test_data,\n",
    "    deployment_name=deployment_name,  # name is required as default deployment is not set\n",
    "    params_override=[{\"mini_batch_size\": \"10\"}, {\"compute.instance_count\": \"1\"}],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the details of the job\n",
    "job_name = job.name\n",
    "batch_job = ml_client.jobs.get(name=job_name)\n",
    "print(batch_job.status)\n",
    "# stream the job logs\n",
    "ml_client.jobs.stream(name=job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.6 Download and review output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_dir = \"./scoring_results\"\n",
    "ml_client.jobs.download(\n",
    "    name=job.name, output_name=deployment.output_file_name, download_path=output_dir\n",
    ")\n",
    "\n",
    "# Print the first five lines of the output\n",
    "result_file = os.path.join(output_dir, deployment.output_file_name)\n",
    "with open(result_file) as f:\n",
    "    for x in range(5):\n",
    "        print(next(f))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.7 Choose a random file for visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import json\n",
    "\n",
    "with open(result_file, \"r\") as f:\n",
    "    contents = f.readlines()\n",
    "rand_file = contents[random.randrange(len(contents))]\n",
    "prediction = json.loads(rand_file)\n",
    "print(prediction[\"filename\"])\n",
    "print(prediction[\"probs\"])\n",
    "print(prediction[\"labels\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download the image file from the datastore\n",
    "file_path_list_on_vm = prediction[\"filename\"].split(\"/\")\n",
    "local_file_path = (\n",
    "    f\"./data/fridgeObjects/{file_path_list_on_vm[-2]}/{file_path_list_on_vm[-1]}\"\n",
    ")\n",
    "local_file_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "\n",
    "IMAGE_SIZE = (18, 12)\n",
    "plt.figure(figsize=IMAGE_SIZE)\n",
    "img_np = mpimg.imread(local_file_path)\n",
    "img = Image.fromarray(img_np.astype(\"uint8\"), \"RGB\")\n",
    "x, y = img.size\n",
    "\n",
    "fig, ax = plt.subplots(1, figsize=(15, 15))\n",
    "# Display the image\n",
    "ax.imshow(img_np)\n",
    "\n",
    "label_index = np.argmax(prediction[\"probs\"])\n",
    "label = prediction[\"labels\"][label_index]\n",
    "conf_score = prediction[\"probs\"][label_index]\n",
    "\n",
    "display_text = \"{} ({})\".format(label, round(conf_score, 3))\n",
    "print(display_text)\n",
    "\n",
    "color = \"red\"\n",
    "plt.text(30, 30, display_text, color=color, fontsize=30)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Delete the endpoint\n",
    "Delete the endpoint to release the resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.batch_endpoints.begin_delete(name=batch_endpoint_name)"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  },
  "kernelspec": {
   "display_name": "Python 3.10 - SDK V2",
   "language": "python",
   "name": "python310-sdkv2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}