sdk/jobs/automl-standalone-jobs/automl-nlp-text-classification-multiclass-task-sentiment-analysis/automl-nlp-text-classification-multiclass-task-sentiment-mlflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AutoML - Sentiment Analysis scenario: Train \"the best\" NLP Text Classification Multi-class model for a 'Sentiment Labeled Sentences' dataset.\n",
    "\n",
    "**Requirements** - In order to benefit from this tutorial, you will need:\n",
    "- A basic understanding of Machine Learning\n",
    "- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
    "- A Compute Cluster. [Check this notebook to create a compute cluster](../../../resources/compute/compute.ipynb)\n",
    "- A python environment\n",
    "- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../../README.md) - check the getting started section"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Connect to Azure Machine Learning Workspace\n",
    "\n",
    "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the job will be run.\n",
    "\n",
    "## 1.1. Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852261599
    },
    "name": "automl-import"
   },
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "from azure.identity import DefaultAzureCredential\n",
    "from azure.ai.ml import automl, Input, MLClient\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "\n",
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2. Configure workspace details and get a handle to the workspace\n",
    "\n",
    "To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852261744
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "mlclient-setup",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "credential = DefaultAzureCredential()\n",
    "ml_client = None\n",
    "try:\n",
    "    ml_client = MLClient.from_config(credential)\n",
    "except Exception as ex:\n",
    "    print(ex)\n",
    "    # Enter details of your AML workspace\n",
    "    subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "    resource_group = \"<RESOURCE_GROUP>\"\n",
    "    workspace = \"<AML_WORKSPACE_NAME>\"\n",
    "    ml_client = MLClient(credential, subscription_id, resource_group, workspace)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show Azure ML Workspace information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "workspace = ml_client.workspaces.get(name=ml_client.workspace_name)\n",
    "\n",
    "output = {}\n",
    "output[\"Workspace\"] = ml_client.workspace_name\n",
    "output[\"Subscription ID\"] = ml_client.connections._subscription_id\n",
    "output[\"Resource Group\"] = workspace.resource_group\n",
    "output[\"Location\"] = workspace.location\n",
    "output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Data\n",
    "\n",
    "**NOTE:** In this PRIVATE PREVIEW we're defining the MLTable in a separate folder and .YAML file.\n",
    "In later versions, you'll be able to do it all in Python APIs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib\n",
    "from zipfile import ZipFile\n",
    "\n",
    "# Destination folders\n",
    "training_mltable_path = \"./training-mltable-folder/\"\n",
    "validation_mltable_path = \"./validation-mltable-folder/\"\n",
    "test_mltable_path = \"./test-mltable-folder/\"\n",
    "\n",
    "if not os.path.exists(test_mltable_path):\n",
    "    os.mkdir(test_mltable_path)\n",
    "\n",
    "# Download dataset files and copy within each MLTable folder\n",
    "\n",
    "training_download_url = \"https://raw.githubusercontent.com/dotnet/spark/main/examples/Microsoft.Spark.CSharp.Examples/MachineLearning/Sentiment/Resources/yelptrain.csv\"\n",
    "training_data_file = training_mltable_path + \"yelp_training_set.csv\"\n",
    "urllib.request.urlretrieve(training_download_url, filename=training_data_file)\n",
    "\n",
    "valid_download_url = \"https://raw.githubusercontent.com/dotnet/spark/main/examples/Microsoft.Spark.CSharp.Examples/MachineLearning/Sentiment/Resources/yelptest.csv\"\n",
    "valid_data_file = validation_mltable_path + \"yelp_validation_set.csv\"\n",
    "urllib.request.urlretrieve(valid_download_url, filename=valid_data_file)\n",
    "\n",
    "test_download_url = \"https://raw.githubusercontent.com/dotnet/spark/main/examples/Microsoft.Spark.CSharp.Examples/MachineLearning/Sentiment/Resources/yelptest.csv\"\n",
    "test_data_file = test_mltable_path + \"yelp_test_set.csv\"\n",
    "urllib.request.urlretrieve(test_download_url, filename=test_data_file)\n",
    "\n",
    "print(\"Dataset files downloaded.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "data-load"
   },
   "outputs": [],
   "source": [
    "# Training MLTable defined locally, with local data to be uploaded\n",
    "my_training_data_input = Input(type=AssetTypes.MLTABLE, path=training_mltable_path)\n",
    "\n",
    "# Validation MLTable defined locally, with local data to be uploaded\n",
    "my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=validation_mltable_path)\n",
    "\n",
    "# WITH REMOTE PATH: If available already in the cloud/workspace-blob-store\n",
    "# my_training_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml://datastores/workspaceblobstore/paths/my_training_mltable\")\n",
    "# my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml://datastores/workspaceblobstore/paths/my_validation_mltable\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Configure and run the AutoML NLP Text Classification Multiclass training job\n",
    "In this section we will configure and run the AutoML job, for training the model.    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# general job parameters\n",
    "compute_name = \"gpu-cluster\"\n",
    "exp_name = \"dpv2-nlp-text-classification-experiment\"\n",
    "dataset_language_code = \"eng\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852262026
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "text-classification-configuration",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# Create the AutoML job with the related factory-function.\n",
    "\n",
    "text_classification_job = automl.text_classification(\n",
    "    compute=compute_name,\n",
    "    # name=\"dpv2-nlp-text-classification-multiclass-job-01\",\n",
    "    experiment_name=exp_name,\n",
    "    training_data=my_training_data_input,\n",
    "    validation_data=my_validation_data_input,\n",
    "    target_column_name=\"Sentiment\",\n",
    "    primary_metric=\"accuracy\",\n",
    "    tags={\"my_custom_tag\": \"My custom value\"},\n",
    ")\n",
    "\n",
    "text_classification_job.set_limits(timeout_minutes=120)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "language-setting"
   },
   "outputs": [],
   "source": [
    "text_classification_job.set_featurization(dataset_language=dataset_language_code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Run the Command\n",
    "Using the `MLClient` created earlier, we will now run this Command as a job in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1634852267930
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "job-submit",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# Submit the AutoML job\n",
    "\n",
    "returned_job = ml_client.jobs.create_or_update(\n",
    "    text_classification_job\n",
    ")  # submit the job to the backend\n",
    "\n",
    "print(f\"Created job: {returned_job}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.stream(returned_job.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Retrieve the Best Trial (Best Model's trial/run)\n",
    "Use the MLFLowClient to access the results (such as Models, Artifacts, Metrics) of a previously completed AutoML Trial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize MLFlow Client\n",
    "The models and artifacts that are produced by AutoML can be accessed via the MLFlow interface. \n",
    "Initialize the MLFlow client here, and set the backend as Azure ML, via. the MLFlow Client.\n",
    "\n",
    "*IMPORTANT*, you need to have installed the latest MLFlow packages with:\n",
    "\n",
    "    pip install azureml-mlflow\n",
    "\n",
    "    pip install mlflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Obtain the tracking URI for MLFlow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlflow\n",
    "\n",
    "# Obtain the tracking URL from MLClient\n",
    "MLFLOW_TRACKING_URI = ml_client.workspaces.get(\n",
    "    name=ml_client.workspace_name\n",
    ").mlflow_tracking_uri\n",
    "\n",
    "print(MLFLOW_TRACKING_URI)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the MLFLOW TRACKING URI\n",
    "mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)\n",
    "print(\"\\nCurrent tracking uri: {}\".format(mlflow.get_tracking_uri()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlflow.tracking.client import MlflowClient\n",
    "\n",
    "# Initialize MLFlow client\n",
    "mlflow_client = MlflowClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the AutoML parent Job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "job_name = returned_job.name\n",
    "\n",
    "# Example if providing an specific Job name/ID\n",
    "# job_name = \"b4e95546-0aa1-448e-9ad6-002e3207b4fc\"\n",
    "\n",
    "# Get the parent run\n",
    "mlflow_parent_run = mlflow_client.get_run(job_name)\n",
    "\n",
    "print(\"Parent Run: \")\n",
    "print(mlflow_parent_run)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the AutoML best child run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the best model's child run\n",
    "best_child_run_id = mlflow_parent_run.data.tags[\"automl_best_child_run_id\"]\n",
    "print(\"Found best child run id: \", best_child_run_id)\n",
    "\n",
    "best_run = mlflow_client.get_run(best_child_run_id)\n",
    "\n",
    "print(\"Best child run: \")\n",
    "print(best_run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "best_run.data.metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download the best model locally\n",
    "\n",
    "Access the results (such as Models, Artifacts, Metrics) of a previously completed AutoML Run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create local folder\n",
    "local_dir = \"./artifact_downloads\"\n",
    "if not os.path.exists(local_dir):\n",
    "    os.mkdir(local_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download run's artifacts/outputs\n",
    "local_path = mlflow_client.download_artifacts(\n",
    "    best_run.info.run_id, \"outputs\", local_dir\n",
    ")\n",
    "print(\"Artifacts downloaded in: {}\".format(local_path))\n",
    "print(\"Artifacts: {}\".format(os.listdir(local_path)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show the contents of the MLFlow model folder\n",
    "os.listdir(\"./artifact_downloads/outputs/mlflow-model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next Steps\n",
    "You can see further examples of other AutoML tasks such as Regression, Image-Object-Detection, NLP-Text-Classification, Time-Series-Forcasting, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python3-azureml"
  },
  "kernelspec": {
   "display_name": "Python 3.10 - SDK V2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "microsoft": {
   "host": {
    "AzureML": {
     "notebookHasBeenCompleted": true
    }
   }
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}