Tuning Hyperparameters with Hyperdrive in Azure Machine Learning
Description
In the world of data science, model parameters are the elements generated from training a dataset. In contrast, a hyperparameter is a parameter used to control the outcome of training the model. Machine learning models are deep and complex and require several hyperparameters to build the best model. There is no logical formula for obtaining the best hyperparameter values. These values must be tweaked and analyzed. Like tuning a musical instrument, data scientists must tune their training models to achieve the best possible outcome.
Hyperparameter optimization can become a tedious task of tweaking values and re-running experiments. Hyperdrive is a Python package that automates this process in Azure Machine Learning. Deploying experiments with Hyperdrive dramatically reduces the process of manually tweaking the hyperparameters used for each experiment.
In this lab, you will dive into Azure Notebooks and launch a Jupyter notebook to create a Hyperdrive experiment and perform hyperparameter tuning against a regression training model.
Learning Objectives
Upon completion of this lab you will be able to:
- Use Hyperdrive to tune hyperparameters
- Find a model that has optimal hyperparameter values
- Create and run Azure Notebooks
- Manage a workspace using the Azure Machine Learning SDK
Intended Audience
This lab is intended for:
- Individuals studying to take the Azure DP-100 exam
- Anyone interested in learning how to use the Azure Machine Learning SDK
Lab Prerequisites
You should be familiar with:
- Basic concepts of Azure Machine Learning
- Experience with Python is not required but preferred
Updates
September 11th, 2023 - Updated the instructions and screenshots to reflect the latest UI
March 31st, 2022 - Updated the lab's notebook to the latest kernel and Azure ML library versions
March 1st, 2021 - Updated screenshots to match the latest Portal experience and expanded upon the definition of hyperparameters in the lab notebook