Hyperparameter Tuning with Internal Tuning

Subclass to conduct only internal hyperparameter tuning for a mlr3::Learner.

Note

The selected mlr3::Measure does not influence the tuning result. To change the loss-function for the internal tuning, consult the hyperparameter documentation of the tuned mlr3::Learner.

Dictionary

This Tuner can be instantiated with the associated sugar function tnr():

tnr("internal")

Progress Bars

$optimize() supports progress bars via the package progressr combined with a bbotk::Terminator. Simply wrap the function in progressr::with_progress() to enable them. We recommend to use package progress as backend; enable with progressr::handlers("progress").

Logging

All Tuners use a logger (as implemented in lgr) from package bbotk. Use lgr::get_logger("bbotk") to access and control the logger.

Resources

There are several sections about hyperparameter optimization in the mlr3book.

• Getting started with hyperparameter optimization.

• An overview of all tuners can be found on our website.

• Tune a support vector machine on the Sonar data set.

• Learn about tuning spaces.

• Estimate the model performance with nested resampling.

• Learn about multi-objective optimization.

• Simultaneously optimize hyperparameters and use early stopping with XGBoost.

• Automate the tuning.

The gallery features a collection of case studies and demos about optimization.

• Learn more advanced methods with the Practical Tuning Series.

• Learn about hotstarting models.

• Run the default hyperparameter configuration of learners as a baseline.

• Use the Hyperband optimizer with different budget parameters.

The cheatsheet summarizes the most important functions of mlr3tuning.

See also

Other Tuner: Tuner, mlr_tuners, mlr_tuners_cmaes, mlr_tuners_design_points, mlr_tuners_gensa, mlr_tuners_grid_search, mlr_tuners_irace, mlr_tuners_nloptr, mlr_tuners_random_search

Super classes

mlr3tuning::Tuner -> mlr3tuning::TunerBatch -> TunerBatchInternal

Methods

Public methods

• TunerBatchInternal$new()

• TunerBatchInternal$clone()

Inherited methods

• mlr3tuning::Tuner$format()
• mlr3tuning::Tuner$help()
• mlr3tuning::Tuner$print()
• mlr3tuning::TunerBatch$optimize()

---

Method new()

Creates a new instance of this R6 class.

Usage

TunerBatchInternal$new()

---

Method clone()

The objects of this class are cloneable with this method.

Usage

TunerBatchInternal$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

library(mlr3learners)

# Retrieve task
task = tsk("pima")

# Load learner and set search space
learner = lrn("classif.xgboost",
  nrounds = to_tune(upper = 1000, internal = TRUE),
  early_stopping_rounds = 10,
  validate = "test",
  eval_metric = "merror"
)

# Internal hyperparameter tuning on the pima indians diabetes data set
instance = tune(
  tnr("internal"),
  tsk("iris"),
  learner,
  rsmp("cv", folds = 3),
  msr("internal_valid_score", minimize = TRUE, select = "merror")
)

# best performing hyperparameter configuration
instance$result_learner_param_vals
#> $eval_metric
#> [1] "merror"
#> 
#> $nrounds
#> [1] 3
#> 
#> $nthread
#> [1] 1
#> 
#> $verbose
#> [1] 0
#> 

instance$result_learner_param_vals$internal_tuned_values
#> NULL