Skip to contents

Extract Inner Tuning Results

Source: R/extract_inner_tuning_results.R
extract_inner_tuning_results.Rd

Extract inner tuning results of nested resampling. Implemented for mlr3::ResampleResult and mlr3::BenchmarkResult.

Usage

extract_inner_tuning_results(x, tuning_instance, ...)

# S3 method for ResampleResult
extract_inner_tuning_results(x, tuning_instance = FALSE, ...)

# S3 method for BenchmarkResult
extract_inner_tuning_results(x, tuning_instance = FALSE, ...)

Arguments

x

(mlr3::ResampleResult | mlr3::BenchmarkResult).

tuning_instance

(logical(1))
If TRUE, tuning instances are added to the table.

...

(any)
Additional arguments.

Value

data.table::data.table().

Details

The function iterates over the AutoTuner objects and binds the tuning results to a data.table::data.table(). The AutoTuner must be initialized with store_tuning_instance = TRUE and mlr3::resample() or mlr3::benchmark() must be called with store_models = TRUE. Optionally, the tuning instance can be added for each iteration.

Data structure

The returned data table has the following columns:

  • experiment (integer(1))
    Index, giving the according row number in the original benchmark grid.

  • iteration (integer(1))
    Iteration of the outer resampling.

  • One column for each hyperparameter of the search spaces.

  • One column for each performance measure.

  • learner_param_vals (list())
    Hyperparameter values used by the learner. Includes fixed and proposed hyperparameter values.

  • x_domain (list())
    List of transformed hyperparameter values.

  • tuning_instance (TuningInstanceSingleCrit | TuningInstanceMultiCrit)
    Optionally, tuning instances.

  • task_id (character(1)).

  • learner_id (character(1)).

  • resampling_id (character(1)).

Examples

# Nested Resampling on Palmer Penguins Data Set

learner = lrn("classif.rpart",
  cp = to_tune(1e-04, 1e-1, logscale = TRUE))

# create auto tuner
at = auto_tuner(
  tuner = tnr("random_search"),
  learner = learner,
  resampling = rsmp ("holdout"),
  measure = msr("classif.ce"),
  term_evals = 4)

resampling_outer = rsmp("cv", folds = 2)
rr = resample(tsk("iris"), at, resampling_outer, store_models = TRUE)

# extract inner results
extract_inner_tuning_results(rr)
#>    iteration        cp classif.ce learner_param_vals  x_domain task_id
#>        <int>     <num>      <num>             <list>    <list>  <char>
#> 1:         1 -8.418866       0.08          <list[2]> <list[1]>    iris
#> 2:         2 -9.058200       0.00          <list[2]> <list[1]>    iris
#>             learner_id resampling_id
#>                 <char>        <char>
#> 1: classif.rpart.tuned            cv
#> 2: classif.rpart.tuned            cv