Extract inner tuning results of nested resampling. Implemented for mlr3::ResampleResult and mlr3::BenchmarkResult. The function iterates over the AutoTuner objects and binds the tuning results to a data.table::data.table(). AutoTuner must be initialized with store_tuning_instance = TRUE and resample() or benchmark() must be called with store_models = TRUE.

extract_inner_tuning_results(x)

Arguments

x

(mlr3::ResampleResult | mlr3::BenchmarkResult).

Value

data.table::data.table().

Data structure

The returned data table has the following columns:

  • experiment (integer(1))
    Index, giving the according row number in the original benchmark grid.

  • iteration (integer(1))
    Iteration of the outer resampling.

  • One column for each hyperparameter of the search spaces.

  • One column for each performance measure.

  • learner_param_vals (list())
    Hyperparameter values used by the learner. Includes fixed and proposed hyperparameter values.

  • x_domain (list())
    List of transformed hyperparameter values.

  • task_id (character(1)).

  • learner_id (character(1)).

  • resampling_id (character(1)).

Examples

learner = lrn("classif.rpart", cp = to_tune(1e-04, 1e-1, logscale = TRUE)) at = auto_tuner( method = "grid_search", learner = learner, resampling = rsmp ("holdout"), measure = msr("classif.ce"), term_evals = 4) resampling_outer = rsmp("cv", folds = 2) rr = resample(tsk("iris"), at, resampling_outer, store_models = TRUE) extract_inner_tuning_results(rr)
#> iteration cp classif.ce learner_param_vals x_domain task_id #> 1: 1 -8.442812 0.12 <list[2]> <list[1]> iris #> 2: 2 -7.675284 0.00 <list[2]> <list[1]> iris #> learner_id resampling_id #> 1: classif.rpart.tuned cv #> 2: classif.rpart.tuned cv