The `ArchiveAsyncTuning“ stores all evaluated hyperparameter configurations and performance scores in a rush::Rush database.
Details
The ArchiveAsyncTuning is a connector to a rush::Rush database.
Data Structure
The table ($data) has the following columns:
One column for each hyperparameter of the search space (
$search_space).One (list-)column for the
internal_tuned_valuesOne column for each performance measure (
$codomain).x_domain(list())
Lists of (transformed) hyperparameter values that are passed to the learner.runtime_learners(numeric(1))
Sum of training and predict times logged in learners per mlr3::ResampleResult / evaluation. This does not include potential overhead time.timestamp(POSIXct)
Time stamp when the evaluation was logged into the archive.batch_nr(integer(1))
Hyperparameters are evaluated in batches. Each batch has a unique batch number.
Analysis
For analyzing the tuning results, it is recommended to pass the ArchiveAsyncTuning to as.data.table().
The returned data table contains the mlr3::ResampleResult for each hyperparameter evaluation.
S3 Methods
as.data.table.ArchiveTuning(x, unnest = "x_domain", exclude_columns = "uhash", measures = NULL)
Returns a tabular view of all evaluated hyperparameter configurations.
ArchiveAsyncTuning ->data.table::data.table()x(ArchiveAsyncTuning)unnest(character())
Transforms list columns to separate columns. Set toNULLif no column should be unnested.exclude_columns(character())
Exclude columns from table. Set toNULLif no column should be excluded.measures(List of mlr3::Measure)
Score hyperparameter configurations on additional measures.
Super classes
bbotk::Archive -> bbotk::ArchiveAsync -> ArchiveAsyncTuning
Active bindings
internal_search_space(paradox::ParamSet)
The search space containing those parameters that are internally optimized by themlr3::Learner.benchmark_result(mlr3::BenchmarkResult)
Benchmark result.
Methods
Inherited methods
bbotk::Archive$format()bbotk::Archive$help()bbotk::ArchiveAsync$best()bbotk::ArchiveAsync$clear()bbotk::ArchiveAsync$data_with_state()bbotk::ArchiveAsync$nds_selection()bbotk::ArchiveAsync$pop_point()bbotk::ArchiveAsync$push_failed_point()bbotk::ArchiveAsync$push_points()bbotk::ArchiveAsync$push_result()bbotk::ArchiveAsync$push_running_point()
Method new()
Creates a new instance of this R6 class.
Usage
ArchiveAsyncTuning$new(
search_space,
codomain,
rush,
internal_search_space = NULL
)Arguments
search_space(paradox::ParamSet)
Hyperparameter search space. IfNULL(default), the search space is constructed from the paradox::TuneToken of the learner's parameter set (learner$param_set).codomain(bbotk::Codomain)
Specifies codomain of objective function i.e. a set of performance measures. Internally created from provided mlr3::Measures.rush(
Rush)
If a rush instance is supplied, the tuning runs without batches.internal_search_space(paradox::ParamSet or
NULL)
The internal search space.check_values(
logical(1))
IfTRUE(default), hyperparameter configurations are check for validity.
Method learner()
Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Learner does not contain a model. Use $learners() to get learners with models.
Method learners()
Retrieve list of trained mlr3::Learner objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Method learner_param_vals()
Retrieve param values of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Method predictions()
Retrieve list of mlr3::Prediction objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Method resample_result()
Retrieve mlr3::ResampleResult of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.