Experiment Docs

Class Experiment

Main class to create, manage, and search experimental results.

Parameters:

• name - (string) name of experiment.
• params - (dict) dicitionary of parameter names to their random sampling functions.
• directory - (string) directory in which the experiment will be saved. Default: randopt_results

Return type: n/a

Example:

e = ro.Experiment('exp_name', {
    'batch_size' : ro.Uniform(low=5.0, high=150.0, dtype='int'),
    'iterations': ro.Normal(mean=1000.0, std=150.0, dtype='int'),
    'learning_rate' : ro.Uniform(low=0.0001, high=0.01, dtype='float'),
})

add_result

def add_result(self, result, data=None, attachment=None)

Generates a randomly sampled value for all specified parameters

Parameters:

• result - (float) value for the current set of hyperparameters.
• data - (dict) additional logging data.
• attachment - (dict) attachment data excluded from JSON summary.

Return type: n/a

Example:

e.add_result(loss)

all

def all(self)

Alias for Experiment.all_results()

Example:

e.all()

all_results

def all_results(self)

Iterates through all previous results in no specific order

Parameters: n/a

Return type: iterator

Example:

for res in e.all_results():
    print(res.result)
    print(res.params)

count

def count(self)

Returns the number of JSON summaries.

Parameters: n/a

Return type: int

Example:

e.count()

list

def list(self)

Returns a SummaryList of all results.

Parameters: n/a

Return type: SummaryList

Example:

summaries = exp.list()
summaries.filter(lambda x: x.result > 0.1)

maximum

def maximum(self)

Returns the maximum result from saved results.

Parameters: n/a

Return type: float

Example:

e.maximum()

minimum

def minimum(self)

Returns the minimum result from saved results.

Parameters: n/a

Return type: float

Example:

e.minimum()

sample

def sample(self, key)

Generates, sets, and returns a randomly sampled value for given parameter.

Parameters:

• key - (string) name of randomly sampled parameter

Return type: float/int

Example:

e.sample('iterations')

sample_all_params

def sample_all_params(self)

Generates a randomly sampled value for all specified parameters.

Parameters: n/a

Return type: dict of parameters and values.

Example:

e.sample_all_params()

save_state

def save_state(self, path)

Saves the state of the random variables into a file.

Parameters:

• path - (string) target filepath

Return type: n/a

Example:

e.save_state(states/curr_state.pk)

seed

def seed(self, seed)

Manually set a seed value.

Parameters:

• seed - (int) random seed.

Return type: n/a

Example:

e.seed(1234)

set

def set(self, key, value)

set_state

def set_state(self, path)

Sets the state of random variables from a file

Parameters:

• path - (string) target filepath

Return type: n/a

Example:

e.set_state(states/curr_state.pk)

top

def top(self, count, fn=<function <lambda> at 0x7fde8af338c8>)

Returns the top count best results. By default, minimum.

Parameters:

• count - (int) number of results to return.
• fn - (function) comparison function. Default: leq

Return type: dict of parameters

Example:

e.top(3)