betty.engine

Unlike traditional automatic differentiation techniques that calculate analytic Jacobian for each operation, multilevel optimization requires approximating best-response Jacobian for each level optimization problem. Below is the list of approximation techniques that are supported by Betty.

Engine

class betty.engine.Engine(problems, config=None, dependencies=None, env=None)

Engine handles a dataflow graph based on the user-provided hierarchical problem dependencies. It also provides a primitive for executing multilevel optimization.

check_leaf(problem)

Check whether the given problem is a leaf problem or not.

Parameters:

problem (Problem) – Problem in multilevel optimization

Returns:

True or False

Return type:

bool

configure_systems()

Configure basic systems set-up like distributed training and device placement.

do_validation()

Check whether to run validation.

eval()

Set all problems in multilevel optimization to the eval mode.

find_paths(src, dst)

Find all paths from src to dst with a modified depth-first search algorithm.

Parameters:

• src (Problem) – The end point of the upper-to-lower edge.

• dst (Problem) – The start point of the upper-to-lower edge.

Returns:

List of all paths from src to dst.

initialize()

Initialize dependencies (computational graph) between problems.

is_implemented(fn_name)

Check whether fn_name method is implemented in the class.

Parameters:

fn_name (str) – class method name

Return type:

bool

is_rank_zero()

Check whether the current process is rank 0

parse_config()

Parse EngineConfig.

parse_dependency()

Parse user-provided u2l and l2u dependencies to figure out 1) topological order for multilevel optimization execution, and 2) backpropagation path(s) for each problem. A modified depth-first search algorithm is used.

run()

Execute multilevel optimization by running gradient descent for leaf problems.

set_dependency(dependencies)

set_problem_attr(problem)

Set class attribute for the given problem based on their names

Parameters:

problem (Problem) – Problem in multilevel optimization

Returns:

problem name

Return type:

str

train()

Set all problems in multilevel optimization to the train mode.

train_step()

Running one-step gradient descent for all leaf problems.