Parameter or hyperparameter?

Background

Two kinds of "knob" exist in a neural net, and the distinction matters:

• Trainable parameters — the weights and biases (W1, b1, W2, b2, …). Backprop computes $\partial L/\partial\theta$ for each and gradient descent updates them.
• Hyperparameters — settings you fix outside the gradient loop: learning rate, batch size, epochs, depth, width, activation, dropout, weight decay, the loss choice. They are not updated by $\partial L/\partial\theta$ on the network.

Problem statement

Implement is_hyperparameter(name) returning True if name is a hyperparameter, False if it is a trainable parameter.

Input

• name — a string drawn from this fixed vocabulary:
  • Hyperparameters: "learning_rate", "batch_size", "epochs", "num_layers", "depth", "hidden_size", "width", "activation", "dropout", "weight_decay", "lambda", "loss".
  • Trainable parameters: "W1", "b1", "W2", "b2", "weight", "bias", "weights", "biases".

Output

Returns a bool.

Examples

Example 1

Input:  name = "learning_rate"
Output: True

Example 2

Input:  name = "W1"
Output: False

Constraints

• Return True only for the hyperparameter names listed above.
• Trainable-parameter names (and the weight/bias family) return False.

Notes

• The rule of thumb: if backprop's gradient descent moves it on the training loss, it's a parameter; if you choose it before/between runs, it's a hyperparameter.