Pretraining objective per family

Background

Each family is pretrained with a different self-supervised objective:

• encoder-only → masked language modeling (predict masked tokens from both sides).
• decoder-only → next-token prediction (predict each token from the left).
• encoder-decoder → text-to-text (map an input sequence to an output sequence).

Problem statement

Implement pretraining_objective(family) returning the objective string.

Input

• family — "encoder-only", "decoder-only", or "encoder-decoder".

Output

Returns one of:

• "masked-language-modeling",
• "next-token-prediction",
• "text-to-text".

Examples

Example 1

Input:  family = "decoder-only"
Output: "next-token-prediction"

Example 2

Input:  family = "encoder-only"
Output: "masked-language-modeling"

Constraints

• Map each family to its exact objective string above.

Notes

• Next-token prediction's appeal: it puts a learning signal on every token, scales to any text, and reframes most tasks as "continue the sequence" — a big reason decoder-only won.