Sequential steps — serial vs parallel

Background

RNNs are inherently serial along time: computing position $t$ requires position $t-1$ to be done, so a length-$T$ sequence needs $T$ sequential steps. Attention computes all positions in one big parallel operation, so its sequential depth is $1$ — which is exactly why it saturates GPUs.

Problem statement

Implement sequential_steps(seq_len, arch) returning the number of sequential (non-parallelizable) steps to process the sequence.

Input

• seq_len — int, sequence length $T$.
• arch — "rnn" or "attention".

Output

Returns an int: seq_len for "rnn", 1 for "attention".

Examples

Example 1

Input:  seq_len = 100, arch = "rnn"
Output: 100

Example 2

Input:  seq_len = 100, arch = "attention"
Output: 1

Constraints

• "rnn" → seq_len; "attention" → 1.
• Return a plain int.

Notes

• Fewer sequential steps means more work can run in parallel on a GPU — the throughput advantage that let transformers scale.