Quiz

# ============================================================
# Q1) from_logits=True vs False — which loss + last layer combo?
# ============================================================
# ANSWER:
#   loss = SparseCategoricalCrossentropy(from_logits=True)
#   last layer = Dense(N)       (no softmax)
# OR
#   loss = SparseCategoricalCrossentropy(from_logits=False)
#   last layer = Dense(N, activation='softmax')
# Mixing them (softmax + from_logits=True) silently degrades training.

# ============================================================
# Q2) Training loss decreasing but val loss flat — what to do?
# ============================================================
# ANSWER:
# - Add regularisation (dropout, weight decay)
# - Reduce learning rate (ReduceLROnPlateau)
# - Add more data or augmentation
# - Early stop with restore_best_weights=True
# - Check leakage: val set must be representative AND disjoint from train

# ============================================================
# Q3) Mixed precision — when to enable?
# ============================================================
# ANSWER: on any GPU with TensorCores (V100, A100, H100, RTX 30+ series).
# tf.keras.mixed_precision.set_global_policy('mixed_float16')
# Halves memory, ~1.5-2x throughput, ~no accuracy change on most tasks.

# ============================================================
# Q4) GPU at 30% utilisation — bottleneck?
# ============================================================
# ANSWER: usually data pipeline.
# - Add .cache().prefetch(tf.data.AUTOTUNE) to the dataset
# - Increase num_parallel_calls on .map(...)
# - Use TFRecord on disk instead of decoded images
# - Profile via TensorBoard 'profile' tab

# ============================================================
# Q5) Distributed training — single GPU vs MultiGPU vs MultiHost?
# ============================================================
# ANSWER:
# - 1 GPU             plain tf.keras
# - 1 host, N GPUs    MirroredStrategy (1 line)
# - N hosts           MultiWorkerMirroredStrategy + TF_CONFIG
# - TPU pod           TPUStrategy
# Scale incrementally; do not chase MultiHost until 1 host saturates.

# ============================================================
# Q6) Saving a model — .keras vs SavedModel vs .h5?
# ============================================================
# ANSWER:
# - .keras       single-file Keras 3 default. Use for shipping in TF.
# - SavedModel   directory format, for TF Serving / TF Hub.
# - .h5          legacy. Avoid for new code.
# Convert to TFLite for mobile, ONNX for interop, TF.js for browser.

# ============================================================
# Bonus — why is your custom loss returning NaN at epoch 1?
# ============================================================
# ANSWER: log of 0 or division by tiny number. Add epsilon:
#   loss = -tf.reduce_mean(y_true * tf.math.log(y_pred + 1e-7))
# Also check inputs aren't all-zero after preprocessing.

# ============================================================
# Scoring
# ============================================================
# 6 / 6 -> ready for production TF
# 4 / 6 -> bookmark tensorflow/cheatsheet
# < 4   -> revisit the official Keras tutorial

# 3 questions per lesson.