iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Exercises

By · Reviewed & last updated · Part of TensorFlow Tutorial

Five TensorFlow drills that exercise the gotchas: dataset pipelines, loss/last-layer pairing, callbacks, mixed precision, and SavedModel. Try first.

Five TF exercises

EXAMPLE
# ============================================================
# Drill 1 — Fix the GPU-starvation loader
# ============================================================
# GIVEN:
# ds = tf.data.Dataset.from_tensor_slices((X, y)).batch(64)
# GPU sits at 35% during training. Add three knobs.
#
# ANSWER:
ds = (tf.data.Dataset.from_tensor_slices((X, y))
        .shuffle(10_000)
        .batch(64)
        .cache()                                   # cache decoded items in memory
        .prefetch(tf.data.AUTOTUNE))               # overlap producer/consumer
# If you read from disk, .map with num_parallel_calls=tf.data.AUTOTUNE too.

# ============================================================
# Drill 2 — Loss + last-layer pairing
# ============================================================
# GIVEN:
# model = tf.keras.Sequential([..., tf.keras.layers.Dense(10, activation='softmax')])
# model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True))
# Accuracy is suspiciously low. Why?
#
# ANSWER: from_logits=True expects RAW logits. With activation='softmax' you
# softmax twice. Either:
# - Drop the activation, keep from_logits=True
# - Keep softmax, change to from_logits=False

# ============================================================
# Drill 3 — Add EarlyStopping + checkpoint
# ============================================================
# Save best weights, restore at end, stop when val_loss stops improving.
#
# ANSWER:
cbs = [
    tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3,
                                       restore_best_weights=True),
    tf.keras.callbacks.ModelCheckpoint('best.keras', monitor='val_loss',
                                         save_best_only=True),
    tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=2),
    tf.keras.callbacks.TensorBoard('logs/run-' + datetime.now().strftime('%Y%m%d-%H%M%S')),
]
model.fit(ds_train, validation_data=ds_val, epochs=50, callbacks=cbs)

# ============================================================
# Drill 4 — Mixed precision on an A100
# ============================================================
# ANSWER:
tf.keras.mixed_precision.set_global_policy('mixed_float16')
# Final Dense should output float32 for numerical stability:
# tf.keras.layers.Dense(num_classes, dtype='float32', activation='softmax')

# ============================================================
# Drill 5 — Save + load + serve a SavedModel
# ============================================================
# Train -> save -> reload -> run inference
#
# ANSWER:
model.save('saved_model_dir')                      # SavedModel directory
loaded = tf.keras.models.load_model('saved_model_dir')
preds = loaded.predict(X_test)

# For TF Serving:
# docker run -p 8501:8501 -v $(pwd)/saved_model_dir:/models/my/1 \
#   -e MODEL_NAME=my tensorflow/serving
# curl -X POST -d '{"instances": [...]}' http://localhost:8501/v1/models/my:predict

# ============================================================
# Bonus — train deterministically
# ============================================================
# ANSWER:
tf.keras.utils.set_random_seed(42)
tf.config.experimental.enable_op_determinism()
# Cost: ~10-30% slower. Use only when reproducibility matters more than speed.

# ============================================================
# Scoring
# ============================================================
# 5 / 5 -> production-ready TF
# 3 / 5 -> bookmark the cheatsheet
# < 3   -> read the Keras Functional API tutorial

Why it matters

`.cache().prefetch(tf.data.AUTOTUNE)` is the duo that fixes "GPU is idle 60% of the time" reports without changing the model. Add `.map(..., num_parallel_calls=tf.data.AUTOTUNE)` when decoding from disk, and the input pipeline stops being the bottleneck — your GPU does the work it was bought for.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
# Fill in: model.____(Xtr, ytr, epochs=10)
Try it Yourself »

Discussion

Loading…