Boto3 Retries: Why Your AWS Scripts Fail Silently

Originally published on DevToolHub.

Boto3's default retry mode is legacy, not standard — and legacy mode only retries a narrow list of errors: ConnectionError, Throttling, ThrottlingException, and ProvisionedThroughputExceededException. If your script fails on a transient RequestTimeout or SlowDown error that standard mode would have retried automatically, that's why.

boto3 Retry Modes: Legacy, Standard, and Adaptive

Legacy (the default) retries 5 attempts against a short error list with exponential backoff, base factor 2.

Standard retries 3 attempts by default, adds circuit-breaking, and covers a longer error list — RequestTimeout, TooManyRequestsException, SlowDown, plus HTTP 500/502/503/504. Backoff caps at 20 seconds.

Adaptive builds on standard and adds client-side rate limiting via a token bucket. AWS marks it experimental.

Setting the Retry Mode: The Syntax That Trips People Up

In code, with a Config object:

from botocore.config import Config
import boto3

config = Config(
    retries={
        "total_max_attempts": 10,
        "mode": "standard",
    }
)

client = boto3.client("ec2", config=config)

In ~/.aws/config, different key names:

[myConfigProfile]
region = us-east-1
max_attempts = 10
retry_mode = standard

Mix the two formats' keys and boto3 won't error — it just ignores what it doesn't recognize.

Catching ClientError the Right Way

import botocore.exceptions

try:
    client.describe_stream(StreamName="myDataStream")
except botocore.exceptions.ClientError as error:
    error_code = error.response["Error"]["Code"]

⚠️ Note: standard mode already retries throttling errors. A manual time.sleep() loop on top is usually redundant.

Checking Whether a Retry Actually Happened

response = client.some_operation()
print(response["ResponseMetadata"]["RetryAttempts"])

Why Third-Party Retry Wrappers Exist at All

Packages like botobackoff and awsretry exist because plenty of scripts were written against the legacy default and never revisited.

Frequently Asked Questions

Q: What's the default boto3 retry mode? A: legacy.

Q: Does standard mode retry throttling errors automatically? A: Yes, several throttling exceptions are retried with backoff.

Q: What's the difference between total_max_attempts and max_attempts? A: Same concept, different key name depending on where you set it.

Q: Should I use adaptive mode in production? A: It's experimental; standard mode is the stable choice.

Q: How do I check if a boto3 call actually retried? A: Read response["ResponseMetadata"]["RetryAttempts"].

Quick Summary:

• Default retry mode is legacy, not standard
• Config uses mode/total_max_attempts; the AWS config file uses retry_mode/max_attempts
• Standard mode already retries throttling — a manual loop on top is usually redundant
• Catch botocore.exceptions.ClientError, read error.response["Error"]["Code"]
• Check ResponseMetadata["RetryAttempts"] to confirm retries happen