Wal Fetch Guide: Restore PostgreSQL WAL Files from Cloud Storage

Wal fetch is a critical command in the WAL-G toolkit used to retrieve Write-Ahead Log (WAL) files from remote storage—such as Amazon S3, Google Cloud Storage, or Azure Blob—during PostgreSQL database recovery. When configured correctly in postgresql.conf as the restore_command, wal fetch enables seamless point-in-time recovery (PITR) and continuous archiving for production databases.

The standard restore_command for PostgreSQL using wal-g wal-fetch looks like: restore_command = 'wal-g wal-fetch "%f" "%p"'. Here, %f is the WAL filename and %p is the destination path where PostgreSQL expects the file to land.

WAL-G wal-fetch supports prefetching of WAL files into a /.wal-g/prefetch directory, significantly speeding up recovery by downloading upcoming WAL segments ahead of time. This is especially useful during long restore operations over S3.

Error handling is built-in: wal-fetch returns exit code 74 (EX_IOERR) when a WAL file is not found, which signals the end of WAL restoration in PostgreSQL rather than triggering a fatal error. This allows smooth recovery completion.

Security is supported through GPG/PGP encryption. Set WALG_PGP_KEY or WALG_PGP_KEY_PATH in your environment to enable automatic decryption during wal-fetch operations.

Performance tuning is possible with the WALG_DOWNLOAD_CONCURRENCY environment variable, which controls how many goroutines fetch WAL files in parallel. By default, WAL-G uses the minimum of the number of files to extract and 10.

“Wal fetch is the backbone of continuous archiving—get it right and your database recovery becomes fast, reliable, and fully automated.”