Image and thumbnail upload database flow

Last updated: 10 September 2026

1. Result

Images are uploaded directly to evequality/incoming/..., processed by an image Lambda with Sharp, written back to the shared bucket as responsive WebP plus a compatibility JPEG or PNG, verified and atomically published.

The primary compressed image key is stored in media_versions.object_key. media_assets.active_version_id selects it. For legacy video, audio and PDF thumbnails, publication also writes a full CDN URL to the relevant legacy column.

Video HLS processing does not extract a thumbnail automatically. The video file and its thumbnail are two separate assets, uploads, jobs and versions.

Normal images use Lambda. Oversized or timed-out work is safely rerouted to the scale-to-zero EC2 heavy worker, which runs the same Sharp processor.

This document describes the code and Terraform currently implemented in the repository; deployment and production validation are separate steps.

2. Supported image roles

The stable identity is always legacy table + legacy ID + role. A video content asset and video thumbnail asset can therefore advance independently without overwriting each other's active version.

3. Main AWS and database flow

4. Stage-by-stage database changes

Stage A — authorize image upload

The administrator selects the exact image asset and uploads jpg, jpeg, png or webp. The application checks the legacy permission corresponding to that asset and inserts:

Example temporary object:

The browser sends bytes directly to S3 through signed requests. Laravel is control-plane only.

Stage B — confirm and create image job

Laravel verifies S3 metadata, exact byte count, checksum and actual image signature before it accepts the upload.

The asset/revision/profile idempotency key prevents duplicate jobs when confirmation is repeated.

Stage C — outbox and image queue

The scheduled dispatcher performs:

The image queue uses SQS-managed encryption, 20-second long polling, 180-second visibility, four-day message retention and redrive after three receives to a DLQ retained for 14 days.

Its Lambda event-source mapping has batch size one and partial-batch failure reporting.

Stage D — image Lambda lease

The ARM64 image Lambda has:

Claiming work changes:

At least three times the source size, with a 100 MB minimum, must be free in the temporary work directory.

Stage E — Sharp optimization

The processor auto-rotates the image, never enlarges it, respects the 60-million-pixel guard and generates:

• WebP quality 82 at configured widths below the source width, plus the source width capped at 1,600 px;
• one compatibility image at the largest allowed width;
• PNG compatibility when transparency exists, otherwise mozjpeg JPEG quality 85.

Generic image filenames are:

The largest generated WebP becomes media_versions.object_key. There must be a primary WebP and at least two total variants.

For a video thumbnail, the same data is named for legacy compatibility and placed beside the video's HLS files:

The largest WebP is exactly thumbnail.webp. Audio and PDF thumbnails use their EVE-compatible audio/... or pdf/... catalogue prefix with the generic w*.webp names. A book cover, whose legacy row is not handled by the catalogue path builder, uses media/image/{asset_ulid}/....

For replacement versions, versions/r{revision} is added when that asset already has a version.

The verifier checks every stored variant key, non-empty object size, allowed WebP/JPEG/PNG signature, valid width/height and a minimum of two variants.

Stage F — atomic publication

After verification, a transaction rejects stale generation/revision/lease state, cancellation or a blocked asset, and then commits:

For example:

The final modern key is relative to the shared S3 bucket. The compatibility legacy field is a full CDN URL.

For bookCollection/cover, the current transaction still publishes media_versions and changes media_assets.active_version_id, but it does not replace bookCollection.book_image. Any migration from the legacy BLOB to modern cover delivery needs an explicit reader/compatibility change before the BLOB can be retired.

Important media_versions fields

5. Database relationship for thumbnails

Prisma model names differ in capitalization, but the physical MySQL tables are video, audio, pdfSolutions, media_assets, media_uploads, media_jobs, media_job_attempts, outbox_messages, media_versions and publication_audits.

6. Image status timeline

When confirmation verification fails, Laravel clears confirmation_started_at and returns the upload to authorized, allowing another confirmation attempt before expiry.

7. Heavy-worker fallback

When the source exceeds the default 25 MiB Lambda limit or the Sharp command times out:

The conversion ASG starts from zero to one EC2 worker. That worker claims a new attempt and invokes the same ImageProcessor/Sharp script with the long heavy-media timeout. The application EC2 never performs image compression.

8. Failures, retry, rollback and cleanup

• Decode warnings, pixel-limit violations, missing variants, invalid signatures/dimensions, storage failures and process failures become attempt error codes.
• Retryable failures change media_jobs back to dispatched with exponential delay; exhausted jobs become failed.
• The SQS message is retried and ultimately redriven to the media-specific DLQ after three receives.
• Uploaded candidate variants from a failed attempt are deleted best-effort.
• Expired database leases are recovered every minute.
• Manual retry creates a new generation and outbox row; stale messages cannot publish.
• Rollback revalidates the complete stored variant inventory, changes the active version, refreshes the compatible legacy URL where supported and writes publication_audits.
• The hourly source purger removes only the matching incoming/image/... source after the active version is verified and published for the default 24-hour retention period.
• Purge claim, success and failure are recorded in media_uploads.source_delete_started_at, source_deleted_at and source_delete_error_code.

9. Final table ownership