Audio upload and compression database flow

Last updated: 10 September 2026

1. Result

A new audio file is uploaded directly to the shared evequality bucket, queued through a durable MySQL outbox, compressed by the audio Lambda with FFmpeg, verified, and atomically published.

The final compressed audio key is stored in media_versions.object_key. media_assets.active_version_id selects the live version. Publication also writes the full CDN URL to the legacy audio.audio column.

Normal audio uses Lambda. If the source is larger than the configured Lambda limit or the Lambda process times out, the same job is safely rerouted to the scale-to-zero EC2 heavy worker.

This describes the implemented repository state, not proof that the Terraform resources are already deployed.

2. Main AWS and database flow

3. Database identity

Each legacy audio row has two independent media assets:

For an audio content row with audio.id = 45, the stable identity is:

media_assets.external_code contains audio.audioCode. Replacements create new source revisions and versions under the same content asset.

4. Stage-by-stage database changes

Stage A — upload authorization

The admin UI posts asset ID, filename, size, MIME and checksum contract to POST /admin/api/media/uploads.

Allowed source extensions are mp3, m4a, wav, flac and ogg. Laravel checks the administrator's audios.update permission and inserts:

Example temporary key:

Files at or above the default 100 MiB multipart threshold upload in parts. The browser uploads bytes directly to S3; Laravel supplies signed requests but does not proxy the file.

Stage B — confirmation and job creation

After upload, the browser confirms the upload. Laravel checks S3 byte count, content type, checksum and real signature. One transaction then changes or inserts:

The job is idempotent for asset + source revision + profile, so repeated confirmation does not create duplicate jobs.

Stage C — outbox and audio SQS

The one-minute outbox dispatcher changes:

The audio queue is a Standard encrypted SQS queue with 660-second visibility, 20-second long polling, four-day message retention and redrive to a 14-day DLQ after three receives. The event-source mapping invokes only the audio Lambda, with batch size one and partial-batch failure reporting.

Stage D — Lambda lease and processing attempt

The audio Lambda is an ARM64 container configured with:

Before conversion, the Lambda claims a durable lease:

The pipeline requires local free space of at least three times the expected source size, with a 100 MB minimum.

Stage E — FFmpeg compression and verification

The processor:

1. Downloads the exact confirmed S3 object/version.
2. Uses FFprobe to require a positive source duration.
3. Runs FFmpeg with AAC at 128 kbit/s, removes video streams and creates fast-start M4A.
4. Uses FFprobe to require AAC and compare output duration to source within one second or one percent.
5. Fully decodes the output to a null sink to detect corruption.
6. Uploads the result as audio.m4a.

An example final key is:

For later replacements, the directory can include versions/r{revision}. The exact hierarchy is derived from legacy board, class, subject, service, unit and chapter records.

The candidate verifier checks that the S3 object exists, has a valid M4A/AAC signature, has positive duration and reports the AAC codec.

Stage F — atomic publication

Publication is allowed only when the lease, generation, source revision and asset state are still authoritative.

Example:

The modern table deliberately stores a relative S3 key. The legacy column receives a full CDN URL for compatibility.

Important media_versions fields

5. Audio job status timeline

If upload confirmation fails its S3 or signature checks, media_uploads changes from uploaded back to authorized, clears confirmation_started_at, and may be confirmed again before expires_at.

6. Heavy-worker fallback

Fallback is automatic and keeps the same media_jobs.id:

1. If media_uploads.expected_bytes exceeds the default 250 MiB Lambda threshold, or an audio process times out, the Lambda attempt becomes failed with error_code = heavy_fallback.
2. media_jobs.processor changes from audio_lambda to heavy_worker.
3. media_jobs.generation increases, status returns to pending, retry budget is extended and fallback_reason records source_size_limit or lambda_timeout.
4. A new outbox_messages row targets the video/heavy queue.
5. The conversion ASG starts an EC2 worker if necessary.
6. The EC2 worker runs the same AudioProcessor with the heavy-media timeout, then publishes normally.

Old Lambda/SQS messages contain the old generation and therefore cannot publish after rerouting.

7. Table ownership summary

8. Failure, DLQ and cleanup

• A retryable processing error marks the attempt failed, records a safe error code/context and returns the job to dispatched with exponential delay.
• The Lambda response marks retryable SQS records as failed; successful or stale-generation records are acknowledged by omission.
• The job becomes failed when its attempt budget is exhausted. SQS moves a message to the audio DLQ after three receives.
• Candidate output uploaded by a failed attempt is deleted best-effort.
• Expired processing leases are marked lease_expired by the one-minute recovery command.
• A manual retry increases the job generation and creates a fresh outbox row.
• Rollback re-verifies a previously published version before changing active_version_id and the legacy URL.
• The temporary incoming/audio/... source is deleted only after the exact revision is verified, published, active and at least the configured retention age, default 24 hours.
• Source cleanup records source_delete_started_at, source_deleted_at or source_delete_error_code in media_uploads.

9. Audio thumbnail

The audio image is not produced by FFmpeg. It is a separate media_assets row with media_type = image and role = thumbnail. The administrator uploads it through the image and thumbnail upload flow. Successful image publication updates audio.audioThumbnail.