The S3 module supplies typed tasks for buckets and objects. All task types live in io.github.gregwhitaker.awsemulator.aws.s3 and use the AWS Emulator - S3 task group. Each task starts the selected emulator on demand and waits until it is ready.
Register Resource Tasks
Make your test task depend on provisioning, then opt that same consumer into the emulator. Provisioning tasks default to the shared managed instance.
Kotlin DSL
import io.github.gregwhitaker.awsemulator.aws.s3.CreateS3BucketTask
plugins {
java
id("io.github.gregwhitaker.aws-emulator") version "0.3.0"
}
val createBucket = tasks.register<CreateS3BucketTask>("createTestBucket") {
bucketName.set("application-test-data")
}
val tests = tasks.named<Test>("test") {
dependsOn(createBucket)
}
awsEmulator {
useWith(tests)
}
Groovy DSL
import io.github.gregwhitaker.awsemulator.aws.s3.CreateS3BucketTask
plugins {
id 'java'
id 'io.github.gregwhitaker.aws-emulator' version '0.3.0'
}
def createBucket = tasks.register('createTestBucket', CreateS3BucketTask) {
bucketName.set('application-test-data')
}
def tests = tasks.named('test') {
dependsOn(createBucket)
}
awsEmulator {
useWith(tests)
}
|
Note
|
A managed provisioning task invoked alone creates resources that disappear when the build finishes. To keep resources available for a development session, select DEVELOPMENT mode.
|
Task Properties
All properties use Gradle’s lazy property API.
| Task type | Required | Optional |
|---|---|---|
|
|
|
|
|
|
|
None |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Shared property | Default | Meaning |
|---|---|---|
|
|
Select the build-owned runtime or the persistent |
|
|
Select a declared named runtime. Its name and lifecycle mode must match the consumer; see named runtimes. |
|
|
Positive timeout for each SDK operation, including retries. |
Provisioning tasks always execute and are never restored from the build cache because they operate on live emulator state. The plugin supplies its own SDK dependencies for these tasks; the application declares its own dependencies.
S3 Bucket Behavior
CreateS3BucketTask and CreateS3BucketsTask create general-purpose buckets in awsEmulator.region. For us-east-1, it omits the location constraint; for other regions, it sends the configured region.
If an accessible bucket already exists, the task reuses it without another CreateBucket call. Its objects and configuration remain intact. A region mismatch reported by HeadBucket fails the task.
These creation tasks do not configure directory buckets or access points. Use the configuration tasks below for policies, versioning, and notifications, and the object tasks to seed bucket contents.
API: CreateS3BucketTask.
Bulk Creation
import io.github.gregwhitaker.awsemulator.aws.s3.CreateS3BucketsTask
val createBuckets = tasks.register<CreateS3BucketsTask>("createTestBuckets") {
buckets.set(listOf("application-data", "application-uploads"))
}
buckets is a ListProperty<String>. Bulk creation uses the singular task’s region and reuse checks. All names are validated before starting the emulator. Duplicate names are processed once.
API: CreateS3BucketsTask.
Bucket Policies And Versioning
Configure an existing bucket after its creation task. Configuration tasks do not create a missing bucket. They use the selected runtime and mode, always execute against live state, and close their SDK clients after use.
import io.github.gregwhitaker.awsemulator.aws.s3.*
import software.amazon.awssdk.services.s3.model.BucketVersioningStatus
import io.github.gregwhitaker.awsemulator.model.EmulatorMode
val policy = tasks.register<PutS3BucketPolicyTask>("configureBucketPolicy") {
dependsOn(createBucket)
bucketName.set(createBucket.flatMap { it.bucketName })
policyFile.set(layout.projectDirectory.file("fixtures/bucket-policy.json"))
}
val versioning = tasks.register<PutS3BucketVersioningTask>("versionBucket") {
dependsOn(createBucket)
bucketName.set(createBucket.flatMap { it.bucketName })
status.set(BucketVersioningStatus.ENABLED)
}
awsEmulator { useWith(tasks.named<Test>("test"), EmulatorMode.MANAGED, policy, versioning) }
PutS3BucketPolicyTask accepts exactly one of inline policy (Property<String>) or policyFile (RegularFileProperty). Files are UTF-8 and read during execution, including on configuration-cache reuse. Both forms accept a JSON object of at most 20 KiB. Malformed JSON, duplicate keys, and trailing JSON documents fail before connecting; diagnostics omit policy values. Policy semantics are validated by the emulator.
The policy operation replaces the entire policy; it does not merge statements. For example, fixtures/bucket-policy.json can grant object reads:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::000000000000:root"},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::application-test-data/*"
}]
}
Match the resource ARN to your bucket and the principal to your account. This task preserves S3 Block Public Access settings; public policies can be rejected by the emulator. DeleteS3BucketPolicyTask removes only the policy and succeeds when the existing bucket has no policy. A missing bucket or service error fails the task.
PutS3BucketVersioningTask.status is a required Property<BucketVersioningStatus> accepting ENABLED or SUSPENDED, without a default. Suspension retains existing versions; it does not return a bucket to its original unversioned state. The task leaves MFA Delete unspecified and exposes no MFA settings. See versioning behavior.
Register configuration tasks explicitly; the module registers only the policy-deletion convenience command. The registered tasks above also accept command-line overrides:
./gradlew configureBucketPolicy --policyFile=fixtures/bucket-policy.json
./gradlew versionBucket --status=SUSPENDED
./gradlew deleteS3BucketPolicy --bucketName=application-test-data
The first two commands use their configured mode, initially MANAGED. For persistent configuration, select DEVELOPMENT for both the configuration task and its bucket-creation prerequisites; setting awsEmulator.defaultMode on the build covers custom tasks together. The deletion convenience command defaults to DEVELOPMENT.
Bucket Notifications
PutS3BucketNotificationConfigurationTask.notificationConfiguration is a required Property<NotificationConfiguration> using the AWS SDK v2 model. It accepts queue, topic, Lambda function, and EventBridge configurations, including SDK prefix/suffix filters. There is no default configuration.
For a standard SQS destination:
import software.amazon.awssdk.services.s3.model.Event
import software.amazon.awssdk.services.s3.model.NotificationConfiguration
import software.amazon.awssdk.services.s3.model.QueueConfiguration
import io.github.gregwhitaker.awsemulator.aws.s3.PutS3BucketNotificationConfigurationTask
import io.github.gregwhitaker.awsemulator.model.EmulatorMode
val notifications = tasks.register<PutS3BucketNotificationConfigurationTask>("notifyUploads") {
dependsOn(createBucket, createNotificationQueue)
bucketName.set(createBucket.flatMap { it.bucketName })
notificationConfiguration.set(NotificationConfiguration.builder().queueConfigurations(
QueueConfiguration.builder().id("uploads")
.queueArn("arn:aws:sqs:us-east-1:000000000000:application-upload-events")
.events(Event.S3_OBJECT_CREATED).build()).build())
}
awsEmulator { useWith(tasks.named<Test>("test"), EmulatorMode.MANAGED, notifications) }
createNotificationQueue must provision that queue with an SQS resource policy allowing s3.amazonaws.com to perform sqs:SendMessage, restricted to the source bucket ARN. Match destination ARNs to the actual account and region. The basic example provides the full queue policy and verifies delivery with the pinned emulator’s account 000000000000. SNS destinations similarly need publishing permission, and Lambda destinations need invocation permission. This task does not create destinations, modify their policies, or bypass destination validation. Direct SQS and SNS FIFO destinations are rejected; use standard destinations.
The notification operation replaces the entire configuration, including destinations not mentioned in the new value. To remove all notifications, explicitly supply NotificationConfiguration.builder().build(). To enable EventBridge delivery, include EventBridgeConfiguration.builder().build() through the SDK builder’s eventBridgeConfiguration(…) method. Do not register competing configuration tasks for the same bucket unless their execution order is explicit.
Each destination requires an ARN for its service and at least one recognized S3 event. Supplied IDs must be nonblank and unique across destinations. Filters allow at most one prefix and one suffix, each with a value. Structural validation occurs before connecting; destination permissions, filter overlap, and other service constraints are checked by the emulator. Docker integration tests verify SQS object-created delivery; availability and fidelity of other destination workflows depend on the emulator release.
The three PutS3Bucket* task types are setup tasks and participate in resource sets and Spring Boot automatic setup discovery. Deletion tasks are excluded from automatic setup. Groovy DSL uses the same SDK builders and properties with tasks.register('notifyUploads', PutS3BucketNotificationConfigurationTask) { … }.
Upload And Inspect Objects
Object tasks require an existing bucket. Use the bucket task from the registration example as a dependency:
import io.github.gregwhitaker.awsemulator.aws.s3.UploadS3DirectoryTask
val uploadAssets = tasks.register<UploadS3DirectoryTask>("uploadTestAssets") {
dependsOn(createBucket)
bucketName.set(createBucket.flatMap { it.bucketName })
sourceDirectory.set(layout.projectDirectory.dir("fixtures/assets"))
prefix.set("assets")
}
tasks.named<Test>("test") {
dependsOn(uploadAssets)
}
UploadS3DirectoryTask preserves relative paths under the configured prefix. UploadS3ObjectTask uploads one file to an explicit object key. Uploads replace matching keys and preserve other objects; they do not synchronize deletions.
ListS3ObjectsTask follows all response pages and prints object keys and sizes. DownloadS3ObjectTask writes to an explicit outputFile, replacing it only after the full download succeeds. Object tasks execute again on subsequent builds, including configuration-cache reuse.
See S3 data fixtures for content types, metadata, directory handling, and development datasets.
ExportS3ObjectsTask captures current object bytes, keys, content types, and user metadata in a portable JSON fixture; ReplayS3ObjectsTask writes that fixture to an explicit existing bucket. Export requires an already running runtime and never starts an owned emulator. See export and replay for the format, size limit, and exclusions.
Module Convenience Tasks
The S3 module registers these tasks in the AWS Emulator - S3 group. They use the persistent development emulator by default:
| Task | CLI options |
|---|---|
|
None required |
|
|
|
|
|
|
|
|
Every resource task accepts --mode=MANAGED or --mode=DEVELOPMENT and --runtime <name>. Custom instances inherit awsEmulator.defaultMode, initially MANAGED. Ensure provisioning and consuming tasks select the same runtime name and mode.
./gradlew listS3Buckets
./gradlew listS3Objects --bucketName=application-data --prefix=assets/
./gradlew purgeS3Buckets --buckets=application-data
./gradlew deleteS3Buckets --buckets=application-data --force
Listing And Cleanup
S3 listing follows all response pages and prints bucket names and creation dates. Purging retains the bucket but removes objects, historical versions, and delete markers, including when versioning is suspended. The task checks individual deletion errors even when the service accepts the batch. Deletion requires an empty bucket unless force is true; an already absent bucket succeeds. Purging and deleting require explicit bucket names.