Download a file(s) from an S3 bucket.
This task can operate in two modes:
- Single file mode: when providing only the 'key' parameter, it downloads a specific file from S3.
- Multiple files mode: when using filtering parameters (prefix, delimiter, regexp), it downloads multiple files matching the criteria.
In single file mode, the output contains the properties of a single file (uri, contentLength, etc.). In multiple files mode, the output contains maps that associate each file key with its properties (uris, contentLengths, etc.).
type: "io.kestra.plugin.aws.s3.download"id: aws_s3_download
namespace: company.team
tasks:
- id: download
type: io.kestra.plugin.aws.s3.Download
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
bucket: "my-bucket"
key: "path/to/file"
The S3 bucket name.
Access Key Id in order to connect to AWS.
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
falseIf set to true, the task will use the AWS S3 DefaultAsyncClient instead of the S3CrtAsyncClient, which better integrates with S3-compatible services but restricts uploads and downloads to 2GB.
A character used to group keys
When specified, the task switches to multiple files mode. The API returns all keys that share a common prefix up to the delimiter.
The endpoint with which the SDK should communicate.
This property allows you to use a different S3 compatible storage backend.
The account ID of the expected bucket owner
Requests will fail with a Forbidden error (access denied) if the bucket is owned by a different account.
Force path style access.
Must only be used when compatibilityMode is enabled.
The key of a file to download
When specified without filtering options (prefix, delimiter, regexp), the task will download a single file.
Used for pagination in multiple files mode
This is the key at which a previous listing ended.
1000The maximum number of keys to include in the response in multiple files mode
The prefix of files to download
When specified, the task switches to multiple files mode and downloads all files with keys starting with this prefix.
A regular expression to filter the keys of the objects to download
When specified, the task switches to multiple files mode and only downloads files matching the pattern.
AWS region with which the SDK should communicate.
Sets the value of the RequestPayer property for this object.
Secret Key Id in order to connect to AWS.
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource.
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
The AWS STS endpoint with which the SDKClient should communicate.
AWS STS Role.
The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
AWS STS External Id.
A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.
PT15MdurationAWS STS Session duration.
The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.
AWS STS Session name.
This property is only used when an stsRoleArn is defined.
The specific version of the object
This property is only applicable when downloading a single file with the key parameter.
The size of the file in bytes
The MIME type of the file
An ETag is an opaque identifier assigned by a web server to a specific version of a resource found at a URL.
The metadata of the file
uriThe URI of the downloaded file in Kestra's storage
The version ID of the file