Create clusters in Google Cloud Dataproc.
yaml
type: "io.kestra.plugin.gcp.dataproc.clusters.Create"Creates a cluster in Google Cloud Dataproc.
yaml
id: gcp_dataproc_cluster_create
namespace: company.team
tasks:
- id: cluster_create
type: io.kestra.plugin.gcp.dataproc.clusters.Create
clusterName: YOUR_CLUSTER_NAME
region: YOUR_REGION
zone: YOUR_ZONE
masterMachineType: n1-standard-2
workerMachineType: n1-standard-2
workers: 2
bucket: YOUR_BUCKET_NAME
Creates a cluster in Google Cloud Dataproc with specific disk size.
yaml
id: gcp_dataproc_cluster_create
namespace: company.team
tasks:
- id: create_cluster_with_certain_disk_size
type: io.kestra.plugin.gcp.dataproc.clusters.Create
clusterName: YOUR_CLUSTER_NAME
region: YOUR_REGION
zone: YOUR_ZONE
masterMachineType: n1-standard-2
masterDiskSizeGB: 500
workerMachineType: n1-standard-2
workerDiskSizeGB: 200
workers: 2
bucket: YOUR_BUCKET_NAM
Min length
1The cluster name.
Min length
1The region.
The GCS bucket name.
The Dataproc image URI.
The Compute Engine image resource used for cluster instances.
The GCP service account to impersonate.
The disk size in GB for each master node.
The master machine type.
The GCP project ID.
SubType string
Default
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used.
The GCP service account.
The disk size in GB for each worker node.
The worker machine type.
The number of workers.
The zone.
The cluster name.
Default
falseWhether cluster was created successfully.