1. Allow to change shielded config defaults for 2.1+ images
2. Support batches filtering in list API
3. Support Trino jobs on 2.1+ image clusters
4. Support batch TTL
5. Support custom staging bucket for batches
6. Expose approximate and current batches resources usage
7. Support Hudi and Trino components
PiperOrigin-RevId: 511550277
This includes the following:
1. The new start and stop cluster methods.
2. The ability to specify a metastore config in a cluster.
3. The ability to specify a (BETA) GKE cluster when creating a Dataproc cluster.
4. The ability to configure the behavior for private IPv6 cluster networking.
5. The ability to specify node affinity groups for clusters.
6. The ability to specify shielded VM configurations for clusters.
7. Support for service-account based secure multi-tenancy.
8. The ability to specify cluster labels for picking which cluster should run a job.
9. Components for DOCKER, DRUID, FLINK, HBASE, RANGER, and SOLR
10. The ability to specify a DAG timeout for workflows.
Committer: @ojarjur
PiperOrigin-RevId: 367518225
This change includes the following updates:
1. There is a new `temp_bucket` field for clusters.
2. There is a new `endpoint_config` field for clusters.
3. There is a new `preemptibility` field for instance group configs.
4. There are various updates to the doc comments.
PiperOrigin-RevId: 323829608
