文档

批处理框架

概述

MinIO 批处理框架允许您使用 YAML 格式的作业定义文件(“批处理文件”)创建、管理、监控和执行作业。批处理作业直接在 MinIO 部署上运行,以利用服务器端处理能力,而无需您运行 MinIO 客户端 的本地机器的限制。

批处理文件定义一个作业任务。

启动后,MinIO 开始处理作业。完成时间取决于部署可用的资源。

如果作业的任何部分失败,MinIO 会重试作业,直到达到作业定义中定义的次数。

MinIO 批处理框架支持以下作业类型

作业类型

描述

复制

执行从一个 MinIO 位置到另一个 MinIO 位置的一次性复制过程。

密钥轮换

执行一次性过程,以循环对象上的 sse-s3 或 sse-kms 加密密钥。

过期

执行存储桶中对象的立即过期的一次性过程。

MinIO 批处理 CLI

The mc batch commands include

mc batch generate

The mc batch generate command creates a basic YAML-formatted template file for the specified job type.

mc batch start

The mc batch start command launches a batch job from a job batch YAML file.

mc batch list

The mc batch list command outputs a list of the batch jobs currently in progress on a deployment.

mc batch status

The mc batch status command outputs summaries of job events on a MinIO server.

Changed in version mc: RELEASE.2024-07-03T20-17-25Z

Batch status displays summaries for active, in-progress jobs or any batch job completed in the previous three (3) days.

mc batch describe

The mc batch describe command outputs the job definition for a specified job ID.

mc batch cancel

The mc batch cancel stops an ongoing batch job.

Access to mc batch

Each batch job executes using the credentials specified in the batch definition. The success of a given batch job depends on those credentials having the appropriate permissions to perform all requested actions.

The user executing the batch job must have the following permissions. You can alternatively restrict users from accessing these functions by blocking or limiting access to these actions

admin:ListBatchJobs

Grants the user the ability to see batch jobs currently in process.

admin:DescribeBatchJobs

Grants the user the ability to see the definition details of batch job currently in process.

admin:StartBatchJob

Grants the user the ability to start a batch job. The job may be further restricted by the credentials the job uses to access either the source or target deployments.

admin:CancelBatchJob

Allows the user to stop a batch job currently in progress.

You can assign any of these actions to users independently or in any combination.

The built-in ConsoleAdmin policy includes sufficient access to perform all of these types of batch job actions.

Local Deployment

You run a batch job against a particular deployment by passing an alias to the mc batch command. The deployment you specify in the command becomes the local deployment within the context of that batch job.