文档

批处理框架

概述

MinIO 批处理框架允许您使用 YAML 格式的作业定义文件(“批处理文件”)创建、管理、监控和执行作业。批处理作业直接在 MinIO 部署上运行,以利用服务器端的处理能力,而无需您运行 MinIO 客户端 的本地机器的限制。

批处理文件定义一个作业任务。

启动后,MinIO 开始处理作业。完成时间取决于部署可用的资源。

如果作业的任何部分失败,MinIO 会根据作业定义中定义的次数重试作业。

MinIO 批处理框架支持以下作业类型

作业类型

描述

replicate

执行一次性复制过程,将一个 MinIO 位置复制到另一个 MinIO 位置。

keyrotate

执行一次性过程,循环对象上的 sse-s3 或 sse-kms 密码密钥。

expire

执行一次性立即过期操作,以过期存储桶中的对象。

MinIO 批处理 CLI

mc batch 命令包括

mc batch generate

mc batch generate 命令为指定的作业类型创建一个基本的 YAML 格式的模板文件。

mc batch start

mc batch start 命令从作业批处理 YAML 文件启动批处理作业。

mc batch list

The mc batch list command outputs a list of the batch jobs currently in progress on a deployment.

mc batch status

The mc batch status command outputs summaries of job events on a MinIO server.

Changed in version mc: RELEASE.2024-07-03T20-17-25Z

Batch status displays summaries for active, in-progress jobs or any batch job completed in the previous three (3) days.

mc batch describe

The mc batch describe command outputs the job definition for a specified job ID.

mc batch cancel

The mc batch cancel stops an ongoing batch job.

Access to mc batch

Each batch job executes using the credentials specified in the batch definition. The success of a given batch job depends on those credentials having the appropriate permissions to perform all requested actions.

The user executing the batch job must have the following permissions. You can alternatively restrict users from accessing these functions by blocking or limiting access to these actions

admin:ListBatchJobs

Grants the user the ability to see batch jobs currently in process.

admin:DescribeBatchJobs

Grants the user the ability to see the definition details of batch job currently in process.

admin:StartBatchJob

Grants the user the ability to start a batch job. The job may be further restricted by the credentials the job uses to access either the source or target deployments.

admin:CancelBatchJob

Allows the user to stop a batch job currently in progress.

You can assign any of these actions to users independently or in any combination.

The built-in ConsoleAdmin policy includes sufficient access to perform all of these types of batch job actions.

Local Deployment

You run a batch job against a particular deployment by passing an alias to the mc batch command. The deployment you specify in the command becomes the local deployment within the context of that batch job.