Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,104 +2,125 @@
{
"title": "ALTER ROUTINE LOAD",
"language": "en",
"description": "This syntax is used to modify an existing routine load job. Only jobs in PAUSED state can be modified."
"description": "Modifies a paused Routine Load job, including its Kafka target table, job properties, and Kafka source properties."
}
---

## Description

This syntax is used to modify an existing routine load job. Only jobs in PAUSED state can be modified.
This statement modifies an existing Routine Load job. Only jobs in the `PAUSED` state can be modified. You can pause a Routine Load job by using [PAUSE ROUTINE LOAD](./PAUSE-ROUTINE-LOAD.md).

After a successful modification, you can:

- Use [SHOW ROUTINE LOAD](./SHOW-ROUTINE-LOAD.md) to check the modified job details.
- Use [RESUME ROUTINE LOAD](./RESUME-ROUTINE-LOAD.md) to restart the job.

## Syntax

```sql
ALTER ROUTINE LOAD FOR [<db>.]<job_name>
[<job_properties>]
FROM [<data_source>]
[<data_source_properties>]
[SET TARGET TABLE = "<new_table_name>"]
[PROPERTIES (
"<job_property>" = "<value>"
[, ...]
)]
[FROM <data_source> (
"<data_source_property>" = "<value>"
[, ...]
)];
```

## Required Parameters
## Parameters

### 1. `[<db>.]<job_name>`

Specifies the name of the job to modify. An identifier must start with a letter and cannot contain spaces or special characters unless the entire identifier is enclosed in backticks.

An identifier cannot use a reserved keyword. For more information, see [Identifier Requirements](../../../basic-element/object-identifiers.md) and [Reserved Keywords](../../../basic-element/reserved-keywords.md).

**1. `[<db>.]<job_name>`**
### 2. `SET TARGET TABLE = "<new_table_name>"`

> Specifies the name of the job to be modified. The identifier must begin with a letter character and cannot contain spaces or special characters unless the entire identifier string is enclosed in backticks.
>
> The identifier cannot use reserved keywords. For more details, please refer to identifier requirements and reserved keywords.
Specifies the new target table for the load job.

## Optional Parameters
Switching only the target table does not move historical data. Existing batches remain in the old table. After the job resumes, Doris continues consuming from the retained Kafka offsets and writes new batches to the new table.

**1. `<job_properties>`**
The target table must be in the same database as the job and must be a non-temporary OLAP table. Target-table switching supports only single-table Kafka Routine Load jobs.

> Specifies the job parameters to be modified. Currently supported parameters include:
>
> - desired_concurrent_number
> - max_error_number
> - max_batch_interval
> - max_batch_rows
> - max_batch_size
> - jsonpaths
> - json_root
> - strip_outer_array
> - strict_mode
> - timezone
> - num_as_string
> - fuzzy_parse
> - partial_columns
> - max_filter_ratio
### 3. `<job_properties>`

**2. `<data_source_properties>`**
Specifies the job properties to modify. The following properties are currently supported:

> Properties related to the data source. Currently supports:
>
> - `<kafka_partitions>`
> - `<kafka_offsets>`
> - `<kafka_broker_list>`
> - `<kafka_topic>`
> - Custom properties, such as `<property.group.id>`
- `desired_concurrent_number`
- `max_error_number`
- `max_batch_interval`
- `max_batch_rows`
- `max_batch_size`
- `jsonpaths`
- `json_root`
- `strip_outer_array`
- `strict_mode`
- `timezone`
- `num_as_string`
- `fuzzy_parse`
- `partial_columns`
- `max_filter_ratio`

**3. `<data_source>`**
### 4. `<data_source>`

> The type of data source. Currently supports:
>
> - KAFKA
Specifies the data source type. When used together with `SET TARGET TABLE`, currently only the following type is supported:

- `KAFKA`

### 5. `<data_source_properties>`

Specifies the data source properties to modify. The following properties are currently supported:

- `kafka_broker_list`
- `kafka_topic`
- Custom properties, such as `property.group.id`
- `kafka_partitions` and `kafka_offsets`, which modify the offsets of Kafka partitions to be consumed. When `kafka_topic` is not changed, only currently consumed partitions can be modified; new partitions cannot be added.

## Access Control Requirements

Users executing this SQL command must have at least the following privileges:
The user executing this SQL command must have at least the following privileges:

| Privilege | Object | Notes |
| :-------- | :----- | :---- |
| LOAD_PRIV | Table | ALTER ROUTINE LOAD requires LOAD privilege on the table |
| --- | --- | --- |
| `LOAD_PRIV` | Current target table | Required when modifying a single-table Routine Load job. |
| `LOAD_PRIV` | Database | Required when modifying a multi-table Routine Load job. |
| `LOAD_PRIV` | New target table | Additionally required when using `SET TARGET TABLE`. |

## Examples

## Notes
### Modify `desired_concurrent_number` to `1`

- `kafka_partitions` and `kafka_offsets` are used to modify the offset of kafka partitions to be consumed, and can only modify currently consumed partitions. New partitions cannot be added.
```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "1"
);
```

## Examples
### Modify `desired_concurrent_number`, partition offsets, and group ID

```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "10"
)
FROM KAFKA
(
"kafka_partitions" = "0, 1, 2",
"kafka_offsets" = "100, 200, 100",
"property.group.id" = "new_group"
);
```

### Switch the target table to `new_table_name`

- Modify `desired_concurrent_number` to 1

```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "1"
);
```

- Modify `desired_concurrent_number` to 10, modify partition offsets, and modify group id

```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "10"
)
FROM kafka
(
"kafka_partitions" = "0, 1, 2",
"kafka_offsets" = "100, 200, 100",
"property.group.id" = "new_group"
);
```
```sql
ALTER ROUTINE LOAD FOR db1.label1
SET TARGET TABLE = "new_table_name";
```
Original file line number Diff line number Diff line change
Expand Up @@ -2,105 +2,125 @@
{
"title": "ALTER ROUTINE LOAD",
"language": "zh-CN",
"description": "该语法用于修改已经创建的例行导入作业。只能修改处于 PAUSED 状态的作业。"
"description": "修改已暂停的 Routine Load 作业,说明 Kafka 单表作业切换目标表、修改作业属性、更新 Kafka 数据源属性与消费 offset 的语法、权限要求和使用示例。"
}
---

## 描述

该语法用于修改已经创建的例行导入作业。只能修改处于 PAUSED 状态的作业。
该语法用于修改已经创建的 Routine Load 导入作业。只能修改处于 `PAUSED` 状态的作业。您可以通过 [PAUSE ROUTINE LOAD](./PAUSE-ROUTINE-LOAD.md) 暂停 Routine Load 导入作业。

修改成功后,您可以:

- 通过 [SHOW ROUTINE LOAD](./SHOW-ROUTINE-LOAD.md) 检查修改后的作业详情。
- 通过 [RESUME ROUTINE LOAD](./RESUME-ROUTINE-LOAD.md) 重启该导入作业。

## 语法

```sql
ALTER ROUTINE LOAD FOR [<db>.]<job_name>
[<job_properties>]
FROM [<data_source>]
[<data_source_properties>]
[SET TARGET TABLE = "<new_table_name>"]
[PROPERTIES (
"<job_property>" = "<value>"
[, ...]
)]
[FROM <data_source> (
"<data_source_property>" = "<value>"
[, ...]
)];
```

## 必选参数
## 参数说明

### 1. `[<db>.]<job_name>`

指定要修改的作业名称。标识符必须以字母字符开头,并且不能包含空格或特殊字符,除非整个标识符字符串用反引号括起来。

标识符不能使用保留关键字。有关更多详细信息,请参阅[标识符要求](../../../basic-element/object-identifiers.md)和[保留关键字](../../../basic-element/reserved-keywords.md)。

**1. `[<db>.]<job_name>`**
### 2. `SET TARGET TABLE = "<new_table_name>"`

> 指定要修改的作业名称。标识符必须以字母字符开头,并且不能包含空格或特殊字符,除非整个标识符字符串用反引号括起来。
>
> 标识符不能使用保留关键字。有关更多详细信息,请参阅标识符要求和保留关键字。
指定要切换到的导入目标表。

## 可选参数
纯切表不会搬迁历史数据。旧批次仍保留在旧表中;作业恢复后,Doris 从保留的 Kafka offset 继续消费,并把新批次写入新表。

**1. `<job_properties>`**
目标表必须与作业位于同一数据库,且为非临时 OLAP 表。只有单表 Kafka Routine Load 作业支持切换目标表。

> 指定需要修改的作业参数。目前支持修改的参数包括:
>
> - desired_concurrent_number
> - max_error_number
> - max_batch_interval
> - max_batch_rows
> - max_batch_size
> - jsonpaths
> - json_root
> - strip_outer_array
> - strict_mode
> - timezone
> - num_as_string
> - fuzzy_parse
> - partial_columns
> - max_filter_ratio
### 3. `<job_properties>`

**2. `<data_source_properties>`**
指定需要修改的作业参数。目前支持修改的参数包括:

> 数据源的相关属性。目前支持:
>
> - kafka_partitions
> - kafka_offsets
> - kafka_broker_list
> - kafka_topic
> - 自定义 property,如 property.group.id
- `desired_concurrent_number`
- `max_error_number`
- `max_batch_interval`
- `max_batch_rows`
- `max_batch_size`
- `jsonpaths`
- `json_root`
- `strip_outer_array`
- `strict_mode`
- `timezone`
- `num_as_string`
- `fuzzy_parse`
- `partial_columns`
- `max_filter_ratio`

**3. `<data_source>`**
### 4. `<data_source>`

> 数据源的类型。当前支持:
>
> - KAFKA
数据源的类型。与 `SET TARGET TABLE` 组合使用时,当前仅支持:

- `KAFKA`

### 5. `<data_source_properties>`

数据源的相关属性。目前仅支持:

- `kafka_broker_list`
- `kafka_topic`
- 自定义 `property`,如 `property.group.id`
- `kafka_partitions` 和 `kafka_offsets`,用于修改待消费的 Kafka partition 的 offset。未修改 `kafka_topic` 时,仅能修改当前已经消费的 partition,不能新增 partition。

## 权限控制

执行此 SQL 命令的用户必须至少具有以下权限:

| 权限(Privilege) | 对象(Object) | 说明(Notes) |
| :---------------- | :------------- | :---------------------------- |
| LOAD_PRIV | 表(Table) | SHOW ROUTINE LOAD 需要对表有LOAD权限 |
| 权限 | 对象 | 说明 |
| --- | --- | --- |
| `LOAD_PRIV` | 当前目标表 | 修改单表 Routine Load 作业时需要。 |
| `LOAD_PRIV` | 数据库 | 修改 multi-table Routine Load 作业时需要。 |
| `LOAD_PRIV` | 新目标表 | 使用 `SET TARGET TABLE` 时还需要此权限。 |

## 注意事项
## 示例

- `kafka_partitions` 和 `kafka_offsets` 用于修改待消费的 kafka partition 的 offset,仅能修改当前已经消费的 partition。不能新增 partition。
### 将 `desired_concurrent_number` 修改为 `1`

## 示例
```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "1"
);
```

- 将 `desired_concurrent_number` 修改为 1

```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "1"
);
```

- 将 `desired_concurrent_number` 修改为 10,修改 partition 的 offset,修改 group id

```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "10"
)
FROM kafka
(
"kafka_partitions" = "0, 1, 2",
"kafka_offsets" = "100, 200, 100",
"property.group.id" = "new_group"
);
```
### 将 `desired_concurrent_number` 修改为 `10`,并修改 partition offset 和 group ID

```sql
ALTER ROUTINE LOAD FOR db1.label1
PROPERTIES
(
"desired_concurrent_number" = "10"
)
FROM KAFKA
(
"kafka_partitions" = "0, 1, 2",
"kafka_offsets" = "100, 200, 100",
"property.group.id" = "new_group"
);
```

### 将目标表修改为 `new_table_name`

```sql
ALTER ROUTINE LOAD FOR db1.label1
SET TARGET TABLE = "new_table_name";
```
Loading