Connector Admin CLI
The pulsar-admin
tool helps you manage Pulsar connectors.
sources
β
An interface for managing Pulsar IO sources (ingress data into Pulsar).
$ pulsar-admin sources subcommands
Subcommands are:
-
create
-
update
-
delete
-
get
-
status
-
list
-
stop
-
start
-
restart
-
localrun
-
available-sources
-
reload
create
β
Submit a Pulsar IO source connector to run in a Pulsar cluster.
Usageβ
$ pulsar-admin sources create options
Optionsβ
Flag | Description |
---|---|
-a , --archive | The path to the NAR archive for the source. It also supports url-path (http/https/file [file protocol assumes that file already exists on worker host]) from which worker can download the package. |
--classname | The source's class name if archive is file-url-path (file://). |
--cpu | The CPU (in cores) that needs to be allocated per source instance (applicable only to Docker runtime). |
--deserialization-classname | The SerDe classname for the source. |
--destination-topic-name | The Pulsar topic to which data is sent. |
--disk | The disk (in bytes) that needs to be allocated per source instance (applicable only to Docker runtime). |
--name | The source's name. |
--namespace | The source's namespace. |
--parallelism | The source's parallelism factor, that is, the number of source instances to run. |
--processing-guarantees | The processing guarantees (aka delivery semantics) applied to the source. Possible Values: ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE. |
--ram | The RAM (in bytes) that needs to be allocated per source instance (applicable only to the process and Docker runtimes). |
-st , --schema-type | The schema type. Either a builtin schema (for example, AVRO and JSON) or custom schema class name to be used to encode messages emitted from source. |
--source-config | Source config key/values. The source-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--source-config-file | The path to a YAML config file specifying the source's configuration. |
-t , --source-type | The source's connector provider. The source-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--tenant | The source's tenant. |
update
β
Update a already submitted Pulsar IO source connector.
Usageβ
$ pulsar-admin sources update options
Optionsβ
Flag | Description |
---|---|
-a , --archive | The path to the NAR archive for the source. It also supports url-path (http/https/file [file protocol assumes that file already exists on worker host]) from which worker can download the package. |
--classname | The source's class name if archive is file-url-path (file://). |
--cpu | The CPU (in cores) that needs to be allocated per source instance (applicable only to Docker runtime). |
--deserialization-classname | The SerDe classname for the source. |
--destination-topic-name | The Pulsar topic to which data is sent. |
--disk | The disk (in bytes) that needs to be allocated per source instance (applicable only to Docker runtime). |
--name | The source's name. |
--namespace | The source's namespace. |
--parallelism | The source's parallelism factor, that is, the number of source instances to run. |
--processing-guarantees | The processing guarantees (aka delivery semantics) applied to the source. Possible Values: ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE. |
--ram | The RAM (in bytes) that needs to be allocated per source instance (applicable only to the process and Docker runtimes). |
-st , --schema-type | The schema type. Either a builtin schema (for example, AVRO and JSON) or custom schema class name to be used to encode messages emitted from source. |
--source-config | Source config key/values. |
--source-config-file | The path to a YAML config file specifying the source's configuration. |
-t , --source-type | The source's connector provider. The source-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--tenant | The source's tenant. |
--update-auth-data | Whether or not to update the auth data. Default value: false. |
delete
β
Delete a Pulsar IO source connector.
Usageβ
$ pulsar-admin sources delete options
Optionβ
Flag | Description |
---|---|
--name | The source's name. |
--namespace | The source's namespace. |
--tenant | The source's tenant. |
get
β
Get the information about a Pulsar IO source connector.
Usageβ
$ pulsar-admin sources get options
Optionsβ
Flag | Description |
---|---|
--name | The source's name. |
--namespace | The source's namespace. |
--tenant | The source's tenant. |
status
β
Check the current status of a Pulsar Source.
Usageβ
$ pulsar-admin sources status options
Optionsβ
Flag | Description |
---|---|
--instance-id | The source ID. If instance-id is not provided, Pulsar gets status of all instances. |
--name | The source's name. |
--namespace | The source's namespace. |
--tenant | The source's tenant. |
list
β
List all running Pulsar IO source connectors.
Usageβ
$ pulsar-admin sources list options
Optionsβ
Flag | Description |
---|---|
--namespace | The source's namespace. |
--tenant | The source's tenant. |
stop
β
Stop a source instance.
Usageβ
$ pulsar-admin sources stop options
Optionsβ
Flag | Description |
---|---|
--instance-id | The source instanceID. If instance-id is not provided, Pulsar stops all instances. |
--name | The source's name. |
--namespace | The source's namespace. |
--tenant | The source's tenant. |
start
β
Start a source instance.
Usageβ
$ pulsar-admin sources start options
Optionsβ
Flag | Description |
---|---|
--instance-id | The source instanceID. If instance-id is not provided, Pulsar starts all instances. |
--name | The source's name. |
--namespace | The source's namespace. |
--tenant | The source's tenant. |
restart
β
Restart a source instance.
Usageβ
$ pulsar-admin sources restart options
Optionsβ
Flag | Description |
---|---|
--instance-id | The source instanceID. If instance-id is not provided, Pulsar restarts all instances. |
--name | The source's name. |
--namespace | The source's namespace. |
--tenant | The source's tenant. |
localrun
β
Run a Pulsar IO source connector locally rather than deploying it to the Pulsar cluster.
Usageβ
$ pulsar-admin sources localrun options
Optionsβ
Flag | Description |
---|---|
-a , --archive | The path to the NAR archive for the Source. It also supports url-path (http/https/file [file protocol assumes that file already exists on worker host]) from which worker can download the package. |
--broker-service-url | The URL for the Pulsar broker. |
--classname | The source's class name if archive is file-url-path (file://). |
--client-auth-params | Client authentication parameter. |
--client-auth-plugin | Client authentication plugin using which function-process can connect to broker. |
--cpu | The CPU (in cores) that needs to be allocated per source instance (applicable only to the Docker runtime). |
--deserialization-classname | The SerDe classname for the source. |
--destination-topic-name | The Pulsar topic to which data is sent. |
--disk | The disk (in bytes) that needs to be allocated per source instance (applicable only to the Docker runtime). |
--hostname-verification-enabled | Enable hostname verification. Default value: false. |
--name | The sourceβs name. |
--namespace | The sourceβs namespace. |
--parallelism | The sourceβs parallelism factor, that is, the number of source instances to run). |
--processing-guarantees | The processing guarantees (aka delivery semantics) applied to the source. Available values: ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE. |
--ram | The RAM (in bytes) that needs to be allocated per source instance (applicable only to the Docker runtime). |
-st , --schema-type | The schema type. Either a builtin schema (for example, AVRO and JSON) or custom schema class name to be used to encode messages emitted from source. |
--source-config | Source config key/values. |
--source-config-file | The path to a YAML config file specifying the sourceβs configuration. |
--source-type | The source's connector provider. The source-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--tenant | The sourceβs tenant. |
--tls-allow-insecure | Allow insecure tls connection. Default value: false. |
--tls-trust-cert-path | The tls trust cert file path. |
--use-tls | Use tls connection. Default value: false. |
available-sources
β
Get the list of Pulsar IO connector sources supported by Pulsar cluster.
Usageβ
$ pulsar-admin sources available-sources
reload
β
Reload the available built-in connectors.
Usageβ
$ pulsar-admin sources reload
sinks
β
An interface for managing Pulsar IO sinks (egress data from Pulsar).
$ pulsar-admin sinks subcommands
Subcommands are:
-
create
-
update
-
delete
-
get
-
status
-
list
-
stop
-
start
-
restart
-
localrun
-
available-sinks
-
reload
create
β
Submit a Pulsar IO sink connector to run in a Pulsar cluster.
Usageβ
$ pulsar-admin sinks create options
Optionsβ
Flag | Description |
---|---|
-a , --archive | The path to the archive file for the sink. It also supports url-path (http/https/file [file protocol assumes that file already exists on worker host]) from which worker can download the package. |
--auto-ack | Whether or not the framework will automatically acknowledge messages. |
--classname | The sink's class name if archive is file-url-path (file://). |
--cpu | The CPU (in cores) that needs to be allocated per sink instance (applicable only to Docker runtime). |
--custom-schema-inputs | The map of input topics to schema types or class names (as a JSON string). |
--custom-serde-inputs | The map of input topics to SerDe class names (as a JSON string). |
--disk | The disk (in bytes) that needs to be allocated per sink instance (applicable only to Docker runtime). |
-i, --inputs | The sink's input topic or topics (multiple topics can be specified as a comma-separated list). |
--name | The sink's name. |
--namespace | The sink's namespace. |
--parallelism | The sink's parallelism factor, that is, the number of sink instances to run. |
--processing-guarantees | The processing guarantees (aka delivery semantics) applied to the sink. Possible Values: ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE. |
--ram | The RAM (in bytes) that needs to be allocated per sink instance (applicable only to the process and Docker runtimes). |
--retain-ordering | Sink consumes and sinks messages in order. |
--sink-config | sink config key/values. |
--sink-config-file | The path to a YAML config file specifying the sink's configuration. |
-t , --sink-type | The sink's connector provider. The sink-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--subs-name | Pulsar source subscription name if user wants a specific subscription-name for input-topic consumer. |
--tenant | The sink's tenant. |
--timeout-ms | The message timeout in milliseconds. |
--topics-pattern | TopicsPattern to consume from list of topics under a namespace that match the pattern. --input and --topics-Pattern are mutually exclusive. Add SerDe class name for a pattern in --customSerdeInputs (supported for java fun only). |
update
β
Update a Pulsar IO sink connector.
Usageβ
$ pulsar-admin sinks update options
Optionsβ
Flag | Description |
---|---|
-a , --archive | The path to the archive file for the sink. It also supports url-path (http/https/file [file protocol assumes that file already exists on worker host]) from which worker can download the package. |
--auto-ack | Whether or not the framework will automatically acknowledge messages. |
--classname | The sink's class name if archive is file-url-path (file://). |
--cpu | The CPU (in cores) that needs to be allocated per sink instance (applicable only to Docker runtime). |
--custom-schema-inputs | The map of input topics to schema types or class names (as a JSON string). |
--custom-serde-inputs | The map of input topics to SerDe class names (as a JSON string). |
--disk | The disk (in bytes) that needs to be allocated per sink instance (applicable only to Docker runtime). |
-i, --inputs | The sink's input topic or topics (multiple topics can be specified as a comma-separated list). |
--name | The sink's name. |
--namespace | The sink's namespace. |
--parallelism | The sink's parallelism factor, that is, the number of sink instances to run. |
--processing-guarantees | The processing guarantees (aka delivery semantics) applied to the sink. Possible Values: ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE. |
--ram | The RAM (in bytes) that needs to be allocated per sink instance (applicable only to the process and Docker runtimes). |
--retain-ordering | Sink consumes and sinks messages in order. |
--sink-config | sink config key/values. |
--sink-config-file | The path to a YAML config file specifying the sink's configuration. |
-t , --sink-type | The sink's connector provider. The sink-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--subs-name | Pulsar source subscription name if user wants a specific subscription-name for input-topic consumer. |
--tenant | The sink's tenant. |
--timeout-ms | The message timeout in milliseconds. |
--topics-pattern | TopicsPattern to consume from list of topics under a namespace that match the pattern. --input and --topics-Pattern are mutually exclusive. Add SerDe class name for a pattern in --customSerdeInputs (supported for java fun only). |
--update-auth-data | Whether or not to update the auth data. Default value: false. |
delete
β
Delete a Pulsar IO sink connector.
Usageβ
$ pulsar-admin sinks delete options
Optionβ
Flag | Description |
---|---|
--name | The sink's name. |
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
get
β
Get the information about a Pulsar IO sink connector.
Usageβ
$ pulsar-admin sinks get options
Optionsβ
Flag | Description |
---|---|
--name | The sink's name. |
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
status
β
Check the current status of a Pulsar sink.
Usageβ
$ pulsar-admin sinks status options
Optionsβ
Flag | Description |
---|---|
--instance-id | The sink ID. If instance-id is not provided, Pulsar gets status of all instances. |
--name | The sink's name. |
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
list
β
List all running Pulsar IO sink connectors.
Usageβ
$ pulsar-admin sinks list options
Optionsβ
Flag | Description |
---|---|
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
stop
β
Stop a sink instance.
Usageβ
$ pulsar-admin sinks stop options
Optionsβ
Flag | Description |
---|---|
--instance-id | The sink instanceID. If instance-id is not provided, Pulsar stops all instances. |
--name | The sink's name. |
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
start
β
Start a sink instance.
Usageβ
$ pulsar-admin sinks start options
Optionsβ
Flag | Description |
---|---|
--instance-id | The sink instanceID. If instance-id is not provided, Pulsar starts all instances. |
--name | The sink's name. |
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
restart
β
Restart a sink instance.
Usageβ
$ pulsar-admin sinks restart options
Optionsβ
Flag | Description |
---|---|
--instance-id | The sink instanceID. If instance-id is not provided, Pulsar restarts all instances. |
--name | The sink's name. |
--namespace | The sink's namespace. |
--tenant | The sink's tenant. |
localrun
β
Run a Pulsar IO sink connector locally rather than deploying it to the Pulsar cluster.
Usageβ
$ pulsar-admin sinks localrun options
Optionsβ
Flag | Description |
---|---|
-a , --archive | The path to the archive file for the sink. It also supports url-path (http/https/file [file protocol assumes that file already exists on worker host]) from which worker can download the package. |
--auto-ack | Whether or not the framework will automatically acknowledge messages. |
--broker-service-url | The URL for the Pulsar broker. |
--classname | The sink's class name if archive is file-url-path (file://). |
--client-auth-params | Client authentication parameter. |
--client-auth-plugin | Client authentication plugin using which function-process can connect to broker. |
--cpu | The CPU (in cores) that needs to be allocated per sink instance (applicable only to the Docker runtime). |
--custom-schema-inputs | The map of input topics to Schema types or class names (as a JSON string). |
--custom-serde-inputs | The map of input topics to SerDe class names (as a JSON string). |
--disk | The disk (in bytes) that needs to be allocated per sink instance (applicable only to the Docker runtime). |
--hostname-verification-enabled | Enable hostname verification. Default value: false. |
-i , --inputs | The sink's input topic or topics (multiple topics can be specified as a comma-separated list). |
--name | The sinkβs name. |
--namespace | The sinkβs namespace. |
--parallelism | The sinkβs parallelism factor, that is, the number of sink instances to run). |
--processing-guarantees | The processing guarantees (aka delivery semantics) applied to the sink. Available values: ATLEAST_ONCE, ATMOST_ONCE, EFFECTIVELY_ONCE. |
--ram | The RAM (in bytes) that needs to be allocated per sink instance (applicable only to the Docker runtime). |
--retain-ordering | Sink consumes and sinks messages in order. |
--sink-config | sink config key/values. |
--sink-config-file | The path to a YAML config file specifying the sinkβs configuration. |
--sink-type | The sink's connector provider. The sink-type parameter of the currently built-in connectors is determined by the setting of the name parameter specified in the pulsar-io.yaml file. |
--subs-name | Pulsar source subscription name if user wants a specific subscription-name for input-topic consumer. |
--tenant | The sinkβs tenant. |
--timeout-ms | The message timeout in milliseconds. |
--tls-allow-insecure | Allow insecure tls connection. Default value: false. |
--tls-trust-cert-path | The tls trust cert file path. |
--topics-pattern | TopicsPattern to consume from list of topics under a namespace that match the pattern. --input and --topics-Pattern are mutually exclusive. Add SerDe class name for a pattern in --customSerdeInputs (supported for java fun only). |
--use-tls | Use tls connection. Default value: false. |
available-sinks
β
Get the list of Pulsar IO connector sinks supported by Pulsar cluster.
Usageβ
$ pulsar-admin sinks available-sinks
reload
β
Reload the available built-in connectors.
Usageβ
$ pulsar-admin sinks reload