Kafka CLI and kcatBeta
To connect to a deployed Cloud Kafka instance from a terminal, use Kafka CLI or kcat. Choose a tool based on your task:
- Kafka CLI — to administer Kafka and send and read messages.
- kcat — to check the connection and send and read messages without installing Java.
-
Go to your VK Cloud account.
-
Go to Data Platform → Service instances.
-
Click the Cloud Kafka instance name.
-
On the General information tab, copy the connection string for the option you need:
- Bootstrap connection string with TLS — for a TLS connection;
- Bootstrap connection string — for a connection without TLS.
-
Download the certificate chain for a TLS connection: on the General information tab, click ••• → Download certificate chain in the certificate card.
-
Have the username and password for your Cloud Kafka account ready. You can add an account on the service instance page.
-
Open a terminal and set variables with your Cloud Kafka account username and password:
K_USERNAME="<USERNAME>"K_PASSWORD="<PASSWORD>" -
Create the
/tmp/client.propertiesconfiguration file for a connection with or without TLS:TLSWithout TLScat > /tmp/client.properties <<EOFsecurity.protocol=SASL_SSLsasl.mechanism=SCRAM-SHA-512sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="${K_USERNAME}" password="${K_PASSWORD}";ssl.truststore.type=PEMssl.truststore.location=<CERTIFICATE_CHAIN_PATH>ssl.endpoint.identification.algorithm=request.timeout.ms=30000EOFHere,
<CERTIFICATE_CHAIN_PATH>is the path to the certificate chain file downloaded from the General information tab during preparation. -
Set the environment variables:
export K_BOOTSTRAP="<BOOTSTRAP_SERVER_ADDRESS_AND_PORT>"export K_CLI_BASE="<KAFKA_CLI_DIRECTORY_PATH>"export K_CONFIG=/tmp/client.propertiesHere:
<BOOTSTRAP_SERVER_ADDRESS_AND_PORT>: the connection string copied during preparation;<KAFKA_CLI_DIRECTORY_PATH>: the directory where Kafka CLI is installed. Example:/opt/homebrew/opt/kafka/bin.
-
Get the list of topics:
"${K_CLI_BASE}/kafka-topics.sh" \--command-config "${K_CONFIG}" \--bootstrap-server "${K_BOOTSTRAP}" \--listThe terminal displays the list of available topics. If no topics are available, the output may be empty.
-
Open PowerShell and set variables with your Cloud Kafka account username and password:
$env:K_USERNAME = "<USERNAME>"$env:K_PASSWORD = "<PASSWORD>" -
Create a client configuration file for a connection with TLS (traffic is encrypted) or without TLS (traffic is not encrypted):
TLSWithout TLS@"security.protocol=SASL_SSLsasl.mechanism=SCRAM-SHA-512sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="$env:K_USERNAME" password="$env:K_PASSWORD";ssl.truststore.type=PEMssl.truststore.location=<CERTIFICATE_CHAIN_PATH>ssl.endpoint.identification.algorithm=request.timeout.ms=30000"@ | Set-Content -Path "$env:TEMP\client.properties" -Encoding ASCIIHere,
<CERTIFICATE_CHAIN_PATH>is the path to the certificate chain file downloaded from the General information tab during preparation. Use forward slashes in the path, for exampleC:/certs/ca.crt. -
Set the environment variables:
$env:K_BOOTSTRAP = "<BOOTSTRAP_SERVER_ADDRESS_AND_PORT>"$env:K_CLI_BASE = "<KAFKA_CLI_DIRECTORY_PATH>"$env:K_CONFIG = "$env:TEMP\client.properties"Here:
<BOOTSTRAP_SERVER_ADDRESS_AND_PORT>: the connection string copied during preparation;<KAFKA_CLI_DIRECTORY_PATH>: the directory containing Kafka CLI scripts, for exampleC:\kafka\bin\windows.
Use a directory with a short path for Kafka CLI, for example
C:\kafka. A long path may cause the script to fail with the errorThe input line is too long. -
Get the list of topics:
& "$env:K_CLI_BASE\kafka-topics.bat" `--command-config "$env:K_CONFIG" `--bootstrap-server "$env:K_BOOTSTRAP" `--listThe terminal displays the list of available topics. If no topics are available, the output may be empty.
Open a terminal and run the command for the selected connection option.
kcat -b "<BOOTSTRAP_SERVER_ADDRESS_AND_PORT>" -X security.protocol=SASL_SSL -X sasl.mechanism=SCRAM-SHA-512 -X sasl.username="<USERNAME>" -X sasl.password="<PASSWORD>" -X ssl.ca.location="<CERTIFICATE_CHAIN_PATH>" -L
The command displays information about the brokers and topics of the Cloud Kafka instance.
Here:
<BOOTSTRAP_SERVER_ADDRESS_AND_PORT>: the connection string copied during preparation.<USERNAME>and<PASSWORD>: the Cloud Kafka user's credentials.<CERTIFICATE_CHAIN_PATH>: the path to the certificate chain file downloaded from the General information tab during preparation. Used only for TLS connections.