From b99fd2cafa7495488254b04b275b1a62206dca05 Mon Sep 17 00:00:00 2001 From: Paulo Alberto Simoes Date: Tue, 27 Aug 2024 00:13:29 -0300 Subject: [PATCH 1/3] Add topic name format requirement. Signed-off-by: Paulo Alberto Simoes --- README.md | 81 +++++++++++-------- .../src/main/resources/config.properties | 2 +- .../src/main/resources/config.properties | 2 +- 3 files changed, 51 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index 7eee562..9ca92dc 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -# Kafka Java Client for Oracle Transactional Event Queues +# Kafka Java Client for Oracle Transactional Event Queues ## Building the Kafka Java Client for Oracle TxEventQ distribution @@ -9,14 +9,13 @@ You need to have [Gradle 7.3 or above](http://www.gradle.org/installation) and [ This distribution contains version 23.4.0.0 of the `Kafka Java Client for Oracle Transactional Event Queues` project. It will be referred as OKafka-23.4.0.0 henceforth. This is tested with JDK 11.0.22 but we recommend using the latest version. -The Kafka Java Client works with Oracle Database 23ai Free version as well as Oracle Database 23ai available on Oracle Autonomous Cloud platform. +The Kafka Java Client works with Oracle Database 23ai Free version as well as Oracle Database 23ai available on Oracle Autonomous Cloud platform. -To test this distribution in free Oracle Cloud environment create [Oracle Cloud account](https://docs.cloud.oracle.com/en-us/iaas/Content/FreeTier/freetier.htm) then create [Oracle Autonomous Transaction Processing Database instance](https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/tutorial-getting-started-autonomous-db/index.html) in cloud. +To test this distribution in free Oracle Cloud environment create [Oracle Cloud account](https://docs.cloud.oracle.com/en-us/iaas/Content/FreeTier/freetier.htm) then create [Oracle Autonomous Transaction Processing Database instance](https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/tutorial-getting-started-autonomous-db/index.html) in cloud. A database user should be created and should be granted the privileges mentioned in Database user configuration section. Then create a Transactional Event Queue to produce and consume messages. - -### Database user configuration ### +### Database user configuration To run `OKafka application` against Oracle Database, a database user must be created and must be granted below privileges. @@ -46,7 +45,7 @@ It is preferred in general to assign or grant a specific quota on a tablespace t ALTER USER user QUOTA UNLIMITED /* or size-clause */ on tablespace_name; ``` -Once user is created and above privileges are granted, connect to Oracle Database as this user and create a Transactional Event Queue using below PL/SQL script. One can also use `KafkaAdmin` interface as shown in `CreateTopic.java` in `examples` directory to create a Transactional Event Queue. +Once user is created and above privileges are granted, connect to Oracle Database as this user and create a Topic using below PL/SQL script. ```roomsql -- Create an OKafka topic named 'TXEQ' with 5 partition and retention time of 7 days. @@ -55,50 +54,69 @@ begin end; ``` -#### Connection configuration #### +> NOTE: A Topic can also be created using OKAFKA Administration methods. Or, through the Producer interface which creates a new topic if it was not previously created. + +#### Connection configuration `OKafka` uses JDBC(thin driver) connection to connect to Oracle Database instance using any one of two security protocols. - -1. PLAINTEXT -2. SSL +1. PLAINTEXT +2. SSL 1.PLAINTEXT: In this protocol a JDBC connection is setup by providing username and password in plain text in ojdbc.prperties file. To use PLAINTEXT protocol user must provide following properties through application. - security.protocol = "PLAINTEXT" - bootstrap.servers = "host:port" - oracle.service.name = "name of the service running on the instance" - oracle.net.tns_admin = "location of ojdbc.properties file" - +```text +security.protocol = "PLAINTEXT" +bootstrap.servers = "host:port" +oracle.service.name = "name of the service running on the instance" +oracle.net.tns_admin = "location of ojdbc.properties file" +``` + `ojdbc.properties` file must have below properties - - user(in lowercase)=DatabaseUserName - password(in lowercase)=Password + +''' +user(in lowercase)=DatabaseUserName +password(in lowercase)=Password +''' 2.SSL: This protocol requires that, while connecting to Oracle Database, the JDBC driver authenticates database user using Oracle Wallet or Java KeyStore(JKS) files. This protocol is typically used to o connect to Oracle database 23ai instance in Oracle Autonomous cloud. To use this protocol `Okafka` application must specify following properties. - security.protocol = "SSL" - oracle.net.tns_admin = "location containing Oracle Wallet, tnsname.ora and ojdbc.properties file" - tns.alias = "alias of connection string in tnsnames.ora" +```text +security.protocol = "SSL" +oracle.net.tns_admin = "location containing Oracle Wallet, tnsname.ora and ojdbc.properties file" +tns.alias = "alias of connection string in tnsnames.ora" +``` + +Directory location provided in `oracle.net.tns_admin` property should have: -Directory location provided in `oracle.net.tns_admin` property should have 1. Oracle Wallet 2. tnsnames.ora file 3. ojdbc.properties file (optional) This depends on how the Oracle Wallet is configured. Learn more about [JDBC Thin Connections with a Wallet (mTLS)](https://docs.oracle.com/en/cloud/paas/atp-cloud/atpug/connect-jdbc-thin-wallet.html#GUID-5ED3C08C-1A84-4E5A-B07A-A5114951AA9E) to establish secured JDBC connections. - -Note: tnsnames.ora file in wallet downloaded from Oracle Autonomous Database contains JDBC connection string which is used for establishing JDBC connection. - + +> Note: tnsnames.ora file in wallet downloaded from Oracle Autonomous Database contains JDBC connection string which is used for establishing JDBC connection. + +#### APIs configuration + +You can get a detailed description of the Producer, Consumer and Administration APIs in the [Kafka APIs for Oracle Transactional Event Queues Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/23/adque/Kafka_cient_interface_TEQ.html#GUID-5549915E-6509-4065-B05E-E96338F4742C). + +> NOTE: Topic name property should be provided in UPPERCASE. +> +>> ```text +>> topic.name= +>> ``` + ### Building okafka.jar Simplest way to build the `okafka.jar` file is by using Gradle build tool. This distribution contains gradle build files which will work for Gradle 7.3 or higher. -``` +```shell ./gradle jar ``` + This generates `okafka-23.4.0.0.jar` in `okafka_source_dir/clients/build/libs`. **Project Dependency:** @@ -119,18 +137,17 @@ All these jars are downloaded from Maven Repository during gradle build. To build the `okafka.jar` file which includes all the dependent jar files in itself. -``` +```shell ./gradle fullJar ``` -This generates `okafka-full-23.4.0.0.jar` in `okafka_source_dir/clients/build/libs`. - -## Build javadoc +This genddsferates `okafka-full-23.4.0.0.jar` in `okafka_source_dir/clients/build/libs`. +## Build javadoc This command generates javadoc in `okafka_source_dir/clients/build/docs/javadoc` -``` +```shell gradle javadoc ``` @@ -142,7 +159,7 @@ Repository contains 2 common OKafka application examples in `examples` folder. Produces 10 messages into TxEQ topic. `2. ConsumerOKafka.java` -Consumes 10 messages from TxEQ topic. +Consumes 10 messages from TxEQ topic. ## Kafka Java Client APIs supported diff --git a/examples/consumer/src/main/resources/config.properties b/examples/consumer/src/main/resources/config.properties index 65ae47b..da2bc36 100644 --- a/examples/consumer/src/main/resources/config.properties +++ b/examples/consumer/src/main/resources/config.properties @@ -13,7 +13,7 @@ oracle.net.tns_admin= #tns.alias= # Application specific OKafka consumer properties -topic.name= +topic.name= group.id= enable.auto.commit=true diff --git a/examples/producer/src/main/resources/config.properties b/examples/producer/src/main/resources/config.properties index 3111efc..6867697 100644 --- a/examples/producer/src/main/resources/config.properties +++ b/examples/producer/src/main/resources/config.properties @@ -13,7 +13,7 @@ oracle.net.tns_admin= #tns.alias= #Appliction specific OKafka Producer properties -topic.name= +topic.name= batch.size=200 linger.ms=100 From 9605df55c8b5fc3a91be926a57ce60c3627582e2 Mon Sep 17 00:00:00 2001 From: Paulo Alberto Simoes Date: Tue, 27 Aug 2024 00:16:17 -0300 Subject: [PATCH 2/3] Fix format of code block Signed-off-by: Paulo Alberto Simoes --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9ca92dc..9f17058 100644 --- a/README.md +++ b/README.md @@ -74,10 +74,10 @@ oracle.net.tns_admin = "location of ojdbc.properties file" `ojdbc.properties` file must have below properties -''' +```text user(in lowercase)=DatabaseUserName password(in lowercase)=Password -''' +``` 2.SSL: This protocol requires that, while connecting to Oracle Database, the JDBC driver authenticates database user using Oracle Wallet or Java KeyStore(JKS) files. This protocol is typically used to o connect to Oracle database 23ai instance in Oracle Autonomous cloud. To use this protocol `Okafka` application must specify following properties. From b267192be231a79ff731898d3f5c425a09fad698 Mon Sep 17 00:00:00 2001 From: Paulo Alberto Simoes Date: Tue, 27 Aug 2024 00:17:48 -0300 Subject: [PATCH 3/3] text format Signed-off-by: Paulo Alberto Simoes --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9f17058..288f6f9 100644 --- a/README.md +++ b/README.md @@ -54,7 +54,7 @@ begin end; ``` -> NOTE: A Topic can also be created using OKAFKA Administration methods. Or, through the Producer interface which creates a new topic if it was not previously created. +> Note: A Topic can also be created using OKAFKA Administration methods. Or, through the Producer interface which creates a new topic if it was not previously created. #### Connection configuration @@ -102,7 +102,7 @@ Learn more about [JDBC Thin Connections with a Wallet (mTLS)](https://docs.oracl You can get a detailed description of the Producer, Consumer and Administration APIs in the [Kafka APIs for Oracle Transactional Event Queues Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/23/adque/Kafka_cient_interface_TEQ.html#GUID-5549915E-6509-4065-B05E-E96338F4742C). -> NOTE: Topic name property should be provided in UPPERCASE. +> Note: Topic name property should be provided in UPPERCASE. > >> ```text >> topic.name=