camel-sql-kafka-connector source configuration

When using camel-sql-kafka-connector as source make sure to use the following Maven dependency to have support for the connector:

<dependency>
  <groupId>org.apache.camel.kafkaconnector</groupId>
  <artifactId>camel-sql-kafka-connector</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel Kafka connector version -->
</dependency>

To use this Source connector in Kafka connect you’ll need to set the following connector.class

connector.class=org.apache.camel.kafkaconnector.sql.CamelSqlSourceConnector

The camel-sql source connector supports 49 options, which are listed below.

Name	Description	Default	Required	Priority
camel.source.path.query	Sets the SQL query to perform. You can externalize the query by using file: or classpath: as prefix and specify the location of the file.	null	true	HIGH
camel.source.endpoint.allowNamedParameters	Whether to allow using named parameters in the queries.	true	false	MEDIUM
camel.source.endpoint.dataSource	Sets the DataSource to use to communicate with the database.	null	false	MEDIUM
camel.source.endpoint.dataSourceRef	Sets the reference to a DataSource to lookup from the registry, to use for communicating with the database.	null	false	LOW
camel.source.endpoint.outputClass	Specify the full package and class name to use as conversion when outputType=SelectOne.	null	false	MEDIUM
camel.source.endpoint.outputHeader	Store the query result in a header instead of the message body. By default, outputHeader == null and the query result is stored in the message body, any existing content in the message body is discarded. If outputHeader is set, the value is used as the name of the header to store the query result and the original message body is preserved.	null	false	MEDIUM
camel.source.endpoint.outputType	Make the output of consumer or producer to SelectList as List of Map, or SelectOne as single Java object in the following way: a) If the query has only single column, then that JDBC Column object is returned. (such as SELECT COUNT( ) FROM PROJECT will return a Long object. b) If the query has more than one column, then it will return a Map of that result. c) If the outputClass is set, then it will convert the query result into an Java bean object by calling all the setters that match the column names. It will assume your class has a default constructor to create instance with. d) If the query resulted in more than one rows, it throws an non-unique result exception. StreamList streams the result of the query using an Iterator. This can be used with the Splitter EIP in streaming mode to process the ResultSet in streaming fashion. One of: [SelectOne] [SelectList] [StreamList]	"SelectList"	false	MEDIUM
camel.source.endpoint.separator	The separator to use when parameter values is taken from message body (if the body is a String type), to be inserted at # placeholders. Notice if you use named parameters, then a Map type is used instead. The default value is comma	","	false	MEDIUM
camel.source.endpoint.breakBatchOnConsumeFail	Sets whether to break batch if onConsume failed.	false	false	MEDIUM
camel.source.endpoint.bridgeErrorHandler	Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.	false	false	MEDIUM
camel.source.endpoint.expectedUpdateCount	Sets an expected update count to validate when using onConsume.	-1	false	MEDIUM
camel.source.endpoint.maxMessagesPerPoll	Sets the maximum number of messages to poll	null	false	MEDIUM
camel.source.endpoint.onConsume	After processing each row then this query can be executed, if the Exchange was processed successfully, for example to mark the row as processed. The query can have parameter.	null	false	MEDIUM
camel.source.endpoint.onConsumeBatchComplete	After processing the entire batch, this query can be executed to bulk update rows etc. The query cannot have parameters.	null	false	MEDIUM
camel.source.endpoint.onConsumeFailed	After processing each row then this query can be executed, if the Exchange failed, for example to mark the row as failed. The query can have parameter.	null	false	MEDIUM
camel.source.endpoint.routeEmptyResultSet	Sets whether empty resultset should be allowed to be sent to the next hop. Defaults to false. So the empty resultset will be filtered out.	false	false	MEDIUM
camel.source.endpoint.sendEmptyMessageWhenIdle	If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.	false	false	MEDIUM
camel.source.endpoint.transacted	Enables or disables transaction. If enabled then if processing an exchange failed then the consumer breaks out processing any further exchanges to cause a rollback eager.	false	false	MEDIUM
camel.source.endpoint.useIterator	Sets how resultset should be delivered to route. Indicates delivery as either a list or individual object. defaults to true.	true	false	MEDIUM
camel.source.endpoint.exceptionHandler	To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.	null	false	MEDIUM
camel.source.endpoint.exchangePattern	Sets the exchange pattern when the consumer creates an exchange. One of: [InOnly] [InOut] [InOptionalOut]	null	false	MEDIUM
camel.source.endpoint.pollStrategy	A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel.	null	false	MEDIUM
camel.source.endpoint.processingStrategy	Allows to plugin to use a custom org.apache.camel.component.sql.SqlProcessingStrategy to execute queries when the consumer has processed the rows/batch.	null	false	MEDIUM
camel.source.endpoint.alwaysPopulateStatement	If enabled then the populateStatement method from org.apache.camel.component.sql.SqlPrepareStatementStrategy is always invoked, also if there is no expected parameters to be prepared. When this is false then the populateStatement is only invoked if there is 1 or more expected parameters to be set; for example this avoids reading the message body/headers for SQL queries with no parameters.	false	false	MEDIUM
camel.source.endpoint.basicPropertyBinding	Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities	false	false	MEDIUM
camel.source.endpoint.parametersCount	If set greater than zero, then Camel will use this count value of parameters to replace instead of querying via JDBC metadata API. This is useful if the JDBC vendor could not return correct parameters count, then user may override instead.	null	false	MEDIUM
camel.source.endpoint.placeholder	Specifies a character that will be replaced to in SQL query. Notice, that it is simple String.replaceAll() operation and no SQL parsing is involved (quoted strings will also change).	"#"	false	MEDIUM
camel.source.endpoint.prepareStatementStrategy	Allows to plugin to use a custom org.apache.camel.component.sql.SqlPrepareStatementStrategy to control preparation of the query and prepared statement.	null	false	MEDIUM
camel.source.endpoint.synchronous	Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).	false	false	MEDIUM
camel.source.endpoint.templateOptions	Configures the Spring JdbcTemplate with the key/values from the Map	null	false	MEDIUM
camel.source.endpoint.usePlaceholder	Sets whether to use placeholder and replace all placeholder characters with sign in the SQL queries.	true	false	MEDIUM
camel.source.endpoint.backoffErrorThreshold	The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.	null	false	MEDIUM
camel.source.endpoint.backoffIdleThreshold	The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.	null	false	MEDIUM
camel.source.endpoint.backoffMultiplier	To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.	null	false	MEDIUM
camel.source.endpoint.delay	Milliseconds before the next poll.	500L	false	MEDIUM
camel.source.endpoint.greedy	If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.	false	false	MEDIUM
camel.source.endpoint.initialDelay	Milliseconds before the first poll starts.	1000L	false	MEDIUM
camel.source.endpoint.repeatCount	Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever.	0L	false	MEDIUM
camel.source.endpoint.runLoggingLevel	The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. One of: [TRACE] [DEBUG] [INFO] [WARN] [ERROR] [OFF]	"TRACE"	false	MEDIUM
camel.source.endpoint.scheduledExecutorService	Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool.	null	false	MEDIUM
camel.source.endpoint.scheduler	To use a cron scheduler from either camel-spring or camel-quartz component. Use value spring or quartz for built in scheduler	"none"	false	MEDIUM
camel.source.endpoint.schedulerProperties	To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler.	null	false	MEDIUM
camel.source.endpoint.startScheduler	Whether the scheduler should be auto started.	true	false	MEDIUM
camel.source.endpoint.timeUnit	Time unit for initialDelay and delay options. One of: [NANOSECONDS] [MICROSECONDS] [MILLISECONDS] [SECONDS] [MINUTES] [HOURS] [DAYS]	"MILLISECONDS"	false	MEDIUM
camel.source.endpoint.useFixedDelay	Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.	true	false	MEDIUM
camel.component.sql.dataSource	Sets the DataSource to use to communicate with the database.	null	false	MEDIUM
camel.component.sql.bridgeErrorHandler	Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.	false	false	MEDIUM
camel.component.sql.basicPropertyBinding	Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities	false	false	LOW
camel.component.sql.usePlaceholder	Sets whether to use placeholder and replace all placeholder characters with sign in the SQL queries. This option is default true	true	false	MEDIUM

Name

Description

Default

Required

Priority

camel.source.path.query

Sets the SQL query to perform. You can externalize the query by using file: or classpath: as prefix and specify the location of the file.

null

true

HIGH

camel.source.endpoint.allowNamedParameters

Whether to allow using named parameters in the queries.

true

false

MEDIUM

camel.source.endpoint.dataSource

Sets the DataSource to use to communicate with the database.

null

false

MEDIUM

camel.source.endpoint.dataSourceRef

Sets the reference to a DataSource to lookup from the registry, to use for communicating with the database.

null

false

LOW

camel.source.endpoint.outputClass

Specify the full package and class name to use as conversion when outputType=SelectOne.

null

false

MEDIUM

camel.source.endpoint.outputHeader

Store the query result in a header instead of the message body. By default, outputHeader == null and the query result is stored in the message body, any existing content in the message body is discarded. If outputHeader is set, the value is used as the name of the header to store the query result and the original message body is preserved.

null

false

MEDIUM

camel.source.endpoint.outputType

Make the output of consumer or producer to SelectList as List of Map, or SelectOne as single Java object in the following way: a) If the query has only single column, then that JDBC Column object is returned. (such as SELECT COUNT( ) FROM PROJECT will return a Long object. b) If the query has more than one column, then it will return a Map of that result. c) If the outputClass is set, then it will convert the query result into an Java bean object by calling all the setters that match the column names. It will assume your class has a default constructor to create instance with. d) If the query resulted in more than one rows, it throws an non-unique result exception. StreamList streams the result of the query using an Iterator. This can be used with the Splitter EIP in streaming mode to process the ResultSet in streaming fashion. One of: [SelectOne] [SelectList] [StreamList]

"SelectList"

false

MEDIUM

camel.source.endpoint.separator

The separator to use when parameter values is taken from message body (if the body is a String type), to be inserted at # placeholders. Notice if you use named parameters, then a Map type is used instead. The default value is comma

","

false

MEDIUM

camel.source.endpoint.breakBatchOnConsumeFail

Sets whether to break batch if onConsume failed.

false

MEDIUM

camel.source.endpoint.bridgeErrorHandler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

MEDIUM

camel.source.endpoint.expectedUpdateCount

Sets an expected update count to validate when using onConsume.

-1

false

MEDIUM

camel.source.endpoint.maxMessagesPerPoll

Sets the maximum number of messages to poll

null

false

MEDIUM

camel.source.endpoint.onConsume

After processing each row then this query can be executed, if the Exchange was processed successfully, for example to mark the row as processed. The query can have parameter.

null

false

MEDIUM

camel.source.endpoint.onConsumeBatchComplete

After processing the entire batch, this query can be executed to bulk update rows etc. The query cannot have parameters.

null

false

MEDIUM

camel.source.endpoint.onConsumeFailed

After processing each row then this query can be executed, if the Exchange failed, for example to mark the row as failed. The query can have parameter.

null

false

MEDIUM

camel.source.endpoint.routeEmptyResultSet

Sets whether empty resultset should be allowed to be sent to the next hop. Defaults to false. So the empty resultset will be filtered out.

false

MEDIUM

camel.source.endpoint.sendEmptyMessageWhenIdle

If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.

false

MEDIUM

camel.source.endpoint.transacted

Enables or disables transaction. If enabled then if processing an exchange failed then the consumer breaks out processing any further exchanges to cause a rollback eager.

false

MEDIUM

camel.source.endpoint.useIterator

Sets how resultset should be delivered to route. Indicates delivery as either a list or individual object. defaults to true.

true

false

MEDIUM

camel.source.endpoint.exceptionHandler

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

null

false

MEDIUM

camel.source.endpoint.exchangePattern

Sets the exchange pattern when the consumer creates an exchange. One of: [InOnly] [InOut] [InOptionalOut]

null

false

MEDIUM

camel.source.endpoint.pollStrategy

A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel.

null

false

MEDIUM

camel.source.endpoint.processingStrategy

Allows to plugin to use a custom org.apache.camel.component.sql.SqlProcessingStrategy to execute queries when the consumer has processed the rows/batch.

null

false

MEDIUM

camel.source.endpoint.alwaysPopulateStatement

If enabled then the populateStatement method from org.apache.camel.component.sql.SqlPrepareStatementStrategy is always invoked, also if there is no expected parameters to be prepared. When this is false then the populateStatement is only invoked if there is 1 or more expected parameters to be set; for example this avoids reading the message body/headers for SQL queries with no parameters.

false

MEDIUM

camel.source.endpoint.basicPropertyBinding

Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities

false

MEDIUM

camel.source.endpoint.parametersCount

If set greater than zero, then Camel will use this count value of parameters to replace instead of querying via JDBC metadata API. This is useful if the JDBC vendor could not return correct parameters count, then user may override instead.

null

false

MEDIUM

camel.source.endpoint.placeholder

Specifies a character that will be replaced to in SQL query. Notice, that it is simple String.replaceAll() operation and no SQL parsing is involved (quoted strings will also change).

"#"

false

MEDIUM

camel.source.endpoint.prepareStatementStrategy

Allows to plugin to use a custom org.apache.camel.component.sql.SqlPrepareStatementStrategy to control preparation of the query and prepared statement.

null

false

MEDIUM

camel.source.endpoint.synchronous

Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).

false

MEDIUM

camel.source.endpoint.templateOptions

Configures the Spring JdbcTemplate with the key/values from the Map

null

false

MEDIUM

camel.source.endpoint.usePlaceholder

Sets whether to use placeholder and replace all placeholder characters with sign in the SQL queries.

true

false

MEDIUM

camel.source.endpoint.backoffErrorThreshold

The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.

null

false

MEDIUM

camel.source.endpoint.backoffIdleThreshold

The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.

null

false

MEDIUM

camel.source.endpoint.backoffMultiplier

To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.

null

false

MEDIUM

camel.source.endpoint.delay

Milliseconds before the next poll.

500L

false

MEDIUM

camel.source.endpoint.greedy

If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.

false

MEDIUM

camel.source.endpoint.initialDelay

Milliseconds before the first poll starts.

1000L

false

MEDIUM

camel.source.endpoint.repeatCount

Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever.

false

MEDIUM

camel.source.endpoint.runLoggingLevel

The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. One of: [TRACE] [DEBUG] [INFO] [WARN] [ERROR] [OFF]

"TRACE"

false

MEDIUM

camel.source.endpoint.scheduledExecutorService

Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool.

null

false

MEDIUM

camel.source.endpoint.scheduler

To use a cron scheduler from either camel-spring or camel-quartz component. Use value spring or quartz for built in scheduler

"none"

false

MEDIUM

camel.source.endpoint.schedulerProperties

To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler.

null

false

MEDIUM

camel.source.endpoint.startScheduler

Whether the scheduler should be auto started.

true

false

MEDIUM

camel.source.endpoint.timeUnit

Time unit for initialDelay and delay options. One of: [NANOSECONDS] [MICROSECONDS] [MILLISECONDS] [SECONDS] [MINUTES] [HOURS] [DAYS]

"MILLISECONDS"

false

MEDIUM

camel.source.endpoint.useFixedDelay

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.

true

false

MEDIUM

camel.component.sql.dataSource

Sets the DataSource to use to communicate with the database.

null

false

MEDIUM

camel.component.sql.bridgeErrorHandler

false

MEDIUM

camel.component.sql.basicPropertyBinding

Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities

false

LOW

camel.component.sql.usePlaceholder

Sets whether to use placeholder and replace all placeholder characters with sign in the SQL queries. This option is default true

true

false

MEDIUM

The camel-sql sink connector has no converters out of the box.

The camel-sql sink connector has no transforms out of the box.

The camel-sql sink connector has no aggregation strategies out of the box.