Gora

Since Camel 2.14

Both producer and consumer are supported

Camel-Gora is an Apache Camel component that allows you to work with NoSQL databases using the Apache Gora framework.

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-gora</artifactId>
 <!-- use the same version as your Camel core version -->
 <version>x.x.x</version>
</dependency>

Apache Gora Overview

The Apache Gora open source framework provides an in-memory data model and persistence for big data. Gora supports persisting to column stores, key value stores, document stores and RDBMSs, and analyzing the data with extensive Apache Hadoop™ MapReduce support. Gora uses the Apache Software License v2.0 and graduated from the Apache Incubator in Janauary 2012 to become a top-level Apache project.

Apache Gora currently supports the following datastores: Apache HBase, Apache Cassandra, Apache Accumulo, Amazon DynamoDB and SQL databases such as hsqldb, MySQL and more.

URI format

gora:instanceName[?options]

Hbase examples with mandatory options :

XML

<to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Java DSL

to("gora:foobar?keyClass=java.lang.Long&valueClass=org.apache.camel.component.gora.generated.Pageview&dataStoreClass=org.apache.gora.hbase.store.HBaseStore")

Configuratiion

Using camel-gora needs some configuration. This mainly involve to configure the AvroStore through the gora.properties file and to define the relevant mappings as part of the gora-core module.

Extensive information for this configuration can be found in the apache gora documentation and the gora-conf page.

Options

The Gora component supports 3 options, which are listed below.

Name Description Default Type

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

boolean

autowiredEnabled (advanced)

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

boolean

The Gora endpoint is configured using URI syntax:

gora:name

with the following path and query parameters:

Path Parameters (1 parameters):

Name Description Default Type

name

Required Instance name

String

Query Parameters (22 parameters):

Name Description Default Type

dataStoreClass (common)

The type of the dataStore

String

keyClass (common)

The type class of the key

String

valueClass (common)

The type of the value

String

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

concurrentConsumers (consumer)

Number of concurrent consumers

1

int

endKey (consumer)

The End Key

Object

endTime (consumer)

The End Time

long

fields (consumer)

The Fields

Strings

keyRangeFrom (consumer)

The Key Range From

Object

keyRangeTo (consumer)

The Key Range To

Object

limit (consumer)

The Limit

long

startKey (consumer)

The Start Key

Object

startTime (consumer)

The Start Time

long

timeRangeFrom (consumer)

The Time Range From

long

timeRangeTo (consumer)

The Time Range To

long

timestamp (consumer)

The Timestamp

long

exceptionHandler (consumer)

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

ExceptionHandler

exchangePattern (consumer)

Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut

ExchangePattern

flushOnEveryOperation (producer)

Flush on every operation

true

boolean

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

boolean

hadoopConfiguration (advanced)

Hadoop Configuration

Configuration

synchronous (advanced)

Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).

false

boolean

Supported Gora Operations

Supported operations include : put, get, delete, getSchemaName, deleteSchema, createSchema, query, deleteByQuery, schemaExists.

Some of the operations require arguments while some others no. The arguments to operations could be either the body of the in message or defined in a header property. Below there is a list with some additional info for each operation.

Property Description

put

Inserts the persistent object with the given key.

get

Returns the object corresponding to the given key fetching all the fields.

delete

Deletes the object with the given key.

getSchemaName

Returns the schema name given to this DataStore.

deleteSchema

Deletes the underlying schema or table (or similar) in the datastore that holds the objects.

createSchema

Creates the optional schema or table (or similar) in the datastore to hold the objects.

query

Executes the given query and returns the results.

deleteByQuery

Deletes all the objects matching the query.

schemaExists

Returns whether the schema that holds the data exists in the datastore.

Gora Headers

Property Description

GoraOperation

Used in order to define the operation to execute.

GoraKey

Used in order to define the datum key for the operations need it.

Usage examples

Create Schema (XML DSL):

<setHeader name="GoraOperation">
 <constant>CreateSchema</constant>
</setHeader>
<to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

SchemaExists (XML DSL):

<setHeader name="GoraOperation">
 <constant>SchemaExists</constant>
</setHeader>
 <to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Put (XML DSL):

<setHeader name="GoraOperation">
 <constant>put</constant>
</setHeader>
<setHeader name="GoraKey">
 <constant>22222</constant>
</setHeader>
<to uri="gora:foo?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Get (XML DSL):

<setHeader name="GoraOperation">
 <constant>GET</constant>
</setHeader>
<setHeader name="GoraKey">
 <constant>10101</constant>
</setHeader>
<to uri="gora:bar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Delete (XML DSL):

<setHeader name="GoraOperation">
 <constant>DELETE</constant>
</setHeader>
<setHeader name="GoraKey">
 <constant>22222</constant>
</setHeader>
<to uri="gora:bar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Query (XML DSL):

<to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

The full usage examples in the form of integration tests can be found at camel-gora-examples repository.

More resources

For more please information and in depth configuration refer to the Apache Gora Documentation and the Apache Gora Tutorial.

Spring Boot Auto-Configuration

When using gora with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-gora-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

The component supports 4 options, which are listed below.

Name Description Default Type

camel.component.gora.autowired-enabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

Boolean

camel.component.gora.bridge-error-handler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

Boolean

camel.component.gora.enabled

Whether to enable auto configuration of the gora component. This is enabled by default.

Boolean

camel.component.gora.lazy-start-producer

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

Boolean