Gora :: Apache Camel

Apache Gora Overview

The Apache Gora open source framework provides an in-memory data model and persistence for big data. Gora supports persisting to column stores, key value stores, document stores and RDBMSs, and analyzing the data with extensive Apache Hadoop™ MapReduce support. Gora uses the Apache Software License v2.0 and graduated from the Apache Incubator in Janauary 2012 to become a top-level Apache project.

Apache Gora currently supports the following datastores: Apache HBase, Apache Cassandra, Apache Accumulo, Amazon DynamoDB and SQL databases such as hsqldb, MySQL and more.

URI format

gora:instanceName[?options]

Hbase examples with mandatory options :

XML

<to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Java DSL

to("gora:foobar?keyClass=java.lang.Long&valueClass=org.apache.camel.component.gora.generated.Pageview&dataStoreClass=org.apache.gora.hbase.store.HBaseStore")

Configuratiion

Using camel-gora needs some configuration. This mainly involve to configure the AvroStore through the gora.properties file and to define the relevant mappings as part of the gora-core module.

Extensive information for this configuration can be found in the apache gora documentation and the gora-conf page.

Options

The Gora component supports 3 options, which are listed below.

Name	Description	Default	Type
bridgeErrorHandler (consumer)	Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.	false	boolean
lazyStartProducer (producer)	Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.	false	boolean
autowiredEnabled (advanced)	Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.	true	boolean

Name

Description

Default

Type

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

boolean

autowiredEnabled (advanced)

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

boolean

The Gora endpoint is configured using URI syntax:

gora:name

with the following path and query parameters:

Path Parameters (1 parameters):

Name	Description	Default	Type
name	Required Instance name		String

Name

Description

Default

Type

name

Required Instance name

String

Query Parameters (22 parameters):

Name	Description	Default	Type
dataStoreClass (common)	The type of the dataStore		String
keyClass (common)	The type class of the key		String
valueClass (common)	The type of the value		String
bridgeErrorHandler (consumer)	Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.	false	boolean
concurrentConsumers (consumer)	Number of concurrent consumers	1	int
endKey (consumer)	The End Key		Object
endTime (consumer)	The End Time		long
fields (consumer)	The Fields		Strings
keyRangeFrom (consumer)	The Key Range From		Object
keyRangeTo (consumer)	The Key Range To		Object
limit (consumer)	The Limit		long
startKey (consumer)	The Start Key		Object
startTime (consumer)	The Start Time		long
timeRangeFrom (consumer)	The Time Range From		long
timeRangeTo (consumer)	The Time Range To		long
timestamp (consumer)	The Timestamp		long
exceptionHandler (consumer)	To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.		ExceptionHandler
exchangePattern (consumer)	Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut		ExchangePattern
flushOnEveryOperation (producer)	Flush on every operation	true	boolean
lazyStartProducer (producer)	Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.	false	boolean
hadoopConfiguration (advanced)	Hadoop Configuration		Configuration
synchronous (advanced)	Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).	false	boolean

Name

Description

Default

Type

dataStoreClass (common)

The type of the dataStore

String

keyClass (common)

The type class of the key

String

valueClass (common)

The type of the value

String

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

concurrentConsumers (consumer)

Number of concurrent consumers

1

int

endKey (consumer)

The End Key

Object

endTime (consumer)

The End Time

long

fields (consumer)

The Fields

Strings

keyRangeFrom (consumer)

The Key Range From

Object

keyRangeTo (consumer)

The Key Range To

Object

limit (consumer)

The Limit

long

startKey (consumer)

The Start Key

Object

startTime (consumer)

The Start Time

long

timeRangeFrom (consumer)

The Time Range From

long

timeRangeTo (consumer)

The Time Range To

long

timestamp (consumer)

The Timestamp

long

exceptionHandler (consumer)

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

ExceptionHandler

exchangePattern (consumer)

Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut

ExchangePattern

flushOnEveryOperation (producer)

Flush on every operation

true

boolean

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

boolean

hadoopConfiguration (advanced)

Hadoop Configuration

Configuration

synchronous (advanced)

Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).

false

boolean

Supported Gora Operations

Supported operations include : put, get, delete, getSchemaName, deleteSchema, createSchema, query, deleteByQuery, schemaExists.

Some of the operations require arguments while some others no. The arguments to operations could be either the body of the in message or defined in a header property. Below there is a list with some additional info for each operation.

Property	Description
put	Inserts the persistent object with the given key.
get	Returns the object corresponding to the given key fetching all the fields.
delete	Deletes the object with the given key.
getSchemaName	Returns the schema name given to this DataStore.
deleteSchema	Deletes the underlying schema or table (or similar) in the datastore that holds the objects.
createSchema	Creates the optional schema or table (or similar) in the datastore to hold the objects.
query	Executes the given query and returns the results.
deleteByQuery	Deletes all the objects matching the query.
schemaExists	Returns whether the schema that holds the data exists in the datastore.

Property

Description

put

Inserts the persistent object with the given key.

get

Returns the object corresponding to the given key fetching all the fields.

delete

Deletes the object with the given key.

getSchemaName

Returns the schema name given to this DataStore.

deleteSchema

Deletes the underlying schema or table (or similar) in the datastore that holds the objects.

createSchema

Creates the optional schema or table (or similar) in the datastore to hold the objects.

query

Executes the given query and returns the results.

deleteByQuery

Deletes all the objects matching the query.

schemaExists

Returns whether the schema that holds the data exists in the datastore.

Gora Headers

Property	Description
GoraOperation	Used in order to define the operation to execute.
GoraKey	Used in order to define the datum key for the operations need it.

Property

Description

GoraOperation

Used in order to define the operation to execute.

GoraKey

Used in order to define the datum key for the operations need it.

Usage examples

Create Schema (XML DSL):

<setHeader name="GoraOperation">
 <constant>CreateSchema</constant>
</setHeader>
<to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

SchemaExists (XML DSL):

<setHeader name="GoraOperation">
 <constant>SchemaExists</constant>
</setHeader>
 <to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Put (XML DSL):

<setHeader name="GoraOperation">
 <constant>put</constant>
</setHeader>
<setHeader name="GoraKey">
 <constant>22222</constant>
</setHeader>
<to uri="gora:foo?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Get (XML DSL):

<setHeader name="GoraOperation">
 <constant>GET</constant>
</setHeader>
<setHeader name="GoraKey">
 <constant>10101</constant>
</setHeader>
<to uri="gora:bar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Delete (XML DSL):

<setHeader name="GoraOperation">
 <constant>DELETE</constant>
</setHeader>
<setHeader name="GoraKey">
 <constant>22222</constant>
</setHeader>
<to uri="gora:bar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

Query (XML DSL):

<to uri="gora:foobar?keyClass=java.lang.Long&amp;valueClass=org.apache.camel.component.gora.generated.Pageview&amp;dataStoreClass=org.apache.gora.hbase.store.HBaseStore"/>

The full usage examples in the form of integration tests can be found at camel-gora-examples repository.

More resources

For more please information and in depth configuration refer to the Apache Gora Documentation and the Apache Gora Tutorial.

Spring Boot Auto-Configuration

When using gora with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-gora-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

The component supports 4 options, which are listed below.

Name	Description	Default	Type
camel.component.gora.autowired-enabled	Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.	true	Boolean
camel.component.gora.bridge-error-handler	Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.	false	Boolean
camel.component.gora.enabled	Whether to enable auto configuration of the gora component. This is enabled by default.		Boolean
camel.component.gora.lazy-start-producer	Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.	false	Boolean

Name

Description

Default

Type

camel.component.gora.autowired-enabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

Boolean

camel.component.gora.bridge-error-handler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

Boolean

camel.component.gora.enabled

Whether to enable auto configuration of the gora component. This is enabled by default.

Boolean

camel.component.gora.lazy-start-producer

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

Boolean