Artifact IDs

The artifactId for each of the Milyn components is as follows:

• Smooks Core: "milyn-smooks-core"
• Smooks Cartridges:
  • CSV: "milyn-smooks-csv"
  • EDI: "milyn-smooks-edi"
  • Javabean: "milyn-smooks-javabean"
  • JSON: "milyn-smooks-json"
  • Routing: "milyn-smooks-routing"
  • Templating: "milyn-smooks-templating"
  • CSS: "milyn-smooks-css"
  • Servlet: "milyn-smooks-servlet"
• Commons: "milyn-commons" (all components depend on Commons, so no need to specify a dependency on this module if your dependent on one of the others)
• EdiSax: "milyn-edisax"
• Magger: "milyn-magger"
• Tinak: "milyn-tinak"

Note: All cartridges depend on Smooks Core (i.e. "milyn-smooks-core"). Therefore, if your project depends on one of the cartridges, there's no need to specify the dependency on Smooks Core.

See the POMs in the Tutorials as examples of Maven based applications that depend on different Smooks Cartridges.

Smooks v1.1+

The Maven repo groupId for all Milyn components (Smooks v1.1+) is "org.milyn".

Smooks v1.1 is not released yet, but SNAPSHOTs are available from "http://snapshots.repository.codehaus.org". You need to list this Maven repo in your project POM e.g.

<repositories>
    .....
    <repository>
        <id>codehaus.m2.snapshots</id>
        <url>http://snapshots.repository.codehaus.org</url>
        <releases>
            <enabled>false</enabled>
        </releases>
    </repository>
</repositories>

Once Smooks v1.1 is released, the binaries will be available from the maven2 repo on ibiblio.

Smooks v1.0/v1.0.x

The Maven repo groupId for all Milyn components (pre Smooks v1.1) is "milyn".

Also note that the v1.0 and v1.0.1 binaries are not available from ibiblio. You need to list this Maven repo in your project POM e.g.

<repositories>
    .....
    <repository>
        <id>codehaus</id>
        <url>http://dist.codehaus.org</url>
    </repository>
</repositories>

Ant Users

See Getting Smooks Binaries from the Maven2 Repository Using Ant.

XSL Template Driven Transformations

Smooks allows you to perform fragment based XML to XML transformations via the XslContentHandlerFactory, which adds support for "xsl" resource types into Smooks.

As with everything in Smooks, your XSL Stylesheet is defined as a <resource-config> with the resource being either an inlined XSLT, or a URI reference to an external XSL Stylesheet. The configuration of an XSL Stylesheet resource is as follows:

 <resource-config selector="target-element">
     <resource type="xsl">XSL Resource - Inline or URI</resource>

     <!-- (Optional) The action to be applied on the template content. Should the content
          generated by the template:
          1. replace ("replace") the target element, or
          2. be added to ("addto") the target element, or
          3. be inserted before ("insertbefore") the target element, or
          4. be inserted after ("insertafter") the target element.
          5. be bound to ("bindto") an ExecutionContext variable named by the "bindId" param.
          Default "replace".-->
     <param name="action">replace/addto/insertbefore/insertafter/bindto</param>

     <!-- (Optional) Is this XSL template resource a complete XSL template, or just a "Templatelet".
          Only relevant for inlined XSL resources.  URI based resource are always assumed to NOT be templatelets.
          Default "false" (for inline resources).-->
     <param name="is-xslt-templatelet">true/false</param>

     <!-- (Optional) Should the template be applied before (true) or
             after (false) Smooks visits the child elements of the target element.
             Default "false".-->
     <param name="applyTemplateBefore">true/false</param>

     <!-- (Optional) The name of the OutputStreamResource
             to which the result should be written. If set, the "action" param is ignored. -->
     <param name="outputStreamResource">xyzResource</param>

     <!-- (Optional) Template encoding.
          Default "UTF-8".-->
     <param name="encoding">encoding</param>

     <!-- (Optional) bindId when "action" is "bindto".
     <param name="bindId">xxxx</param>

     <!-- (Optional) Fail on XSL Transformer Warning.
          Default "true".-->
     <param name="failOnWarning">false</param> <!-- Default "true" -->

 </resource-config>

By default, XSLTs are applied to a fragment after Smooks has already processed the child content of that fragment, with the default action being that of replacing the processed fragment with the result of the templating operation. Both of these behaviors can be overridden by configuration parameters (see configuration above).

Another action of note is the "bindto" action, which allows you to bind the templating result to Smooks ExecutionContext under the key specified in the "bindId" <param>. This action is available in support of fragment routing via the Routing Cartridge.

<!-- /example/BasicJavaTransformer.xsl -->
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
				version="1.0">

    <xsl:output method="xml" encoding="UTF-8" />

    <xsl:template match="b">
        <xxx></xxx>
    </xsl:template>

</xsl:stylesheet>

<?xml version="1.0"?>
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">

    <resource-config selector="c b">
	<resource>/example/BasicXslTransform.xsl</resource>
        <param name="is-xslt-templatelet">false</param>
    </resource-config>

</smooks-resource-list>

<a>
    <b>
        <c>
            <b></b>
        </c>
    </b>
</a>

<a>
    <b>
        <c>
            <xxx></xxx>
        </c>
    </b>
</a>

// Instantiate Smooks with the config...
Smooks smooks = new Smooks("smooks-config.xml");

// Filter the input message to the outputWriter...
smooks.filter(new StreamSource(messageInStream), new StreamResult(messageOutStream));

FreeMarker Template Driven Transformations

The FreeMarker templating engine, in conjunction with the Java Binding capabilities of the Javabean Cartridge, can be used to transform messages into a range of different formats (XML, EDI, CSV etc). Basically, this works by merging a Java Object Model (POJO) with a FreeMarker template. This has the effect of making your transforms very clear to understand and visualize and would be our preferred method of Template based transforms.

Lets walk though an example to show what we mean. We begin with our Java Object Model, our POJO, which simulates a PatientRecord (getters and setters are left out):

 public class PatientRecord implement Serializable{
     private String name;
}

Next we look at our FreeMarker template (saved in a file named recordTemplate.ftl) :

<record>
    <firstName>${patient.name}</firstname>
     ...
</record>

Here, "patient" refers to a beanId previously bound in the ExcecutionContext. This could have been bound by a previous transformation or been set as the JavaSource input like this:

//    create the input for the transformation
Map<String, Object> beanMap = new HashMap<String, Object>();
JavaSource source = new JavaSource();
source.setBeans( beanMap );
beanMap.put( "patient", patientRecord );

//    create tranformation ouput result
StreamResult result = new StreamResult( new StringWriter() );

//    transform
smooks.filter( source, result, executionContext );

The next step is to create a Smooks configuration:

<?xml version="1.0"?>
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">

    <resource-config selector="global-parameters">
        <param name="stream.filter.type">SAX</param>
    </resource-config>

    <resource-config selector="$document">
        <resource>recordTemplate.ftl</resource>
    </resource-config>

</smooks-resource-list>

In the example above, the Java objects used in the FreeMarker template are manually injected into the Smooks Execution context (via the BeanAccessor class). While this is a valid usecase, it's not expected to be the normal usage pattern.

In most cases, the Java Object model should be populated from the Source message using the Javabean Cartridge using its Java Binding capabilities. Used in this way, the Javabean Cartridge is used to populate data from the source message into a Java Object model, which is then applied to a FreeMarker template to produce output in the result (or to a OutputStreamResource - see Routing). We refer to this method of transformation as "Model Driven Transformation" and a good example of this approach can be seen in the model-driven-basic tutorial.

One downside to the Model Driven Transformation approach can be the requirement to create an Java Object Model. To help here, the Smooks Javabean Cartridge has support for what we call "Virtual Object Models", whereby normal java.util.Map implementations can be used to fill the roll of a node in the model hierarchy. An example of this approach can be seen in the model-driven-basic-virtual tutorial.

Java or Groovy Transformations

Smooks always allows you to hook in low level logic, targeting specific events/fragments in the source data stream. This can be really useful for working around situations not covered out of the box by the Smooks Cartridges.

For more on this please look at the Tutorials.

Data binding

Because you probably want something more than an empty Java Object Model, we will now discuss how to get the data from the XML into your Java Objects. The BeanPopulator resource configurations provides the bindings parameter for this purpose. The bindings parameter accepts binding nodes. A binding node defines the binding between an XML node or attribute and a property of the bean.

The binding node configuration attributes are as follows:

The following example shows how three node values are bound with three bean properties:

<resource-config selector="order-item">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItem</param>
    <param name="beanClass">example.model.OrderItem</param>
    <param name="bindings">
        <binding property="productId" type="Long" selector="order-item/product" />
        <binding property="quantity" type="Integer" selector="order-item/quantity" />
        <binding property="price" type="Double" selector="order-item/price" />
    </param>
</resource-config>

The following things happen in the example above:

1. When the Smooks filter encounters the start of an order-item element in the source XML (on the visitBefore event), it creates an instance of the example.model.OrderItem class and puts it in the bean map under the beanId orderItem.
2. When the Smooks filter encounters the end of an order-item/product element in the source XML (on the visitAfter event), the text data in the element is decoded into a Long. The OrderItem bean under the beanId orderItem is retrieved from the beanMap. The OrderItem#setProductId(Long) is called on the bean with the decoded Long value as the argument.
3. When the Smooks filter encounters the end of an order-item/quantity element in the source XML (on the visitAfter event), the text data in the element is decoded into an Integer. The OrderItem bean under the beanId orderItem is retrieved from the beanMap. The OrderItem#setQuantity(Integer) is called on the bean with the decoded Integer value as argument.
4. When the Smooks filter encounters the end of an order-item/price element in the source XML (on the visitAfter event), the text data in the element is decoded into a Double. The OrderItem bean under the beanId orderItem is retrieved from the beanMap. The OrderItem#setPrice(Double) is called on the bean with the decoded Double value as argument.

<resource-config selector="header">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanClass">org.milyn.javabean.Header</param>
    <param name="bindings">
        <binding property="date" type="OrderDateLong" selector="header/date"/>
        <binding property="customerNumber" type="Long" selector="header/customer/@number" />
        <binding property="customerName" selector="header/customer" />
    </param>
</resource-config>

<resource-config selector="decoder:OrderDateLong">
    <resource>org.milyn.javabean.decoders.DateDecoder</resource>
    <param name="format">EEE MMM dd HH:mm:ss z yyyy</param>
    <param name="locale-language">en</param>
    <param name="locale-country">IE</param>
</resource-config>

Javabean Wiring

In the previous chapters we looked at how to convert some XML data to a single Javabean Object. But you probably want something more then one single big object. This chapter will look at how you can create a complete object model using the Javabean Cartridge using  javabean wiring.

Javabean Wiring makes it possible to reference beans from the bean map to each other. This is done via a special selector in the binding configuration. By using the ${beanId} notation, the cartridge knows that you want to wire another bean, from the bean map, to that property. At the moment that the visitor visits the selected node, the bean that you want to wire, can or may not already exist. If the target  bean doesn't exist then the target bean will be wired as soon as it gets created later on in the filter process. This is called late wiring.

Here is an example of bean wiring:

<!-- The normal bindings are left out in this example -->
<resource-config selector="order">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">order</param>
    <param name="beanClass">example.model.Order</param>
    <param name="bindings">
        <binding property="header" selector="${header}" />
    </param>
</resource-config>

<resource-config selector="header">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">header</param>
    <param name="beanClass">example.model.Header</param>
    <param name="bindings">
        <binding property="order" selector="${order}" />
    </param>
</resource-config>

In this example the first resource configuration has one binding with the selector configuration ${header}. That configuration means that it wants to wire another bean with the beanId header. The second resource configuration has the header beanId. That resource configuration has a binding of it's own but for the order beanId of the previous configuration. You can guess what the end result will be. The Order object has a reference to the Header object and the Header object has a reference to the Order object.

In the case when the bean of the targeted beanId already exists in the bean map then the setter method of the target property will only be called once. But in the case of late wiring then it is possible that the setter method will be called multiple times. Because every time the bean with the beanId is created, the javabean cartridge will call the setter method. This makes it possible to call some 'add object to collection' method. The following example shows this situation:

<resource-config selector="order-items">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItemList</param>
    <param name="beanClass">example.model.OrderItemList</param>
    <param name="bindings">
        <binding setterMethod="addOrderItem" selector="${orderItem}" />
    </param>
</resource-config>

<resource-config selector="order-item">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItem</param>
    <param name="beanClass">example.model.OrderItem</param>
    <param name="bindings">
	<!-- bindings -->
    </param>
</resource-config>

Everytime that the <order-item> node is visited, the created example.model.OrderItem object will be added to the example.model.OrderItemList by calling the OrderItemList#addOrderItem(OrderItem) method.

Bean wiring doesn't only work for nested XML structures like this:

<root>
   <a>
      <b/>
      <b/>
   </a>
   <a>
      <b/>
      <b/>
   </a>
</root>

but also for flat XML structures like this:

<root>
   <a/>
   <b/>
   <b/>

   <a/>
   <b/>
   <b/>
</root>

 

You don't have to worry the the second two <b> elements of previous example get wired to the first <a> node. The cartridge makes sure that the first <a> node resource configuration stops wiring new B objects as soon as the new A object, of the second <a> element, gets added to the bean map. If you want to know more about how this works then take a look in the Smooks Developer Guide in the  Javabean wiring internalssection.

With a javabean wiring binding the rules for the other binding configurations are a bit different:

<resource-config selector="order-items">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItemList</param>
    <param name="beanClass">java.util.ArrayList</param>
    <param name="bindings">
        <binding selector="${orderItem}" />
    </param>
</resource-config>

<resource-config selector="order-item">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItem</param>
    <param name="beanClass">example.model.OrderItem</param>
    <param name="bindings">
	<!-- bindings -->
    </param>
</resource-config>

<resource-config selector="order-items">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItemList</param>
    <param name="beanClass">example.model.OrderItem[]</param>
    <param name="bindings">
        <binding selector="${orderItem}" />
    </param>
</resource-config>

<resource-config selector="order-item">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItem</param>
    <param name="beanClass">example.model.OrderItem</param>
    <param name="bindings">
	<!-- bindings -->
    </param>
</resource-config>

<resource-config selector="order-item">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItemMap</param>
    <param name="beanClass">java.util.HashMap</param>
    <param name="bindings">
        <binding property="productId" type="Long" selector="order-item product" />
        <binding property="quantity" type="Integer" selector="order-item quantity" />
        <binding property="price" type="Double" selector="order-item price" />
    </param>
</resource-config>

<root>
   <property name="key1">value1</property>
   <property name="key2">value2</property>
   <property name="key3">value3</property>
</root>

<resource-config selector="a">
   <resource>org.milyn.javabean.BeanPopulator</resource>
   <param name="beanClass">java.util.HashMap</param>
   <param name="bindings">
      <binding property="@name" selector="property" />
   </param>
</resource-config>

<resource-config selector="order">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">order</param>
    <param name="beanClass">java.util.HashMap</param>
    <param name="bindings">
        <binding property="header" selector="${header}" />
        <binding property="orderItems" selector="${orderItems}" />
    </param>
</resource-config>

<resource-config selector="header">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">header</param>
    <param name="beanClass">java.util.HashMap</param>
    <param name="bindings">
	<binding property="order" selector="${order}" />
        <binding property="date" type="OrderDateLong" selector="header date" />
        <binding property="customerNumber" type="Integer" selector="header customer @number" />
        <binding property="customerName" type="String" selector="header customer" />
    </param>
</resource-config>

 <resource-config selector="order-items">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItems</param>
    <param name="beanClass">java.util.ArrayList</param>
    <param name="bindings">
        <binding selector="${orderItem}" />
    </param>
</resource-config>

<resource-config selector="order-item">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">orderItem</param>
    <param name="beanClass">java.util.HashMap</param>
    <param name="bindings">
        <binding property="order" selector="${order}" />
        <binding property="productId" type="Long" selector="order-item product" />
        <binding property="quantity" type="Integer" selector="order-item quantity" />
        <binding property="price" type="Double" selector="order-item price" />
    </param>
</resource-config>

Expression Based Bindings

Sometimes your Source message can represent the underlying data differently to the Java Object Model to which you are binding e.g. your Source Order message may contain "quantity" and "price", while your Target model represents this data as "total" (i.e. price * quantity).

You can handle this usecase in Smooks by using an "Expression Based Binding". This feature allows you to set the binding value based on the result of an expression evaluation (on the bean context). Therefore, it can be used to perform all sorts of complex binding operations, including the one outlined above.

The following is an example of a config to merge field values together. Imagine you have a message like this:

<message>
    <time>17:45</time>
    <day>25</day>
    <month>12</month>
    <year>1999</year>
</message>

And you need to merge the date field values together and decode the merged value into the "date" (type: java.util.Date) property on your target bean. Here's the config snippet...

<!-- Concat and decode the date field values into the "date" property of the Message bean... -->
<resource-config selector="message">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">message</param>
    <param name="beanClass">org.milyn.javabean.expressionbinding.Message</param>
    <param name="bindings">
        <binding property="date" type="MessageDate">
            messageDate.time + " " + messageDate.day + "/" + messageDate.month + "/" + messageDate.year
        </binding>
    </param>
</resource-config>

<!-- Capture the date field values into a Map.  Used above... -->
<resource-config selector="message">
    <resource>org.milyn.javabean.BeanPopulator</resource>
    <param name="beanId">messageDate</param>
    <param name="beanClass">java.util.HashMap</param>
    <param name="bindings">
        <binding property="time" selector="message/time" />
        <binding property="day" selector="message/day" />
        <binding property="month" selector="message/month" />
        <binding property="year" selector="message/year" />
    </param>
</resource-config>

<!-- Date decoder... -->
<resource-config selector="decoder:MessageDate">
    <resource>org.milyn.javabean.decoders.DateDecoder</resource>
    <param name="format">HH:mm dd/MM/yyyy</param>
</resource-config>

Merging Multiple Data Entities Into a Single Binding

This can be achieved using Expression Based Bindings (previous section).

Generating the Smooks Binding Configuration

The Javabean Cartridge contains the org.milyn.javabean.gen.ConfigGenerator utility class that can be used to generate a binding configuration template. This template can then be used as the basis for defining a binding.

From the commandline:

$JAVA_HOME/bin/java -classpath <classpath> org.milyn.javabean.gen.ConfigGenerator -c <rootBeanClass> -o <outputFilePath> [-p <propertiesFilePath>]

• The "-c" commandline arg specifies the root class of the model whose binding config is to be generated.
• The "-o" commandline arg specifies the path and filename for the generated config output.
• The "-p" commandline arg specifies the path and filename optional binding configuration file that specifies aditional binding parameters.

The optional "-p" properties file parameter allows specification of additional config parameters:

• packages.included: Semi-colon seperated list of packages scoping classes to be included in the binding generation.
• packages.excluded: Semi-colon seperated list of packages scoping classes to be excluded in the binding generation.

After running this utility against the target class, you typically need to perform the following follow-up tasks in order to make the binding configuration work for your Source data model.

1. For each <resource-config>, set the selector to the event element against which the bean instance should be created i.e. on which the beans lifecycle should be associated.
2. Update the selector on each <binding> to select the event element/attribute supplying the binding data for that bean property.
3. Check the type attribute on the bindings. Not all will be set; depending on the actual property type. These must be configured by hand e.g. you may need to configure a custom decoder for the field (e.g. for date fields).
4. Double check that <binding> config template elements have been added for all relevant properties. Check this against the actual code.

Determining the selector values can sometimes be difficult, especially for non XML Sources (Java etc). The Html Reporting tool can be a great help here because it helps you visualise the input message model (against which the selectors will be applied) as seen by Smooks. So, first off, generate a report using your Source data, but with an empty transformation configuration. In the report, you can see the model against which you need to add your configurations. Add the configurations one at a time, rerunning the report to check they are being applied.

Source and Target Object Models

The required mappings from the source to target Object models are as follows:

Source Model Event Stream

Using the Html Reporting tool we can see that the SAX Event Stream produced by the source Object Model is as follows:

<example.srcmodel.Order>
    <header>
        <customerNumber>
        </customerNumber>
        <customerName>
        </customerName>
    </header>
    <orderItems>
        <example.srcmodel.OrderItem>
            <productId>
            </productId>
            <quantity>
            </quantity>
            <price>
            </price>
        </example.srcmodel.OrderItem>
    </orderItems>
</example.srcmodel.Order>

So we need to target the Smooks Javabean resources at this event stream. This is shown in the Smooks Configuration.

Smooks Configuration

The Smooks configuration for performing this transform ("smooks-config.xml") is as follows (see the Source Model Event Stream above):

<?xml version="1.0"?>
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">

    <resource-config selector="global-parameters">
        <param name="stream.filter.type">SAX</param>
    </resource-config>

    <resource-config selector="example.srcmodel.Order">
        <resource>org.milyn.javabean.BeanPopulator</resource>
        <param name="beanId">lineOrder</param>
        <param name="beanClass">example.trgmodel.LineOrder</param>
        <param name="bindings">
            <binding property="customerId" selector="header/customerNumber" />
            <binding property="customerName" selector="header/customerName" />
            <binding property="lineItems" selector="${lineItems}" />
        </param>
    </resource-config>

    <resource-config selector="orderItems">
        <resource>org.milyn.javabean.BeanPopulator</resource>
        <param name="beanId">lineItems</param>
        <param name="beanClass">example.trgmodel.LineItem[]</param>
        <param name="bindings">
            <binding selector="${lineItem}" />
        </param>
    </resource-config>

    <resource-config selector="example.srcmodel.OrderItem">
        <resource>org.milyn.javabean.BeanPopulator</resource>
        <param name="beanId">lineItem</param>
        <param name="beanClass">example.trgmodel.LineItem</param>
        <param name="bindings">
            <binding property="productCode" selector="example.srcmodel.OrderItem/productId" />
            <binding property="unitQuantity" type="Integer" selector="example.srcmodel.OrderItem/quantity" />
            <binding property="unitPrice" type="BigDecimal" selector="example.srcmodel.OrderItem/price" />
        </param>
    </resource-config>

</smooks-resource-list>

Smooks Execution

The source object model is provided to Smooks via a org.milyn.delivery.JavaSource Object. This object is created by passing the constructor the root object of the source model. The resulting JavaSource object is used in the Smooks#filter method. The resulting code could look like as follows:

protected LineOrder runSmooksTransform(Order srcOrder) throws IOException, SAXException {
    Smooks smooks = new Smooks("smooks-config.xml");
    ExecutionContext executionContext = smooks.createExecutionContext();

    // Transform the source Order to the target LineOrder via a
    // JavaSource and JavaResult instance...
    JavaSource source = new JavaSource(srcOrder);
    JavaResult result = new JavaResult();

    // Configure the execution context to generate a report...
    executionContext.setEventListener(new HtmlReportGenerator("target/report/report.html"));

    smooks.filter(source, result, executionContext);

    return (LineOrder) result.getBean("lineOrder");
}

One-to-One Transformation

If the requirement is to process a huge message by transforming it into a single message of another format, the easiest mechanism with Smooks is to apply multiple templating operations (using XSL, FreeMarker etc) to the Source message Event Stream, outputing to the Result stream.

The following diagram tries to illustrate the process involved when using FreeMarker as the templating technology.  The assumption here is that you're familiar with the Javabean Cartridge and how it can be used to populate the Object Models used in the FreeMarker templates (including how to define Virtual Object Models).

So in this example, you use the Source events up to the end of the header to gather <header> information from the source message and populate a Virtual Object Model, which is then used in the FreeMarker template that's triggered by the <header> element's visitAfter event.  The same basic process is utilized for transforming the <order-items>.  At the end we need to apply a static template to complete the Result message.

This approach to performing a One-to-One Transformation of a huge message works simply because the only objects in memory at any one time are the order header details and the current <order-item> details (in the Virtual Object Model).  Obviously it can't work if the transformation is so obscure as to always require full access to all the data in the source message e.g. if the messages needs to have all the order items reversed in order (or sorted).  In such a case however, you do have the option of routing the order details and items to a database and then using the database's storage, query and paging features to perform the transformation.

Splitting & Routing

Another common approach to processing large/huge messages is to split them out into smaller messages that can be processed independently. Of course Splitting and Routing is not just a solution for processing huge messages. It's often needed with smaller messages too (message size may be irrelevant) where, for example, order items in an an order message need to be split out and routed (based on content - "Content Base Routing") to different departments or partners for processing. Under these conditions, the message formats required at the different destinations may also vary e.g.

• "destination1" required XML via the file system,
• "destination2" requires Java objects via a JMS Queue,
• "destination3" picks the messages up from a table in a Database etc.
• "destination4" requires EDI messages via a JMS Queue,
• etc etc

With Smooks v1.0, all of the above is possible. You can perform multiple splitting and routing operations to multiple destinations (of different types) in a single pass over a message.

The key to processing huge messages is to make sure that you always maintain a small memory footprint. You can do this using the Javabean Cartridge by making sure you're only binding the most relevant message data (into the bean context) at any one time. In the following sections, the examples are all based on splitting and routing of order-items out of an order message. The solutions shown all work for huge messages because the Smooks Javabean Cartridge binding configurations are implemented such that the only data held in memory at any given time is the main order details (order header etc) and the "current" order item details.

Complex splitting operations are supported through use of the Javabean Cartridge to extract the data for the split-message. In this way, you can extract and recombine data from across different sub-hierarchies of the Source message, to produce the split messages. It also means you can (through the use of templating) easily generate the split messages in a range of different formats. More on this later.

Routing to File

The example outlined in this section illustrates how you can combine the following Smooks functionality to split a message out into smaller messages on the file system.

1. The Javabean Cartridge for extracting data from the message and holding it in variables in the bean context.
2. The FileOutputStreamResource for managing file system streams (naming, opening, closing, throttling creation etc).
3. FreeMarker Templating for generating the individual split messages from data bound in the bean context, and written to the FileOutputStreamResource.

In the example, we want to process a huge order message and route the individual order item details to file. The following illustrates what we want to achieve. As you can see, the split messages don't just contain data from the order item fragments. They also contain data from the order header and root elements.

To achieve this with Smooks, we assemble the following solution.

Smooks Resource configuration #1 and #2 define the Java Bindings for extracting the order header information (config #1) and the order-item information (config #2). This is the key to processing a huge message; making sure that we only have the current order item in memory at any one time. The Smooks Javabean Cartridge manages all this for you, creating and recreating the orderItem beans as the <order-item> fragments are being processed.

Smooks Resource configuration #3 manages the generation of the files on the file system. As you can see from the configuration, the file names can be dynamically constructed from data in the bean context. You can also see that it can throttle the creation of the files via the "highWaterMark" configuration parameter. This helps you manage file creation so as not to overwhelm the target file system.

Smooks Resource configuration #4 defines the FreeMarker templating resource used to write the split messages to the OutputStream created by the FileOutputStreamResource (config #3). See how config #4 references the FileOutputStreamResource resource.

<resource-config selector="order-item">
    <resource>org.milyn.routing.jms.JMSRouter</resource>
    <param name="beanId">orderItem</param>
    <param name="destinationName">/queue/smooksRouterQueue</param>
    <param name="correlationIdPattern">${order.orderId}-${order.orderItem.itemId}</param>
    <param name="highWaterMark">50</param>
</resource-config>

Routing to a Database

Routing to a Database is also quite easy. Please read the "Routing to File" section above before reading this section.

So we take the same scenario as with the File Routing example above, but this time we want to route the order and order item data to a Database. This is what we want to achieve:

To achieve this with Smooks, we assemble the following solution.

The main points of note with this example are:

1. How to configure a DataSource resource (config #1).
2. How to reference and use the DataSource resource from the SQLExecutor resource configurations (configs #2, #3 and #4).

For more on the SQLExecutor, please read the Persistence section.

Check out the db-extract-transform-load example.

Configuration Example

<resource-config selector="customer-details">
        <resource>org.milyn.routing.db.SQLExecutor</resource>
        <param name="executeBefore">true</param>
        <param name="datasource">DBExtractTransformLoadDS</param>
        <param name="statement">select ORDERNUMBER from ORDERS where ORDERNUMBER = ${order.orderNum}</param>
        <param name="resultSetName">orderExistsRS</param>
</resource-config>