Path expressions consist of two parts: a selection criteria (or an input path) and an output path:

  inputPath => outputPath 

The inputPath part defines an expression for the path of a node that is to be sequenced. Input paths consist of '/' separated segments, where each segment represents a pattern for a single node's name (including the same-name-sibling indexes) and '@' signifies a property name.

Let's first look at some simple examples:

  //(*.(jpg|jpeg|gif|bmp|pcx|png)[*])/jcr:content[@jcr:data] => /images/$1 

A sequencer that extracts metadata from JPEG, GIF, BMP, PCX, PNG, IFF, RAS, PBM, PGM, PPM and PSD image files. This sequencer extracts the file format, image resolution, number of bits per pixel and optionally number of images, comments and physical resolution, and then writes this information into the repository using the following structure:

This structure could be extended in the future to add EXIF and IPTC metadata as child nodes. For example, EXIF metadata is structured as tags in directories, where the directories form something like namespaces, and which are used by different camera vendors to store custom metadata. This structure could be mapped with each directory (e.g. "EXIF" or "Nikon Makernote" or "IPTC") as the name of a child node, with the EXIF tags values stored as either properties or child nodes.

To use this sequencer, simply include the dna-sequencer-images JAR in your application and configure the Sequencing Service to use this sequencer using something similar to:

String name = "Image Sequencer";
String desc = "Sequences image files to extract the characteristics of the image";
String classname = "org.jboss.dna.sequencer.images.ImageMetadataSequencer";
String[] classpath = null; // Use the current classpath
String[] pathExpressions = {"//(*.(jpg|jpeg|gif|bmp|pcx|png|iff|ras|pbm|pgm|ppm|psd)[*])/jcr:content[@jcr:data] => /images/$1"};
SequencerConfig sequencerConfig = new SequencerConfig(name, desc, classname, 
                                                      classpath, pathExpressions);
sequencingService.addSequencer(sequencerConfig);

This sequencer is included in JBoss DNA and processes Microsoft Office documents, including Excel spreadsheets and PowerPoint presentations. With presentations, the sequencer extracts the slides, titles, text and slide thumbnails. With spreadsheets, the sequencer extracts the names of the sheets. And, the sequencer extracts for all the files the general file information, including the name of the author, title, keywords, subject, comments, and various dates.

To use this sequencer, simply include the dna-sequencer-msoffice JAR and all of the POI JARs in your application and configure the Sequencing Service to use this sequencer using something similar to:

String name = "Microsoft Office Document Sequencer";
String desc = "Sequences MS Office documents, including spreadsheets and presentations";
String classname = "org.jboss.dna.sequencer.msoffice.MSOfficeMetadataSequencer";
String[] classpath = null; // Use the current classpath
String[] pathExpressions = {"//(*.(doc|docx|ppt|pps|xls)[*])/jcr:content[@jcr:data] => /msoffice/$1"};
SequencerConfig sequencerConfig = new SequencerConfig(name, desc, classname, 
                                                      classpath, pathExpressions);
sequencingService.addSequencer(sequencerConfig);

The ZIP file sequencer is included in JBoss DNA and extracts the files and folders contained in the ZIP archive file, extracting the files and folders into the repository using JCR's nt:file and nt:folder node types.

To use this sequencer, simply include the dna-sequencer-zip JAR in your application and configure the Sequencing Service to use this sequencer using something similar to:

String name = "ZIP Sequencer";
String desc = "Sequences ZIP archives to extract the files and folders";
String classname = "org.jboss.dna.sequencer.zip.ZipSequencer";
String[] pathExpressions = {"//(*.zip[*])/jcr:content[@jcr:data] => /zips/$1"};
SequencerConfig sequencerConfig = new SequencerConfig(name, desc, classname, 
                                                      classpath, pathExpressions);
this.sequencingService.addSequencer(sequencerConfig);

One of the sequencers that included in JBoss DNA is the dna-sequencer-java subproject. This sequencer parses Java source code added to the repository and extracts the basic structure of the classes and enumerations defined in the code. This structure includes: the package structures, class declarations, class and member attribute declarations, class and member method declarations with signature (but not implementation logic), enumerations with each enumeration literal value, annotations, and JavaDoc information for all of the above. After extracting this information from the source code, the sequencer then writes this structure into the repository, where it can be further processed, analyzed, searched, navigated, or referenced.

To use this sequencer, simply include the dna-sequencer-java JAR (plus all of the JARs that it is dependent upon) in your application and configure the Sequencing Service to use this sequencer using something similar to:

String name = "Java Sequencer";
String desc = "Sequences java files to extract the characteristics of the Java source";
String classname = "org.jboss.dna.sequencer.java.JavaMetadataSequencer";
String[] classpath = null; // Use the current classpath
String[] pathExpressions = {"//(*.java[*])/jcr:content[@jcr:data] => /java/$1"};
SequencerConfig sequencerConfig = new SequencerConfig(name, desc, classname, 
                                                      classpath, pathExpressions);
this.sequencingService.addSequencer(sequencerConfig);

Another sequencer that is included in JBoss DNA is the dna-sequencer-mp3 sequencer project. This sequencer processes MP3 audio files added to a repository and extracts the ID3 metadata for the file, including the track's title, author, album name, year, and comment. After extracting this information from the audio files, the sequencer then writes this structure into the repository, where it can be further processed, analyzed, searched, navigated, or referenced.

To use this sequencer, simply include the dna-sequencer-mp3 JAR and the JAudioTagger library in your application and configure the Sequencing Service to use this sequencer using something similar to:

String name = "MP3 Sequencer";
String desc = "Sequences MP3 files to extract the ID3 tags of the audio file";
String classname = "org.jboss.dna.sequencer.mp3.Mp3MetadataSequencer";
String[] pathExpressions = {"//(*.mp3[*])/jcr:content[@jcr:data] => /mp3s/$1"};
SequencerConfig sequencerConfig = new SequencerConfig(name, desc, classname, 
                                                      classpath, pathExpressions);
this.sequencingService.addSequencer(sequencerConfig);

This sequencer is incomplete and is not currently usable. The purpose is to sequence JCR Compact Node Definition (CND) files to extract the node definitions with their property definitions, and inserting these into the repository using JCR standard notation.

The first step is to create the Maven 2 project that you can use to compile your code and build the JARs. Maven 2 automates a lot of the work, and since you're already set up to use Maven, using Maven for your project will save you a lot of time and effort. Of course, you don't have to use Maven 2, but then you'll have to get the required libraries and manage the compiling and building process yourself.

You can create your Maven project any way you'd like. For examples, see the Maven 2 documentation. Once you've done that, just add the dependencies in your project's pom.xml dependencies section:

<dependency>

  <groupId>org.jboss.dna</groupId>

  <artifactId>dna-common</artifactId>

  <version>0.1</version>

</dependency>

<dependency>

  <groupId>org.jboss.dna</groupId>

  <artifactId>dna-graph</artifactId>

  <version>0.1</version>

</dependency>

<dependency>

  <groupId>org.slf4j</groupId>

  <artifactId>slf4j-api</artifactId>

</dependency>

     

These are minimum dependencies required for compiling a sequencer. Of course, you'll have to add other dependencies that your sequencer needs.

As for testing, you probably will want to add more dependencies, such as those listed here:

<dependency>

  <groupId>junit</groupId>

  <artifactId>junit</artifactId>

  <version>4.4</version>

  <scope>test</scope>

</dependency>

<dependency>

  <groupId>org.hamcrest</groupId>

  <artifactId>hamcrest-library</artifactId>

  <version>1.1</version>

  <scope>test</scope>

</dependency>

<!-- Logging with Log4J -->

<dependency>

  <groupId>org.slf4j</groupId>

  <artifactId>slf4j-log4j12</artifactId>

  <version>1.4.3</version>

  <scope>test</scope>

</dependency>

<dependency>

  <groupId>log4j</groupId>

  <artifactId>log4j</artifactId>

  <version>1.2.14</version>

  <scope>test</scope>

</dependency>

     

Testing JBoss DNA sequencers does not require a JCR repository or the JBoss DNA services. (For more detail, see the testing section.) However, if you want to do integration testing with a JCR repository and the JBoss DNA services, you'll need additional dependencies for these libraries.

<dependency>

  <groupId>org.jboss.dna</groupId>

  <artifactId>dna-repository</artifactId>

  <version>0.1</version>

  <scope>test</scope>

</dependency>

<!-- Java Content Repository API -->

<dependency>

  <groupId>javax.jcr</groupId>

  <artifactId>jcr</artifactId>

  <version>1.0.1</version>

  <scope>test</scope>

</dependency>

<!-- Apache Jackrabbit (JCR Implementation) -->

<dependency>

  <groupId>org.apache.jackrabbit</groupId>

  <artifactId>jackrabbit-api</artifactId>

  <version>1.3.3</version>

  <scope>test</scope>

  <!-- Exclude these since they are included in JDK 1.5 -->

  <exclusions>

    <exclusion>

      <groupId>xml-apis</groupId>

      <artifactId>xml-apis</artifactId>

    </exclusion>

    <exclusion>

      <groupId>xerces</groupId>

      <artifactId>xercesImpl</artifactId>

    </exclusion>

  </exclusions>

</dependency>

<dependency>

  <groupId>org.apache.jackrabbit</groupId>

  <artifactId>jackrabbit-core</artifactId>

  <version>1.3.3</version>

  <scope>test</scope>

  <!-- Exclude these since they are included in JDK 1.5 -->

  <exclusions>

    <exclusion>

      <groupId>xml-apis</groupId>

      <artifactId>xml-apis</artifactId>

    </exclusion>

    <exclusion>

      <groupId>xerces</groupId>

      <artifactId>xercesImpl</artifactId>

    </exclusion>

  </exclusions>

</dependency>

     

At this point, your project should be set up correctly, and you're ready to move on to writing the Java implementation for your sequencer.

After creating the project and setting up the dependencies, the next step is to create a Java class that implements the StreamSequencer interface. This interface is very straightforward and involves a single method:

public interface StreamSequencer {

    /**
     * Sequence the data found in the supplied stream, placing the output 
     * information into the supplied map.
     *
     * @param stream the stream with the data to be sequenced; never null
     * @param output the output from the sequencing operation; never null
     * @param progressMonitor the progress monitor that should be kept 
     *   updated with the sequencer's progress and that should be
     *   frequently consulted as to whether this operation has been cancelled.
     */
    void sequence( InputStream stream, SequencerOutput output, ProgressMonitor progressMonitor );

The job of a stream sequencer is to process the data in the supplied stream, and place into the SequencerOutput any information that is to go into the JCR repository. JBoss DNA figures out when your sequencer should be called (of course, using the sequencing configuration you'll add in a bit), and then makes sure the generated information is saved in the correct place in the repository.

The SequencerOutput class is fairly easy to use. There are basically two methods you need to call. One method sets the property values, while the other sets references to other nodes in the repository. Use these methods to describe the properties of the nodes you want to create, using relative paths for the nodes and valid JCR property names for properties and references. JBoss DNA will ensure that nodes are created or updated whenever they're needed.

public interface SequencerOutput {

  /**
   * Set the supplied property on the supplied node.  The allowable
   * values are any of the following:
   *   - primitives (which will be autoboxed)
   *   - String instances
   *   - String arrays
   *   - byte arrays
   *   - InputStream instances
   *   - Calendar instances
   *
   * @param nodePath the path to the node containing the property; 
   * may not be null
   * @param property the name of the property to be set
   * @param values the value(s) for the property; may be empty if 
   * any existing property is to be removed
   */
  void setProperty( String nodePath, String property, Object... values );

  /**
   * Set the supplied reference on the supplied node.
   *
   * @param nodePath the path to the node containing the property; 
   * may not be null
   * @param property the name of the property to be set
   * @param paths the paths to the referenced property, which may be
   * absolute paths or relative to the sequencer output node;
   * may be empty if any existing property is to be removed
   */
  void setReference( String nodePath, String property, String... paths );
}

JBoss DNA will create nodes of type nt:unstructured unless you specify the value for the jcr:primaryType property. You can also specify the values for the jcr:mixinTypes property if you want to add mixins to any node.

For a complete example of a sequencer, let's look at the ImageMetadataSequencer implementation:

public class ImageMetadataSequencer implements StreamSequencer {

    public static final String METADATA_NODE = "image:metadata";
    public static final String IMAGE_PRIMARY_TYPE = "jcr:primaryType";
    public static final String IMAGE_MIXINS = "jcr:mixinTypes";
    public static final String IMAGE_MIME_TYPE = "jcr:mimeType";
    public static final String IMAGE_ENCODING = "jcr:encoding";
    public static final String IMAGE_FORMAT_NAME = "image:formatName";
    public static final String IMAGE_WIDTH = "image:width";
    public static final String IMAGE_HEIGHT = "image:height";
    public static final String IMAGE_BITS_PER_PIXEL = "image:bitsPerPixel";
    public static final String IMAGE_PROGRESSIVE = "image:progressive";
    public static final String IMAGE_NUMBER_OF_IMAGES = "image:numberOfImages";
    public static final String IMAGE_PHYSICAL_WIDTH_DPI = "image:physicalWidthDpi";
    public static final String IMAGE_PHYSICAL_HEIGHT_DPI = "image:physicalHeightDpi";
    public static final String IMAGE_PHYSICAL_WIDTH_INCHES = "image:physicalWidthInches";
    public static final String IMAGE_PHYSICAL_HEIGHT_INCHES = "image:physicalHeightInches";

    /**
     * {@inheritDoc}
     */
    public void sequence( InputStream stream, SequencerOutput output, 
                          ProgressMonitor progressMonitor ) {
        progressMonitor.beginTask(10, ImageSequencerI18n.sequencerTaskName);

        ImageMetadata metadata = new ImageMetadata();
        metadata.setInput(stream);
        metadata.setDetermineImageNumber(true);
        metadata.setCollectComments(true);

        // Process the image stream and extract the metadata ...
        if (!metadata.check()) {
            metadata = null;
        }
        progressMonitor.worked(5);
        if (progressMonitor.isCancelled()) return;

        // Generate the output graph if we found useful metadata ...
        if (metadata != null) {
            // Place the image metadata into the output map ...
            output.setProperty(METADATA_NODE, IMAGE_PRIMARY_TYPE, "image:metadata");
            // output.psetProperty(METADATA_NODE, IMAGE_MIXINS, "");
            output.setProperty(METADATA_NODE, IMAGE_MIME_TYPE, metadata.getMimeType());
            // output.setProperty(METADATA_NODE, IMAGE_ENCODING, "");
            output.setProperty(METADATA_NODE, IMAGE_FORMAT_NAME, metadata.getFormatName());
            output.setProperty(METADATA_NODE, IMAGE_WIDTH, metadata.getWidth());
            output.setProperty(METADATA_NODE, IMAGE_HEIGHT, metadata.getHeight());
            output.setProperty(METADATA_NODE, IMAGE_BITS_PER_PIXEL, metadata.getBitsPerPixel());
            output.setProperty(METADATA_NODE, IMAGE_PROGRESSIVE, metadata.isProgressive());
            output.setProperty(METADATA_NODE, IMAGE_NUMBER_OF_IMAGES, metadata.getNumberOfImages());
            output.setProperty(METADATA_NODE, IMAGE_PHYSICAL_WIDTH_DPI, metadata.getPhysicalWidthDpi());
            output.setProperty(METADATA_NODE, IMAGE_PHYSICAL_HEIGHT_DPI, metadata.getPhysicalHeightDpi());
            output.setProperty(METADATA_NODE, IMAGE_PHYSICAL_WIDTH_INCHES, metadata.getPhysicalWidthInch());
            output.setProperty(METADATA_NODE, IMAGE_PHYSICAL_HEIGHT_INCHES, metadata.getPhysicalHeightInch());
        }

        progressMonitor.done();
    }
}

Notice how the image metadata is extracted and the output graph is generated. A single node is created with the name image:metadata and with the image:metadata node type. No mixins are defined for the node, but several properties are set on the node using the values obtained from the image metadata. After this method returns, the constructed graph will be saved to the repository in all of the places defined by its configuration. (This is why only relative paths are used in the sequencer.)

Also note how the progress monitor is used. Reporting progress through the supplied ProgressMonitor> is very easy, and it ensures that JBoss DNA can accurately monitor and report the status of sequencing activities to the users. At the beginning of the operation, call beginTask(...) with a meaningful message describing the operation and a total for the amount of work that will be done by this sequencer. Then perform the sequencing work, periodically reporting work by specifying the incremental amount of work with the worked(double) method, or by creating a subtask with the createSubtask(double) method and reporting work against that subtask monitor.

Your method should periodically use the ProgressMonitor's isCancelled() method to check whether the operation has been cancelled.. If this method returns true, the implementation should abort all work as soon as possible and close any resources that were acquired or opened.

Finally, when your sequencing operation is completed, it should call done() on the progress monitor.

The sequencing framework was designed to make testing sequencers much easier. In particular, the StreamSequencer interface does not make use of the JCR API. So instead of requiring a fully-configured JCR repository and JBoss DNA system, unit tests for a sequencer can focus on testing that the content is processed correctly and the desired output graph is generated.

The following code fragment shows one way of testing a sequencer, using JUnit 4.4 assertions and some of the classes made available by JBoss DNA. Of course, this example code does not do any error handling and does not make all the assertions a real test would.

StreamSequencer sequencer = new ImageMetadataSequencer();
MockSequencerOutput output = new MockSequencerOutput();
ProgressMonitor progress = new SimpleProgressMonitor("Test activity");
InputStream stream = null;
try {
    stream = this.getClass().getClassLoader().getResource("caution.gif").openStream();
    sequencer.sequence(stream,output,progress);   // writes to 'output'
    assertThat(output.getPropertyValues("image:metadata", "jcr:primaryType"), 
               is(new Object[] {"image:metadata"}));
    assertThat(output.getPropertyValues("image:metadata", "jcr:mimeType"), 
               is(new Object[] {"image/gif"}));
    // ... make more assertions here
    assertThat(output.hasReferences(), is(false));
} finally {
    stream.close();
}

It's also useful to test that a sequencer produces no output for something it should not understand:

 sequencer = new ImageMetadataSequencer();
MockSequencerOutput output = new MockSequencerOutput();
ProgressMonitor progress = new SimpleProgressMonitor("Test activity");
InputStream stream = null;
try {
    stream = this.getClass().getClassLoader().getResource("caution.pict").openStream();
    sequencer.sequence(stream,output,progress);   // writes to 'output'
    assertThat(output.hasProperties(), is(false));
    assertThat(output.hasReferences(), is(false));
} finally {
    stream.close();
}

These are just two simple tests that show ways of testing a sequencer. Some tests may get quite involved, especially if a lot of output data is produced.

It may also be useful to create some integration tests that configure JBoss DNA to use a custom sequencer, and to then upload content using the JCR API, verifying that the custom sequencer did run. However, remember that JBoss DNA runs sequencers asynchronously in the background, and you must synchronize your tests to ensure that the sequencers have a chance to run before checking the results. (One way of doing this (although, granted, not always reliable) is to wait for a second after uploading your content, shutdown the SequencingService and await its termination, and then check that the sequencer output has been saved to the JCR repository. For an example of this technique, see the SequencingClientTest unit test in the example application.)

The first step of deploying a sequencer consists of adding/changing the sequencer configuration (e.g., SequencerConfig) in the SequencingService. This was covered in the previous chapter.

The second step is to make the sequencer implementation available to JBoss DNA. At this time, the JAR containing your new sequencer, as well as any JARs that your sequencer depends on, should be placed on your application classpath.