General Usage:

This project builds an extension to Doxia. Doxia is used by maven-site-plugin to build project sites within a Maven project. The Macro defined by this extension can be used as soon as it was registered within your own project's pom.xml (Maven 2 - Project Object Model).

Prerequisites

  • Maven Version >= 2.2
  • JDK Version >= 5
  • "maven-site-plugin" Version >= 2.1 (Doxia >= 1.1 is required)

1. Define the dependencies inside the "pom.xml":

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
    ...
  
    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <artifactId>maven-site-plugin</artifactId>
                    <version>2.1</version>
  
                    <dependencies>
                        <!-- additional dependencies required by "source-class" implementations are added here -->
                        <dependency>
                            <groupId>org.tinyjee.dim</groupId>
                            <artifactId>doxia-include-macro</artifactId>
                            <version>1.1</version>
                        </dependency>
                    </dependencies>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

The include macro's artifact has to be added as a dependency to the site plugin's dependency list, in order to let Maven Doxia recognize it when generating a project site.
Any additionally required 3rd party dependencies have to be added here when using the "source-class" parameter with an implementation that depends on a certain library that is not in the default site building classpath (or project classpath when the goal initialize was called before calling the macro).

Available Versions: An overview on public available versions of "doxia-include-macro" can be obtained from here:
http://mvnrepository.com/artifact/org.tinyjee.dim/doxia-include-macro

Macro Configuration & Multi-Module Projects:

With non standard path configuration or multi-module projects, the macro requires the following addition in the base POM to resolve the project paths correctly. Even though optional, it is in general advisable to include this build plugin as it improves site building stability and provides access to the configured POM dependencies and the resulting classpath.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<project>
    <build>
        <plugins>
            <!-- Optionally initializes & configures the macro (in particular adds support for multimodule projects )-->
            <!-- This also fixes the basedir issue in multimodule builds (see http://jira.codehaus.org/browse/DOXIA-373)-->
            <plugin>
                <groupId>org.tinyjee.dim</groupId>
                <artifactId>doxia-include-macro</artifactId>
                <version>1.1</version>
                <executions>
                    <execution>
                        <id>initialize-doxia-include-macro</id>
                        <!-- Use "initialize" when the 'site' was not given as 'phase'. -->
                        <!-- phase>initialize</phase -->
                        <phase>pre-site</phase>
                        <goals>
                            <goal>initialize</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Besides configuring the macro, the build plugin sets a system property that is currently picked up by Doxia to set the base path for macros.
That works not only for this but also for all other macros that attempt to resolve resources relative to the given base directory. As a consequence of setting a system property, parallel site building is unsupported.

2. Use in XDoc or APT documents

Usage within APT

%{include|source=org.tinyjee.maven.dim.IncludeMacro}
%{include|source=org/tinyjee/maven/dim/IncludeMacro.java}
%{include|source=template.html|verbatim=false}

Usage within XDoc

<macro name="include">
  <param name="source" value="org/tinyjee/maven/dim/IncludeMacro.java"/>
  ...
</macro>

Parameter Reference:

The following list shows all parameters that can be used with the Macro:

Parameter Name Since Description
"source" 1.0.7 Sets the content to include using a source path, source URL or the fully qualified java class name.

Examples:
source=path/to/file
source=my.package.MyClass
source=/absolute/path/to/file
source=http://somehost/someservice#.xml
source=classpath:/bundled/template.vm

Relative source paths are resolved against multiple base locations including site, java (test)source and resource directories. Non existing absolute paths starting with the project's base directory are stripped and retried using the relative paths resolver.
When remote content is specified using URLs with schemes like http or ftp, the content is downloaded using a network connection and cached locally for the time of a single build (= subsequent requests to the same remote source will not download the source again).

Executing the goal "initialize" within the maven build enables the following additional syntax for referencing a module file location:
[artifactId]:path/to/file
[groupId:artifactId]:path/to/file
[top]:path/to/file/in/project/root

Finally sources can also be loaded from the module's or the site building classpath using "classpath:/path/to/file".
The classpath is constructed out of the module sources and test-sources folder with the parent classloader being defined by the dependency list of the maven-site-plugin. When the goal "initialize" is executed the classpath includes all dependencies that are defined inside the pom.xml.
"source-class" 1.0.7 Sets a class to use as dynamic source for content, request parameters or both.

A class that is set using this parameter must implement the interface java.util.Map (or derive from a Map implementation like java.util.HashMap). All entries of this map are treated as if they had been specified as request parameters, on the macro call (therefore all parameters except 'source-class' can be set dynamically).

The source class may choose to use one of the 4 constructor signatures: Default, Constructor(java.io.File basePath), Constructor(java.util.Map requestProperties) or Constructor(java.io.File basePath, java.util.Map requestProperties).

The default behaviour of a source class should be to provide properties for a velocity template that is specified by either setting the "source" key to a bundled template or require the user to set the parameter when using the macro with this class.
Alternatively it can also directly provide the content to include (instead of loading the template specified with "source"). To do so the map key "source-content" must either return a String or Map<Integer, List<String>> (firstLine=>lines) containing the content to include.

The classpath that is used to load the class is similar to the classpath used when loading sources. (see "source" parameter for more information)

Simple Example:
package my.package;
public class MySource extends HashMap<String, Object> {
    public MySource(java.io.File basePath, java.util.Map requestProperties) {
        putAll(requestProperties);
        put("verbatim", false);
        put("source-content", "<b>Hello World!</b>");
    }
}
Macro Call:
%{include|source-class=my.package.MySource}
"source-content" 1.0.7 Sets the source content inline with the macro call. (This feature may become more useful when DOXIA-445 gets implemented. Voting for it may help.)

The purpose of this parameter is to allow the inline definition of templates used to render output of source classes. Unless specified, "source-is-template" is set to "true" when this parameter is set with the macro call (it is not implicitly set when the parameter was set by a source class).

Example: Including the main java doc comment of a class (using the JavaSourceLoader extension)
%{include|source-content=$comment|source-java=my.package.MyClass}
"file" 1.0.7 Sets the source path using java.io.File instead of evaluating it against URL or File paths.

Under normal conditions it's easier to set the "source" parameter instead. However there may be occasions where it's necessary to force that the path is treated as a file path.
"source-is-template" 1.0.7 Toggles whether the source content is treated as a Velocity template. (Default: auto (= true if file extension ends with ".vm"))
"source-content-type" 1.0.7 Sets the content type of the source. (Default: auto (= detected by file extensions ".apt", ".fml", ".xdoc"))

Setting this property has no effect for verbatim content. With non-verbatim content this information is used to select a doxia parser that transcodes the the content when including it into the site. Using a parser allows to use xdoc templates inside apt documents and vice versa.

Values that will select a parser are "apt", "fml", "xdoc", any other values will have no effect.
"charset" 1.0.7 Defines the charset to use when loading the source content. (Default: context specific, used to be "UTF-8" in pre 1-1-alpha)

When no charset is specified the following actions apply:
  • If auto detection is enabled, the detected charset is used.
  • If source is using the HTTP protocol, any given charset is used.
  • The build charset is used when the goal initialize was executed.
  • The systems default charset is used if nothing else is defined.
"charset-autodetect" 1.0.7 Toggles whether charset detection is used when loading source content. (Default: true)

This value can have multiple values:
  • false: Disable any auto detection.
  • true: Enable auto detection, externally specified charsets override detected values.
  • force: Use auto detection result even if confidence is low.
"verbatim" 1.0.7 Toggles whether the content is verbatim (included as source) or non-verbatim (included as markup and interpreted). (Default: auto)
"id" 1.0.7 Deprecated:

Sets the snippet-id to use for extracting snippets out of the specified source.

Snippets are processed on a line by line basis and multiple snippets with the same name may appear in one file. In order to define a section using a snippet id, surround a block with:
// START SNIPPET: snippet-id
... code to include ...
// END SNIPPET: snippet-id

Note: From version 1.1 this parameter is deprecated, use snippet instead. Calls like "%{include|id=SnippetId|..}" translate to %{include|snippet=#SnippetId|..}
"snippet" 1.1-alpha Sets one or more comma separated expression to select the content snippets to include.

Snippet selection allows to define what content is included or excluded from the loaded or generated source. This operation is always the last thing to happen before including and highlighting content and can therefore be combined with any other options that the macro offers.

The general format of this parameter is "prefix:expression1, prefix:expression2, .." to include matching content and "!prefix:expression1, !prefix:expression2, .." to exclude matching content.

Examples:
  • Include snippet by ID: 'snippet=#SnippetId'
  • Exclude all snippet id definitions: 'snippet=!snippet-ids'
  • Include by ID, exclude any definitions: 'snippet=#SnippetId, !snippet-ids'
  • 'grep' like syntax: 'snippet=grep:Token,!grep:NotToken'
  • 'Regular Expression' syntax: 'snippet=re:([0-9,.;\s]+)$' (all lines ending with numeric expressions)
  • 'XPath' syntax: 'snippet=xp:/html/head/title'
  • 'AOP'-like syntax: 'snippet=aj:..InnerClass, !aj:..toString()'
  • 'Token & Brace' syntax: 'snippet=tb:"json-key"{, !tb:"inner-array"['
  • Line range: 'snippet=lines:50-100'

Some snippet selectors (e.g. XPath or AOP) require that the source content is well formed (XML/JSON or Java in this example) and may fail if applied to other content.
A complete overview on available expressions can be seen by looking after implementations of SnippetSelector.
"snippet-grow-offset" 1.1-alpha Optional parameter that can be used to grow snippet selections by the given amount of lines.
"snippet-start-offset" 1.1-alpha Optional parameter that can be used to grow the start of snippet selections by the given amount of lines.
"snippet-end-offset" 1.1-alpha Optional parameter that can be used to grow the end of snippet selections by the given amount of lines.
"snippet-grow-excludes" 1.1-alpha Optional boolean parameter that toggles whether growing is applied to exclusions as well. (Default: false)
"show-gutter" 1.0.7 Toggles whether the gutter including line numbers is shown for verbatim content. (Default: true)
"set-first-line" 1.0.7 Sets the line number to use for the first line (if gutter is shown). (Default: auto)
"pad-line-numbers" 1.0.7 Pads all line numbers by the specified amount of zeros (if gutter is shown).
"highlight" 1.0.7 Deprecated:

Sets a list of lines to highlight inside verbatim content.

Lines can be specified as a comma separated list of line numbers, as a range using x-y or as a mixture of the 2.
Example usage: highlight=1,5,10,20-25

As of version 1.1-beta this option is deprecated and was replaced by "highlight-lines". The example usage from above translates to "highlight-lines=lines:1,5,10,20-25"
"highlight-lines" 1.1-beta Defines lines to highlight inside verbatim content using the same selection logic as used within the "snippet" parameter. Selected lines are highlighted by the use of a background color.

Examples:
  • "highlight-lines=grep:org.tinyjee.dim+3"
  • "highlight-lines=xp://license"
  • "highlight-lines=lines:1,5,10,20-25"
"highlight-type" 1.0.7 Sets the highlighter (brush) to use when highlighting verbatim content. (Default: auto ; uses source content or extension to detect)

DIM bundles the brushes bat, properties, clojure and velocity in addition to what is available by default inside the syntax highlighting library. A set of known brushes includes: java, js, jsp, cpp, csharp, sql, xml, html, etc. Check the documentation of SyntaxHighlighter to get a complete list.

Highlighting of verbatim content can be disabled by setting this property to none.

Specifying the highlight type as "html/[brushname]" allows to highlight HTML with embedded scripts. A common use case is "html/js" which highlights any HTML-like code with embedded javascript (Note: files with an extension of ".xhtml" or ".html" default to this highlighting type).
"highlight-theme" 1.0.7 Sets the highlighting color theme to use. (Default: default)

Possible values: "default", "django", "eclipse", "emacs", "fadeToGrey", "MDUltra", "midnight", "RDark". (See: http://alexgorbatchev.com/SyntaxHighlighter/manual/themes/)
The theme can be set only once per page. The first highlighted inclusion will set the theme for all others.
"no-cache" 1.0.7 Forces the re-loading of source content when set to true.

Content is cached for the lifetime of the build to avoid double downloads of remote resources.
"site-directory" 1.0.7 Deprecated:

Sets the path of the site source directory.

Setting this property is not required if the site directory uses standard paths or is located directly below the module's base directory.

As of version 1.1-beta this option is deprecated and was replaced by running the "initialize" goal using the macro as build plugin.

Snippet Selectors:

Implementation Description
RegularExpressionSnippetSelector Implements a simple regular expression based snippet selector using the expression prefix "RE:".

Expression samples:
  • re:(INFO|WARN|ERROR)
  • %{include|source=build.log|snippet=RE:(?i)(INFO|WARN.*|ERROR)|snippet-grow-offset=5}

By default this selector works case-sensitive, unless "case-sensitive=false" is specified with the macro call or the case in-sensitive flag "(?i)" is set in the regular expression (as used in the second example).
IdSnippetSelector Implements the default ID based selector already known from the classic snippet macro that is part of Doxia.

In difference to the snippet macro, IDs must be prefixed with "#" and are matched fully. Sub-id matching is not allowed unless hierarchically built IDs use one of the defined delimiting chars IdSnippetSelector#ID_DELIMITER_CHARS) (e.g. "ParentId.ChildId").

Snippets are processed on a line by line basis and multiple snippets with the same name may appear in one file. In order to define a section using a snippet id, surround a block with:
// START SNIPPET: snippet-id
... code to include ...
// END SNIPPET: snippet-id
Expression samples:
  • #snippet-id
  • #MySnippetIdToInclude
  • !#MySnippetIdToExclude
  • #FirstSnippet, #SecondSnippet, #ThirdSnippet
  • #ParentId (Matches "ParentId" and "ParentId.[ANYTHING]")
  • #ParentId.ChildId
This selector is always case-insensitive.
GrepLikeSnippetSelector Implements a token based snippet selector working similar as the unix command 'grep' using the expression prefix "grep:".

Expression samples:
  • grep:case-insensitive-token
  • grep:ERROR
  • %{include|source=build.log|snippet=GREP:Exception, GREP:org.tinyjee.dim|snippet-grow-offset=5}
  • %{include|source=build.log|snippet=GREP:Exception+3, GREP:org.tinyjee.dim+5}

By default this selector works case-insensitive, unless "case-sensitive=true" is set.

The grep selector allows to define "snippet-end-offset" inline by adding "+lines" to the end of a statement. In order to match "+5" use an escaped expression like "\+5".
XPathSnippetSelector Implements a XPath expression based snippet selector supporting XML and JSON formatted input, using the expression prefix "XP:".

Well formed content required: This snippet selector requires well formed XML or JSON input. Anything else will cause failures.

Example Expressions:
  • xp:/html/head/title
  • xp://title
  • 'xp:/default:html/default:head' (with namespace awareness enabled the prefix for the default XML-NS is 'default')

Notes:
  • XPath support for JSON is enabled if the source content's file extension is ".json" or "source-content-type=json" is set. See json document builder for details how json translates to XML.
  • XML namespace awareness can be turned on by setting "namespace-aware=true" (does not exist in JSON).
AOPLikeSnippetSelector Implements an AOP expression like snippet selector using the expression prefix "AJ:".

Well formed content required: This snippet selector requires well formed Java source input. Anything else will cause failures.

Expression samples:
  • AJ:..toString()
  • AJ:..fieldName
  • AJ:public void my..MyClass.myMethod(..)
  • AJ:public void my..MyClass.myMethod(java.lang.String, ..)
  • AJ:public void my..MyClass.myMethod(..String, ..)
  • AJ:@SuppressWarnings(value="unchecked")
  • AJ:@my..Annotation(..value="something"..)

See also AbstractSelectableJavaEntitiesList for more details.

The selector uses AbstractSelectableJavaEntitiesList#selectAnnotated(String) to retrieve matching elements of the given java source in case of the expression starts with an '@', otherwise AbstractSelectableJavaEntitiesList#selectMatching(String) is used to find matching java entities.

Comments and bean properties are expanded if "expand-snippets=true" is set. Effectively this means when matching a field, getter or setter all linked code parts are automatically included, as well as the javadoc comments.
LineRangeSnippetSelector Implements a selector based on a line range definition to select snippets using the expression prefix "lines:".

The primary purpose of this selector is to cut-off content that should not be in the output (e.g. using !lines:100+).

Example expressions:
  • 'lines:50-100' (from 50 to 100)
  • 'lines:50+' (from 50 to EOF)
  • 'lines:50-' (from 1 to 50)
  • 'lines:1,3,10-12,1000+' (selects 1, 3, 10, 11, 12 and 1000 to EOF)
TokenAndBraceSnippetSelector Selects snippets using an activation token and open/close brace counting, using the expression prefix "TB:".

This selector works with any C based language constructs of the form Activation-TOKEN .. { ... content ... } as well as other content that uses hierarchical braces of one of the supported formats (see below).

Expression samples:
  • tb:methodName{
  • tb:activationToken[
  • tb:activationToken(
  • tb:activationToken<
  • tb:XmlCommentNextToThisToken<!--
  • tb:<!--
  • tb:/*
  • tb:#*

If the brace is omitted, the selector defaults to the braces {}.
In case of only a brace but no token is specified, the selector matches all content that is surrounded by the given brace. E.g. <!-- matches all comments of an XML/HTML file.

Activation tokens of the format [line-number] (e.g. [132]) activate the brace matching algorithm when the given line number is reached instead of a string-match of the token is found.

By default this selector works case-insensitive, unless "case-sensitive=true" is set.
IdDefinitionSelector Implements a selector that matches all snippet ID definitions triggered by a single expression "snippet-ids". The purpose of this selector is to exclude any snippet id definitions from matched content.

Example: "%{include|snippet=#id-to-include, !snippet-ids}".

Debugging / Logging:

Whether debug logging is enabled when using the commandline option "-X" depends on the site plugin and Doxia version that is used (by experience, in most cases it is not).
Adding the commandline option "-Ddim.verbose=true" does always enable debug logging inside the include macro and allows to trace issues that may prevent content from being rendered correctly.

Velocity Template Processing Reference:

Template Processing:

Any included content is processed by the Apache Velocity template engine, as soon as the file name of the source ends with .vm or source-is-template is set to true. A reference on how to work with templates can be obtained from here: http://velocity.apache.org/engine/releases/velocity-1.5/user-guide.html

In most cases the velocity engine version will be 1.5 and it is not affected by adding the include macro to the site plugin's dependencies. Instead the exact version of the engine depends on what was set inside the site plugin's dependency graph.

Any parameters that were given with the macro call and any parameters that were added by RequestParameterTransformer's or source classes automatically turn into variables within velocity templates.

Default Engine Configuration:

19
20
21
22
23
24
25
26
27
28
29
30
31
32
public void configureEngine(VelocityEngine engine) {
    // Register a custom resource loader that uses DIM's path / url resolve logic
    engine.setProperty(RESOURCE_LOADER, "dim-resource");
    engine.setProperty("dim-resource.resource.loader.class",
            "org.tinyjee.maven.dim.sources.TemplateSource$UrlResourceLoader");
  
    // Enable template-local macro definitions
    if (engine.getProperty(VM_PERM_ALLOW_INLINE) == null) engine.setProperty(VM_PERM_ALLOW_INLINE, true);
  
    // Setup macro libraries
    String existingLibrary = (String) engine.getProperty(VM_LIBRARY);
    engine.setProperty(VM_LIBRARY, "classpath:/templates/dim/macro-library.vm," + VM_LIBRARY_DEFAULT +
            (existingLibrary == null ? "" : ',' + existingLibrary));
}

Default Context Configuration:

55
56
57
58
59
60
61
62
63
64
public void configureContext(Map<String, Object> parameters, Context templateContext) {
    // Note: The given parameters and tools configuration is already present, applying
    //       additional customizations:
    if (!templateContext.containsKey("globals")) templateContext.put("globals", Globals.getInstance());
    if (!templateContext.containsKey("macroParameters")) templateContext.put("macroParameters", parameters);
    if (!templateContext.containsKey("context")) templateContext.put("context", templateContext);
  
    // 'sourceUrl' is set when the context is constructed and points to the actual template file,
    // there's no need to construct and set it here.
}

The default context is configured with an instance of $globals that offers access to logging, base path and the ability to attach content (css, scripts and arbitrary text-based) to a page.

Enabled Velocity Tools (Version 2.0):

The include macro uses the following code snippet to include several additional tools that help building content with velocity:

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
public void configureTools(EasyFactoryConfiguration toolsConfiguration) {
    toolsConfiguration.toolbox("application").
            tool("org.apache.velocity.tools.generic.AlternatorTool").
            tool("org.apache.velocity.tools.generic.ClassTool").
            tool("org.apache.velocity.tools.generic.ComparisonDateTool").
            tool("org.apache.velocity.tools.generic.ConversionTool").
            tool("org.apache.velocity.tools.generic.DateTool").
            tool("org.apache.velocity.tools.generic.DisplayTool").
            tool("org.apache.velocity.tools.generic.EscapeTool").
            tool("org.apache.velocity.tools.generic.FieldTool").
            tool("org.apache.velocity.tools.generic.MathTool").
            tool("org.apache.velocity.tools.generic.NumberTool").
            tool("org.apache.velocity.tools.generic.RenderTool").
            tool("org.apache.velocity.tools.generic.SortTool").
            tool("org.apache.velocity.tools.generic.ListTool").
            tool("org.apache.velocity.tools.generic.ResourceTool").
            tool("org.apache.velocity.tools.generic.MarkupTool").
            tool(VelocityHighlightingTool.class).
            tool(VelocitySourceClassInvocationTool.class);
}

A complete reference on how to work with velocity tools can be obtained from here: http://velocity.apache.org/tools/releases/2.0/summary.html

Macro Related Tools

Besides the standard Apache tools, 'doxia-include-macro' bundles a couple of custom tools to reuse the macro features inside a velocity template.

Implementation Description
VelocityHighlightingTool Implements a custom velocity tool that can be used to trigger code highlighting within templates.

Usage within a template:
Highlighted content: $highlighter.highlight("public class MyClass {...}")

Quick Example - Embed the comment of a class with all code blocks (<code><pre>) being highlighted:
%{include|source-content=$highlighter.highlightCodeBlocks($comment)|source-java=my.package.MyClass}

Reference:
  • $highlighter.java("..java-code...")
  • $highlighter.js("..js-code...")
  • $highlighter.xml("..xml-code...")
  • $highlighter.css("..css-code...")
  • $highlighter.jsp("..jsp-code...")
  • $highlighter.highlight("..autodetect-code...")
  • $highlighter.highlight("..brush-name..e.g. 'java'", "..code...")
  • $highlighter.highlightCodeBlocks("Non-highlighted content. <code><pre>public class MyClass {...}</pre></code>")
  • $highlighter.highlightCodeBlocks($comment)
VelocitySourceClassInvocationTool Implements a custom velocity tool that can be used to invoke other source classes.

Usage within a template:
#set ($result = $invoker.clear().put('paramA', 'value').invokeClass('my.package.MyClass'))
Result: $result.get("key")

Custom Tools

Source classes (used via source-class) are free to set variables with custom implementation inside the template context (= request parameter map) which may be accessed and used from a velocity template in exactly the same way as tools are accessed. E.g. if a source class implements the code snippet "put("myCallable", new Callable() { ... });" the template can use "$myCallable.call()" to invoke the "call()" method in this example and retrieve the result.

In case of there's a need to adjust the tool configuration itself, look after Extension Reference and create a macro extension that performs this task.