Chapter 2Running XML Calabash

The easiest way to run XML Calabash is directly from the jar file:

java -jar xmlcalabash-3.0.0-alpha28.jar options…

Run this way, all of the dependencies included in the lib directory are automatically included on the classpath.

You can also run the main class directly, for example in a build system or a shell script:

java com.xmlcalabash.app.Main options…

But if you choose this form, you must ensure that the classpath contains both the XML Calabash jar file and all of the necessary dependencies. One feature of the second form is that it allows you to change or update the dependencies.

In the fullness of time, XML Calabash will be available through Maven which has extensive features for managing dependencies. In the short term, it’s just a bit messy.

Upgrading Saxon

The one library that you are likely to want to change in Saxon. If you have a license for Saxon PE or Saxon EE, it makes sense to swap out the Saxon HE library that ships with XML Calabash for PE or EE.

To save you the trouble of constructing and managing a large classpath, the alpha distribution of XML Calabash is configured with a hack. Simply delete the Saxon-HE-12.5.jar file from the lib directory and copy in your PE or EE jar instead. You must use a 12.5 jar file for this purpose! Be careful to make sure there’s only one Saxon jar file in the lib directory. Having multiple versions of the same library on the classpath is an invitation for subtle and mysterious crashes.

Saxon generally searches for a license file on the classpath, which isn’t going to work with this hack. Instead, you can use the SAXON_HOME environment variable, as described in the documentation.

Elsewhere in this document, we assume that the command “xmlcalabash” runs XML Calabash. This can be replaced by the java -jar … version above or by your own script.

XML Calabash has a command driven command-line interface. It supports three commands: run, to run a pipeline; version, to print the version, and help to display help. If no command is given, the run command is assumed.

The run command

Where the options and arguments are:

run, selects the run command

If the command is omitted, run is assumed.

--configuration:configuration-file, identifies a configuration file

When XML Calabash begins, it reads configuration settings from a configuration file. If you don’t specify a configuration file, it will search first for .xmlcalabash3 in the current directory and then in your home directory.

--pipe:boolean, sets piped mode

If piped mode is enabled, XML Calabash will behave like a Unix pipeline. If there is no binding for the primary input port, it will read it from standard input. If there is no binding for the primary output port, it will write it to standard output. If not specified, the default setting comes from the piped-io configuration setting.

--input:[type@]port=uri, identifies an input

This option associates the resource identified by the uri with the input port named port. The pipeline must have an input port named port.

If type is provided, it is the content type that will be used to parse the input; it must be a valid content type. If no content type is provided, the content type will be inferred from the URI, if possible.

If the --input option is repeated for the same port, the resources become a sequence of documents on that port, in the order specified.

If the input uri provided is “-”, input will be read from standard input. When standard input is read, if no type is provided, it will be parsed as the first usable content type listed on the named input port. It will be parsed as XML if the input port doesn’t specify any usable content types.

Note

A content type is “usable” if it is not a negated type, if it doesn’t use wildcards for the type or subtype, or if it’s recognized as a “+xml” or “+json” content type.

--output:port=filespec, identifies an output

In the simplest case, at most one document appears on the output port and filespec is just a filename. The filespec can also be a template. The output documents are numbered, starting at 1. Within the filespec, the string “%d” will be replaced by the document number. (The string “%h” will be replaced by the document number in hexadecimal and “%o” will be replaced by the document number in octal, for no good reason.) Zeros can be added to pad the value, “%00d” will produce a document number that is at least two digits long, padded on the left with 0’s if there are fewer than 10 documents. For completeness, a literal “%” can be added to the filename with “%%” in the filespec.

If the filespec does not contain a number template, then all of the outputs on the port will be concatenated to the same file. You may not repeat the --output option for the same port.

The pipeline must have an output port named port.

If the output filespec provided is “-”, output from that port will be written to standard output. At most one output port may be explicitly bound to standard output.

In piped mode, at most one port will write to standard output. If there is no explicit binding to standard output, the primary output port writes to standard output. If the primary output port is bound to some other output, no output will be sent to standard output.

If piped mode is not enabled, and no port is explicitly bound to standard output, the pipeline will write the output from all otherwise unbound output ports to standard output. If exactly one port is being written to standard output, and if that port cannot produce a sequence, then the output is written directly to standard output.

If more than one port may appear on standard output, or if a sequence of documents may appear, and if standard output appears to be going to a terminal window, “decoration” is added as an aid to comprehension:

  • A header is printed before the output identifying the port name, document number, and base URI.

  • A line of equal signs is printed as a separator between documents (if more than one document is output).

  • When XML or HTML content is output, the XML declaration is omitted.

  • Indentation is turned on.

The method for determining whether output is going to a terminal or being redirected isn’t terribly sophisticated and may be wrong in some circumstances. It’s safer to explicitly enable piped mode or use --output to write to a file if you want to save the output.

--namespace:prefix=uri, identifies a namespace binding

Binds the specified prefix to the uri. This has no effect on the pipeline; these bindings are only used when evaluating the --step option and in expressions used to define options.

The default namespace bindings on the command line are:

PrefixNamespace URI
arrayhttp://www.w3.org/2005/xpath-functions/array
cxhttp://xmlcalabash.com/ns/extensions
fnhttp://www.w3.org/2005/xpath-functions
maphttp://www.w3.org/2005/xpath-functions/map
mathhttp://www.w3.org/2005/xpath-functions/math
phttp://www.w3.org/ns/xproc
saxonhttp://saxon.sf.net/
xshttp://www.w3.org/2001/XMLSchema

The --namespace option can add to, or change, these bindings.

--init:class-name, Saxon configuration initializer

Attempts to load and execute the class named class-name, which must be available on the class path and must implement the net.sf.saxon.lib.Initializer interface. (This is analagous to the -init: option on the Saxon command line API.)

--graphs:graph-output-directory, SVG graph outputs

This option writes hyperlinked SVG descriptions of pipelines, and their corresponding graphs, to the graph-output-directory. The descriptions are “boxes and arrows” diagrams of the connections between the steps. One SVG diagram is produced for each declared pipeline and its corresponding graph. An HTML index in the graph-output-directory makes them easy to browse. See Chapter 5, Pipelines vs. Graphs for more details.

The processor assumes that it “owns” the graph-output-directory; it will erase files and directories before creating the graph output.

Note

Some browsers have better support for SVG than others. If the diagrams are difficult to view, or if links don’t work correctly, a good first step is to try a different browser.

--verbosity:verbosity

There are five levels of verbosity. The level of verbosity determines how much detail is printed about the progress of a running pipeline.

trace

Lots of detail, show everything.

debug

Lots of detail

info

Show relevant status messages.

warn

Show only warnings and errors.

error

No messages except fatal errors

Setting the verbosity also sets the logging level.

--explain, explain errors

Enables error explanations. If error explanations are enabled, when a pipeline fails, in addition to the error message, a short explanation of the cause will be provided.

--visualizer:name

Selects a visualizer for reporting pipeline progress. There are three defined visualizers.

silent

Shows no progress reports.

plain

Reports the name of each step when it begins running. Most steps manufactured automatically during graph construction are omitted. There is one option, indent which determines whether or not, and to what extent, reports are indented when they are nested inside compound steps.

detail

Reports the name of each step and may report on the documents produced.

If the steps option is true, the progress of steps is recorded. (Defaults to true.)

If the documents option is true, the documents produced during execution are recorded. (Defaults to false.)

Options can be specified on the command line as key=value pairs after the name. The name and any options are separated by a question mark (?). Key-value pairs are separted by semicolons (;). For example: --visualizer=detail?steps=true;documents=true.

--trace:output-file, defines a runtime trace file

A runtime trace of the pipeline execution will be written to output-file. (If the --trace-documents option is given but the --trace option is not, the default output file is named trace.xml in the --trace-documents output directory.)

For more information about tracking, see Appendix B, Tracing execution.

--trace-documents:output-directory, defines a runtime trace directory

If the --trace-documents option is given, the runtime trace will write a copy of every document that flows through the pipeline into the specified directory. These documents are identified in the --trace file.

--assertions:level

By default, assertions are disabled. You can enable them by setting an appropriate level:

error

Schematron reports are output and Schematron errors are treated as errors. (They cause the step to throw Q{http://xmlcalabash.com/ns/error}XI0041.)

warning (or warn)

Schematron reports are output and Schematron errors are treated as warnings.

ignore

Assertions are ignored.

--licensed

Requests a licensed processor. In practice, a licensed processor is used by default, if one is available. However, --licensed:false can be used to explicitly request an unlicensed processor when Saxon PE or Saxon EE are on the classpath.

Schema-aware processing requires Saxon EE and a valid Saxon license.

--debug

If debugging is enabled, the verbosity is set to at least the debug level and the backend logging framework’s “log level” is set to the verbosity.

This produces a lot of messages, typically on the standard error output. XML Calabash uses SLF4J and Logback for logging. You can configure it in the usual ways.

--debugger

Start an interactive debugging session on the pipeline. See Chapter 8, The interactive debugger.

--nogo

Compile the pipeline, and produce graphs if they’re requested, but don’t actually run the pipeline.

--catalog:catalog-file-uri

Adds the specified catalog to the list of XML Catalogs that are used during resource resolution.

--xml-schema:xml-schema-file-uri

Adds the specified XML Schema to the validation context. This schema will be available in both implicit validation and when the p:validate-with-xml-schema step is used.

--validation-mode:mode

This option enables implicit validation. The mode must be either “lax” or “strict”.

----try-namespaces

This option enables attempting to retrieve a schema using its namespace URI during implicit validation.

--use-location-hints

This option enables attempting to retrieve a schema using location hints during implicit validation.

--help

This is equivalent to issuing the help command. It’s provided as an option for convenience.

--step:step, identifies a step

The --step option is a little bit overloaded. Its interpretation depends on whether or not a pipeline is specified.

If a pipeline is specified, it must identify a p:library. The value of the --step option is interpreted as the name of a step in that library. The named step will be run.

For example:

$ xmlcalabash --step:helloWorld --library:examples.xpl …

If no pipeline is specified, the value of the --step option is interpreted as the type of a step. The (atomic) step with the specified type will be run. All of the inputs, outputs, and options specified apply to that step.

For example:

$ xmlcalabash --step:p:xslt …

The p and cx namespaces are bound by default. The --namespace option can be used to change the namespace bindings.

pipeline.xpl, the pipeline to run

This identifies the pipeline to run. If the root element is p:declare-step, then that pipeline will be run. If the root element is p:library, the first pipeline in the library will be run, unless the step option specifies an alternate pipeline.

option=value, sets an option

You can provide values for pipeline options on the command line. These override any defaults declared in the pipeline. There must be a pipeline option (or a static option) named option.

If the option name includes “::”, then it is treated as a shortcut for setting the value in a map. In this case, the name before the “::” is the name of the actual parameter and the name after the “::” is the name of a key in that map. (With the additional special case that if the name begins with “::”, then the option name is take to be default option name, currently parameters.) This is intended to be somewhat reminiscent of XPath axes.

In both cases, the name can be an EQName or it can use a prefix previously defined with --namespace; if the name is a simple NCName, it is not in a namespace.

If the value begins with “?”, what follows is taken to be an XPath expression. That expression is evaluated using the namespace bindings defined. The context item is undefined. The result of evaluating the expression is the value of the option. If the value does not begin with a “?”, the whole string becomes the value as an xs:untypedAtomic.

If multiple assignments to the same option (or map item) appear, the value will be a sequence with those values in the order specified.

For example, suppose that the following sequences of options are given:

a=1 b=2

Sets the option a to 1 and the option b to 2.

a=1 b=2+3

Sets the option a to 1 and the option b to “2+3”.

a=1 a=2 b=?2+3

Sets the option a to (1, 2) and the option b to 5.

a=1 parameters="map{'key': 'value'}"

Sets the option a to 1 and the option parameters to a map with the QName key “key” and the value “value”.

a=1 parameters::key=value

Sets the option a to 1 and the option parameters to a map with the QName key “key” and the value “value”.

a=1 parameters::key=value serialization::method=xml serialization::indent=true

Sets the option a to 1 and the option parameters to a map with the QName key “key” and the value “value” and the option serialization to a map with the QName key “method” with the value “xml” and the QName key “indent” with the value “true”.

a=1 ::key=value serialization::method=xml serialization::indent=true

Sets the option a to 1 and the option parameters to a map with the QName key “key” and the value “value” and the option serialization to a map with the QName key “method” with the value “xml” and the QName key “indent” with the value “true”.

Note that the use of serialization in these examples assumes that the step being run has an option named serialization. This doesn’t otherwise have any effect on the serialization options of any output port.

The help command

xmlcalabash help

Displays a short summary of the command line options and arguments, not dissimilar to the preceding section. If help is requested, all of the other command line arguments are ignored.

The version command

xmlcalabash version [--verbosity:verbosity]

Displays the XML Calabash version and the version of Saxon:

$ xmlcalabash version
XML Calabash version 3.0.0-alpha28 (build 798883f.194.1a1008, 26 Apr 2025)
Running with Saxon HE version 12.5 using at most 1 of 12 available threads

Most options are ignored when the version command is used, but if the debug level of --verbosity is requested, the version summary will include details about third party dependencies such as the HTML parser and XML resolver. In this case, the output is formatted in a way that can more easily be parsed, for example by a shell script.

$ xmlcalabash --debug
PRODUCT_NAME=XML Calabash
VERSION=3.0.0-alpha28
BUILD_DATE=2025-04-26
BUILD_ID=798883f.194.1a1008
SAXON_EDITION=HE
VENDOR_NAME=Norm Tovey-Walsh
VENDOR_URI=https://xmlcalabash.com/
THREADS=1
MAX_THREADS=12
DEPENDENCY_brotliDec=0.1.2
DEPENDENCY_commonsCodec=1.17.0
DEPENDENCY_commonsCompress=1.27.1
DEPENDENCY_flexmarkAll=0.64.8
DEPENDENCY_graalvmJS=23.1.5
DEPENDENCY_htmlparser=1.4.16
DEPENDENCY_httpClient=5.3.1
DEPENDENCY_jing=20241231
DEPENDENCY_jsonSchemaValidator=1.5.4
DEPENDENCY_saxon=12.5
DEPENDENCY_schxslt2=1.3.1
DEPENDENCY_sinclude=5.3.0
DEPENDENCY_slf4j=2.0.16
DEPENDENCY_tukaaniXz=1.10
DEPENDENCY_uuidCreator=6.0.0
DEPENDENCY_xercesImpl=2.12.2
DEPENDENCY_xmlResolver=6.0.12
Important

These are the compile-time dependencies, the versions that the processor expected. The versions actually used are controlled by what appears on the classpath at runtime.