Chapter 2. Running XML Calabash
The easiest way to run XML Calabash is directly from the jar file:
java -jar xmlcalabash-3.0.0-alpha28.jar options…
Run this way, all of the dependencies included in the
lib
directory are automatically included on the classpath.
You can also run the main class directly, for example in a build system or a shell script:
java com.xmlcalabash.app.Main options…
But if you choose this form, you must ensure that the classpath contains both the XML Calabash jar file and all of the necessary dependencies. One feature of the second form is that it allows you to change or update the dependencies.
In the fullness of time, XML Calabash will be available through Maven which has extensive features for managing dependencies. In the short term, it’s just a bit messy.
The one library that you are likely to want to change in Saxon. If you have a license for Saxon PE or Saxon EE, it makes sense to swap out the Saxon HE library that ships with XML Calabash for PE or EE.
To save you the trouble of constructing and managing a large classpath, the
alpha distribution of XML Calabash is configured with a hack. Simply
delete the Saxon-HE-12.5.jar
file from the
lib
directory and copy in your PE or EE jar instead. You
must use a 12.5 jar file for this purpose!
Be careful to make sure there’s only one Saxon jar file in the
lib
directory. Having multiple versions of the same library on the classpath
is an invitation for subtle and mysterious crashes.
Saxon generally searches for a license file on the classpath, which isn’t going
to work with this hack. Instead, you can use the SAXON_HOME
environment
variable, as described in
the
documentation.
Elsewhere in this document, we assume that the command “xmlcalabash
”
runs XML Calabash. This can be replaced by the java -jar …
version above or
by your own script.
XML Calabash has a command driven command-line interface. It supports
three commands: run
, to run a pipeline;
version
, to print the version, and
help
to display help. If no command is
given, the run
command is assumed.
The run
command
[--pipe] [--input:[type@]port=uri…] [--output:port=filespec…]
[--namespace:prefix=uri…] [--init:class-name…]
[--graphs:graph-output-directory]
[--verbosity:verbosity] [--explain] [--visualizer:name]
[--trace:output-file] [--trace-documents:output-directory]
[--assertions:level]
[--licensed] [--debug] [--debugger] [--nogo]
[--catalog:catalog-file-uri…] [--xml-schema:xml-schema-file-uri…]
[--validation-mode:mode] [--try-namespaces] [--use-location-hints] [--help]
[--step:step-name] [pipeline.xpl] [option=value…]
Where the options and arguments are:
run
, selects the run commandIf the command is omitted,
run
is assumed.--configuration:configuration-file
, identifies a configuration fileWhen XML Calabash begins, it reads configuration settings from a configuration file. If you don’t specify a configuration file, it will search first for
.xmlcalabash3
in the current directory and then in your home directory.--pipe:boolean
, sets piped modeIf piped mode is enabled, XML Calabash will behave like a Unix pipeline. If there is no binding for the primary input port, it will read it from standard input. If there is no binding for the primary output port, it will write it to standard output. If not specified, the default setting comes from the
piped-io
configuration setting.--input:[type@]port=uri
, identifies an inputThis option associates the resource identified by the uri with the input port named port. The pipeline must have an input port named port.
If type is provided, it is the content type that will be used to parse the input; it must be a valid content type. If no content type is provided, the content type will be inferred from the URI, if possible.
If the
--input
option is repeated for the same port, the resources become a sequence of documents on that port, in the order specified.If the input uri provided is “
-
”, input will be read from standard input. When standard input is read, if no type is provided, it will be parsed as the first usable content type listed on the named input port. It will be parsed as XML if the input port doesn’t specify any usable content types.ⓘNoteA content type is “usable” if it is not a negated type, if it doesn’t use wildcards for the type or subtype, or if it’s recognized as a “+xml” or “+json” content type.
--output:port=filespec
, identifies an outputIn the simplest case, at most one document appears on the output port and filespec is just a filename. The filespec can also be a template. The output documents are numbered, starting at 1. Within the filespec, the string “
%d
” will be replaced by the document number. (The string “%h
” will be replaced by the document number in hexadecimal and “%o
” will be replaced by the document number in octal, for no good reason.) Zeros can be added to pad the value, “%00d
” will produce a document number that is at least two digits long, padded on the left with 0’s if there are fewer than 10 documents. For completeness, a literal “%” can be added to the filename with “%%
” in the filespec.If the filespec does not contain a number template, then all of the outputs on the port will be concatenated to the same file. You may not repeat the
--output
option for the same port.The pipeline must have an output port named port.
If the output filespec provided is “
-
”, output from that port will be written to standard output. At most one output port may be explicitly bound to standard output.In piped mode, at most one port will write to standard output. If there is no explicit binding to standard output, the primary output port writes to standard output. If the primary output port is bound to some other output, no output will be sent to standard output.
If piped mode is not enabled, and no port is explicitly bound to standard output, the pipeline will write the output from all otherwise unbound output ports to standard output. If exactly one port is being written to standard output, and if that port cannot produce a sequence, then the output is written directly to standard output.
If more than one port may appear on standard output, or if a sequence of documents may appear, and if standard output appears to be going to a terminal window, “decoration” is added as an aid to comprehension:
A header is printed before the output identifying the port name, document number, and base URI.
A line of equal signs is printed as a separator between documents (if more than one document is output).
When XML or HTML content is output, the XML declaration is omitted.
Indentation is turned on.
The method for determining whether output is going to a terminal or being redirected isn’t terribly sophisticated and may be wrong in some circumstances. It’s safer to explicitly enable piped mode or use
--output
to write to a file if you want to save the output.--namespace:prefix=uri
, identifies a namespace bindingBinds the specified prefix to the uri. This has no effect on the pipeline; these bindings are only used when evaluating the
--step
option and in expressions used to define options.The default namespace bindings on the command line are:
Prefix Namespace URI array
http://www.w3.org/2005/xpath-functions/array
cx
http://xmlcalabash.com/ns/extensions
fn
http://www.w3.org/2005/xpath-functions
map
http://www.w3.org/2005/xpath-functions/map
math
http://www.w3.org/2005/xpath-functions/math
p
http://www.w3.org/ns/xproc
saxon
http://saxon.sf.net/
xs
http://www.w3.org/2001/XMLSchema
The
--namespace
option can add to, or change, these bindings.--init:class-name
, Saxon configuration initializerAttempts to load and execute the class named class-name, which must be available on the class path and must implement the
net.sf.saxon.lib.Initializer
interface. (This is analagous to the-init:
option on the Saxon command line API.)--graphs:graph-output-directory
, SVG graph outputsThis option writes hyperlinked SVG descriptions of pipelines, and their corresponding graphs, to the graph-output-directory. The descriptions are “boxes and arrows” diagrams of the connections between the steps. One SVG diagram is produced for each declared pipeline and its corresponding graph. An HTML index in the graph-output-directory makes them easy to browse. See Chapter 5, Pipelines vs. Graphs for more details.
The processor assumes that it “owns” the graph-output-directory; it will erase files and directories before creating the graph output.
ⓘNoteSome browsers have better support for SVG than others. If the diagrams are difficult to view, or if links don’t work correctly, a good first step is to try a different browser.
--verbosity:verbosity
There are five levels of verbosity. The level of verbosity determines how much detail is printed about the progress of a running pipeline.
trace
Lots of detail, show everything.
debug
Lots of detail
info
Show relevant status messages.
warn
Show only warnings and errors.
error
No messages except fatal errors
Setting the verbosity also sets the logging level.
--explain
, explain errorsEnables error explanations. If error explanations are enabled, when a pipeline fails, in addition to the error message, a short explanation of the cause will be provided.
--visualizer:name
Selects a visualizer for reporting pipeline progress. There are three defined visualizers.
silent
Shows no progress reports.
plain
Reports the name of each step when it begins running. Most steps manufactured automatically during graph construction are omitted. There is one option,
indent
which determines whether or not, and to what extent, reports are indented when they are nested inside compound steps.detail
Reports the name of each step and may report on the documents produced.
If the
steps
option istrue
, the progress of steps is recorded. (Defaults to true.)If the
documents
option istrue
, the documents produced during execution are recorded. (Defaults to false.)
Options can be specified on the command line as
key=value
pairs after the name. The name and any options are separated by a question mark (?
). Key-value pairs are separted by semicolons (;
). For example:--visualizer=detail?steps=true;documents=true
.--trace:output-file
, defines a runtime trace fileA runtime trace of the pipeline execution will be written to
output-file
. (If the--trace-documents
option is given but the--trace
option is not, the default output file is namedtrace.xml
in the--trace-documents
output directory.)For more information about tracking, see Appendix B, Tracing execution.
--trace-documents:output-directory
, defines a runtime trace directoryIf the
--trace-documents
option is given, the runtime trace will write a copy of every document that flows through the pipeline into the specified directory. These documents are identified in the--trace
file.--assertions:level
By default, assertions are disabled. You can enable them by setting an appropriate level:
error
Schematron reports are output and Schematron errors are treated as errors. (They cause the step to throw
Q{http://xmlcalabash.com/ns/error}XI0041
.)warning
(orwarn
)Schematron reports are output and Schematron errors are treated as warnings.
ignore
Assertions are ignored.
--licensed
Requests a licensed processor. In practice, a licensed processor is used by default, if one is available. However,
--licensed:false
can be used to explicitly request an unlicensed processor when Saxon PE or Saxon EE are on the classpath.Schema-aware processing requires Saxon EE and a valid Saxon license.
--debug
If debugging is enabled, the verbosity is set to at least the
debug
level and the backend logging framework’s “log level” is set to the verbosity.This produces a lot of messages, typically on the standard error output. XML Calabash uses SLF4J and Logback for logging. You can configure it in the usual ways.
--debugger
Start an interactive debugging session on the pipeline. See Chapter 8, The interactive debugger.
--nogo
Compile the pipeline, and produce graphs if they’re requested, but don’t actually run the pipeline.
--catalog:catalog-file-uri
Adds the specified catalog to the list of XML Catalogs that are used during resource resolution.
--xml-schema:xml-schema-file-uri
Adds the specified XML Schema to the validation context. This schema will be available in both implicit validation and when the
p:validate-with-xml-schema
step is used.--validation-mode:mode
This option enables implicit validation. The mode must be either “lax” or “strict”.
----try-namespaces
This option enables attempting to retrieve a schema using its namespace URI during implicit validation.
--use-location-hints
This option enables attempting to retrieve a schema using location hints during implicit validation.
--help
This is equivalent to issuing the
help
command. It’s provided as an option for convenience.--step:step
, identifies a stepThe
--step
option is a little bit overloaded. Its interpretation depends on whether or not a pipeline is specified.If a pipeline is specified, it must identify a
p:library
. The value of the--step
option is interpreted as the name of a step in that library. The named step will be run.For example:
$ xmlcalabash --step:helloWorld --library:examples.xpl …
If no pipeline is specified, the value of the
--step
option is interpreted as the type of a step. The (atomic) step with the specified type will be run. All of the inputs, outputs, and options specified apply to that step.For example:
$ xmlcalabash --step:p:xslt …
The
p
andcx
namespaces are bound by default. The--namespace
option can be used to change the namespace bindings.pipeline.xpl
, the pipeline to runThis identifies the pipeline to run. If the root element is
p:declare-step
, then that pipeline will be run. If the root element isp:library
, the first pipeline in the library will be run, unless thestep
option specifies an alternate pipeline.option=value
, sets an optionYou can provide values for pipeline options on the command line. These override any defaults declared in the pipeline. There must be a pipeline option (or a static option) named option.
If the option name includes “::”, then it is treated as a shortcut for setting the value in a map. In this case, the name before the “::” is the name of the actual parameter and the name after the “::” is the name of a key in that map. (With the additional special case that if the name begins with “::”, then the option name is take to be default option name, currently
parameters
.) This is intended to be somewhat reminiscent of XPath axes.In both cases, the name can be an EQName or it can use a prefix previously defined with
--namespace
; if the name is a simpleNCName
, it is not in a namespace.If the value begins with “?”, what follows is taken to be an XPath expression. That expression is evaluated using the namespace bindings defined. The context item is undefined. The result of evaluating the expression is the value of the option. If the value does not begin with a “?”, the whole string becomes the value as an
xs:untypedAtomic
.If multiple assignments to the same
option
(or map item) appear, the value will be a sequence with those values in the order specified.For example, suppose that the following sequences of options are given:
a=1 b=2
Sets the option
a
to 1 and the optionb
to 2.a=1 b=2+3
Sets the option
a
to 1 and the optionb
to “2+3”.a=1 a=2 b=?2+3
Sets the option
a
to (1, 2) and the optionb
to 5.a=1 parameters="map{'key': 'value'}"
Sets the option
a
to 1 and the optionparameters
to a map with the QName key “key” and the value “value”.a=1 parameters::key=value
Sets the option
a
to 1 and the optionparameters
to a map with the QName key “key” and the value “value”.a=1 parameters::key=value serialization::method=xml serialization::indent=true
Sets the option
a
to 1 and the optionparameters
to a map with the QName key “key” and the value “value” and the optionserialization
to a map with the QName key “method” with the value “xml” and the QName key “indent” with the value “true”.a=1 ::key=value serialization::method=xml serialization::indent=true
Sets the option
a
to 1 and the optionparameters
to a map with the QName key “key” and the value “value” and the optionserialization
to a map with the QName key “method” with the value “xml” and the QName key “indent” with the value “true”.
Note that the use of
serialization
in these examples assumes that the step being run has an option namedserialization
. This doesn’t otherwise have any effect on the serialization options of any output port.
The help
command
Displays a short summary of the command line options and arguments, not dissimilar to the preceding section. If help is requested, all of the other command line arguments are ignored.
The version
command
Displays the XML Calabash version and the version of Saxon:
$ xmlcalabash version
XML Calabash version 3.0.0-alpha28 (build 798883f.194.1a1008, 26 Apr 2025)
Running with Saxon HE version 12.5 using at most 1 of 12 available threads
Most options are ignored when the version
command is
used, but if the debug
level of --verbosity
is requested, the version summary will include details about third party
dependencies such as the HTML parser and XML resolver. In this case, the output
is formatted in a way that can more easily be parsed, for example by a shell script.
$ xmlcalabash --debug
PRODUCT_NAME=XML Calabash
VERSION=3.0.0-alpha28
BUILD_DATE=2025-04-26
BUILD_ID=798883f.194.1a1008
SAXON_EDITION=HE
VENDOR_NAME=Norm Tovey-Walsh
VENDOR_URI=https://xmlcalabash.com/
THREADS=1
MAX_THREADS=12
DEPENDENCY_brotliDec=0.1.2
DEPENDENCY_commonsCodec=1.17.0
DEPENDENCY_commonsCompress=1.27.1
DEPENDENCY_flexmarkAll=0.64.8
DEPENDENCY_graalvmJS=23.1.5
DEPENDENCY_htmlparser=1.4.16
DEPENDENCY_httpClient=5.3.1
DEPENDENCY_jing=20241231
DEPENDENCY_jsonSchemaValidator=1.5.4
DEPENDENCY_saxon=12.5
DEPENDENCY_schxslt2=1.3.1
DEPENDENCY_sinclude=5.3.0
DEPENDENCY_slf4j=2.0.16
DEPENDENCY_tukaaniXz=1.10
DEPENDENCY_uuidCreator=6.0.0
DEPENDENCY_xercesImpl=2.12.2
DEPENDENCY_xmlResolver=6.0.12
These are the compile-time dependencies, the versions that the processor expected. The versions actually used are controlled by what appears on the classpath at runtime.