1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of six parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal>, <literal>log</literal>
62 and <literal>zoom</literal>.
67 The <literal>torus</literal> element specifies target profiles
68 and takes the following content:
72 <term>attribute <literal>url</literal></term>
75 URL of Web service to be used when fetch target profiles from
76 a remote service (Torus normally).
79 The sequence <literal>%query</literal> is replaced with a CQL
80 query for the Torus search.
83 The special sequence <literal>%realm</literal> is replaced by value
84 of attribute <literal>realm</literal> or by realm DATABASE argument.
87 The special sequence <literal>%db</literal> is replaced with
88 a single database while searching. Note that this sequence
89 is no longer needed, because the <literal>%query</literal> can already
90 query for a single database by using CQL query
91 <literal>udb==...</literal>.
96 <term>attribute <literal>content_url</literal></term>
99 URL of Web service to be used to fetch target profile
100 for a given database (udb) of type content. Semantics otherwise like
101 <literal>url</literal> attribute above.
106 <term>attribute <literal>realm</literal></term>
109 The default realm value. Used for %realm in URL, unless
110 specified in DATABASE argument.
115 <term>attribute <literal>proxy</literal></term>
118 HTTP proxy to bse used for fetching target profiles.
123 <term>attribute <literal>xsldir</literal></term>
126 Directory that is searched for XSL stylesheets. Stylesheets
127 are specified in the target profile by the
128 <literal>transform</literal> element.
133 <term>attribute <literal>element_transform</literal></term>
136 Specifies the element that triggers retrieval and transform using
137 the parameters elementSet, recordEncoding, requestSyntax, transform
138 from the target profile. Default value
139 is "pz2", due to the fact that for historical reasons the
140 common format is that used in Pazpar2.
145 <term>attribute <literal>element_raw</literal></term>
148 Specifies an element that triggers retrieval using the
149 parameters elementSet, recordEncoding, requestSyntax from the
150 target profile. Same actions as for element_transform, but without
151 the XSL transform. Useful for debugging.
152 The default value is "raw".
157 <term>attribute <literal>explain_xsl</literal></term>
160 Specifies a stylesheet that converts one or more Torus records
161 to ZeeExplain records. The content of recordData is assumed to be
162 holding each Explain record.
167 <term>attribute <literal>record_xsl</literal></term>
170 Specifies a stylesheet that converts retrieval records after
171 transform/literal operations.
174 When Metaproxy creates a content proxy session, the XSL parameter
175 <literal>cproxyhost</literal> is passed to the transform.
180 <term>element <literal>records</literal></term>
183 Local target profiles. This element may includes zero or
184 more <literal>record</literal> elements (one per target
185 profile). See section TARGET PROFILE.
191 <refsect2 id="fieldmap">
192 <title>fieldmap</title>
194 The <literal>fieldmap</literal> may be specified zero or more times and
195 specifies the map from CQL fields to CCL fields and takes the
200 <term>attribute <literal>cql</literal></term>
203 CQL field that we are mapping "from".
208 <term>attribute <literal>ccl</literal></term>
211 CCL field that we are mapping "to".
217 <refsect2 id="cclmap_base">
218 <title>cclmap</title>
220 The third part of the configuration consists of zero or more
221 <literal>cclmap</literal> elements that specifies
222 <emphasis>base</emphasis> CCL profile to be used for all targets.
223 This configuration, thus, will be combined with cclmap-definitions
224 from the target profile.
228 <title>contentProxy</title>
230 The <literal>contentProxy</literal> element controls content proxy'in.
232 is optional and must only be defined if content proxy'ing is enabled.
236 <term>attribute <literal>config_file</literal></term>
239 Specifies the file that configures the cf-proxy system. Metaproxy
240 uses setting <literal>sessiondir</literal> and
241 <literal>proxyhostname</literal> from that file to configure
242 name of proxy host and directory of parameter files for the cf-proxy.
247 <term>attribute <literal>server</literal></term>
250 Specifies the content proxy host. The host is of the form
251 host[:port]. That is without a method (such as HTTP) and optional
256 This setting is deprecated. Use the config_file (above)
257 to inform about the proxy server.
263 <term>attribute <literal>tmp_file</literal></term>
266 Specifies a filename of a session file for content proxy'ing. The
267 file should be an absolute filename that includes
268 <literal>XXXXXX</literal> which is replaced by a unique filename
269 using the mkstemp(3) system call. The default value of this
270 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
274 This setting is deprecated. Use the config_file (above)
275 to inform about the session file area.
285 The <literal>log</literal> element controls logging for the
290 <term>attribute <literal>apdu</literal></term>
293 If the value of apdu is "true", then protocol packages
294 (APDUs and HTTP packages) from the ZOOM filter will be
295 logged to the yaz_log system. A value of "false" will
296 not perform logging of protocol packages (the default
307 The <literal>zoom</literal> element controls settings for the
312 <term>attribute <literal>timeout</literal></term>
315 Is an integer that specifies, in seconds, how long an operation
316 may take before ZOOM gives up. Default value is 40.
325 <title>QUERY HANDLING</title>
327 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
331 Queries are converted in two separate steps. In the first step
332 the input query is converted to RPN/Type-1. This is always
333 the common internal format between step 1 and step 2.
334 In step 2 the query is converted to the native query type of the target.
337 Step 1: for RPN, the query is passed un-modified to the target.
340 Step 1: for CCL, the query is converted to RPN via
341 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
342 the target profile as well as
343 <link linkend="cclmap_base">base CCL maps</link>.
346 Step 1: For CQL, the query is converted to CCL. The mappings of
347 CQL fields to CCL fields are handled by
348 <link linkend="fieldmap"><literal>fieldmap</literal></link>
349 elements as part of the target profile. The resulting query, CCL,
350 is the converted to RPN using the schema mentioned earlier (via
351 <literal>cclmap</literal>).
354 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
355 If the target is SRU-based, the RPN will be converted to CQL.
356 If the target is SOLR-based, the RPN will be converted to SOLR's query
362 <title>SORTING</title>
364 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
365 which was introduced in SRU version 1.2. The conversion from SORTBY clause
366 to native sort for some target is driven by the two parameters:
367 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
368 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
371 If a sort field that does not have an equivalent
372 <literal>sortmap_</literal>-mapping is passed un-modified through the
373 conversion. It doesn't throw a diagnostic.
378 <title>TARGET PROFILE</title>
380 The ZOOM module is driven by a number of settings that specifies how
381 to handle each target.
382 Note that unknown elements are silently <emphasis>ignored</emphasis>.
385 The elements, in alphabetical order, are:
389 <term id="zoom-torus-authentication">authentication</term><listitem>
391 Authentication parameters to be sent to the target. For
392 Z39.50 targets, this will be sent as part of the
393 Init Request. Authentication consists of two components: username
394 and password, separated by a slash.
397 If this value is omitted or empty no authentication information is sent.
402 <varlistentry id="cclmap">
403 <term>cclmap_<replaceable>field</replaceable></term><listitem>
405 This value specifies CCL field (qualifier) definition for some
406 field. For Z39.50 targets this most likely will specify the
407 mapping to a numeric use attribute + a structure attribute.
408 For SRU targets, the use attribute should be string based, in
409 order to make the RPN to CQL conversion work properly (step 2).
415 <term>cfAuth</term><listitem>
417 When cfAuth is defined, its value will be used as authentication
418 to backend target and authentication setting will be specified
419 as part of a database. This is like a "proxy" for authentication and
420 is used for Connector Framework based targets.
426 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
428 Specifies HTTP proxy for the target in the form
429 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
435 <term>cfSubDB</term><listitem>
437 Specifies sub database for a Connector Framework based target.
442 <varlistentry id="zoom-torus-contentConnector">
443 <term>contentConnector</term><listitem>
445 Specifies a database for content-based proxy'ing.
451 <term>elementSet</term><listitem>
453 Specifies the elementSet to be sent to the target if record
454 transform is enabled (not to be confused' with the record_transform
455 module). The record transform is enabled only if the client uses
456 record syntax = XML and a element set determined by
457 the <literal>element_transform</literal> /
458 <literal>element_raw</literal> from the configuration.
459 By default that is the element sets <literal>pz2</literal>
460 and <literal>raw</literal>.
461 If record transform is not enabled, this setting is
462 not used and the element set specified by the client
469 <term>literalTransform</term><listitem>
471 Specifies a XSL stylesheet to be used if record
472 transform is anabled; see description of elementSet.
473 The XSL transform is only used if the element set is set to the
474 value of <literal>element_transform</literal> in the configuration.
477 The value of literalTransform is the XSL - string encoded.
483 <term>piggyback</term><listitem>
485 A value of 1/true is a hint to the ZOOM module that this Z39.50
486 target supports piggyback searches, ie Search Response with
487 records. Any other value (false) will prevent the ZOOM module
488 to make use of piggyback (all records part of Present Response).
494 <term>queryEncoding</term><listitem>
496 If this value is defined, all queries will be converted
497 to this encoding. This should be used for all Z39.50 targets that
498 do not use UTF-8 for query terms.
504 <term>recordEncoding</term><listitem>
506 Specifies the character encoding of records that are returned
507 by the target. This is primarily used for targets were records
508 are not UTF-8 encoded already. This setting is only used
509 if the record transform is enabled (see description of elementSet).
515 <term>requestSyntax</term><listitem>
517 Specifies the record syntax to be specified for the target
518 if record transform is enabled; see description of elementSet.
519 If record transform is not enabled, the record syntax of the
520 client is passed verbatim to the target.
525 <varlistentry id="sortmap">
526 <term>sortmap_<replaceable>field</replaceable></term><listitem>
528 This value the native field for a target. The form of the value is
529 given by <link linkend="sortStrategy">sortStrategy</link>.
534 <varlistentry id="sortStrategy">
535 <term>sortStrategy</term><listitem>
537 Specifies sort strategy for a target. One of:
538 <literal>z3950</literal>, <literal>type7</literal>,
539 <literal>cql</literal>, <literal>sru11</literal> or
540 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
541 or CQL sortby depending on whether Type-1 or CQL is
542 actually sent to the target.
548 <term>sru</term><listitem>
550 If this setting is set, it specifies that the target is web service
551 based and must be one of : <literal>get</literal>,
552 <literal>post</literal>, <literal>soap</literal>
553 or <literal>solr</literal>.
559 <term>sruVersion</term><listitem>
561 Specifies the SRU version to use. It unset, version 1.2 will be
562 used. Some servers do not support this version, in which case
563 version 1.1 or even 1.0 could be set it.
569 <term>transform</term><listitem>
571 Specifies a XSL stylesheet filename to be used if record
572 transform is anabled; see description of elementSet.
573 The XSL transform is only used if the element set is set to the
574 value of <literal>element_transform</literal> in the configuration.
580 <term>udb</term><listitem>
582 This value is required and specifies the unique database for
583 this profile . All target profiles should hold a unique database.
588 <varlistentry id="urlRecipe">
589 <term>urlRecipe</term><listitem>
591 The value of this field is a string that generates a dynamic link
592 based on record content. If the resulting string is non-zero in length
593 a new field, <literal>metadata</literal> with attribute
594 <literal>type="generated-url"</literal> is generated.
595 The contents of this field is the result of the URL recipe conversion.
596 The urlRecipe value may refer to an existing metadata element by
597 ${field[pattern/result/flags]}, which will take content
598 of field and perform a regular expression conversion using the pattern
599 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
600 metadata element <literal>title</literal> and converts one or more
601 spaces to a plus character.
607 <term>zurl</term><listitem>
609 This is setting is mandatory and specifies the ZURL of the
610 target in the form of host/database. The HTTP method should
611 not be provided as this is guessed from the "sru" attribute value.
618 <title>DATABASE parameters</title>
620 Extra information may be carried in the Z39.50 Database or SRU path,
621 such as authentication to be passed to backend etc. Some of
622 the parameters override TARGET profile values. The format is
625 udb,parm1=value1&parm2=value2&...
628 Where udb is the unique database recognised by the backend and parm1,
629 value1, .. are parameters to be passed. The following describes the
630 supported parameters. Like form values in HTTP the parameters and
631 values are URL encoded. The separator, though, between udb and parameters
632 is a comma rather than a question mark. What follows question mark are
633 HTTP arguments (in this case SRU arguments).
640 Specifies user to be passed to backend. If this parameter is
641 omitted, the user will be taken from TARGET profile setting
642 <link linkend="zoom-torus-authentication">
643 <literal>authentication</literal>
650 <term>password</term>
653 Specifies password to be passed to backend. If this parameters is
654 omitted, the password will be taken from TARGET profile setting
655 <link linkend="zoom-torus-authentication">
656 <literal>authentication</literal>
666 Specifies one or more proxies for backend. If this parameter is
667 omitted, the proxy will be taken from TARGET profile setting
668 <link linkend="zoom-torus-cfproxy">
669 <literal>cfProxy</literal></link>.
670 The parameter is a list of comma-separated host:port entries.
671 Bost host and port must be given for each proxy.
676 <term>cproxysession</term>
679 Session ID for content proxy. This parameter is, generally,
680 not used by anything but the content proxy itself.
685 <term>nocproxy</term>
688 If this parameter is specified, content-proyxing is disabled
697 Session realm to be used for this target, changed the resulting
698 URL to be used for getting a target profile, by changing the
699 value that gets substituted for the %realm string.
707 All parameters that has prefix x, dash are passed verbatim
715 <title>SCHEMA</title>
716 <literallayout><xi:include
717 xi:href="../xml/schema/filter_zoom.rnc"
719 xmlns:xi="http://www.w3.org/2001/XInclude" />
724 <title>EXAMPLES</title>
726 The following configuration illustrates most of the
731 url="http://torus.indexdata.com/src/records/?query=%query"
732 proxy="localhost:3128"
734 <fieldmap cql="cql.anywhere"/>
735 <fieldmap cql="cql.serverChoice"/>
736 <fieldmap cql="dc.creator" ccl="au"/>
737 <fieldmap cql="dc.title" ccl="ti"/>
738 <fieldmap cql="dc.subject" ccl="su"/>
742 <attr type="u" value="12"/>
743 <attr type="s" value="107"/>
756 <title>SEE ALSO</title>
759 <refentrytitle>metaproxy</refentrytitle>
760 <manvolnum>1</manvolnum>
765 <refentrytitle>virt_db</refentrytitle>
766 <manvolnum>3mp</manvolnum>
774 <!-- Keep this comment at the end of the file