1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of six parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal>, <literal>log</literal>
62 and <literal>zoom</literal>.
67 The <literal>torus</literal> element specifies target profiles
68 and takes the following content:
72 <term>attribute <literal>url</literal></term>
75 URL of Web service to be used when fetch target profiles from
76 a remote service (Torus normally).
79 The sequence <literal>%query</literal> is replaced with a CQL
80 query for the Torus search.
83 The special sequence <literal>%realm</literal> is replaced by value
84 of attribute <literal>realm</literal> or by realm DATABASE argument.
87 The special sequence <literal>%db</literal> is replaced with
88 a single database while searching. Note that this sequence
89 is no longer needed, because the <literal>%query</literal> can already
90 query for a single database by using CQL query
91 <literal>udb==...</literal>.
96 <term>attribute <literal>content_url</literal></term>
99 URL of Web service to be used to fetch target profile
100 for a given database (udb) of type content. Semantics otherwise like
101 <literal>url</literal> attribute above.
106 <term>attribute <literal>realm</literal></term>
109 The default realm value. Used for %realm in URL, unless
110 specified in DATABASE argument.
115 <term>attribute <literal>proxy</literal></term>
118 HTTP proxy to bse used for fetching target profiles.
123 <term>attribute <literal>xsldir</literal></term>
126 Directory that is searched for XSL stylesheets. Stylesheets
127 are specified in the target profile by the
128 <literal>transform</literal> element.
133 <term>attribute <literal>element_transform</literal></term>
136 Specifies the element that triggers retrieval and transform using
137 the parameters elementSet, recordEncoding, requestSyntax, transform
138 from the target profile. Default value
139 is "pz2", due to the fact that for historical reasons the
140 common format is that used in Pazpar2.
145 <term>attribute <literal>element_raw</literal></term>
148 Specifies an element that triggers retrieval using the
149 parameters elementSet, recordEncoding, requestSyntax from the
150 target profile. Same actions as for element_transform, but without
151 the XSL transform. Useful for debugging.
152 The default value is "raw".
157 <term>attribute <literal>explain_xsl</literal></term>
160 Specifies a stylesheet that converts one or more Torus records
161 to ZeeExplain records. The content of recordData is assumed to be
162 holding each Explain record.
167 <term>attribute <literal>record_xsl</literal></term>
170 Specifies a stylesheet that converts retrieval records after
171 transform/literal operations.
174 When Metaproxy creates a content proxy session, the XSL parameter
175 <literal>cproxyhost</literal> is passed to the transform.
180 <term>element <literal>records</literal></term>
183 Local target profiles. This element may includes zero or
184 more <literal>record</literal> elements (one per target
185 profile). See section TARGET PROFILE.
191 <refsect2 id="fieldmap">
192 <title>fieldmap</title>
194 The <literal>fieldmap</literal> may be specified zero or more times and
195 specifies the map from CQL fields to CCL fields and takes the
200 <term>attribute <literal>cql</literal></term>
203 CQL field that we are mapping "from".
208 <term>attribute <literal>ccl</literal></term>
211 CCL field that we are mapping "to".
217 <refsect2 id="cclmap_base">
218 <title>cclmap</title>
220 The third part of the configuration consists of zero or more
221 <literal>cclmap</literal> elements that specifies
222 <emphasis>base</emphasis> CCL profile to be used for all targets.
223 This configuration, thus, will be combined with cclmap-definitions
224 from the target profile.
228 <title>contentProxy</title>
230 The <literal>contentProxy</literal> element controls content proxy'in.
232 is optional and must only be defined if content proxy'ing is enabled.
236 <term>attribute <literal>config_file</literal></term>
239 Specifies the file that configures the cf-proxy system. Metaproxy
240 uses setting <literal>sessiondir</literal> and
241 <literal>proxyhostname</literal> from that file to configure
242 name of proxy host and directory of parameter files for the cf-proxy.
247 <term>attribute <literal>server</literal></term>
250 Specifies the content proxy host. The host is of the form
251 host[:port]. That is without a method (such as HTTP) and optional
256 This setting is deprecated. Use the config_file (above)
257 to inform about the proxy server.
263 <term>attribute <literal>tmp_file</literal></term>
266 Specifies a filename of a session file for content proxy'ing. The
267 file should be an absolute filename that includes
268 <literal>XXXXXX</literal> which is replaced by a unique filename
269 using the mkstemp(3) system call. The default value of this
270 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
274 This setting is deprecated. Use the config_file (above)
275 to inform about the session file area.
285 The <literal>log</literal> element controls logging for the
290 <term>attribute <literal>apdu</literal></term>
293 If the value of apdu is "true", then protocol packages
294 (APDUs and HTTP packages) from the ZOOM filter will be
295 logged to the yaz_log system. A value of "false" will
296 not perform logging of protocol packages (the default
307 The <literal>zoom</literal> element controls settings for the
312 <term>attribute <literal>timeout</literal></term>
315 Is an integer that specifies, in seconds, how long an operation
316 may take before ZOOM gives up. Default value is 40.
321 <term>attribute <literal>proxy_timeout</literal></term>
324 Is an integer that specifies, in seconds, how long an operation
325 a proxy check will wait before giving up. Default value is 1.
334 <title>QUERY HANDLING</title>
336 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
340 Queries are converted in two separate steps. In the first step
341 the input query is converted to RPN/Type-1. This is always
342 the common internal format between step 1 and step 2.
343 In step 2 the query is converted to the native query type of the target.
346 Step 1: for RPN, the query is passed un-modified to the target.
349 Step 1: for CCL, the query is converted to RPN via
350 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
351 the target profile as well as
352 <link linkend="cclmap_base">base CCL maps</link>.
355 Step 1: For CQL, the query is converted to CCL. The mappings of
356 CQL fields to CCL fields are handled by
357 <link linkend="fieldmap"><literal>fieldmap</literal></link>
358 elements as part of the target profile. The resulting query, CCL,
359 is the converted to RPN using the schema mentioned earlier (via
360 <literal>cclmap</literal>).
363 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
364 If the target is SRU-based, the RPN will be converted to CQL.
365 If the target is SOLR-based, the RPN will be converted to SOLR's query
371 <title>SORTING</title>
373 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
374 which was introduced in SRU version 1.2. The conversion from SORTBY clause
375 to native sort for some target is driven by the two parameters:
376 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
377 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
380 If a sort field that does not have an equivalent
381 <literal>sortmap_</literal>-mapping is passed un-modified through the
382 conversion. It doesn't throw a diagnostic.
387 <title>TARGET PROFILE</title>
389 The ZOOM module is driven by a number of settings that specifies how
390 to handle each target.
391 Note that unknown elements are silently <emphasis>ignored</emphasis>.
394 The elements, in alphabetical order, are:
398 <term id="zoom-torus-authentication">authentication</term><listitem>
400 Authentication parameters to be sent to the target. For
401 Z39.50 targets, this will be sent as part of the
402 Init Request. Authentication consists of two components: username
403 and password, separated by a slash.
406 If this value is omitted or empty no authentication information is sent.
411 <varlistentry id="cclmap">
412 <term>cclmap_<replaceable>field</replaceable></term><listitem>
414 This value specifies CCL field (qualifier) definition for some
415 field. For Z39.50 targets this most likely will specify the
416 mapping to a numeric use attribute + a structure attribute.
417 For SRU targets, the use attribute should be string based, in
418 order to make the RPN to CQL conversion work properly (step 2).
424 <term>cfAuth</term><listitem>
426 When cfAuth is defined, its value will be used as authentication
427 to backend target and authentication setting will be specified
428 as part of a database. This is like a "proxy" for authentication and
429 is used for Connector Framework based targets.
435 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
437 Specifies HTTP proxy for the target in the form
438 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
444 <term>cfSubDB</term><listitem>
446 Specifies sub database for a Connector Framework based target.
451 <varlistentry id="zoom-torus-contentConnector">
452 <term>contentConnector</term><listitem>
454 Specifies a database for content-based proxy'ing.
460 <term>elementSet</term><listitem>
462 Specifies the elementSet to be sent to the target if record
463 transform is enabled (not to be confused' with the record_transform
464 module). The record transform is enabled only if the client uses
465 record syntax = XML and a element set determined by
466 the <literal>element_transform</literal> /
467 <literal>element_raw</literal> from the configuration.
468 By default that is the element sets <literal>pz2</literal>
469 and <literal>raw</literal>.
470 If record transform is not enabled, this setting is
471 not used and the element set specified by the client
478 <term>literalTransform</term><listitem>
480 Specifies a XSL stylesheet to be used if record
481 transform is anabled; see description of elementSet.
482 The XSL transform is only used if the element set is set to the
483 value of <literal>element_transform</literal> in the configuration.
486 The value of literalTransform is the XSL - string encoded.
492 <term>piggyback</term><listitem>
494 A value of 1/true is a hint to the ZOOM module that this Z39.50
495 target supports piggyback searches, ie Search Response with
496 records. Any other value (false) will prevent the ZOOM module
497 to make use of piggyback (all records part of Present Response).
503 <term>queryEncoding</term><listitem>
505 If this value is defined, all queries will be converted
506 to this encoding. This should be used for all Z39.50 targets that
507 do not use UTF-8 for query terms.
513 <term>recordEncoding</term><listitem>
515 Specifies the character encoding of records that are returned
516 by the target. This is primarily used for targets were records
517 are not UTF-8 encoded already. This setting is only used
518 if the record transform is enabled (see description of elementSet).
524 <term>requestSyntax</term><listitem>
526 Specifies the record syntax to be specified for the target
527 if record transform is enabled; see description of elementSet.
528 If record transform is not enabled, the record syntax of the
529 client is passed verbatim to the target.
534 <varlistentry id="sortmap">
535 <term>sortmap_<replaceable>field</replaceable></term><listitem>
537 This value the native field for a target. The form of the value is
538 given by <link linkend="sortStrategy">sortStrategy</link>.
543 <varlistentry id="sortStrategy">
544 <term>sortStrategy</term><listitem>
546 Specifies sort strategy for a target. One of:
547 <literal>z3950</literal>, <literal>type7</literal>,
548 <literal>cql</literal>, <literal>sru11</literal> or
549 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
550 or CQL sortby depending on whether Type-1 or CQL is
551 actually sent to the target.
557 <term>sru</term><listitem>
559 If this setting is set, it specifies that the target is web service
560 based and must be one of : <literal>get</literal>,
561 <literal>post</literal>, <literal>soap</literal>
562 or <literal>solr</literal>.
568 <term>sruVersion</term><listitem>
570 Specifies the SRU version to use. It unset, version 1.2 will be
571 used. Some servers do not support this version, in which case
572 version 1.1 or even 1.0 could be set it.
578 <term>transform</term><listitem>
580 Specifies a XSL stylesheet filename to be used if record
581 transform is anabled; see description of elementSet.
582 The XSL transform is only used if the element set is set to the
583 value of <literal>element_transform</literal> in the configuration.
589 <term>udb</term><listitem>
591 This value is required and specifies the unique database for
592 this profile . All target profiles should hold a unique database.
597 <varlistentry id="urlRecipe">
598 <term>urlRecipe</term><listitem>
600 The value of this field is a string that generates a dynamic link
601 based on record content. If the resulting string is non-zero in length
602 a new field, <literal>metadata</literal> with attribute
603 <literal>type="generated-url"</literal> is generated.
604 The contents of this field is the result of the URL recipe conversion.
605 The urlRecipe value may refer to an existing metadata element by
606 ${field[pattern/result/flags]}, which will take content
607 of field and perform a regular expression conversion using the pattern
608 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
609 metadata element <literal>title</literal> and converts one or more
610 spaces to a plus character.
616 <term>zurl</term><listitem>
618 This is setting is mandatory and specifies the ZURL of the
619 target in the form of host/database. The HTTP method should
620 not be provided as this is guessed from the "sru" attribute value.
627 <title>DATABASE parameters</title>
629 Extra information may be carried in the Z39.50 Database or SRU path,
630 such as authentication to be passed to backend etc. Some of
631 the parameters override TARGET profile values. The format is
634 udb,parm1=value1&parm2=value2&...
637 Where udb is the unique database recognised by the backend and parm1,
638 value1, .. are parameters to be passed. The following describes the
639 supported parameters. Like form values in HTTP the parameters and
640 values are URL encoded. The separator, though, between udb and parameters
641 is a comma rather than a question mark. What follows question mark are
642 HTTP arguments (in this case SRU arguments).
649 Specifies user to be passed to backend. If this parameter is
650 omitted, the user will be taken from TARGET profile setting
651 <link linkend="zoom-torus-authentication">
652 <literal>authentication</literal>
659 <term>password</term>
662 Specifies password to be passed to backend. If this parameters is
663 omitted, the password will be taken from TARGET profile setting
664 <link linkend="zoom-torus-authentication">
665 <literal>authentication</literal>
675 Specifies one or more proxies for backend. If this parameter is
676 omitted, the proxy will be taken from TARGET profile setting
677 <link linkend="zoom-torus-cfproxy">
678 <literal>cfProxy</literal></link>.
679 The parameter is a list of comma-separated host:port entries.
680 Bost host and port must be given for each proxy.
685 <term>cproxysession</term>
688 Session ID for content proxy. This parameter is, generally,
689 not used by anything but the content proxy itself.
694 <term>nocproxy</term>
697 If this parameter is specified, content-proyxing is disabled
706 Session realm to be used for this target, changed the resulting
707 URL to be used for getting a target profile, by changing the
708 value that gets substituted for the %realm string.
716 All parameters that has prefix x, dash are passed verbatim
724 <title>SCHEMA</title>
725 <literallayout><xi:include
726 xi:href="../xml/schema/filter_zoom.rnc"
728 xmlns:xi="http://www.w3.org/2001/XInclude" />
733 <title>EXAMPLES</title>
735 The following configuration illustrates most of the
740 url="http://torus.indexdata.com/src/records/?query=%query"
741 proxy="localhost:3128"
743 <fieldmap cql="cql.anywhere"/>
744 <fieldmap cql="cql.serverChoice"/>
745 <fieldmap cql="dc.creator" ccl="au"/>
746 <fieldmap cql="dc.title" ccl="ti"/>
747 <fieldmap cql="dc.subject" ccl="su"/>
751 <attr type="u" value="12"/>
752 <attr type="s" value="107"/>
765 <title>SEE ALSO</title>
768 <refentrytitle>metaproxy</refentrytitle>
769 <manvolnum>1</manvolnum>
774 <refentrytitle>virt_db</refentrytitle>
775 <manvolnum>3mp</manvolnum>
783 <!-- Keep this comment at the end of the file