1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of six parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal>, <literal>log</literal>
62 and <literal>zoom</literal>.
67 The <literal>torus</literal> element specifies target profiles
68 and takes the following content:
72 <term>attribute <literal>url</literal></term>
75 URL of Web service to be used when fetch target profiles from
76 a remote service (Torus normally).
79 The sequence <literal>%query</literal> is replaced with a CQL
80 query for the Torus search.
83 The special sequence <literal>%realm</literal> is replaced by value
84 of attribute <literal>realm</literal> or by realm DATABASE argument.
87 The special sequence <literal>%db</literal> is replaced with
88 a single database while searching. Note that this sequence
89 is no longer needed, because the <literal>%query</literal> can already
90 query for a single database by using CQL query
91 <literal>udb==...</literal>.
96 <term>attribute <literal>content_url</literal></term>
99 URL of Web service to be used to fetch target profile
100 for a given database (udb) of type content. Semantics otherwise like
101 <literal>url</literal> attribute above.
106 <term>attribute <literal>realm</literal></term>
109 The default realm value. Used for %realm in URL, unless
110 specified in DATABASE argument.
115 <term>attribute <literal>proxy</literal></term>
118 HTTP proxy to bse used for fetching target profiles.
123 <term>attribute <literal>xsldir</literal></term>
126 Directory that is searched for XSL stylesheets. Stylesheets
127 are specified in the target profile by the
128 <literal>transform</literal> element.
133 <term>attribute <literal>element_transform</literal></term>
136 Specifies the element that triggers retrieval and transform using
137 the parameters elementSet, recordEncoding, requestSyntax, transform
138 from the target profile. Default value
139 is "pz2", due to the fact that for historical reasons the
140 common format is that used in Pazpar2.
145 <term>attribute <literal>element_raw</literal></term>
148 Specifies an element that triggers retrieval using the
149 parameters elementSet, recordEncoding, requestSyntax from the
150 target profile. Same actions as for element_transform, but without
151 the XSL transform. Useful for debugging.
152 The default value is "raw".
157 <term>attribute <literal>explain_xsl</literal></term>
160 Specifies a stylesheet that converts one or more Torus records
161 to ZeeExplain records. The content of recordData is assumed to be
162 holding each Explain record.
167 <term>attribute <literal>record_xsl</literal></term>
170 Specifies a stylesheet that converts retrieval records after
171 transform/literal operations.
174 When Metaproxy creates a content proxy session, the XSL parameter
175 <literal>cproxyhost</literal> is passed to the transform.
180 <term>element <literal>records</literal></term>
183 Local target profiles. This element may includes zero or
184 more <literal>record</literal> elements (one per target
185 profile). See section TARGET PROFILE.
191 <refsect2 id="fieldmap">
192 <title>fieldmap</title>
194 The <literal>fieldmap</literal> may be specified zero or more times and
195 specifies the map from CQL fields to CCL fields and takes the
200 <term>attribute <literal>cql</literal></term>
203 CQL field that we are mapping "from".
208 <term>attribute <literal>ccl</literal></term>
211 CCL field that we are mapping "to".
217 <refsect2 id="cclmap_base">
218 <title>cclmap</title>
220 The third part of the configuration consists of zero or more
221 <literal>cclmap</literal> elements that specifies
222 <emphasis>base</emphasis> CCL profile to be used for all targets.
223 This configuration, thus, will be combined with cclmap-definitions
224 from the target profile.
228 <title>contentProxy</title>
230 The <literal>contentProxy</literal> element controls content proxy'in.
232 is optional and must only be defined if content proxy'ing is enabled.
236 <term>attribute <literal>server</literal></term>
239 Specifies the content proxy host. The host is of the form
240 host[:port]. That is without a method (such as HTTP) and optional
246 <term>attribute <literal>tmp_file</literal></term>
249 Specifies a filename of a session file for content proxy'ing. The
250 file should be an absolute filename that includes
251 <literal>XXXXXX</literal> which is replaced by a unique filename
252 using the mkstemp(3) system call. The default value of this
253 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
262 The <literal>log</literal> element controls logging for the
267 <term>attribute <literal>apdu</literal></term>
270 If the value of apdu is "true", then protocol packages
271 (APDUs and HTTP packages) from the ZOOM filter will be
272 logged to the yaz_log system. A value of "false" will
273 not perform logging of protocol packages (the default
284 The <literal>zoom</literal> element controls settings for the
289 <term>attribute <literal>timeout</literal></term>
292 Is an integer that specifies, in seconds, how long an operation
293 may take before ZOOM gives up. Default value is 40.
302 <title>QUERY HANDLING</title>
304 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
308 Queries are converted in two separate steps. In the first step
309 the input query is converted to RPN/Type-1. This is always
310 the common internal format between step 1 and step 2.
311 In step 2 the query is converted to the native query type of the target.
314 Step 1: for RPN, the query is passed un-modified to the target.
317 Step 1: for CCL, the query is converted to RPN via
318 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
319 the target profile as well as
320 <link linkend="cclmap_base">base CCL maps</link>.
323 Step 1: For CQL, the query is converted to CCL. The mappings of
324 CQL fields to CCL fields are handled by
325 <link linkend="fieldmap"><literal>fieldmap</literal></link>
326 elements as part of the target profile. The resulting query, CCL,
327 is the converted to RPN using the schema mentioned earlier (via
328 <literal>cclmap</literal>).
331 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
332 If the target is SRU-based, the RPN will be converted to CQL.
333 If the target is SOLR-based, the RPN will be converted to SOLR's query
339 <title>SORTING</title>
341 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
342 which was introduced in SRU version 1.2. The conversion from SORTBY clause
343 to native sort for some target is driven by the two parameters:
344 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
345 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
348 If a sort field that does not have an equivalent
349 <literal>sortmap_</literal>-mapping is passed un-modified through the
350 conversion. It doesn't throw a diagnostic.
355 <title>TARGET PROFILE</title>
357 The ZOOM module is driven by a number of settings that specifies how
358 to handle each target.
359 Note that unknown elements are silently <emphasis>ignored</emphasis>.
362 The elements, in alphabetical order, are:
366 <term id="zoom-torus-authentication">authentication</term><listitem>
368 Authentication parameters to be sent to the target. For
369 Z39.50 targets, this will be sent as part of the
370 Init Request. Authentication consists of two components: username
371 and password, separated by a slash.
374 If this value is omitted or empty no authentication information is sent.
379 <varlistentry id="cclmap">
380 <term>cclmap_<replaceable>field</replaceable></term><listitem>
382 This value specifies CCL field (qualifier) definition for some
383 field. For Z39.50 targets this most likely will specify the
384 mapping to a numeric use attribute + a structure attribute.
385 For SRU targets, the use attribute should be string based, in
386 order to make the RPN to CQL conversion work properly (step 2).
392 <term>cfAuth</term><listitem>
394 When cfAuth is defined, its value will be used as authentication
395 to backend target and authentication setting will be specified
396 as part of a database. This is like a "proxy" for authentication and
397 is used for Connector Framework based targets.
403 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
405 Specifies HTTP proxy for the target in the form
406 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
412 <term>cfSubDB</term><listitem>
414 Specifies sub database for a Connector Framework based target.
419 <varlistentry id="zoom-torus-contentConnector">
420 <term>contentConnector</term><listitem>
422 Specifies a database for content-based proxy'ing.
428 <term>elementSet</term><listitem>
430 Specifies the elementSet to be sent to the target if record
431 transform is enabled (not to be confused' with the record_transform
432 module). The record transform is enabled only if the client uses
433 record syntax = XML and a element set determined by
434 the <literal>element_transform</literal> /
435 <literal>element_raw</literal> from the configuration.
436 By default that is the element sets <literal>pz2</literal>
437 and <literal>raw</literal>.
438 If record transform is not enabled, this setting is
439 not used and the element set specified by the client
446 <term>literalTransform</term><listitem>
448 Specifies a XSL stylesheet to be used if record
449 transform is anabled; see description of elementSet.
450 The XSL transform is only used if the element set is set to the
451 value of <literal>element_transform</literal> in the configuration.
454 The value of literalTransform is the XSL - string encoded.
460 <term>piggyback</term><listitem>
462 A value of 1/true is a hint to the ZOOM module that this Z39.50
463 target supports piggyback searches, ie Search Response with
464 records. Any other value (false) will prevent the ZOOM module
465 to make use of piggyback (all records part of Present Response).
471 <term>queryEncoding</term><listitem>
473 If this value is defined, all queries will be converted
474 to this encoding. This should be used for all Z39.50 targets that
475 do not use UTF-8 for query terms.
481 <term>recordEncoding</term><listitem>
483 Specifies the character encoding of records that are returned
484 by the target. This is primarily used for targets were records
485 are not UTF-8 encoded already. This setting is only used
486 if the record transform is enabled (see description of elementSet).
492 <term>requestSyntax</term><listitem>
494 Specifies the record syntax to be specified for the target
495 if record transform is enabled; see description of elementSet.
496 If record transform is not enabled, the record syntax of the
497 client is passed verbatim to the target.
502 <varlistentry id="sortmap">
503 <term>sortmap_<replaceable>field</replaceable></term><listitem>
505 This value the native field for a target. The form of the value is
506 given by <link linkend="sortStrategy">sortStrategy</link>.
511 <varlistentry id="sortStrategy">
512 <term>sortStrategy</term><listitem>
514 Specifies sort strategy for a target. One of:
515 <literal>z3950</literal>, <literal>type7</literal>,
516 <literal>cql</literal>, <literal>sru11</literal> or
517 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
518 or CQL sortby depending on whether Type-1 or CQL is
519 actually sent to the target.
525 <term>sru</term><listitem>
527 If this setting is set, it specifies that the target is web service
528 based and must be one of : <literal>get</literal>,
529 <literal>post</literal>, <literal>soap</literal>
530 or <literal>solr</literal>.
536 <term>sruVersion</term><listitem>
538 Specifies the SRU version to use. It unset, version 1.2 will be
539 used. Some servers do not support this version, in which case
540 version 1.1 or even 1.0 could be set it.
546 <term>transform</term><listitem>
548 Specifies a XSL stylesheet filename to be used if record
549 transform is anabled; see description of elementSet.
550 The XSL transform is only used if the element set is set to the
551 value of <literal>element_transform</literal> in the configuration.
557 <term>udb</term><listitem>
559 This value is required and specifies the unique database for
560 this profile . All target profiles should hold a unique database.
565 <varlistentry id="urlRecipe">
566 <term>urlRecipe</term><listitem>
568 The value of this field is a string that generates a dynamic link
569 based on record content. If the resulting string is non-zero in length
570 a new field, <literal>metadata</literal> with attribute
571 <literal>type="generated-url"</literal> is generated.
572 The contents of this field is the result of the URL recipe conversion.
573 The urlRecipe value may refer to an existing metadata element by
574 ${field[pattern/result/flags]}, which will take content
575 of field and perform a regular expression conversion using the pattern
576 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
577 metadata element <literal>title</literal> and converts one or more
578 spaces to a plus character.
584 <term>zurl</term><listitem>
586 This is setting is mandatory and specifies the ZURL of the
587 target in the form of host/database. The HTTP method should
588 not be provided as this is guessed from the "sru" attribute value.
595 <title>DATABASE parameters</title>
597 Extra information may be carried in the Z39.50 Database or SRU path,
598 such as authentication to be passed to backend etc. Some of
599 the parameters override TARGET profile values. The format is
602 udb,parm1=value1&parm2=value2&...
605 Where udb is the unique database recognised by the backend and parm1,
606 value1, .. are parameters to be passed. The following describes the
607 supported parameters. Like form values in HTTP the parameters and
608 values are URL encoded. The separator, though, between udb and parameters
609 is a comma rather than a question mark. What follows question mark are
610 HTTP arguments (in this case SRU arguments).
617 Specifies user to be passed to backend. If this parameter is
618 omitted, the user will be taken from TARGET profile setting
619 <link linkend="zoom-torus-authentication">
620 <literal>authentication</literal>
627 <term>password</term>
630 Specifies password to be passed to backend. If this parameters is
631 omitted, the password will be taken from TARGET profile setting
632 <link linkend="zoom-torus-authentication">
633 <literal>authentication</literal>
643 Specifies one or more proxies for backend. If this parameter is
644 omitted, the proxy will be taken from TARGET profile setting
645 <link linkend="zoom-torus-cfproxy">
646 <literal>cfProxy</literal></link>.
647 The parameter is a list of comma-separated host:port entries.
648 Bost host and port must be given for each proxy.
653 <term>cproxysession</term>
656 Session ID for content proxy. This parameter is, generally,
657 not used by anything but the content proxy itself.
662 <term>nocproxy</term>
665 If this parameter is specified, content-proyxing is disabled
674 Session realm to be used for this target, changed the resulting
675 URL to be used for getting a target profile, by changing the
676 value that gets substituted for the %realm string.
684 All parameters that has prefix x, dash are passed verbatim
692 <title>SCHEMA</title>
693 <literallayout><xi:include
694 xi:href="../xml/schema/filter_zoom.rnc"
696 xmlns:xi="http://www.w3.org/2001/XInclude" />
701 <title>EXAMPLES</title>
703 The following configuration illustrates most of the
708 url="http://torus.indexdata.com/src/records/?query=%query"
709 proxy="localhost:3128"
711 <fieldmap cql="cql.anywhere"/>
712 <fieldmap cql="cql.serverChoice"/>
713 <fieldmap cql="dc.creator" ccl="au"/>
714 <fieldmap cql="dc.title" ccl="ti"/>
715 <fieldmap cql="dc.subject" ccl="su"/>
719 <attr type="u" value="12"/>
720 <attr type="s" value="107"/>
733 <title>SEE ALSO</title>
736 <refentrytitle>metaproxy</refentrytitle>
737 <manvolnum>1</manvolnum>
742 <refentrytitle>virt_db</refentrytitle>
743 <manvolnum>3mp</manvolnum>
751 <!-- Keep this comment at the end of the file