1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of five parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal> and <literal>log</literal>.
66 The <literal>torus</literal> element specifies target profiles
67 and takes the following content:
71 <term>attribute <literal>url</literal></term>
74 URL of Web service to be used to fetch target profile
75 for a given database (udb) of type searchable. The special sequence
76 <literal>%db</literal> of the URL is replaced by the
77 actual database specified as part of Search.
80 The special sequence <literal>%realm</literal> is replaced by value
81 of attribute <literal>realm</literal> or by realm DATABASE argument.
86 <term>attribute <literal>content_url</literal></term>
89 URL of Web service to be used to fetch target profile
90 for a given database (udb) of type content. Semantics otherwise like
91 <literal>url</literal> attribute above.
96 <term>attribute <literal>realm</literal></term>
99 The default realm value. Used for %realm in URL, unless
100 specified in DATABASE argument.
105 <term>attribute <literal>proxy</literal></term>
108 HTTP proxy to bse used for fetching target profiles.
113 <term>attribute <literal>xsldir</literal></term>
116 Directory that is searched for XSL stylesheets. Stylesheets
117 are specified in the target profile by the
118 <literal>transform</literal> element.
123 <term>attribute <literal>element_transform</literal></term>
126 Specifies the element that triggers retrieval and transform using
127 the parameters elementSet, recordEncoding, requestSyntax, transform
128 from the target profile. Default value
129 is "pz2", due to the fact that for historical reasons the
130 common format is that used in Pazpar2.
135 <term>attribute <literal>element_raw</literal></term>
138 Specifies an element that triggers retrieval using the
139 parameters elementSet, recordEncoding, requestSyntax from the
140 target profile. Same actions as for element_transform, but without
141 the XSL transform. Useful for debugging.
142 The default value is "raw".
147 <term>element <literal>records</literal></term>
150 Local target profiles. This element may includes zero or
151 more <literal>record</literal> elements (one per target
152 profile). See section TARGET PROFILE.
158 <refsect2 id="fieldmap">
159 <title>fieldmap</title>
161 The <literal>fieldmap</literal> may be specified zero or more times and
162 specifies the map from CQL fields to CCL fields and takes the
167 <term>attribute <literal>cql</literal></term>
170 CQL field that we are mapping "from".
175 <term>attribute <literal>ccl</literal></term>
178 CCL field that we are mapping "to".
184 <refsect2 id="cclmap_base">
185 <title>cclmap</title>
187 The third part of the configuration consists of zero or more
188 <literal>cclmap</literal> elements that specifies
189 <emphasis>base</emphasis> CCL profile to be used for all targets.
190 This configuration, thus, will be combined with cclmap-definitions
191 from the target profile.
195 <title>contentProxy</title>
197 The <literal>contentProxy</literal> element controls content proxy'in.
199 is optional and must only be defined if content proxy'ing is enabled.
203 <term>attribute <literal>server</literal></term>
206 Specifies the content proxy host. The host is of the form
207 host[:port]. That is without a method (such as HTTP) and optional
213 <term>attribute <literal>tmp_file</literal></term>
216 Specifies a filename of a session file for content proxy'ing. The
217 file should be an absolute filename that includes
218 <literal>XXXXXX</literal> which is replaced by a unique filename
219 using the mkstemp(3) system call. The default value of this
220 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
229 The <literal>log</literal> element controls logging for the
234 <term>attribute <literal>apdu</literal></term>
237 If the value of apdu is "true", then protocol packages
238 (APDUs and HTTP packages) from the ZOOM filter will be
239 logged to the yaz_log system. A value of "false" will
240 not perform logging of protocol packages (the default
249 <title>QUERY HANDLING</title>
251 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
255 Queries are converted in two separate steps. In the first step
256 the input query is converted to RPN/Type-1. This is always
257 the common internal format between step 1 and step 2.
258 In step 2 the query is converted to the native query type of the target.
261 Step 1: for RPN, the query is passed un-modified to the target.
264 Step 1: for CCL, the query is converted to RPN via
265 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
266 the target profile as well as
267 <link linkend="cclmap_base">base CCL maps</link>.
270 Step 1: For CQL, the query is converted to CCL. The mappings of
271 CQL fields to CCL fields are handled by
272 <link linkend="fieldmap"><literal>fieldmap</literal></link>
273 elements as part of the target profile. The resulting query, CCL,
274 is the converted to RPN using the schema mentioned earlier (via
275 <literal>cclmap</literal>).
278 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
279 If the target is SRU-based, the RPN will be converted to CQL.
280 If the target is SOLR-based, the RPN will be converted to SOLR's query
286 <title>SORTING</title>
288 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
289 which was introduced in SRU version 1.2. The conversion from SORTBY clause
290 to native sort for some target is driven by the two parameters:
291 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
292 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
295 If a sort field that does not have an equivalent
296 <literal>sortmap_</literal>-mapping is passed un-modified through the
297 conversion. It doesn't throw a diagnostic.
302 <title>TARGET PROFILE</title>
304 The ZOOM module is driven by a number of settings that specifies how
305 to handle each target.
306 Note that unknown elements are silently <emphasis>ignored</emphasis>.
309 The elements, in alphabetical order, are:
313 <term id="zoom-torus-authentication">authentication</term><listitem>
315 Authentication parameters to be sent to the target. For
316 Z39.50 targets, this will be sent as part of the
317 Init Request. Authentication consists of two components: username
318 and password, separated by a slash.
321 If this value is omitted or empty no authentication information is sent.
326 <varlistentry id="cclmap">
327 <term>cclmap_<replaceable>field</replaceable></term><listitem>
329 This value specifies CCL field (qualifier) definition for some
330 field. For Z39.50 targets this most likely will specify the
331 mapping to a numeric use attribute + a structure attribute.
332 For SRU targets, the use attribute should be string based, in
333 order to make the RPN to CQL conversion work properly (step 2).
339 <term>cfAuth</term><listitem>
341 When cfAuth is defined, its value will be used as authentication
342 to backend target and authentication setting will be specified
343 as part of a database. This is like a "proxy" for authentication and
344 is used for Connector Framework based targets.
350 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
352 Specifies HTTP proxy for the target in the form
353 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
359 <term>cfSubDB</term><listitem>
361 Specifies sub database for a Connector Framework based target.
366 <varlistentry id="zoom-torus-contentConnector">
367 <term>contentConnector</term><listitem>
369 Specifies a database for content-based proxy'ing.
375 <term>elementSet</term><listitem>
377 Specifies the elementSet to be sent to the target if record
378 transform is enabled (not to be confused' with the record_transform
379 module). The record transform is enabled only if the client uses
380 record syntax = XML and a element set determined by
381 the <literal>element_transform</literal> /
382 <literal>element_raw</literal> from the configuration.
383 By default that is the element sets <literal>pz2</literal>
384 and <literal>raw</literal>.
385 If record transform is not enabled, this setting is
386 not used and the element set specified by the client
393 <term>literalTransform</term><listitem>
395 Specifies a XSL stylesheet to be used if record
396 transform is anabled; see description of elementSet.
397 The XSL transform is only used if the element set is set to the
398 value of <literal>element_transform</literal> in the configuration.
401 The value of literalTransform is the XSL - string encoded.
407 <term>piggyback</term><listitem>
409 A value of 1/true is a hint to the ZOOM module that this Z39.50
410 target supports piggyback searches, ie Search Response with
411 records. Any other value (false) will prevent the ZOOM module
412 to make use of piggyback (all records part of Present Response).
418 <term>queryEncoding</term><listitem>
420 If this value is defined, all queries will be converted
421 to this encoding. This should be used for all Z39.50 targets that
422 do not use UTF-8 for query terms.
428 <term>recordEncoding</term><listitem>
430 Specifies the character encoding of records that are returned
431 by the target. This is primarily used for targets were records
432 are not UTF-8 encoded already. This setting is only used
433 if the record transform is enabled (see description of elementSet).
439 <term>requestSyntax</term><listitem>
441 Specifies the record syntax to be specified for the target
442 if record transform is enabled; see description of elementSet.
443 If record transform is not enabled, the record syntax of the
444 client is passed verbatim to the target.
449 <varlistentry id="sortmap">
450 <term>sortmap_<replaceable>field</replaceable></term><listitem>
452 This value the native field for a target. The form of the value is
453 given by <link linkend="sortStrategy">sortStrategy</link>.
458 <varlistentry id="sortStrategy">
459 <term>sortStrategy</term><listitem>
461 Specifies sort strategy for a target. One of:
462 <literal>z3950</literal>, <literal>type7</literal>,
463 <literal>cql</literal>, <literal>sru11</literal> or
464 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
465 or CQL sortby depending on whether Type-1 or CQL is
466 actually sent to the target.
472 <term>sru</term><listitem>
474 If this setting is set, it specifies that the target is web service
475 based and must be one of : <literal>get</literal>,
476 <literal>post</literal>, <literal>soap</literal>
477 or <literal>solr</literal>.
483 <term>sruVersion</term><listitem>
485 Specifies the SRU version to use. It unset, version 1.2 will be
486 used. Some servers do not support this version, in which case
487 version 1.1 or even 1.0 could be set it.
493 <term>transform</term><listitem>
495 Specifies a XSL stylesheet filename to be used if record
496 transform is anabled; see description of elementSet.
497 The XSL transform is only used if the element set is set to the
498 value of <literal>element_transform</literal> in the configuration.
504 <term>udb</term><listitem>
506 This value is required and specifies the unique database for
507 this profile . All target profiles should hold a unique database.
512 <varlistentry id="urlRecipe">
513 <term>urlRecipe</term><listitem>
515 The value of this field is a string that generates a dynamic link
516 based on record content. If the resulting string is non-zero in length
517 a new field, <literal>metadata</literal> with attribute
518 <literal>type="generated-url"</literal> is generated.
519 The contents of this field is the result of the URL recipe conversion.
520 The urlRecipe value may refer to an existing metadata element by
521 ${field[pattern/result/flags]}, which will take content
522 of field and perform a regular expression conversion using the pattern
523 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
524 metadata element <literal>title</literal> and converts one or more
525 spaces to a plus character.
528 If the <link linkend="zoom-torus-contentConnector">contentConnector</link>
529 setting also defined, the resulting value is
530 augmented with a session string as well as host name of the
531 content proxy server.
537 <term>zurl</term><listitem>
539 This is setting is mandatory and specifies the ZURL of the
540 target in the form of host/database. The HTTP method should
541 not be provided as this is guessed from the "sru" attribute value.
548 <title>DATABASE parameters</title>
550 Extra information may be carried in the Z39.50 Database or SRU path,
551 such as authentication to be passed to backend etc. Some of
552 the parameters override TARGET profile values. The format is
555 udb,parm1=value1&parm2=value2&...
558 Where udb is the unique database recognised by the backend and parm1,
559 value1, .. are parameters to be passed. The following describes the
560 supported parameters. Like form values in HTTP the parameters and
561 values are URL encoded. The separator, though, between udb and parameters
562 is a comma rather than a question mark. What follows question mark are
563 HTTP arguments (in this case SRU arguments).
570 Specifies user to be passed to backend. If this parameter is
571 omitted, the user will be taken from TARGET profile setting
572 <link linkend="zoom-torus-authentication">
573 <literal>authentication</literal>
580 <term>password</term>
583 Specifies password to be passed to backend. If this parameters is
584 omitted, the password will be taken from TARGET profile setting
585 <link linkend="zoom-torus-authentication">
586 <literal>authentication</literal>
596 Specifies proxy to be for backend. If this parameters is
597 omitted, the proxy will be taken from TARGET profile setting
598 <link linkend="zoom-torus-cfproxy">
599 <literal>cfProxy</literal>
606 <term>cproxysession</term>
609 Session ID for content proxy. This parameter is, generally,
610 not used by anything but the content proxy itself.
618 Session realm to be used for this target, changed the resulting
619 URL to be used for getting a target profile, by changing the
620 value that gets substituted for the %realm string.
628 All parameters that has prefix x, dash are passed verbatim
636 <title>SCHEMA</title>
637 <literallayout><xi:include
638 xi:href="../xml/schema/filter_zoom.rnc"
640 xmlns:xi="http://www.w3.org/2001/XInclude" />
645 <title>EXAMPLES</title>
647 The following configuration illustrates most of the
652 url="http://torus.indexdata.com/src/records/?query=udb%3D%db"
653 proxy="localhost:3128"
655 <fieldmap cql="cql.anywhere"/>
656 <fieldmap cql="cql.serverChoice"/>
657 <fieldmap cql="dc.creator" ccl="au"/>
658 <fieldmap cql="dc.title" ccl="ti"/>
659 <fieldmap cql="dc.subject" ccl="su"/>
663 <attr type="u" value="12"/>
664 <attr type="s" value="107"/>
676 <title>SEE ALSO</title>
679 <refentrytitle>metaproxy</refentrytitle>
680 <manvolnum>1</manvolnum>
685 <refentrytitle>virt_db</refentrytitle>
686 <manvolnum>3mp</manvolnum>
694 <!-- Keep this comment at the end of the file