1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
4 <!ENTITY % local SYSTEM "local.ent">
6 <!ENTITY % entities SYSTEM "entities.ent">
8 <!ENTITY % idcommon SYSTEM "common/common.ent">
11 <refentry id="zebrasrv">
13 <productname>zebra</productname>
14 <productnumber>&version;</productnumber>
15 <info><orgname>Index Data</orgname></info>
19 <refentrytitle>zebrasrv</refentrytitle>
20 <manvolnum>8</manvolnum>
21 <refmiscinfo class="manual">Commands</refmiscinfo>
25 <refname>zebrasrv</refname>
26 <refpurpose>Zebra Server</refpurpose>
32 <refsect1><title>DESCRIPTION</title>
33 <para>Zebra is a high-performance, general-purpose structured text indexing
34 and retrieval engine. It reads structured records in a variety of input
35 formats (e.g. email, &acro.xml;, &acro.marc;) and allows access to them through exact
36 boolean search expressions and relevance-ranked free-text queries.
39 <command>zebrasrv</command> is the &acro.z3950; and &acro.sru; frontend
40 server for the <command>Zebra</command> search engine and indexer.
43 On Unix you can run the <command>zebrasrv</command>
44 server from the command line - and put it
45 in the background. It may also operate under the inet daemon.
46 On WIN32 you can run the server as a console application or
51 <title>OPTIONS</title>
54 The options for <command>zebrasrv</command> are the same
55 as those for &yaz;' <command>yaz-ztest</command>.
56 Option <literal>-c</literal> specifies a Zebra configuration
57 file - if omitted <filename>zebra.cfg</filename> is read.
63 <refsect1 id="protocol-support">
64 <title>&acro.z3950; Protocol Support and Behavior</title>
66 <refsect2 id="zebrasrv-initialization">
67 <title>&acro.z3950; Initialization</title>
70 During initialization, the server will negotiate to version 3 of the
71 &acro.z3950; protocol, and the option bits for Search, Present, Scan,
72 NamedResultSets, and concurrentOperations will be set, if requested by
73 the client. The maximum PDU size is negotiated down to a maximum of
79 <refsect2 id="zebrasrv-search">
80 <title>&acro.z3950; Search</title>
83 The supported query type are 1 and 101. All operators are currently
84 supported with the restriction that only proximity units of type "word"
85 are supported for the proximity operator.
86 Queries can be arbitrarily complex.
87 Named result sets are supported, and result sets can be used as operands
89 Searches may span multiple databases.
93 The server has full support for piggy-backed retrieval (see
94 also the following section).
99 <refsect2 id="zebrasrv-present">
100 <title>&acro.z3950; Present</title>
102 The present facility is supported in a standard fashion. The requested
103 record syntax is matched against the ones supported by the profile of
104 each record retrieved. If no record syntax is given, &acro.sutrs; is the
105 default. The requested element set name, again, is matched against any
106 provided by the relevant record profiles.
109 <refsect2 id="zebrasrv-scan">
110 <title>&acro.z3950; Scan</title>
112 The attribute combinations provided with the termListAndStartPoint are
113 processed in the same way as operands in a query (see above).
114 Currently, only the term and the globalOccurrences are returned with
115 the termInfo structure.
118 <refsect2 id="zebrasrv-sort">
119 <title>&acro.z3950; Sort</title>
122 &acro.z3950; specifies three different types of sort criteria.
123 Of these Zebra supports the attribute specification type in which
124 case the use attribute specifies the "Sort register".
125 Sort registers are created for those fields that are of type "sort" in
126 the default.idx file.
127 The corresponding character mapping file in default.idx specifies the
128 ordinal of each character used in the actual sort.
132 &acro.z3950; allows the client to specify sorting on one or more input
133 result sets and one output result set.
134 Zebra supports sorting on one result set only which may or may not
135 be the same as the output result set.
138 <refsect2 id="zebrasrv-close">
139 <title>&acro.z3950; Close</title>
141 If a Close PDU is received, the server will respond with a Close PDU
142 with reason=FINISHED, no matter which protocol version was negotiated
143 during initialization. If the protocol version is 3 or more, the
144 server will generate a Close PDU under certain circumstances,
145 including a session timeout (60 minutes by default), and certain kinds of
146 protocol errors. Once a Close PDU has been sent, the protocol
147 association is considered broken, and the transport connection will be
148 closed immediately upon receipt of further data, or following a short
153 <refsect2 id="zebrasrv-explain">
154 <title>&acro.z3950; Explain</title>
156 Zebra maintains a "classic"
157 <ulink url="&url.z39.50.explain;">&acro.z3950; Explain</ulink> database
159 This database is called <literal>IR-Explain-1</literal> and can be
160 searched using the attribute set <literal>exp-1</literal>.
163 The records in the explain database are of type
164 <literal>grs.sgml</literal>.
165 The root element for the Explain grs.sgml records is
166 <literal>explain</literal>, thus
167 <filename>explain.abs</filename> is used for indexing.
171 Zebra <emphasis>must</emphasis> be able to locate
172 <filename>explain.abs</filename> in order to index the Explain
173 records properly. Zebra will work without it but the information
174 will not be searchable.
179 <refsect1 id="zebrasrv-sru">
180 <title>The &acro.sru; Server</title>
182 In addition to &acro.z3950;, Zebra supports the more recent and
183 web-friendly IR protocol <ulink url="&url.sru;">&acro.sru;</ulink>.
184 &acro.sru; can be carried over &acro.soap; or a &acro.rest;-like protocol
185 that uses HTTP &acro.get; or &acro.post; to request search responses. The request
186 itself is made of parameters such as
187 <literal>query</literal>,
188 <literal>startRecord</literal>,
189 <literal>maximumRecords</literal>
191 <literal>recordSchema</literal>;
192 the response is an &acro.xml; document containing hit-count, result-set
193 records, diagnostics, etc. &acro.sru; can be thought of as a re-casting
194 of &acro.z3950; semantics in web-friendly terms; or as a standardisation
195 of the ad-hoc query parameters used by search engines such as Google
196 and AltaVista; or as a superset of A9's OpenSearch (which it
200 Zebra supports &acro.z3950;, &acro.sru; &acro.get;, SRU &acro.post;, SRU &acro.soap; (&acro.srw;)
201 - on the same port, recognising what protocol is used by each incoming
202 requests and handling them accordingly. This is a achieved through
203 the use of Deep Magic; civilians are warned not to stand too close.
205 <refsect2 id="zebrasrv-sru-run">
206 <title>Running zebrasrv as an &acro.sru; Server</title>
208 Because Zebra supports all protocols on one port, it would
209 seem to follow that the &acro.sru; server is run in the same way as
210 the &acro.z3950; server, as described above. This is true, but only in
211 an uninterestingly vacuous way: a Zebra server run in this manner
212 will indeed recognise and accept &acro.sru; requests; but since it
213 doesn't know how to handle the &acro.cql; queries that these protocols
214 use, all it can do is send failure responses.
218 It is possible to cheat, by having &acro.sru; search Zebra with
219 a &acro.pqf; query instead of &acro.cql;, using the
220 <literal>x-pquery</literal>
222 <literal>query</literal>.
224 <emphasis role="strong">non-standard extension</emphasis>
226 <emphasis role="strong">very naughty</emphasis>
227 thing to do, but it does give you a way to see Zebra serving &acro.sru;
228 ``right out of the box''. If you start your favourite Zebra
229 server in the usual way, on port 9999, then you can send your web
233 http://localhost:9999/Default?version=1.1
234 &operation=searchRetrieve
235 &x-pquery=mineral
237 &maximumRecords=1
240 This will display the &acro.xml;-formatted &acro.sru; response that includes the
241 first record in the result-set found by the query
242 <literal>mineral</literal>. (For clarity, the &acro.sru; URL is shown
243 here broken across lines, but the lines should be joined together
244 to make single-line URL for the browser to submit.)
248 In order to turn on Zebra's support for &acro.cql; queries, it's necessary
249 to have the &yaz; generic front-end (which Zebra uses) translate them
250 into the &acro.z3950; Type-1 query format that is used internally. And
251 to do this, the generic front-end's own configuration file must be
252 used. See <xref linkend="gfs-config"/>;
253 the salient point for &acro.sru; support is that
254 <command>zebrasrv</command>
255 must be started with the
256 <literal>-f frontendConfigFile</literal>
257 option rather than the
258 <literal>-c zebraConfigFile</literal>
260 and that the front-end configuration file must include both a
261 reference to the Zebra configuration file and the &acro.cql;-to-&acro.pqf;
262 translator configuration file.
265 A minimal front-end configuration file that does this would read as
272 <config>zebra.cfg</config>
273 <cql2rpn>../../tab/pqf.properties</cql2rpn>
279 <literal><config></literal>
280 element contains the name of the Zebra configuration file that was
281 previously specified by the
282 <literal>-c</literal>
283 command-line argument, and the
284 <literal><cql2rpn></literal>
285 element contains the name of the &acro.cql; properties file specifying how
286 various &acro.cql; indexes, relations, etc. are translated into Type-1
290 A zebra server running with such a configuration can then be
291 queried using proper, conformant &acro.sru; URLs with &acro.cql; queries:
294 http://localhost:9999/Default?version=1.1
295 &operation=searchRetrieve
296 &query=title=utah and description=epicent*
298 &maximumRecords=1
302 <refsect1 id="zebrasrv-sru-support">
303 <title>&acro.sru; Protocol Support and Behavior</title>
305 Zebra running as an &acro.sru; server supports SRU version 1.1, including
306 &acro.cql; version 1.1. In particular, it provides support for the
307 following elements of the protocol.
310 <refsect2 id="zebrasrvr-search-and-retrieval">
311 <title>&acro.sru; Search and Retrieval</title>
313 Zebra supports the searchRetrieve operation.
316 One of the great strengths of &acro.sru; is that it mandates a standard
317 query language, &acro.cql;, and that all conforming implementations can
318 therefore be trusted to correctly interpret the same queries. It
319 is with some shame, then, that we admit that Zebra also supports
320 an additional query language, our own Prefix Query Format
321 (<ulink url="&url.yaz.pqf;">&acro.pqf;</ulink>).
322 A &acro.pqf; query is submitted by using the extension parameter
323 <literal>x-pquery</literal>,
325 <literal>query</literal>
326 parameter must be omitted, which makes the request not valid &acro.sru;.
327 Please feel free to use this facility within your own
328 applications; but be aware that it is not only non-standard &acro.sru;
329 but not even syntactically valid, since it omits the mandatory
330 <literal>query</literal> parameter.
334 <refsect2 id="zebrasrv-sru-scan">
335 <title>&acro.sru; Scan</title>
337 Zebra supports scan operation.
338 Scanning using &acro.cql; syntax is the default, where the
339 standard <literal>scanClause</literal> parameter is used.
343 mutant form of &acro.sru; scan is supported, using
344 the non-standard <literal>x-pScanClause</literal> parameter in
345 place of the standard <literal>scanClause</literal> to scan on a
346 &acro.pqf; query clause.
350 <refsect2 id="zebrasrv-sru-explain">
351 <title>&acro.sru; Explain</title>
353 Zebra supports explain.
356 The ZeeRex record explaining a database may be requested either
357 with a fully fledged &acro.sru; request (with
358 <literal>operation</literal>=<literal>explain</literal>
359 and version-number specified)
360 or with a simple HTTP &acro.get; at the server's basename.
361 The ZeeRex record returned in response is the one embedded
362 in the &yaz; Frontend Server configuration file that is described in the
363 <xref linkend="gfs-config"/>.
366 Unfortunately, the data found in the
367 &acro.cql;-to-&acro.pqf; text file must be added by hand-craft into the explain
368 section of the &yaz; Frontend Server configuration file to be able
369 to provide a suitable explain record.
370 Too bad, but this is all extreme
371 new alpha stuff, and a lot of work has yet to be done ..
374 There is no linkage whatsoever between the &acro.z3950; explain model
375 and the &acro.sru; explain response (well, at least not implemented
376 in Zebra, that is ..). Zebra does not provide a means using
377 &acro.z3950; to obtain the ZeeRex record.
381 <refsect2 id="zebrasrv-non-sru-ops">
382 <title>Other &acro.sru; operations</title>
384 In the &acro.z3950; protocol, Initialization, Present, Sort and Close
385 are separate operations. In &acro.sru;, however, these operations do not
391 &acro.sru; has no explicit initialization handshake phase, but
392 commences immediately with searching, scanning and explain
398 Neither does &acro.sru; have a close operation, since the protocol is
399 stateless and each request is self-contained. (It is true that
400 multiple &acro.sru; request/response pairs may be implemented as
401 multiple HTTP request/response pairs over a single persistent
402 TCP/IP connection; but the closure of that connection is not a
403 protocol-level operation.)
408 Retrieval in &acro.sru; is part of the
409 <literal>searchRetrieve</literal> operation, in which a search
410 is submitted and the response includes a subset of the records
411 in the result set. There is no direct analogue of &acro.z3950;'s
412 Present operation which requests records from an established
413 result set. In &acro.sru;, this is achieved by sending a subsequent
414 <literal>searchRetrieve</literal> request with the query
415 <literal>cql.resultSetId=</literal><emphasis>id</emphasis> where
416 <emphasis>id</emphasis> is the identifier of the previously
417 generated result-set.
422 Sorting in &acro.cql; is done within the
423 <literal>searchRetrieve</literal> operation - in v1.1, by an
424 explicit <literal>sort</literal> parameter, but the forthcoming
425 v1.2 or v2.0 will most likely use an extension of the query
426 language, <ulink url="&url.cql.sorting;">&acro.cql; sorting</ulink>.
431 It can be seen, then, that while Zebra operating as an &acro.sru; server
432 does not provide the same set of operations as when operating as a
433 &acro.z3950; server, it does provide equivalent functionality.
438 <refsect1 id="zebrasrv-sru-examples">
439 <title>&acro.sru; Examples</title>
441 Surf into <literal>http://localhost:9999</literal>
442 to get an explain response, or use
444 http://localhost:9999/?version=1.1&operation=explain
448 See number of hits for a query
450 http://localhost:9999/?version=1.1&operation=searchRetrieve
451 &query=text=(plant%20and%20soil)
455 Fetch record 5-7 in Dublin Core format
457 http://localhost:9999/?version=1.1&operation=searchRetrieve
458 &query=text=(plant%20and%20soil)
459 &startRecord=5&maximumRecords=2&recordSchema=dc
463 Even search using &acro.pqf; queries using the <emphasis>extended naughty
464 parameter</emphasis> <literal>x-pquery</literal>
466 http://localhost:9999/?version=1.1&operation=searchRetrieve
467 &x-pquery=@attr%201=text%20@and%20plant%20soil
471 Or scan indexes using the <emphasis>extended extremely naughty
472 parameter</emphasis> <literal>x-pScanClause</literal>
474 http://localhost:9999/?version=1.1&operation=scan
475 &x-pScanClause=@attr%201=text%20something
477 <emphasis>Don't do this in production code!</emphasis>
478 But it's a great fast debugging aid.
483 <refsect1 id="gfs-config"><title>&yaz; server virtual hosts</title>
487 <refsect1><title>SEE ALSO</title>
490 <refentrytitle>zebraidx</refentrytitle>
491 <manvolnum>1</manvolnum>
497 <!-- Keep this comment at the end of the file
502 sgml-minimize-attributes:nil
503 sgml-always-quote-attributes:t
506 sgml-parent-document: "idzebra.xml"
507 sgml-local-catalogs: nil
508 sgml-namecase-general:t