1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
3 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
5 <!ENTITY % local SYSTEM "local.ent">
7 <!ENTITY % entities SYSTEM "entities.ent">
9 <!ENTITY % idcommon SYSTEM "common/common.ent">
14 <title>YAZ User's Guide and Reference</title>
16 <author><firstname>Sebastian</firstname><surname>Hammer</surname></author>
17 <author><firstname>Adam</firstname><surname>Dickmeiss</surname></author>
18 <author><firstname>Mike</firstname><surname>Taylor</surname></author>
19 <author><firstname>Heikki</firstname><surname>Levanto</surname></author>
20 <author><firstname>Dennis</firstname><surname>Schafroth</surname></author>
22 <releaseinfo>&version;</releaseinfo>
24 <year>©right-year;</year>
25 <holder>Index Data</holder>
29 This document is the programmer's guide and reference to the &yaz;
30 package version &version;. &yaz; is a compact toolkit that provides
31 access to the Z39.50 and SRU/Solr protocols, as well as a set of
32 higher-level tools for implementing the server and client
34 The documentation can be used on its own, or as a reference when
35 looking at the example applications provided with the package.
40 <imagedata fileref="common/id.png" format="PNG"/>
43 <imagedata fileref="common/id.eps" format="EPS"/>
48 <chapter id="introduction">
49 <title>Introduction</title>
51 &yaz; is a C/C++ library for information retrieval applications
52 using the Z39.50/SRU/Solr protocols for information retrieval.
60 <ulink url="&url.z39.50;">Z39.50</ulink> version 3 support.
61 Amendments and Z39.50-2002 revision is supported.
67 <ulink url="&url.sru;">SRU GET/POST/SOAP</ulink>
68 version 1.1, 1.2 and 2.0 (over HTTP and HTTPS).
73 Includes BER encoders/decoders for the
74 <ulink url="&url.ill;">ISO ILL</ulink>
81 <ulink url="&url.solr;">Solr</ulink> Web Service version 1.4.x
87 Supports the following transports: BER over TCP/IP
88 (<ulink url="&url.ber.over.tcpip;">RFC1729</ulink>),
89 BER over unix local socket, and
90 <ulink url="&url.http.1.1;">HTTP 1.1</ulink>.
95 Secure Socket Layer support using
96 <ulink url="&url.gnutls;">GnuTLS</ulink>.
97 If enabled, &yaz; uses HTTPS transport (for SOAP) or
98 "Secure BER" (for Z39.50).
104 <ulink url="&url.zoom;">ZOOM</ulink> C API implementing
105 Z39.50, SRU and Solr Web Service.
110 The &yaz; library offers a set of useful utilities
111 related to the protocols, such as MARC (ISO2709) parser,
112 CCL (ISO8777) parser,
113 <ulink url="&url.cql;">CQL</ulink>
114 parser, memory management routines, character set conversion.
119 Portable code. &yaz; compiles out-of-the box on most Unixes and
120 on Windows using Microsoft Visual C++.
125 Fast operation. The C based BER encoders/decoders as well
126 as the server component of &yaz; is very fast.
131 Liberal license that allows for commercial use of &yaz;.
137 <sect1 id="introduction.reading">
138 <title>Reading this Manual</title>
140 Most implementors only need to read a fraction of the
141 material in thie manual, so a quick walkthrough of the chapters
147 <xref linkend="installation"/> contains installation
148 instructions for &yaz;. You don't need reading this
149 if you expect to download &yaz; binaries.
150 However, the chapter contains information about how
151 to make <emphasis>your</emphasis> application link
157 <xref linkend="zoom"/> describes the ZOOM API of &yaz;.
158 This is definitely worth a read if you wish to develop a Z39.50/SRU
164 <xref linkend="server"/> describes the generic frontend server
165 and explains how to develop server Z39.50/SRU applications for &yaz;.
166 Obviously worth reading if you're to develop a server.
171 <xref linkend="yaz-client"/> describes how to use the &yaz; Z39.50
172 client. If you're developer and wish to test your server
173 or a server from another party, you might find this chapter
179 <xref linkend="asn"/> documents the most commonly used Z39.50
180 C data structures offered by the &yaz; API. Client
181 developers using ZOOM and non-Z39.50 implementors may skip this.
186 <xref linkend="soap"/> describes how SRU and SOAP is used
187 in &yaz;. Only if you're developing SRU applications
188 this section is a must.
193 <xref linkend="tools"/> contains sections for the various
194 tools offered by &yaz;. Scan through the material quickly
195 and see what's relevant to you! SRU implementors
196 might find the <link linkend="cql">CQL</link> section
202 <xref linkend="odr"/> goes through the details of the
203 ODR module which is the work horse that encodes and decodes
204 BER packages. Implementors using ZOOM only, do <emphasis>not</emphasis>
206 Most other Z39.50 implementors only need to read the first two
207 sections (<xref linkend="odr.introduction"/> and
208 <xref linkend="odr.use"/>).
213 <xref linkend="comstack"/> describes the network layer module
214 COMSTACK. Implementors using ZOOM or the generic frontend server
215 may skip this. Others, presumably, handling client/server
216 communication on their own should read this.
221 <sect1 id="introduction.api">
222 <title>The API</title>
224 The <ulink url="&url.yaz;">&yaz;</ulink>
225 toolkit offers several different levels of access to the
226 <ulink url="&url.z39.50;">ISO23950/Z39.50</ulink>,
227 <ulink url="&url.ill;">ILL</ulink> and
228 <ulink url="&url.sru;">SRU</ulink>
230 The level that you need to use depends on your requirements, and
231 the role (server or client) that you want to implement.
232 If you're developing a client application you should consider the
233 <link linkend="zoom">ZOOM</link> API.
234 It is, by far, the easiest way to develop clients in C.
235 Server implementers should consider the
236 <link linkend="server">generic frontend server</link>.
237 None of those high-level APIs support the whole protocol, but
238 they do include most facilities used in existing Z39.50 applications.
241 If you're using 'exotic' functionality (meaning anything not included in
242 the high-level APIs), developing non-standard extensions to Z39.50 or
243 you're going to develop an ILL application you'll have to learn the lower
247 The YAZ toolkit modules is shown in figure <xref linkend="yaz.layer"/>.
249 <figure id="yaz.layer">
250 <title>YAZ layers</title>
253 <imagedata fileref="apilayer.png" format="PNG"/>
256 <imagedata fileref="apilayer.eps" format="EPS"/>
261 There are four layers.
264 <para>A client or server application (or both).
265 This layer includes ZOOM and the generic frontend server.
270 The second layer provides a C represenation of the
271 protocol units (packages) for Z39.50 ASN.1, ILL ASN.1,
277 The third layer encodes and decodes protocol data units to
278 simple packages (buffer with certain length). The &odr; module
279 encodes and decodes BER whereas the HTTP modules encodes and
280 decodes HTTP ruquests/responses.
285 The lowest layer is &comstack; which exchanges the encoded packages
286 with a peer process over a network.
292 The &asn; module represents the ASN.1 definition of
293 the Z39.50 protocol. It establishes a set of type and
294 structure definitions, with one structure for each of the top-level
295 PDUs, and one structure or type for each of the contained ASN.1 types.
296 For primitive types, or other types that are defined by the ASN.1
297 standard itself (such as the EXTERNAL type), the C representation is
298 provided by the &odr; (Open Data Representation) subsystem.
301 &odr; is a basic mechanism for representing an
302 ASN.1 type in the C programming language, and for implementing BER
303 encoders and decoders for values of that type. The types defined in
304 the &asn; module generally have the prefix <literal>Z_</literal>, and
305 a suffix corresponding to the name of the type in the ASN.1
306 specification of the protocol (generally Z39.50-1995). In the case of
307 base types (those originating in the ASN.1 standard itself), the prefix
308 <literal>Odr_</literal> is sometimes seen. Either way, look for
309 the actual definition in either <filename>z-core.h</filename> (for the types
310 from the protocol), <filename>odr.h</filename> (for the primitive ASN.1
312 The &asn; library also provides functions (which are, in turn,
313 defined using &odr; primitives) for encoding and decoding data values.
314 Their general form is
316 <funcprototype><funcdef>int <function>z_<replaceable>xxx</replaceable></function></funcdef>
317 <paramdef>ODR <parameter>o</parameter></paramdef>
318 <paramdef>Z_<replaceable>xxx</replaceable> **<parameter>p</parameter></paramdef>
319 <paramdef>int <parameter>optional</parameter></paramdef>
320 <paramdef>const char *<parameter>name</parameter></paramdef>
323 (note the lower-case "z" in the function name)
327 If you are using the premade definitions of the &asn; module, and you
328 are not adding new protocol of your own, the only parts of &odr; that you
329 need to worry about are documented in
330 <xref linkend="odr.use"/>.
334 When you have created a BER-encoded buffer, you can use the &comstack;
335 subsystem to transmit (or receive) data over the network. The &comstack;
336 module provides simple functions for establishing a connection
337 (passively or actively, depending on the role of your application),
338 and for exchanging BER-encoded PDUs over that connection. When you
339 create a connection endpoint, you need to specify what transport to
340 use (TCP/IP, SSL or UNIX sockets).
341 For the remainder of the connection's lifetime, you don't have
342 to worry about the underlying transport protocol at all - the &comstack;
343 will ensure that the correct mechanism is used.
346 We call the combined interfaces to &odr;, &asn;, and &comstack; the service
347 level API. It's the API that most closely models the Z39.50
348 service/protocol definition, and it provides unlimited access to all
349 fields and facilities of the protocol definitions.
352 The reason that the &yaz; service-level API is a conglomerate of the
353 APIs from three different submodules is twofold. First, we wanted to allow
354 the user a choice of different options for each major task. For instance,
355 if you don't like the protocol API provided by &odr;/&asn;, you
356 can use SNACC or BERUtils instead, and still have the benefits of the
357 transparent transport approach of the &comstack; module. Secondly,
358 we realize that you may have to fit the toolkit into an existing
359 event-processing structure, in a way that is incompatible with
360 the &comstack; interface or some other part of &yaz;.
364 <chapter id="installation">
365 <title>Compilation and Installation</title>
366 <sect1 id="installation-introduction">
367 <title>Introduction</title>
369 The latest version of the software will generally be found at:
372 <ulink url="&url.yaz.download;"/>
375 We have tried our best to keep the software portable, and on many
376 platforms, you should be able to compile everything with little or
380 The software is regularly tested on
381 <ulink url="&url.debian;">Debian GNU/Linux</ulink>,
382 <ulink url="&url.centos;">CentOS</ulink>,
383 <ulink url="&url.ubuntu;">Ubuntu Linux</ulink>,
384 <ulink url="&url.freebsd;">FreeBSD (i386)</ulink>,
385 <ulink url="&url.macosx;">MAC OSX</ulink>,
386 <ulink url="&url.solaris;">Solaris</ulink>,
387 Windows 7, Windows XP.
390 Some versions have be known to work on HP/UX,
391 DEC Unix, <ulink url="&url.netbsd;">NetBSD</ulink>,
392 <ulink url="&url.openbsd;">OpenBSD</ulink>,
394 Data General DG/UX (with some CFLAGS tinkering),
395 SGI/IRIX, DDE Supermax, Apple Macintosh (using the Codewarrior programming
396 environment and the GUSI socket libraries),
400 If you move the software to other platforms, we'd be grateful if you'd
401 let us know about it. If you run into difficulties, we will try to help
402 if we can, and if you solve the problems, we would be happy to include
403 your fixes in the next release. So far, we have mostly avoided
404 <literal>#ifdefs</literal> for individual platforms, and we'd
405 like to keep it that way as far as it makes sense.
408 We maintain a mailing-list for the purpose of announcing new releases and
409 bug-fixes, as well as general discussion. Subscribe by
411 <ulink url="&url.yaz.mailinglist;">here</ulink>.
412 General questions and problems can be directed at
413 <ulink url="&url.yaz.mail;"/>, or the address given at the top of
417 <sect1 id="installation.unix"><title>UNIX</title>
420 <ulink url="&url.debian;">Debian GNU/Linux</ulink> (i386 and amd64),
421 <ulink url="&url.ubuntu;">Ubuntu</ulink> (i386 and amd64)
423 <ulink url="&url.centos;">CentOS</ulink> (amd64 only) packages for &yaz;.
424 You should be able to create packages for other CPUs by building
425 them from the source package.
428 YAZ is also part of several packages repositories. Some of them are
433 Solaris CSW: <ulink url="http://www.opencsw.org/packages/yaz/"/>
438 Solaris: <ulink url="http://unixpackages.com"/>
443 FreeBSD: <ulink url="http://www.freshports.org/net/yaz"/>
448 Debian: <ulink url="http://packages.debian.org/search?keywords=yaz"/>
453 Ubuntu: <ulink url="https://launchpad.net/ubuntu/+source/yaz"/>
459 <ulink url="http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/yaz/README.html"/>
463 <sect2 id="installation.source.unix">
464 <title>Compiling from source on Unix</title>
466 Note that if your system doesn't have a native ANSI C compiler, you may
467 have to acquire one separately. We recommend
468 <ulink url="&url.gcc;">GCC</ulink>.
471 If you wish to use character set conversion facilities in &yaz; or if you
472 are compiling &yaz; for use with Zebra it is a good idea to ensure that
473 the iconv library is installed. Some Unixes today already have it
475 <ulink url="&url.libiconv;">GNU libiconv</ulink>.
478 YAZ 3.0.16 and later includes a wrapper for the
479 <ulink url="&url.icu;">ICU</ulink>
480 (International Components for Unicode).
481 In order to use this, the developer version of the ICU library
482 must be available. ICU support is recommended for applications
483 such as Pazpar2 and Zebra.
486 The <ulink url="&url.libxslt;">libxslt</ulink>,
487 <ulink url="&url.libxml2;">libxml2</ulink> librararies are required
488 if &yaz; is to support SRU/Solr.
489 These libraries are very portable and should compile out-of-the
490 box on virtually all Unix platforms. It is available in binary
491 forms for Linux and others.
495 <ulink url="&url.autoconf;">Autoconf</ulink>,
496 <ulink url="&url.automake;">Automake</ulink> and
497 <ulink url="&url.libtool;">Libtool</ulink>
498 are used to generate Makefiles and configure &yaz; for the system.
499 You do <emphasis>not</emphasis> these tools unless you're using the
500 Git version of &yaz;.
503 The CQL parser for &yaz; is built using
504 GNU <ulink url="&url.bison;">Bison</ulink>.
505 This tool is only needed if you're using the Git version of &yaz;.
508 &yaz; includes a tiny ASN.1 compiler. This compiler is
509 written in <ulink url="&url.tcl;">Tcl</ulink>.
510 But as for Bison you do not need it unless you're using Git
511 version of &yaz; or you're using the compiler to built own codecs
515 Generally it should be sufficient to run configure without options,
522 The configure script attempts to use use the C compiler specified by
523 the <literal>CC</literal> environment variable. If not set, GNU C will be
524 used if it is available. The <literal>CFLAGS</literal> environment
525 variable holds options to be passed to the C compiler. If you're using
526 Bourne-compatible shell you may pass something like this to use a
527 particular C compiler with optimization enabled:
530 CC=/opt/ccs/bin/cc CFLAGS=-O ./configure
533 To customize &yaz;, the configure script also accepts a set of options.
534 The most important are:
538 <literal>--prefix</literal>=<replaceable>prefix</replaceable>
541 <para>Specifies installation prefix for &yaz;. This is
542 only needed if you run <literal>make install</literal> later to
543 perform a "system" installation. The prefix is
544 <literal>/usr/local</literal> if not specified.
550 <literal>--enable-tcpd</literal>
553 <para>The front end server will be built using Wietse's
554 <ulink url="&url.tcpwrapper;">TCP wrapper library</ulink>.
555 It allows you to allow/deny clients depending on IP number.
556 The TCP wrapper library is often used in GNU/Linux and
560 <refentrytitle>hosts_access</refentrytitle>
561 <manvolnum>5</manvolnum>
565 <refentrytitle>tcpd</refentrytitle>
566 <manvolnum>8</manvolnum>
573 <literal>--enable-threads</literal>
576 <para>&yaz; will be built using POSIX threads.
577 Specifically, <constant>_REENTRANT</constant> will be defined during
584 <literal>--disable-shared</literal>
587 <para>The make process will not create shared
588 libraries (also known as shared objects <filename>.so</filename>).
589 By default, shared libraries are created -
590 equivalent to <literal>--enable-shared</literal>.
596 <literal>--disable-shared</literal>
599 <para>The make process will not create
600 static libraries (<filename>.a</filename>).
601 By default, static libraries are created -
602 equivalent to <literal>--enable-static</literal>.
608 <literal>--with-iconv</literal>[=<replaceable>prefix</replaceable>]
611 <para>Compile &yaz; with iconv library in directory
612 <replaceable>prefix</replaceable>. By default configure will
613 search for iconv on the system. Use this option if it
614 doesn't find iconv. Alternatively,
615 <literal>--without-iconv</literal>, can be uset to force &yaz;
622 <literal>--with-xslt</literal>[=<replaceable>prefix</replaceable>]
625 <para>Compile &yaz; with
626 <ulink url="&url.libxslt;">libxslt</ulink> in directory
627 <replaceable>prefix</replaceable>.
628 Use this option if you want XSLT and XML support.
629 By default, configure will
630 search for libxslt on the system. Use this option if it
631 libxslt is not found automatically. Alternatively,
632 <literal>--without-xslt</literal>, can be used to force &yaz;
639 <literal>--with-xml2</literal>[=<replaceable>prefix</replaceable>]
642 <para>Compile &yaz; with
643 <ulink url="&url.libxml2;">libxml2</ulink> in directory
644 <replaceable>prefix</replaceable>.
645 Use this option if you want &yaz; to use XML and support SRU/Solr.
646 By default, configure will
647 search for libxml2 on the system. Use this option if it
648 libxml2 is not found automatically. Alternatively,
649 <literal>--without-xml2</literal>, can be used to force &yaz;
653 Note that option <literal>--with-xslt</literal>
654 also enables libxml2.
660 <literal>--with-gnutls</literal>[=<replaceable>prefix</replaceable>]
663 <para>&yaz; will be linked with the GNU TLS libraries and
664 an SSL COMSTACK will be provided. By default configure enables
665 SSL support for YAZ if the GNU TLS development libraries are found
672 <literal>--with-icu</literal>[=<replaceable>prefix</replaceable>]
675 <para>&yaz; will be linked the
676 <ulink url="&url.icu;">ICU</ulink> library in the prefix if given.
677 If prefix is not given, the libraries exposed by the script
678 <application>icu-config</application> will be used if found.
685 <literal>--with-libgcrypt</literal>[=<replaceable>prefix</replaceable>]
688 <para>&yaz; will be linked with
689 <ulink url="&url.libgcrypt;">Libgcrypt</ulink> in the prefix if given.
690 If prefix is not given, the libraries exposed by the script
691 <application>libgcrypt-config</application> will be used if found.
697 <literal>--with-memcached</literal>
700 <para>&yaz; will be linked with
701 <ulink url="&url.libmemcached;">libMemcached</ulink> to allow
702 for result-set caching for ZOOM.
703 The prefix can not be given. Note that YAZ will only search
704 for libMemcached if Libgcrypt is also enabled.
711 When configured, build the software by typing:
717 The following files are generated by the make process:
720 <term><filename>src/libyaz.la</filename></term>
722 Main &yaz; library. This is no ordinary library. It's
724 By default, &yaz; creates a static library in
725 <filename>lib/.libs/libyaz.a</filename>.
729 <term><filename>src/libyaz_server.la</filename></term>
731 Generic Frontend server. This is an add-on for libyaz.la.
732 Code in this library uses POSIX threads functions - if POSIX
733 threads are available on the platform.
737 <term><filename>src/libyaz_icu.la</filename></term>
739 Functions that wrap the ICU library.
743 <term><filename>ztest/yaz-ztest</filename></term>
744 <listitem><para>Test Z39.50 server.
748 <term><filename>client/yaz-client</filename></term>
749 <listitem><para>Z39.50 client for testing the protocol.
750 See chapter <link linkend="yaz-client">
751 YAZ client</link> for more information.
755 <term><filename>util/yaz-config</filename></term>
756 <listitem><para>A Bourne-shell script, generated by configure, that
757 specifies how external applications should compile - and link with
762 <term><filename>util/yaz-asncomp</filename></term>
763 <listitem><para>The ASN.1 compiler for &yaz;. Requires the
764 Tcl Shell, <application>tclsh</application>, in
765 <literal>PATH</literal> to operate.
769 <term><filename>util/yaz-iconv</filename></term>
770 <listitem><para>This program converts data in one character set to
771 another. This command exercises the YAZ character set
776 <term><filename>util/yaz-marcdump</filename></term>
777 <listitem><para>This program parses ISO2709 encoded MARC records
778 and prints them in line-format or XML.
782 <term><filename>util/yaz-icu</filename></term>
783 <listitem><para>This program exposes the ICU wrapper library if that
784 is enabled for YAZ. Only if ICU is available this program is
789 <term><filename>util/yaz-url</filename></term>
790 <listitem><para>This program is a simple HTTP page fetcher ala
795 <term><filename>zoom/zoomsh</filename></term>
797 A simple shell implemented on top of the
798 <link linkend="zoom">ZOOM</link> functions.
799 The shell is a command line application that allows you to enter
800 simple commands to perform ZOOM operations.
804 <term><filename>zoom/zoomtst1</filename>,
805 <filename>zoom/zoomtst2</filename>, ..</term>
807 Several small applications that demonstrates the ZOOM API.
813 If you wish to install &yaz; in system directories
814 <filename>/usr/local/bin</filename>,
815 <filename>/usr/local/lib</filename> .. etc, you can type:
821 You probably need to have root access in order to perform this.
822 You must specify the <literal>--prefix</literal> option for configure if
823 you wish to install &yaz; in other directories than the default
824 <filename>/usr/local/</filename>.
827 If you wish to perform an un-installation of &yaz;, use:
833 This will only work if you haven't reconfigured &yaz; (and therefore
834 changed installation prefix). Note that uninstall will not
835 remove directories created by make install, e.g.
836 <filename>/usr/local/include/yaz</filename>.
839 <sect2 id="installation-linking-yaz-unix">
840 <title>How to make apps using YAZ on UNIX</title>
842 This section describes how to compile - and link your own
843 applications using the &yaz; toolkit.
844 If you're used to Makefiles this shouldn't be hard. As for
845 other libraries you have used before, you have to set a proper include
846 path for your C/C++ compiler and specify the location of
847 &yaz; libraries. You can do it by hand, but generally we suggest
848 you use the <filename>yaz-config</filename> that is generated
849 by <filename>configure</filename>. This is especially
850 important if you're using the threaded version of &yaz; which
851 require you to pass more options to your linker/compiler.
854 The <filename>yaz-config</filename> script accepts command line
855 options that makes the <filename>yaz-config</filename> script print
856 options that you should use in your make process.
857 The most important ones are:
858 <literal>--cflags</literal>, <literal>--libs</literal>
859 which prints C compiler flags, and linker flags respectively.
862 A small and complete <literal>Makefile</literal> for a C
863 application consisting of one source file,
864 <filename>myprog.c</filename>, may look like this:
866 YAZCONFIG=/usr/local/bin/yaz-config
867 CFLAGS=`$(YAZCONFIG) --cflags`
868 LIBS=`$(YAZCONFIG) --libs`
870 $(CC) $(CFLAGS) -o myprog myprog.o $(LIBS)
874 The CFLAGS variable consists of a C compiler directive that will set
875 the include path to the <emphasis>parent</emphasis> directory
876 of <filename>yaz</filename>. That is, if &yaz; header files were
877 installed in <filename>/usr/local/include/yaz</filename>,
878 then include path is set to <filename>/usr/local/include</filename>.
879 Therefore, in your applications you should use
881 #include <yaz/proto.h>
883 and <emphasis>not</emphasis>
885 #include <proto.h>
889 For Libtool users, the <filename>yaz-config</filename> script provides
890 a different variant of option <literal>--libs</literal>, called
891 <literal>--lalibs</literal> that returns the name of the
892 Libtool archive(s) for &yaz; rather than the ordinary ones.
895 For applications using the threaded version of &yaz;,
896 specify <literal>threads</literal> after the
897 other options. When <literal>threads</literal> is given,
898 more flags and linker flags will be printed by
899 <filename>yaz-config</filename>. If our previous example was
900 using threads, you'd have to modify the lines that set
901 <literal>CFLAGS</literal> and <literal>LIBS</literal> as
904 CFLAGS=`$(YAZCONFIG) --cflags threads`
905 LIBS=`$(YAZCONFIG) --libs threads`
907 There is no need specify POSIX thread libraries in your Makefile.
908 The <literal>LIBS</literal> variable includes that as well.
912 <sect1 id="installation.win32">
914 <para>The easiest way to install YAZ on Windows is by downloading
916 <ulink url="&url.yaz.download.win32;">here</ulink>.
917 The installer comes with source too - in case you wish to
918 compile YAZ with different compiler options, etc.
921 <sect2 id="installation.win32.source">
922 <title>Compiling from Source on WIN32</title>
924 &yaz; is shipped with "makefiles" for the NMAKE tool that comes
925 with <ulink url="&url.vstudio;">
926 Microsoft Visual Studio</ulink>. It has been tested with
927 Microsoft Visual Studio 2003/2005/2008.
930 Start a command prompt and switch the sub directory
931 <filename>WIN</filename> where the file <filename>makefile</filename>
932 is located. Customize the installation by editing the
933 <filename>makefile</filename> file (for example by using notepad).
934 The following summarizes the most important settings in that file:
937 <term><literal>DEBUG</literal></term>
939 If set to 1, the software is
940 compiled with debugging libraries (code generation is
941 multi-threaded debug DLL).
942 If set to 0, the software is compiled with release libraries
943 (code generation is multi-threaded DLL).
947 <term><literal>HAVE_TCL</literal>, <literal>TCL</literal></term>
949 If <literal>HAVE_TCL</literal> is set to 1, nmake will
950 use the ASN.1 compiler (<ulink url="&url.tcl;">Tcl</ulink> based).
951 You must set <literal>TCL</literal> to the full path of the Tcl
952 interpreter. A Windows version of Tcl is part of
953 <ulink url="&url.gitwindows;">Git for Windows</ulink>.
956 If you do not have Tcl installed, set
957 <literal>HAVE_TCL</literal> to 0.
961 <term><literal>HAVE_BISON</literal>,
962 <literal>BISON</literal></term>
964 If GNU Bison is present, you might set <literal>HAVE_BISON</literal>
965 to 1 and specify the Bison executable in <literal>BISON</literal>.
966 Bison is only required if you use the Git version of
967 YAZ or if you modify the grammar for CQL
968 (<filename>cql.y</filename>).
971 A Windows version of GNU Bison is part of
972 <ulink url="&url.gitwindows;">Git for Windows</ulink>.
976 <term><literal>HAVE_ICONV</literal>,
977 <literal>ICONV_DIR</literal></term>
979 If <literal>HAVE_ICONV</literal> is set to 1, YAZ is compiled
980 with iconv support. In this configuration, set
981 <literal>ICONV_DIR</literal> to the iconv source directory.
985 <term><literal>HAVE_LIBXML2</literal>,
986 <literal>LIBXML2_DIR</literal></term>
989 If <literal>HAVE_LIBXML2</literal> is set to 1, YAZ is compiled
990 with SRU support. In this configuration, set
991 <literal>LIBXML2_DIR</literal> to the
992 <ulink url="&url.libxml2;">libxml2</ulink> source directory
994 <literal>ZLIB_DIR</literal> to the zlib directory.
997 Windows versions of libxslt, libxml2, zlib and iconv can be found
998 <ulink url="&url.libxml2.download.win32;">
999 Igor Zlatković' site</ulink>.
1003 YAZ is not using zlib but libxml2 is depending on it.
1009 <term><literal>HAVE_LIBXSLT</literal>,
1010 <literal>LIBXSLT_DIR</literal></term>
1013 If <literal>HAVE_LIBXSLT</literal> is set to 1, YAZ is compiled
1014 with XSLT support. In this configuration, set
1015 <literal>LIBXSLT_DIR</literal> to the
1016 <ulink url="&url.libxslt;">libxslt</ulink> source directory.
1020 libxslt depends libxml2.
1026 <term><literal>HAVE_ICU</literal>,
1027 <literal>ICU_DIR</literal></term>
1030 If <literal>HAVE_ICU</literal> is set to 1, YAZ is compiled
1031 with <ulink url="&url.icu;">ICU</ulink> support.
1032 In this configuration, set
1033 <literal>ICU_DIR</literal> to the
1034 <ulink url="&url.icu;">ICU</ulink> source directory.
1041 When satisfied with the settings in the makefile, type
1048 If the <filename>nmake</filename> command is not found on your system
1049 you probably haven't defined the environment variables required to
1050 use that tool. To fix that, find and run the batch file
1051 <filename>vcvars32.bat</filename>. You need to run it from within
1052 the command prompt or set the environment variables "globally";
1053 otherwise it doesn't work.
1057 If you wish to recompile &yaz; - for example if you modify
1058 settings in the <filename>makefile</filename> you can delete
1059 object files, etc by running.
1065 The following files are generated upon successful compilation:
1068 <term><filename>bin/yaz&soversion;.dll</filename> /
1069 <filename>bin/yaz&soversion;d.dll</filename></term>
1071 &yaz; Release/Debug DLL.
1075 <term><filename>lib/yaz&soversion;.lib</filename> /
1076 <filename>lib/yaz&soversion;d.lib</filename></term>
1078 Import library for <filename>yaz&soversion;.dll</filename> /
1079 <filename>yaz&soversion;d.dll</filename>.
1083 <term><filename>bin/yaz_cond&soversion;.dll</filename> /
1084 <filename>bin/yaz_cond&soversion;d.dll</filename></term>
1086 Release/Debug DLL for condition variable utilities (condvar.c).
1090 <term><filename>lib/yaz_cond&soversion;.lib</filename> /
1091 <filename>lib/yaz_cond&soversion;d.lib</filename></term>
1093 Import library for <filename>yaz_cond&soversion;.dll</filename> /
1094 <filename>yaz_cond&soversion;d.dll</filename>.
1098 <term><filename>bin/yaz_icu&soversion;.dll</filename> /
1099 <filename>bin/yaz_icu&soversion;d.dll</filename></term>
1101 Release/Debug DLL for the ICU wrapper utility.
1102 Only build if HAVE_ICU is 1.
1106 <term><filename>lib/yaz_icu&soversion;.lib</filename> /
1107 <filename>lib/yaz_icu&soversion;d.lib</filename></term>
1109 Import library for <filename>yaz_icu&soversion;.dll</filename> /
1110 <filename>yaz_icu&soversion;d.dll</filename>.
1114 <term><filename>bin/yaz-ztest.exe</filename></term>
1116 Z39.50 multi-threaded test/example server. It's a WIN32
1117 console application.
1121 <term><filename>bin/yaz-client.exe</filename></term>
1123 &yaz; Z39.50 client application. It's a WIN32 console application.
1124 See chapter <link linkend="yaz-client">YAZ client</link> for more
1129 <term><filename>bin/yaz-icu.exe</filename></term>
1130 <listitem><para>This program exposes the ICU wrapper library if that
1131 is enabled for YAZ. Only if ICU is available this program is
1136 <term><filename>bin/zoomsh.exe</filename></term>
1138 Simple console application implemented on top of the
1139 <link linkend="zoom">ZOOM</link> functions.
1140 The application is a command line shell that allows you to enter
1141 simple commands to perform ZOOM operations.
1145 <term><filename>bin/zoomtst1.exe</filename>,
1146 <filename>bin/zoomtst2.exe</filename>, ..</term>
1148 Several small applications that demonstrates the ZOOM API.
1155 <sect2 id="installation-linking-yaz-win32">
1156 <title>How to make apps using YAZ on WIN32</title>
1158 This section will go though the process of linking your WIN32
1159 applications with &yaz;.
1162 Some people are confused by the fact that we use the nmake
1163 tool to build &yaz;. They think they have to do that too - in order
1164 to make their WIN32 applications work with &yaz;. The good news is that
1165 you don't have to. You can use the integrated environment of
1166 Visual Studio if desired for your own application.
1169 When setting up a project or Makefile you have to set the following:
1172 <term>include path</term>
1174 Set it to the <filename>include</filename> directory of &yaz;.
1178 <term>import library <filename>yaz&soversion;.lib</filename></term>
1180 You must link with this library. It's located in the
1181 sub directory <filename>lib</filename> of &yaz;.
1182 If you want to link with the debug version of &yaz;, you must
1183 link against <filename>yaz&soversion;d.lib</filename> instead.
1187 <term>dynamic link library
1188 <filename>yaz&soversion;.dll</filename>
1191 This DLL must be in your execution path when you invoke
1192 your application. Specifically, you should distribute this
1193 DLL with your application.
1202 ### Still to document:
1203 ZOOM_connection_errcode(c)
1204 ZOOM_connection_errmsg(c)
1205 ZOOM_connection_addinfo(c)
1206 ZOOM_connection_addinfo(c)
1207 ZOOM_connection_diagset(c);
1208 ZOOM_connection_save_apdu_wrbuf
1209 ZOOM_diag_str(error)
1210 ZOOM_resultset_record_immediate(s, pos)
1211 ZOOM_resultset_cache_reset(r)
1212 ZOOM_options_set_callback(opt, function, handle)
1213 ZOOM_options_create_with_parent2(parent1, parent2)
1214 ZOOM_options_getl(opt, name, len)
1215 ZOOM_options_setl(opt, name, value, len)
1216 ZOOM_options_get_bool(opt, name, defa)
1217 ZOOM_options_get_int(opt, name, defa)
1218 ZOOM_options_set_int(opt, name, value)
1223 &zoom; is an acronym for 'Z39.50 Object-Orientation Model' and is
1224 an initiative started by Mike Taylor (Mike is from the UK, which
1225 explains the peculiar name of the model). The goal of &zoom; is to
1226 provide a common Z39.50 client API not bound to a particular
1227 programming language or toolkit.
1230 From YAZ version 2.1.12, <ulink url="&url.sru;">SRU</ulink> is supported.
1231 You can make SRU ZOOM connections by specifying scheme
1232 <literal>http://</literal> for the hostname for a connection.
1233 The dialect of SRU used is specified by the value of the
1234 connection's <literal>sru</literal> option, which may be SRU over
1235 HTTP GET (<literal>get</literal>),
1236 SRU over HTTP POST (<literal>post</literal>), (SRU over
1237 SOAP) (<literal>soap</literal>) or <literal>solr</literal>
1238 (<ulink url="&url.solr;">Solr</ulink> Web Service).
1239 Using the facility for embedding options in target strings, a
1240 connection can be forced to use SRU rather the SRW (the default) by
1241 prefixing the target string with <literal>sru=get,</literal>, like this:
1242 <literal>sru=get,http://sru.miketaylor.org.uk:80/sru.pl</literal>
1245 <ulink url="&url.solr;">Solr</ulink> protocol support was added to
1246 YAZ in version 4.1.0, as a dialect of a SRU protocol, since both are
1247 HTTP based protocols.
1250 The lack of a simple Z39.50 client API for &yaz; has become more
1251 and more apparent over time. So when the first &zoom; specification
1253 an implementation for &yaz; was quickly developed. For the first time, it is
1254 now as easy (or easier!) to develop clients than servers with &yaz;. This
1255 chapter describes the &zoom; C binding. Before going further, please
1256 reconsider whether C is the right programming language for the job.
1257 There are other language bindings available for &yaz;, and still
1259 are in active development. See the
1260 <ulink url="&url.zoom;">ZOOM web-site</ulink> for
1264 In order to fully understand this chapter you should read and
1265 try the example programs <literal>zoomtst1.c</literal>,
1266 <literal>zoomtst2.c</literal>, .. in the <literal>zoom</literal>
1270 The C language misses features found in object oriented languages
1271 such as C++, Java, etc. For example, you'll have to manually,
1272 destroy all objects you create, even though you may think of them as
1273 temporary. Most objects has a <literal>_create</literal> - and a
1274 <literal>_destroy</literal> variant.
1275 All objects are in fact pointers to internal stuff, but you don't see
1276 that because of typedefs. All destroy methods should gracefully ignore a
1277 <literal>NULL</literal> pointer.
1280 In each of the sections below you'll find a sub section called
1281 protocol behavior, that describes how the API maps to the Z39.50
1284 <sect1 id="zoom-connections">
1285 <title>Connections</title>
1286 <para>The Connection object is a session with a target.
1289 #include <yaz/zoom.h>
1291 ZOOM_connection ZOOM_connection_new(const char *host, int portnum);
1293 ZOOM_connection ZOOM_connection_create(ZOOM_options options);
1295 void ZOOM_connection_connect(ZOOM_connection c, const char *host,
1297 void ZOOM_connection_destroy(ZOOM_connection c);
1300 Connection objects are created with either function
1301 <function>ZOOM_connection_new</function> or
1302 <function>ZOOM_connection_create</function>.
1303 The former creates and automatically attempts to establish a network
1304 connection with the target. The latter doesn't establish
1305 a connection immediately, thus allowing you to specify options
1306 before establishing network connection using the function
1307 <function>ZOOM_connection_connect</function>.
1308 If the port number, <literal>portnum</literal>, is zero, the
1309 <literal>host</literal> is consulted for a port specification.
1310 If no port is given, 210 is used. A colon denotes the beginning of
1311 a port number in the host string. If the host string includes a
1312 slash, the following part specifies a database for the connection.
1315 You can prefix the host with a scheme followed by colon. The
1316 default scheme is <literal>tcp</literal> (Z39.50 protocol).
1317 The scheme <literal>http</literal> selects SRU/get over HTTP by default,
1318 but can overridded to use SRU/post, SRW and the Solr protocol.
1321 You can prefix the scheme-qualified host-string with one or more
1323 <literal><parameter>key</parameter>=<parameter>value</parameter></literal>
1324 sequences, each of which represents an option to be set into the
1325 connection structure <emphasis>before</emphasis> the
1326 protocol-level connection is forged and the initialization
1327 handshake takes place. This facility can be used to provide
1328 authentication credentials, as in host-strings such as:
1329 <literal>user=admin,password=halfAm4n,tcp:localhost:8017/db</literal>
1332 Connection objects should be destroyed using the function
1333 <function>ZOOM_connection_destroy</function>.
1336 void ZOOM_connection_option_set(ZOOM_connection c,
1337 const char *key, const char *val);
1339 void ZOOM_connection_option_setl(ZOOM_connection c,
1341 const char *val, int len);
1343 const char *ZOOM_connection_option_get(ZOOM_connection c,
1345 const char *ZOOM_connection_option_getl(ZOOM_connection c,
1350 The functions <function>ZOOM_connection_option_set</function> and
1351 <function>ZOOM_connection_option_setl</function> allows you to
1352 set an option given by <parameter>key</parameter> to the value
1353 <parameter>value</parameter> for the connection.
1354 For <function>ZOOM_connection_option_set</function>, the
1355 value is assumed to be a 0-terminated string. Function
1356 <function>ZOOM_connection_option_setl</function> specifies a
1357 value of a certain size (len).
1360 Functions <function>ZOOM_connection_option_get</function> and
1361 <function>ZOOM_connection_option_getl</function> returns
1362 the value for an option given by <parameter>key</parameter>.
1364 <table id="zoom-connection-options" frame="top">
1365 <title>ZOOM Connection Options</title>
1367 <colspec colwidth="4*" colname="name"></colspec>
1368 <colspec colwidth="7*" colname="description"></colspec>
1369 <colspec colwidth="3*" colname="default"></colspec>
1372 <entry>Option</entry>
1373 <entry>Description</entry>
1374 <entry>Default</entry>
1379 implementationName</entry><entry>Name of Your client
1380 </entry><entry>none</entry></row>
1382 user</entry><entry>Authentication user name
1383 </entry><entry>none</entry></row>
1385 group</entry><entry>Authentication group name
1386 </entry><entry>none</entry></row>
1388 password</entry><entry>Authentication password.
1389 </entry><entry>none</entry></row>
1391 authenticationMode</entry><entry>How authentication is encoded.
1392 </entry><entry>basic</entry></row>
1394 host</entry><entry>Target host. This setting is "read-only".
1395 It's automatically set internally when connecting to a target.
1396 </entry><entry>none</entry></row>
1398 proxy</entry><entry>Proxy host. If set, the logical host
1399 is encoded in the otherInfo area of the Z39.50 Init PDU
1400 with OID 1.2.840.10003.10.1000.81.1.
1401 </entry><entry>none</entry></row>
1403 clientIP</entry><entry>Client IP. If set, is
1404 encoded in the otherInfo area of a Z39.50 PDU with OID
1405 1.2.840.10003.10.1000.81.3. Holds the original IP addreses
1406 of a client. Is used of ZOOM is used in a gateway of some sort.
1407 </entry><entry>none</entry></row>
1409 async</entry><entry>If true (1) the connection operates in
1410 asynchronous operation which means that all calls are non-blocking
1412 <link linkend="zoom.events"><function>ZOOM_event</function></link>.
1413 </entry><entry>0</entry></row>
1415 maximumRecordSize</entry><entry> Maximum size of single record.
1416 </entry><entry>1 MB</entry></row>
1418 preferredMessageSize</entry><entry> Maximum size of multiple records.
1419 </entry><entry>1 MB</entry></row>
1421 lang</entry><entry> Language for negotiation.
1422 </entry><entry>none</entry></row>
1424 charset</entry><entry> Character set for negotiation.
1425 </entry><entry>none</entry></row>
1427 serverImplementationId</entry><entry>
1428 Implementation ID of server. (The old targetImplementationId
1429 option is also supported for the benefit of old applications.)
1430 </entry><entry>none</entry></row>
1432 targetImplementationName</entry><entry>
1433 Implementation Name of server. (The old
1434 targetImplementationName option is also supported for the
1435 benefit of old applications.)
1436 </entry><entry>none</entry></row>
1438 serverImplementationVersion</entry><entry>
1439 Implementation Version of server. (the old
1440 targetImplementationVersion option is also supported for the
1441 benefit of old applications.)
1442 </entry><entry>none</entry></row>
1444 databaseName</entry><entry>One or more database names
1445 separated by character plus (<literal>+</literal>), which to
1446 be used by subsequent search requests on this Connection.
1447 </entry><entry>Default</entry></row>
1449 piggyback</entry><entry>True (1) if piggyback should be
1450 used in searches; false (0) if not.
1451 </entry><entry>1</entry></row>
1453 smallSetUpperBound</entry><entry>If hits is less than or equal to this
1454 value, then target will return all records using small element set name
1455 </entry><entry>0</entry></row>
1457 largeSetLowerBound</entry><entry>If hits is greater than this
1458 value, the target will return no records.
1459 </entry><entry>1</entry></row>
1461 mediumSetPresentNumber</entry><entry>This value represents
1462 the number of records to be returned as part of a search when when
1463 hits is less than or equal to large set lower bound and if hits
1464 is greater than small set upper bound.
1465 </entry><entry>0</entry></row>
1467 smallSetElementSetName</entry><entry>
1468 The element set name to be used for small result sets.
1469 </entry><entry>none</entry></row>
1471 mediumSetElementSetName</entry><entry>
1472 The element set name to be for medium-sized result sets.
1473 </entry><entry>none</entry></row>
1475 init_opt_search, init_opt_present, init_opt_delSet, etc.</entry><entry>
1476 After a successful Init, these options may be interrogated to
1477 discover whether the server claims to support the specified
1479 </entry><entry>none</entry></row>
1481 <entry>sru</entry><entry>
1482 SRU/Solr transport type. Must be either <literal>soap</literal>,
1483 <literal>get</literal>, <literal>post</literal>, or
1484 <literal>solr</literal>.
1485 </entry><entry>soap</entry></row>
1487 sru_version</entry><entry>
1488 SRU/SRW version. Should be <literal>1.1</literal>, or
1489 <literal>1.2</literal>. This is , prior to connect, the version
1490 to offer (highest version). And following connect (in fact
1491 first operation), holds the negotiated version with the server
1492 (same or lower version).
1493 </entry><entry>1.2</entry></row>
1494 <row id="zoom.facets.option"><entry>
1495 facets</entry><entry>
1496 Requested or recommend facets may be given before a search is sent.
1497 The value of this setting is described in <xref linkend="facets"/>
1498 For inspection of the facets returned, refer to the functions
1499 described in <xref linkend="zoom.facets"/>.
1500 </entry><entry>none</entry></row>
1502 apdulog</entry><entry>
1503 If set to a true value such as "1", a log of low-level
1504 protocol packets is emitted on standard error stream. This
1505 can be very useful for debugging.
1506 </entry><entry>0</entry></row>
1508 saveAPDU</entry><entry>
1509 If set to a true value such as "1", a log of low-level
1510 protocol packets is saved. The log can be retrieved by reading
1511 option APDU. Setting saveAPDU always has the side effect of
1512 resetting the currently saved log. This setting is
1513 <emphasis>write-only</emphasis>. If read, NULL will be returned.
1514 It is only recognized in
1515 <function>ZOOM_connection_option_set</function>.
1516 </entry><entry>0</entry></row>
1519 Returns the log of protocol packets. Will be empty if logging
1520 is not enabled (see saveAPDU above). This setting is
1521 <emphasis>read-only</emphasis>. It is only recognized if used
1522 in call to <function>ZOOM_connection_option_get</function> or
1523 <function>ZOOM_connection_option_getl</function>.
1524 </entry><entry></entry></row>
1526 memcached</entry><entry>
1527 If given and non-empty,
1528 <ulink url="&url.libmemcached;">libMemcached</ulink>
1529 will be configured for the connection.
1530 This option is inspected by ZOOM when a connection is established.
1531 If the <literal>memcached</literal> option is given
1532 and YAZ is compiled without libMemcached support, an internal
1533 diagnostic (10018) will be thrown.
1534 libMemcached support is available for YAZ 5.0.13 or later. If this
1535 option is supplied for an earlier version of YAZ, it is
1536 <emphasis>ignored</emphasis>.
1537 The value of this option is a string passed verbatim to
1538 the <function>memcached</function> function part of libMemcached.
1540 <ulink url="http://manned.org/memcached.3">memcached(3)</ulink>.
1541 Earlier versions of libMemcached
1542 do not offer this function. In this case only the option
1543 <literal>--server=</literal><replaceable>host</replaceable> may
1544 be given (YAZ emulates that part of libMemcached).
1545 </entry><entry>none</entry></row>
1550 If either option <literal>lang</literal> or <literal>charset</literal>
1552 <ulink url="&url.z39.50.charneg;">
1553 Character Set and Language Negotiation</ulink> is in effect.
1556 int ZOOM_connection_error(ZOOM_connection c, const char **cp,
1557 const char **addinfo);
1558 int ZOOM_connection_error_x(ZOOM_connection c, const char **cp,
1559 const char **addinfo, const char **dset);
1562 Function <function>ZOOM_connection_error</function> checks for
1563 errors for the last operation(s) performed. The function returns
1564 zero if no errors occurred; non-zero otherwise indicating the error.
1565 Pointers <parameter>cp</parameter> and <parameter>addinfo</parameter>
1566 holds messages for the error and additional-info if passed as
1567 non-<literal>NULL</literal>. Function
1568 <function>ZOOM_connection_error_x</function> is an extended version
1569 of <function>ZOOM_connection_error</function> that is capable of
1570 returning name of diagnostic set in <parameter>dset</parameter>.
1572 <sect2 id="zoom-connection-z39.50">
1573 <title>Z39.50 Protocol behavior</title>
1575 The calls <function>ZOOM_connection_new</function> and
1576 <function>ZOOM_connection_connect</function> establishes a TCP/IP
1577 connection and sends an Initialize Request to the target if
1578 possible. In addition, the calls waits for an Initialize Response
1579 from the target and the result is inspected (OK or rejected).
1582 If <literal>proxy</literal> is set then the client will establish
1583 a TCP/IP connection with the peer as specified by the
1584 <literal>proxy</literal> host and the hostname as part of the
1585 connect calls will be set as part of the Initialize Request.
1586 The proxy server will then "forward" the PDU's transparently
1587 to the target behind the proxy.
1590 For the authentication parameters, if option <literal>user</literal>
1591 is set and both options <literal>group</literal> and
1592 <literal>pass</literal> are unset, then Open style
1593 authentication is used (Version 2/3) in which case the username
1594 is usually followed by a slash, then by a password.
1595 If either <literal>group</literal>
1596 or <literal>pass</literal> is set then idPass authentication
1597 (Version 3 only) is used. If none of the options are set, no
1598 authentication parameters are set as part of the Initialize Request
1602 When option <literal>async</literal> is 1, it really means that
1603 all network operations are postponed (and queued) until the
1604 function <literal>ZOOM_event</literal> is invoked. When doing so
1605 it doesn't make sense to check for errors after
1606 <literal>ZOOM_connection_new</literal> is called since that
1607 operation "connecting - and init" is still incomplete and the
1608 API cannot tell the outcome (yet).
1611 <sect2 id="zoom.sru.init.behavior">
1612 <title>SRU/Solr Protocol behavior</title>
1614 The HTTP based protocols (SRU, SRW, Solr) doesn't feature an
1615 Inititialize Request, so the connection phase merely establishes a
1616 TCP/IP connection with the HTTP server.
1618 <para>Most of the ZOOM connection options do not
1619 affect SRU/Solr and they are ignored. However, future versions
1620 of &yaz; might honor <literal>implementationName</literal> and
1621 put that as part of User-Agent header for HTTP requests.
1624 The <literal>charset</literal> is used in the Content-Type header
1628 Setting <literal>authentcationMode</literal> specifies how
1629 authentication parameters are encoded for HTTP. The default is
1630 "<literal>basic</literal>" where <literal>user</literal> and
1631 <literal>password</literal> are encoded by using HTTP basic
1635 If <literal>authentcationMode</literal> is "<literal>url</literal>", then
1636 user and password are encoded in the URL by parameters
1637 <literal>x-username</literal> and <literal>x-password</literal> as
1638 given by the SRU standard.
1642 <sect1 id="zoom.query">
1643 <title>Queries</title>
1645 Query objects represents queries.
1648 ZOOM_query ZOOM_query_create(void);
1650 void ZOOM_query_destroy(ZOOM_query q);
1652 int ZOOM_query_prefix(ZOOM_query q, const char *str);
1654 int ZOOM_query_cql(ZOOM_query s, const char *str);
1656 int ZOOM_query_sortby(ZOOM_query q, const char *criteria);
1658 int ZOOM_query_sortby2(ZOOM_query q, const char *strategy,
1659 const char *criteria);
1662 Create query objects using <function>ZOOM_query_create</function>
1663 and destroy them by calling <function>ZOOM_query_destroy</function>.
1664 RPN-queries can be specified in <link linkend="PQF">PQF</link>
1665 notation by using the
1666 function <function>ZOOM_query_prefix</function>.
1667 The <function>ZOOM_query_cql</function> specifies a CQL
1668 query to be sent to the server/target.
1669 More query types will be added in future versions of &yaz;, such as
1670 <link linkend="CCL">CCL</link> to RPN-mapping, native CCL query,
1671 etc. In addition to a search, a sort criteria may be set. Function
1672 <function>ZOOM_query_sortby</function> enables Z39.50 sorting and
1673 it takes sort criteria using the same string notation as
1674 yaz-client's <link linkend="sortspec">sort command</link>.
1676 <para id="zoom.query.sortby2">
1677 <function>ZOOM_query_sortby2</function> is similar to
1678 <function>ZOOM_query_sortby</function> but allows a strategy for
1679 sorting. The reason for the strategy parameter is that some
1680 protocols offers multiple ways of performing sorting.
1681 For example, Z39.50 has the standard sort, which is performed after
1682 search on an existing result set.
1683 It's also possible to use CQL in Z39.50 as the query type and use
1684 CQL's SORTBY keyword. Finally, Index Data's
1685 Zebra server also allows sorting to be specified as part of RPN (Type 7).
1687 <table id="zoom-sort-strategy" frame="top">
1688 <title>ZOOM sort strategy</title>
1690 <colspec colwidth="2*" colname="name"/>
1691 <colspec colwidth="5*" colname="description"/>
1695 <entry>Description</entry>
1700 <entry>z39.50</entry><entry>Z39.50 resultset sort</entry>
1703 <entry>type7</entry><entry>Sorting embedded in RPN(Type-7)</entry>
1706 <entry>cql</entry><entry>CQL SORTBY</entry>
1709 <entry>sru11</entry><entry>SRU sortKeys parameter</entry>
1712 <entry>solr</entry><entry>Solr sort</entry>
1715 <entry>embed</entry><entry>type7 for Z39.50, cql for SRU,
1716 solr for Solr protocol</entry>
1722 <sect1 id="zoom.resultsets"><title>Result sets</title>
1724 The result set object is a container for records returned from
1728 ZOOM_resultset ZOOM_connection_search(ZOOM_connection, ZOOM_query q);
1730 ZOOM_resultset ZOOM_connection_search_pqf(ZOOM_connection c,
1732 void ZOOM_resultset_destroy(ZOOM_resultset r);
1735 Function <function>ZOOM_connection_search</function> creates
1736 a result set given a connection and query.
1737 Destroy a result set by calling
1738 <function>ZOOM_resultset_destroy</function>.
1739 Simple clients may using PQF only may use function
1740 <function>ZOOM_connection_search_pqf</function> in which case
1741 creating query objects is not necessary.
1744 void ZOOM_resultset_option_set(ZOOM_resultset r,
1745 const char *key, const char *val);
1747 const char *ZOOM_resultset_option_get(ZOOM_resultset r, const char *key);
1749 size_t ZOOM_resultset_size(ZOOM_resultset r);
1752 Functions <function>ZOOM_resultset_options_set</function> and
1753 <function>ZOOM_resultset_get</function> sets and gets an option
1754 for a result set similar to <function>ZOOM_connection_option_get</function>
1755 and <function>ZOOM_connection_option_set</function>.
1758 The number of hits also called result-count is returned by
1759 function <function>ZOOM_resultset_size</function>.
1761 <table id="zoom.resultset.options"
1762 frame="top"><title>ZOOM Result set Options</title>
1764 <colspec colwidth="4*" colname="name"></colspec>
1765 <colspec colwidth="7*" colname="description"></colspec>
1766 <colspec colwidth="2*" colname="default"></colspec>
1769 <entry>Option</entry>
1770 <entry>Description</entry>
1771 <entry>Default</entry>
1776 start</entry><entry>Offset of first record to be
1777 retrieved from target. First record has offset 0 unlike the
1778 protocol specifications where first record has position 1.
1779 This option affects ZOOM_resultset_search and
1780 ZOOM_resultset_search_pqf and must be set before any of
1781 these functions are invoked. If a range of
1782 records must be fetched manually after search,
1783 function ZOOM_resultset_records should be used.
1784 </entry><entry>0</entry></row>
1786 count</entry><entry>Number of records to be retrieved.
1787 This option affects ZOOM_resultset_search and
1788 ZOOM_resultset_search_pqf and must be set before any of
1789 these functions are invoked.
1790 </entry><entry>0</entry></row>
1792 presentChunk</entry><entry>The number of records to be
1793 requested from the server in each chunk (present request). The
1794 value 0 means to request all the records in a single chunk.
1795 (The old <literal>step</literal>
1796 option is also supported for the benefit of old applications.)
1797 </entry><entry>0</entry></row>
1799 elementSetName</entry><entry>Element-Set name of records.
1800 Most targets should honor element set name <literal>B</literal>
1801 and <literal>F</literal> for brief and full respectively.
1802 </entry><entry>none</entry></row>
1804 preferredRecordSyntax</entry><entry>Preferred Syntax, such as
1805 <literal>USMARC</literal>, <literal>SUTRS</literal>, etc.
1806 </entry><entry>none</entry></row>
1808 schema</entry><entry>Schema for retrieval, such as
1809 <literal>Gils-schema</literal>, <literal>Geo-schema</literal>, etc.
1810 </entry><entry>none</entry></row>
1812 setname</entry><entry>Name of Result Set (Result Set ID).
1813 If this option isn't set, the ZOOM module will automatically
1814 allocate a result set name.
1815 </entry><entry>default</entry></row>
1817 rpnCharset</entry><entry>Character set for RPN terms.
1818 If this is set, ZOOM C will assume that the ZOOM application is
1819 running UTF-8. Terms in RPN queries are then converted to the
1820 rpnCharset. If this is unset, ZOOM C will not assume any encoding
1821 of RPN terms and no conversion is performed.
1822 </entry><entry>none</entry></row>
1827 For servers that support Search Info report, the following
1828 options may be read using <function>ZOOM_resultset_get</function>.
1829 This detailed information is read after a successful search has
1833 This information is a list of of items, where each item is
1834 information about a term or subquery. All items in the list
1836 <literal>SearchResult.</literal><replaceable>no</replaceable>
1837 where no presents the item number (0=first, 1=second).
1838 Read <literal>searchresult.size</literal> to determine the
1841 <table id="zoom.search.info.report.options"
1842 frame="top"><title>Search Info Report Options</title>
1844 <colspec colwidth="4*" colname="name"></colspec>
1845 <colspec colwidth="7*" colname="description"></colspec>
1848 <entry>Option</entry>
1849 <entry>Description</entry>
1854 <entry>searchresult.size</entry>
1856 number of search result entries. This option is-nonexistant
1857 if no entries are returned by the server.
1861 <entry>searchresult.<replaceable>no</replaceable>.id</entry>
1862 <entry>sub query ID</entry>
1865 <entry>searchresult.<replaceable>no</replaceable>.count</entry>
1866 <entry>result count for item (number of hits)</entry>
1869 <entry>searchresult.<replaceable>no</replaceable>.subquery.term</entry>
1870 <entry>subquery term</entry>
1874 searchresult.<replaceable>no</replaceable>.interpretation.term
1876 <entry>interpretation term</entry>
1880 searchresult.<replaceable>no</replaceable>.recommendation.term
1882 <entry>recommendation term</entry>
1887 <sect2 id="zoom.z3950.resultset.sort">
1888 <title>Z39.50 Result-set Sort</title>
1890 void ZOOM_resultset_sort(ZOOM_resultset r,
1891 const char *sort_type, const char *sort_spec);
1893 int ZOOM_resultset_sort1(ZOOM_resultset r,
1894 const char *sort_type, const char *sort_spec);
1897 <function>ZOOM_resultset_sort</function> and
1898 <function>ZOOM_resultset_sort1</function> both sort an existing
1899 result-set. The sort_type parameter is not use. Set it to "yaz".
1900 The sort_spec is same notation as ZOOM_query_sortby and identical
1901 to that offered by yaz-client's
1902 <link linkend="sortspec">sort command</link>.
1905 These functions only work for Z39.50. Use the more generic utility
1906 <link linkend="zoom.query.sortby2">
1907 <function>ZOOM_query_sortby2</function></link>
1908 for other protocols (and even Z39.50).
1911 <sect2 id="zoom.z3950.resultset.behavior">
1912 <title>Z39.50 Protocol behavior</title>
1914 The creation of a result set involves at least a SearchRequest
1915 - SearchResponse protocol handshake. Following that, if a sort
1916 criteria was specified as part of the query, a SortRequest -
1917 SortResponse handshake takes place. Note that it is necessary to
1918 perform sorting before any retrieval takes place, so no records will
1919 be returned from the target as part of the SearchResponse because these
1920 would be unsorted. Hence, piggyback is disabled when sort criteria
1921 are set. Following Search - and a possible sort - Retrieval takes
1922 place - as one or more Present Requests/Response pairs being
1926 The API allows for two different modes for retrieval. A high level
1927 mode which is somewhat more powerful and a low level one.
1928 The low level is enabled when searching on a Connection object
1929 for which the settings
1930 <literal>smallSetUpperBound</literal>,
1931 <literal>mediumSetPresentNumber</literal> and
1932 <literal>largeSetLowerBound</literal> are set. The low level mode
1933 thus allows you to precisely set how records are returned as part
1934 of a search response as offered by the Z39.50 protocol.
1935 Since the client may be retrieving records as part of the
1936 search response, this mode doesn't work well if sorting is used.
1939 The high-level mode allows you to fetch a range of records from
1940 the result set with a given start offset. When you use this mode
1941 the client will automatically use piggyback if that is possible
1942 with the target and perform one or more present requests as needed.
1943 Even if the target returns fewer records as part of a present response
1944 because of a record size limit, etc. the client will repeat sending
1945 present requests. As an example, if option <literal>start</literal>
1946 is 0 (default) and <literal>count</literal> is 4, and
1947 <literal>piggyback</literal> is 1 (default) and no sorting criteria
1948 is specified, then the client will attempt to retrieve the 4
1949 records as part the search response (using piggyback). On the other
1950 hand, if either <literal>start</literal> is positive or if
1951 a sorting criteria is set, or if <literal>piggyback</literal>
1952 is 0, then the client will not perform piggyback but send Present
1956 If either of the options <literal>mediumSetElementSetName</literal> and
1957 <literal>smallSetElementSetName</literal> are unset, the value
1958 of option <literal>elementSetName</literal> is used for piggyback
1959 searches. This means that for the high-level mode you only have
1960 to specify one elementSetName option rather than three.
1963 <sect2 id="zoom.sru.resultset.behavior">
1964 <title>SRU Protocol behavior</title>
1966 Current version of &yaz; does not take advantage of a result set id
1967 returned by the SRU server. Future versions might do, however.
1968 Since, the ZOOM driver does not save result set IDs any
1969 present (retrieval) is transformed to a SRU SearchRetrieveRequest
1970 with same query but, possibly, different offsets.
1973 Option <literal>schema</literal> specifies SRU schema
1974 for retrieval. However, options <literal>elementSetName</literal> and
1975 <literal>preferredRecordSyntax</literal> are ignored.
1978 Options <literal>start</literal> and <literal>count</literal>
1979 are supported by SRU.
1980 The remaining options
1981 <literal>piggyback</literal>,
1982 <literal>smallSetUpperBound</literal>,
1983 <literal>largeSetLowerBound</literal>,
1984 <literal>mediumSetPresentNumber</literal>,
1985 <literal>mediumSetElementSetName</literal>,
1986 <literal>smallSetElementSetName</literal> are
1990 SRU supports CQL queries, <emphasis>not</emphasis> PQF.
1991 If PQF is used, however, the PQF query is transferred anyway
1992 using non-standard element <literal>pQuery</literal> in
1993 SRU SearchRetrieveRequest.
1996 Solr queries has to be done in Solr query format.
1999 Unfortunately, SRU or Solr does not define a database setting. Hence,
2000 <literal>databaseName</literal> is unsupported and ignored.
2001 However, the path part in host parameter for functions
2002 <function>ZOOM_connecton_new</function> and
2003 <function>ZOOM_connection_connect</function> acts as a
2004 database (at least for the &yaz; SRU server).
2008 <sect1 id="zoom.records">
2009 <title>Records</title>
2011 A record object is a retrieval record on the client side -
2012 created from result sets.
2015 void ZOOM_resultset_records(ZOOM_resultset r,
2017 size_t start, size_t count);
2018 ZOOM_record ZOOM_resultset_record(ZOOM_resultset s, size_t pos);
2020 const char *ZOOM_record_get(ZOOM_record rec, const char *type,
2023 int ZOOM_record_error(ZOOM_record rec, const char **msg,
2024 const char **addinfo, const char **diagset);
2026 ZOOM_record ZOOM_record_clone(ZOOM_record rec);
2028 void ZOOM_record_destroy(ZOOM_record rec);
2031 References to temporary records are returned by functions
2032 <function>ZOOM_resultset_records</function> or
2033 <function>ZOOM_resultset_record</function>.
2036 If a persistent reference to a record is desired
2037 <function>ZOOM_record_clone</function> should be used.
2038 It returns a record reference that should be destroyed
2039 by a call to <function>ZOOM_record_destroy</function>.
2042 A single record is returned by function
2043 <function>ZOOM_resultset_record</function> that takes a
2044 position as argument. First record has position zero.
2045 If no record could be obtained <literal>NULL</literal> is returned.
2048 Error information for a record can be checked with
2049 <function>ZOOM_record_error</function> which returns non-zero
2050 (error code) if record is in error, called <emphasis>Surrogate
2051 Diagnostics</emphasis> in Z39.50.
2054 Function <function>ZOOM_resultset_records</function> retrieves
2055 a number of records from a result set. Parameter <literal>start</literal>
2056 and <literal>count</literal> specifies the range of records to
2057 be returned. Upon completion array
2058 <literal>recs[0], ..recs[count-1]</literal>
2059 holds record objects for the records. The array of records
2060 <literal>recs</literal> should be allocated prior the call
2061 <function>ZOOM_resultset_records</function>. Note that for those
2062 records that couldn't be retrieved from the target
2063 <literal>recs[ ..]</literal> is set to <literal>NULL</literal>.
2065 <para id="zoom.record.get">
2066 In order to extract information about a single record,
2067 <function>ZOOM_record_get</function> is provided. The
2068 function returns a pointer to certain record information. The
2069 nature (type) of the pointer depends on the parameter,
2070 <parameter>type</parameter>.
2073 The <parameter>type</parameter> is a string of the format:
2076 <replaceable>format</replaceable>[;charset=<replaceable>from</replaceable>[/<replaceable>opacfrom</replaceable>][,<replaceable>to</replaceable>]][;format=<replaceable>v</replaceable>]
2079 where <replaceable>format</replaceable> specifies the format of the
2080 returned record, <replaceable>from</replaceable>
2081 specifies the character set of the record in its original form
2082 (as returned by the server), <replaceable>to</replaceable> specifies
2083 the output (returned)
2084 character set encoding.
2085 If <replaceable>to</replaceable> is omitted UTF-8 is assumed.
2086 If charset is not given, then no character set conversion takes place.
2089 <para>OPAC records may be returned in a different
2090 set from the bibliographic MARC record. If this is this the case,
2091 <replaceable>opacfrom</replaceable> should be set to the character set
2092 of the OPAC record part.
2096 Specifying the OPAC record character set requires YAZ 4.1.5 or later.
2100 The format argument controls whether record data should be XML
2101 pretty-printed (post process operation).
2102 It is enabled only if format value <replaceable>v</replaceable> is
2103 <literal>1</literal> and the record content is XML well-formed.
2106 In addition, for certain types, the length
2107 <literal>len</literal> passed will be set to the size in bytes of
2108 the returned information.
2111 The following are the supported values for <replaceable>form</replaceable>.
2113 <varlistentry><term><literal>database</literal></term>
2114 <listitem><para>Database of record is returned
2115 as a C null-terminated string. Return type
2116 <literal>const char *</literal>.
2119 <varlistentry><term><literal>syntax</literal></term>
2120 <listitem><para>The transfer syntax of the record is returned
2121 as a C null-terminated string containing the symbolic name of
2122 the record syntax, e.g. <literal>Usmarc</literal>. Return type
2124 <literal>const char *</literal>.
2127 <varlistentry><term><literal>schema</literal></term>
2128 <listitem><para>The schema of the record is returned
2129 as a C null-terminated string. Return type is
2130 <literal>const char *</literal>.
2133 <varlistentry><term><literal>render</literal></term>
2134 <listitem><para>The record is returned in a display friendly
2135 format. Upon completion buffer is returned
2136 (type <literal>const char *</literal>) and length is stored in
2137 <literal>*len</literal>.
2140 <varlistentry><term><literal>raw</literal></term>
2141 <listitem><para>The record is returned in the internal
2142 YAZ specific format. For GRS-1, Explain, and others, the
2143 raw data is returned as type
2144 <literal>Z_External *</literal> which is just the type for
2145 the member <literal>retrievalRecord</literal> in
2146 type <literal>NamePlusRecord</literal>.
2147 For SUTRS and octet aligned record (including all MARCs) the
2148 octet buffer is returned and the length of the buffer.
2151 <varlistentry><term><literal>xml</literal></term>
2152 <listitem><para>The record is returned in XML if possible.
2153 SRU, Solr and Z39.50 records with transfer syntax XML are
2154 returned verbatim. MARC records are returned in
2155 <ulink url="&url.marcxml;">
2158 (converted from ISO2709 to MARCXML by YAZ).
2159 OPAC records are also converted to XML and the
2160 bibliographic record is converted to MARCXML (when possible).
2161 GRS-1 records are not supported for this form.
2162 Upon completion, the XML buffer is returned
2163 (type <literal>const char *</literal>) and length is stored in
2164 <literal>*len</literal>.
2167 <varlistentry><term><literal>opac</literal></term>
2168 <listitem><para>OPAC information for record is returned in XML
2169 if an OPAC record is present at the position given. If no
2170 OPAC record is present, a NULL pointer is returned.
2173 <varlistentry><term><literal>txml</literal></term>
2174 <listitem><para>The record is returned in TurboMARC if possible.
2175 SRU and Z39.50 records with transfer syntax XML are
2176 returned verbatim. MARC records are returned in
2177 <link linkend="tools.turbomarc">
2180 (converted from ISO2709 to TurboMARC by YAZ).
2181 Upon completion, the XML buffer is returned
2182 (type <literal>const char *</literal>) and length is stored in
2183 <literal>*len</literal>.
2186 <varlistentry><term><literal>json</literal></term>
2187 <listitem><para>Like xml, but MARC records are converted to
2188 <ulink url="&url.marc_in_json;">MARC-in-JSON</ulink>.
2196 <ulink url="&url.marc21;">MARC21</ulink>
2198 <ulink url="&url.marc8;">MARC-8</ulink>
2199 character set encoding.
2200 An application that wishes to display in Latin-1 would use
2202 render; charset=marc8,iso-8859-1
2205 <sect2 id="zoom.z3950.record.behavior">
2206 <title>Z39.50 Protocol behavior</title>
2208 The functions <function>ZOOM_resultset_record</function> and
2209 <function>ZOOM_resultset_records</function> inspects the client-side
2210 record cache. Records not found in cache are fetched using
2212 The functions may block (and perform network I/O) - even though option
2213 <literal>async</literal> is 1, because they return records objects.
2214 (and there's no way to return records objects without retrieving them!).
2217 There is a trick, however, in the usage of function
2218 <function>ZOOM_resultset_records</function> that allows for
2219 delayed retrieval (and makes it non-blocking). By using
2220 a null pointer for <parameter>recs</parameter> you're indicating
2221 you're not interested in getting records objects
2222 <emphasis>now</emphasis>.
2225 <sect2 id="zoom.sru.record.behavior">
2226 <title>SRU/Solr Protocol behavior</title>
2228 The ZOOM driver for SRU/Solr treats records returned by a SRU/Solr server
2229 as if they where Z39.50 records with transfer syntax XML and
2230 no element set name or database name.
2234 <sect1 id="zoom.facets"><title>Facets</title>
2236 Facet operations is not part of the official ZOOM specification, but
2237 is an Index Data extension for YAZ-based Z39.50 targets,
2238 <ulink url="&url.solr;">Solr</ulink> and SRU 2.0 targets.
2240 Facets may be requestd by the
2241 <link linkend="zoom.facets.option">facets</link> option before a
2243 For inspection of the returned facets, the following functions are
2247 ZOOM_facet_field *ZOOM_resultset_facets(ZOOM_resultset r);
2249 ZOOM_facet_field ZOOM_resultset_get_facet_field(ZOOM_resultset r,
2250 const char *facet_name);
2252 ZOOM_facet_field ZOOM_resultset_get_facet_field_by_index(ZOOM_resultset r,
2255 size_t ZOOM_resultset_facets_size(ZOOM_resultset r);
2257 const char *ZOOM_facet_field_name(ZOOM_facet_field facet_field);
2259 size_t ZOOM_facet_field_term_count(ZOOM_facet_field facet_field);
2261 const char *ZOOM_facet_field_get_term(ZOOM_facet_field facet_field,
2262 size_t idx, int *freq);
2265 References to temporary structures are returned by all functions.
2266 They are only valid as long the Result set is valid.
2267 <function>ZOOM_resultset_get_facet_field</function> or
2268 <function>ZOOM_resultset_get_facet_field_by_index</function>.
2269 <function>ZOOM_resultset_facets</function>.
2270 <function>ZOOM_facet_field_name</function>.
2271 <function>ZOOM_facet_field_get_term</function>.
2273 <para id="zoom.resultset.get_facet_field">
2274 A single Facet field is returned by function
2275 <function>ZOOM_resultset_get_facet_field</function> or
2276 <function>ZOOM_resultset_get_facet_field_by_index</function> that takes
2277 a result set and facet name or positive index respectively. First
2278 facet has position zero. If no facet could be obtained (invalid name
2279 or index out of bounds) <literal>NULL</literal> is returned.
2281 <para id="zoom.resultset.facets">
2282 An array of facets field can be returned by
2283 <function>ZOOM_resultset_facets</function>. The length of the array is
2284 given by <function>ZOOM_resultset_facets_size</function>. The array is
2285 zero-based and last entry will be at
2286 <function>ZOOM_resultset_facets_size(result_set)</function>-1.
2288 <para id="zoom.resultset.facets_names">
2289 It is possible to interate over facets by name, by calling
2290 <function>ZOOM_resultset_facets_names</function>.
2291 This will return an const array of char * where each string can be used
2292 as parameter for <function>ZOOM_resultset_get_facet_field</function>.
2295 Function <function>ZOOM_facet_field_name</function> gets the request
2296 facet name from a returned facet field.
2299 Function <function>ZOOM_facet_field_get_term</function> returns the
2300 idx'th term and term count for a facet field.
2301 Idx must between 0 and
2302 <function>ZOOM_facet_field_term_count</function>-1, otherwise the
2303 returned reference will be <literal>NULL</literal>. On a valid idx, the
2304 value of the freq reference will be the term count.
2305 The <literal>freq</literal> parameter must be valid pointer to integer.
2308 <sect1 id="zoom.scan"><title>Scan</title>
2310 This section describes an interface for Scan. Scan is not an
2311 official part of the ZOOM model yet. The result of a scan operation
2312 is the <literal>ZOOM_scanset</literal> which is a set of terms
2313 returned by a target.
2317 The Scan interface is supported for both Z39.50, SRU and Solr.
2321 ZOOM_scanset ZOOM_connection_scan(ZOOM_connection c,
2322 const char *startpqf);
2324 ZOOM_scanset ZOOM_connection_scan1(ZOOM_connection c,
2327 size_t ZOOM_scanset_size(ZOOM_scanset scan);
2329 const char *ZOOM_scanset_term(ZOOM_scanset scan, size_t pos,
2330 size_t *occ, size_t *len);
2332 const char *ZOOM_scanset_display_term(ZOOM_scanset scan, size_t pos,
2333 size_t *occ, size_t *len);
2335 void ZOOM_scanset_destroy(ZOOM_scanset scan);
2337 const char *ZOOM_scanset_option_get(ZOOM_scanset scan,
2340 void ZOOM_scanset_option_set(ZOOM_scanset scan, const char *key,
2344 The scan set is created by function
2345 <function>ZOOM_connection_scan</function> which performs a scan
2346 operation on the connection using the specified
2347 <parameter>startpqf</parameter>.
2348 If the operation was successful, the size of the scan set can be
2349 retrieved by a call to <function>ZOOM_scanset_size</function>.
2350 Like result sets, the items are numbered 0,..size-1.
2351 To obtain information about a particular scan term, call function
2352 <function>ZOOM_scanset_term</function>. This function takes
2353 a scan set offset <literal>pos</literal> and returns a pointer
2354 to a <emphasis>raw term</emphasis> or <literal>NULL</literal> if
2356 If present, the <literal>occ</literal> and <literal>len</literal>
2357 are set to the number of occurrences and the length
2358 of the actual term respectively.
2359 <function>ZOOM_scanset_display_term</function> is similar to
2360 <function>ZOOM_scanset_term</function> except that it returns
2361 the <emphasis>display term</emphasis> rather than the raw term.
2362 In a few cases, the term is different from display term. Always
2363 use the display term for display and the raw term for subsequent
2364 scan operations (to get more terms, next scan result, etc).
2367 A scan set may be freed by a call to function
2368 <function>ZOOM_scanset_destroy</function>.
2369 Functions <function>ZOOM_scanset_option_get</function> and
2370 <function>ZOOM_scanset_option_set</function> retrieves and sets
2371 an option respectively.
2374 The <parameter>startpqf</parameter> is a subset of PQF, namely
2375 the Attributes+Term part. Multiple <literal>@attr</literal> can
2376 be used. For example to scan in title (complete) phrases:
2378 @attr 1=4 @attr 6=2 "science o"
2382 The <function>ZOOM_connecton_scan1</function> is a newer and
2383 more generic alternative to <function>ZOOM_connection_scan</function>
2384 which allows to use both CQL and PQF for Scan.
2386 <table frame="top" id="zoom.scanset.options">
2387 <title>ZOOM Scan Set Options</title>
2389 <colspec colwidth="4*" colname="name"></colspec>
2390 <colspec colwidth="7*" colname="description"></colspec>
2391 <colspec colwidth="2*" colname="default"></colspec>
2394 <entry>Option</entry>
2395 <entry>Description</entry>
2396 <entry>Default</entry>
2401 number</entry><entry>Number of Scan Terms requested in next scan.
2402 After scan it holds the actual number of terms returned.
2403 </entry><entry>20</entry></row>
2405 position</entry><entry>Preferred Position of term in response
2406 in next scan; actual position after completion of scan.
2407 </entry><entry>1</entry></row>
2409 stepSize</entry><entry>Step Size
2410 </entry><entry>0</entry></row>
2412 scanStatus</entry><entry>An integer indicating the Scan Status
2414 </entry><entry>0</entry></row>
2416 rpnCharset</entry><entry>Character set for RPN terms.
2417 If this is set, ZOOM C will assume that the ZOOM application is
2418 running UTF-8. Terms in RPN queries are then converted to the
2419 rpnCharset. If this is unset, ZOOM C will not assume any encoding
2420 of RPN terms and no conversion is performed.
2421 </entry><entry>none</entry></row>
2426 <sect1 id="zoom.extendedservices">
2427 <title>Extended Services</title>
2429 ZOOM offers an interface to a subset of the Z39.50 extended services
2430 as well as a few privately defined ones:
2435 Z39.50 Item Order (ILL).
2436 See <xref linkend="zoom.item.order"/>.
2441 Record Update. This allows a client to insert, modify or delete
2443 See <xref linkend="zoom.record.update"/>.
2448 Database Create. This a non-standard feature. Allows a client
2449 to create a database.
2450 See <xref linkend="zoom.database.create"/>.
2455 Database Drop. This a non-standard feature. Allows a client
2456 to delete/drop a database.
2457 See <xref linkend="zoom.database.drop"/>.
2462 Commit operation. This a non-standard feature. Allows a client
2463 to commit operations.
2464 See <xref linkend="zoom.commit"/>.
2467 <!-- all the ILL PDU options should go here too -->
2470 To create an extended service operation a <literal>ZOOM_package</literal>
2471 must be created. The operation is a five step operation. The
2472 package is created, package is configured by means of options,
2473 the package is send, result is inspected (by means of options),
2474 the package is destroyed.
2477 ZOOM_package ZOOM_connection_package(ZOOM_connection c,
2478 ZOOM_options options);
2480 const char *ZOOM_package_option_get(ZOOM_package p,
2482 void ZOOM_package_option_set(ZOOM_package p, const char *key,
2484 void ZOOM_package_send(ZOOM_package p, const char *type);
2486 void ZOOM_package_destroy(ZOOM_package p);
2489 The <function>ZOOM_connection_package</function> creates a
2490 package for the connection given using the options specified.
2493 Functions <function>ZOOM_package_option_get</function> and
2494 <function>ZOOM_package_option_set</function> gets and sets
2498 <function>ZOOM_package_send</function> sends
2499 the package the via connection specified in
2500 <function>ZOOM_connection_package</function>.
2501 The <parameter>type</parameter> specifies the actual extended service
2502 package type to be sent.
2504 <table frame="top" id="zoom.extendedservices.options">
2505 <title>Extended Service Common Options</title>
2507 <colspec colwidth="4*" colname="name"></colspec>
2508 <colspec colwidth="7*" colname="description"></colspec>
2509 <colspec colwidth="3*" colname="default"></colspec>
2512 <entry>Option</entry>
2513 <entry>Description</entry>
2514 <entry>Default</entry>
2519 <entry>package-name</entry>
2520 <entry>Extended Service Request package name. Must be specified
2521 as part of a request</entry>
2525 <entry>user-id</entry>
2526 <entry>User ID of Extended Service Package. Is a request option</entry>
2530 <entry>function</entry>
2532 Function of package - one of <literal>create</literal>,
2533 <literal>delete</literal>, <literal>modify</literal>. Is
2536 <entry><literal>create</literal></entry>
2539 <entry>waitAction</entry>
2541 Wait action for package. Possible values:
2542 <literal>wait</literal>, <literal>waitIfPossible</literal>,
2543 <literal>dontWait</literal> or <literal>dontReturnPackage</literal>.
2545 <entry><literal>waitIfPossible</literal></entry>
2548 <entry>targetReference</entry>
2550 Target Reference. This is part of the response as returned
2551 by the server. Read it after a successful operation.
2553 <entry><literal>none</literal></entry>
2558 <sect2 id="zoom.item.order">
2559 <title>Item Order</title>
2561 For Item Order, type must be set to <literal>itemorder</literal> in
2562 <function>ZOOM_package_send</function>.
2565 <table frame="top" id="zoom.item.order.options">
2566 <title>Item Order Options</title>
2568 <colspec colwidth="4*" colname="name"></colspec>
2569 <colspec colwidth="7*" colname="description"></colspec>
2570 <colspec colwidth="3*" colname="default"></colspec>
2573 <entry>Option</entry>
2574 <entry>Description</entry>
2575 <entry>Default</entry>
2580 <entry>contact-name</entry>
2581 <entry>ILL contact name</entry>
2585 <entry>contact-phone</entry>
2586 <entry>ILL contact phone</entry>
2590 <entry>contact-email</entry>
2591 <entry>ILL contact email</entry>
2595 <entry>itemorder-item</entry>
2596 <entry>Position for item (record) requested. An integer</entry>
2603 <sect2 id="zoom.record.update">
2604 <title>Record Update</title>
2606 For Record Update, type must be set to <literal>update</literal> in
2607 <function>ZOOM_package_send</function>.
2609 <table frame="top" id="zoom.record.update.options">
2610 <title>Record Update Options</title>
2612 <colspec colwidth="4*" colname="name"></colspec>
2613 <colspec colwidth="7*" colname="description"></colspec>
2614 <colspec colwidth="3*" colname="default"></colspec>
2617 <entry>Option</entry>
2618 <entry>Description</entry>
2619 <entry>Default</entry>
2624 <entry>action</entry>
2626 The update action. One of
2627 <literal>specialUpdate</literal>,
2628 <literal>recordInsert</literal>,
2629 <literal>recordReplace</literal>,
2630 <literal>recordDelete</literal>,
2631 <literal>elementUpdate</literal>.
2633 <entry><literal>specialUpdate (recordInsert for updateVersion=1 which does not support specialUpdate)</literal></entry>
2636 <entry>recordIdOpaque</entry>
2637 <entry>Opaque Record ID</entry>
2641 <entry>recordIdNumber</entry>
2642 <entry>Record ID number</entry>
2646 <entry>record</entry>
2647 <entry>The record itself</entry>
2651 <entry>recordOpaque</entry>
2652 <entry>Specifies an opaque record which is
2653 encoded as an ASN.1 ANY type with the OID as tiven by option
2654 <literal>syntax</literal> (see below).
2655 Option <literal>recordOpaque</literal> is an alternative
2656 to record - and <literal>record</literal> option (above) is
2657 ignored if recordOpaque is set. This option is only available in
2658 YAZ 3.0.35 and later and is meant to facilitate Updates with
2664 <entry>syntax</entry>
2665 <entry>The record syntax (transfer syntax). Is a string that
2666 is a known record syntax.
2668 <entry>no syntax</entry>
2671 <entry>databaseName</entry>
2672 <entry>Database from connection object</entry>
2673 <entry>Default</entry>
2676 <entry>correlationInfo.note</entry>
2677 <entry>Correlation Info Note (string)</entry>
2681 <entry>correlationInfo.id</entry>
2682 <entry>Correlation Info ID (integer)</entry>
2686 <entry>elementSetName</entry>
2687 <entry>Element Set for Record</entry>
2691 <entry>updateVersion</entry>
2692 <entry>Record Update version which holds one of the values
2693 1, 2 or 3. Each version has a distinct OID:
2695 (<ulink url="&url.z39.50.extupdate1;">first version</ulink>) ,
2697 (second version) and
2698 1.2.840.10003.9.5.1.1
2699 (<ulink url="&url.z39.50.extupdate3;">third and
2700 newest version</ulink>).
2710 <sect2 id="zoom.database.create"><title>Database Create</title>
2712 For Database Create, type must be set to <literal>create</literal> in
2713 <function>ZOOM_package_send</function>.
2716 <table frame="top" id="zoom.database.create.options">
2717 <title>Database Create Options</title>
2719 <colspec colwidth="4*" colname="name"></colspec>
2720 <colspec colwidth="7*" colname="description"></colspec>
2721 <colspec colwidth="3*" colname="default"></colspec>
2724 <entry>Option</entry>
2725 <entry>Description</entry>
2726 <entry>Default</entry>
2731 <entry>databaseName</entry>
2732 <entry>Database from connection object</entry>
2733 <entry>Default</entry>
2739 <sect2 id="zoom.database.drop">
2740 <title>Database Drop</title>
2742 For Database Drop, type must be set to <literal>drop</literal> in
2743 <function>ZOOM_package_send</function>.
2745 <table frame="top" id="zoom.database.drop.options">
2746 <title>Database Drop Options</title>
2748 <colspec colwidth="4*" colname="name"></colspec>
2749 <colspec colwidth="7*" colname="description"></colspec>
2750 <colspec colwidth="3*" colname="default"></colspec>
2753 <entry>Option</entry>
2754 <entry>Description</entry>
2755 <entry>Default</entry>
2760 <entry>databaseName</entry>
2761 <entry>Database from connection object</entry>
2762 <entry>Default</entry>
2768 <sect2 id="zoom.commit">
2769 <title>Commit Operation</title>
2771 For Commit, type must be set to <literal>commit</literal> in
2772 <function>ZOOM_package_send</function>.
2775 <sect2 id="zoom.extended.services.behavior">
2776 <title>Protocol behavior</title>
2778 All the extended services are Z39.50-only.
2782 The database create, drop and commit services are privately defined
2784 Refer to <filename>esadmin.asn</filename> in YAZ for the ASN.1
2790 <sect1 id="zoom.options">
2791 <title>Options</title>
2793 Most &zoom; objects provide a way to specify options to change behavior.
2794 From an implementation point of view a set of options is just like
2795 an associative array / hash.
2798 ZOOM_options ZOOM_options_create(void);
2800 ZOOM_options ZOOM_options_create_with_parent(ZOOM_options parent);
2802 void ZOOM_options_destroy(ZOOM_options opt);
2805 const char *ZOOM_options_get(ZOOM_options opt, const char *name);
2807 void ZOOM_options_set(ZOOM_options opt, const char *name,
2811 typedef const char *(*ZOOM_options_callback)
2812 (void *handle, const char *name);
2814 ZOOM_options_callback
2815 ZOOM_options_set_callback(ZOOM_options opt,
2816 ZOOM_options_callback c,
2820 <sect1 id="zoom.queryconversions">
2821 <title>Query conversions</title>
2823 int ZOOM_query_cql2rpn(ZOOM_query s, const char *cql_str,
2824 ZOOM_connection conn);
2826 int ZOOM_query_ccl2rpn(ZOOM_query s, const char *ccl_str,
2828 int *ccl_error, const char **error_string,
2832 <function>ZOOM_query_cql2rpn</function> translates the CQL string,
2833 client-side, into RPN which may be passed to the server.
2834 This is useful for server's that don't themselves
2835 support CQL, for which <function>ZOOM_query_cql</function> is useless.
2836 `conn' is used only as a place to stash diagnostics if compilation
2837 fails; if this information is not needed, a null pointer may be used.
2838 The CQL conversion is driven by option <literal>cqlfile</literal> from
2839 connection conn. This specifies a conversion file (eg pqf.properties)
2840 which <emphasis>must</emphasis> be present.
2843 <function>ZOOM_query_ccl2rpn</function> translates the CCL string,
2844 client-side, into RPN which may be passed to the server.
2845 The conversion is driven by the specification given by
2846 <literal>config</literal>. Upon completion 0 is returned on success; -1
2847 is returned on on failure. Om failure <literal>error_string</literal> and
2848 <literal>error_pos</literal> holds error message and position of
2849 first error in original CCL string.
2852 <sect1 id="zoom.events"><title>Events</title>
2854 If you're developing non-blocking applications, you have to deal
2858 int ZOOM_event(int no, ZOOM_connection *cs);
2861 The <function>ZOOM_event</function> executes pending events for
2862 a number of connections. Supply the number of connections in
2863 <literal>no</literal> and an array of connections in
2864 <literal>cs</literal> (<literal>cs[0] ... cs[no-1]</literal>).
2865 A pending event could be a sending a search, receiving a response,
2867 When an event has occurred for one of the connections, this function
2868 returns a positive integer <literal>n</literal> denoting that an event
2869 occurred for connection <literal>cs[n-1]</literal>.
2870 When no events are pending for the connections, a value of zero is
2872 To ensure that all outstanding requests are performed call this function
2873 repeatedly until zero is returned.
2876 If <function>ZOOM_event</function> returns and returns non-zero, the
2877 last event that occurred can be expected.
2880 int ZOOM_connection_last_event(ZOOM_connection cs);
2883 <function>ZOOM_connection_last_event</function> returns an event type
2884 (integer) for the last event.
2887 <table frame="top" id="zoom.event.ids">
2888 <title>ZOOM Event IDs</title>
2890 <colspec colwidth="4*" colname="name"></colspec>
2891 <colspec colwidth="7*" colname="description"></colspec>
2894 <entry>Event</entry>
2895 <entry>Description</entry>
2900 <entry>ZOOM_EVENT_NONE</entry>
2901 <entry>No event has occurred</entry>
2904 <entry>ZOOM_EVENT_CONNECT</entry>
2905 <entry>TCP/IP connect has initiated</entry>
2908 <entry>ZOOM_EVENT_SEND_DATA</entry>
2909 <entry>Data has been transmitted (sending)</entry>
2912 <entry>ZOOM_EVENT_RECV_DATA</entry>
2913 <entry>Data has been received)</entry>
2916 <entry>ZOOM_EVENT_TIMEOUT</entry>
2917 <entry>Timeout</entry>
2920 <entry>ZOOM_EVENT_UNKNOWN</entry>
2921 <entry>Unknown event</entry>
2924 <entry>ZOOM_EVENT_SEND_APDU</entry>
2925 <entry>An APDU has been transmitted (sending)</entry>
2928 <entry>ZOOM_EVENT_RECV_APDU</entry>
2929 <entry>An APDU has been received</entry>
2932 <entry>ZOOM_EVENT_RECV_RECORD</entry>
2933 <entry>A result-set record has been received</entry>
2936 <entry>ZOOM_EVENT_RECV_SEARCH</entry>
2937 <entry>A search result been received</entry>
2944 <chapter id="server">
2945 <title>Generic server</title>
2946 <sect1 id="server.introduction"><title>Introduction</title>
2948 If you aren't into documentation, a good way to learn how the
2949 back end interface works is to look at the <filename>backend.h</filename>
2950 file. Then, look at the small dummy-server in
2951 <filename>ztest/ztest.c</filename>. The <filename>backend.h</filename>
2952 file also makes a good reference, once you've chewed your way through
2953 the prose of this file.
2956 If you have a database system that you would like to make available by
2957 means of Z39.50 or SRU, &yaz; basically offers your two options. You
2958 can use the APIs provided by the &asn;, &odr;, and &comstack;
2960 create and decode PDUs, and exchange them with a client.
2961 Using this low-level interface gives you access to all fields and
2962 options of the protocol, and you can construct your server as close
2963 to your existing database as you like.
2964 It is also a fairly involved process, requiring
2965 you to set up an event-handling mechanism, protocol state machine,
2966 etc. To simplify server implementation, we have implemented a compact
2967 and simple, but reasonably full-functioned server-frontend that will
2968 handle most of the protocol mechanics, while leaving you to
2969 concentrate on your database interface.
2973 The backend interface was designed in anticipation of a specific
2974 integration task, while still attempting to achieve some degree of
2975 generality. We realize fully that there are points where the
2976 interface can be improved significantly. If you have specific
2977 functions or parameters that you think could be useful, send us a
2978 mail (or better, sign on to the mailing list referred to in the
2979 top-level README file). We will try to fit good suggestions into future
2980 releases, to the extent that it can be done without requiring
2981 too many structural changes in existing applications.
2986 The &yaz; server does not support XCQL.
2990 <sect1 id="server.frontend">
2991 <title>The Database Frontend</title>
2993 We refer to this software as a generic database frontend. Your
2994 database system is the <emphasis>backend database</emphasis>, and the
2995 interface between the two is called the <emphasis>backend API</emphasis>.
2996 The backend API consists of a small number of function handlers and
2997 structure definitions. You are required to provide the
2998 <function>main()</function> routine for the server (which can be
2999 quite simple), as well as a set of handlers to match each of the
3001 The interface functions that you write can use any mechanism you like
3002 to communicate with your database system: You might link the whole
3003 thing together with your database application and access it by
3004 function calls; you might use IPC to talk to a database server
3005 somewhere; or you might link with third-party software that handles
3006 the communication for you (like a commercial database client library).
3007 At any rate, the handlers will perform the tasks of:
3020 Scanning the database index (optional - if you wish to implement SCAN).
3023 Extended Services (optional).
3026 Result-Set Delete (optional).
3029 Result-Set Sort (optional).
3032 Return Explain for SRU (optional).
3036 (more functions will be added in time to support as much of
3037 Z39.50-1995 as possible).
3040 <sect1 id="server.backend">
3041 <title>The Backend API</title>
3043 The header file that you need to use the interface are in the
3044 <filename>include/yaz</filename> directory. It's called
3045 <filename>backend.h</filename>. It will include other files from
3046 the <filename>include/yaz</filename> directory, so you'll
3047 probably want to use the -I option of your compiler to tell it
3048 where to find the files. When you run
3049 <literal>make</literal> in the top-level &yaz; directory,
3050 everything you need to create your server is to link with the
3051 <filename>lib/libyaz.la</filename> library.
3054 <sect1 id="server.main">
3055 <title>Your main() Routine</title>
3057 As mentioned, your <function>main()</function> routine can be quite brief.
3058 If you want to initialize global parameters, or read global configuration
3059 tables, this is the place to do it. At the end of the routine, you should
3063 int statserv_main(int argc, char **argv,
3064 bend_initresult *(*bend_init)(bend_initrequest *r),
3065 void (*bend_close)(void *handle));
3068 The third and fourth arguments are pointers to handlers. Handler
3069 <function>bend_init</function> is called whenever the server receives
3070 an Initialize Request, so it serves as a Z39.50 session initializer. The
3071 <function>bend_close</function> handler is called when the session is
3075 <function>statserv_main</function> will establish listening sockets
3076 according to the parameters given. When connection requests are received,
3077 the event handler will typically <function>fork()</function> and
3078 create a sub-process to handle a new connection.
3079 Alternatively the server may be setup to create threads for each
3081 If you do use global variables and forking, you should be aware, then,
3082 that these cannot be shared between associations, unless you explicitly
3083 disable forking by command line parameters.
3086 The server provides a mechanism for controlling some of its behavior
3087 without using command-line options. The function
3090 statserv_options_block *statserv_getcontrol(void);
3093 will return a pointer to a <literal>struct statserv_options_block</literal>
3094 describing the current default settings of the server. The structure
3095 contains these elements:
3098 <term><literal>int dynamic</literal></term>
3100 A boolean value, which determines whether the server
3101 will fork on each incoming request (TRUE), or not (FALSE). Default is
3102 TRUE. This flag is only read by UNIX-based servers (WIN32 based servers
3107 <term><literal>int threads</literal></term>
3109 A boolean value, which determines whether the server
3110 will create a thread on each incoming request (TRUE), or not (FALSE).
3111 Default is FALSE. This flag is only read by UNIX-based servers
3112 that offer POSIX Threads support.
3113 WIN32-based servers always operate in threaded mode.
3117 <term><literal>int inetd</literal></term>
3119 A boolean value, which determines whether the server
3120 will operates under a UNIX INET daemon (inetd). Default is FALSE.
3124 <term><literal>char logfile[ODR_MAXNAME+1]</literal></term>
3125 <listitem><para>File for diagnostic output ("": stderr).
3129 <term><literal>char apdufile[ODR_MAXNAME+1]</literal></term>
3131 Name of file for logging incoming and outgoing APDUs
3132 ("": don't log APDUs, "-":
3133 <literal>stderr</literal>).
3137 <term><literal>char default_listen[1024]</literal></term>
3138 <listitem><para>Same form as the command-line specification of
3139 listener address. "": no default listener address.
3140 Default is to listen at "tcp:@:9999". You can only
3141 specify one default listener address in this fashion.
3145 <term><literal>enum oid_proto default_proto;</literal></term>
3146 <listitem><para>Either <literal>PROTO_Z3950</literal> or
3147 <literal>PROTO_SR</literal>.
3148 Default is <literal>PROTO_Z39_50</literal>.
3152 <term><literal>int idle_timeout;</literal></term>
3153 <listitem><para>Maximum session idle-time, in minutes. Zero indicates
3154 no (infinite) timeout. Default is 15 minutes.
3158 <term><literal>int maxrecordsize;</literal></term>
3159 <listitem><para>Maximum permissible record (message) size. Default
3160 is 64 MB. This amount of memory will only be allocated if a
3161 client requests a very large amount of records in one operation
3163 Set it to a lower number if you are worried about resource
3164 consumption on your host system.
3168 <term><literal>char configname[ODR_MAXNAME+1]</literal></term>
3169 <listitem><para>Passed to the backend when a new connection is received.
3173 <term><literal>char setuid[ODR_MAXNAME+1]</literal></term>
3174 <listitem><para>Set user id to the user specified, after binding
3175 the listener addresses.
3180 <literal>void (*bend_start)(struct statserv_options_block *p)</literal>
3182 <listitem><para>Pointer to function which is called after the
3183 command line options have been parsed - but before the server
3185 For forked UNIX servers this handler is called in the mother
3186 process; for threaded servers this handler is called in the
3188 The default value of this pointer is NULL in which case it
3189 isn't invoked by the frontend server.
3190 When the server operates as an NT service this handler is called
3191 whenever the service is started.
3196 <literal>void (*bend_stop)(struct statserv_options_block *p)</literal>
3198 <listitem><para>Pointer to function which is called whenever the server
3199 has stopped listening for incoming connections. This function pointer
3200 has a default value of NULL in which case it isn't called.
3201 When the server operates as an NT service this handler is called
3202 whenever the service is stopped.
3206 <term><literal>void *handle</literal></term>
3207 <listitem><para>User defined pointer (default value NULL).
3208 This is a per-server handle that can be used to specify "user-data".
3209 Do not confuse this with the session-handle as returned by bend_init.
3215 The pointer returned by <literal>statserv_getcontrol</literal> points to
3216 a static area. You are allowed to change the contents of the structure,
3217 but the changes will not take effect before you call
3220 void statserv_setcontrol(statserv_options_block *block);
3224 that you should generally update this structure before calling
3225 <function>statserv_main()</function>.
3229 <sect1 id="server.backendfunctions">
3230 <title>The Backend Functions</title>
3232 For each service of the protocol, the backend interface declares one or
3233 two functions. You are required to provide implementations of the
3234 functions representing the services that you wish to implement.
3236 <sect2 id="server.init">
3239 bend_initresult (*bend_init)(bend_initrequest *r);
3242 This handler is called once for each new connection request, after
3243 a new process/thread has been created, and an Initialize Request has
3244 been received from the client. The pointer to the
3245 <function>bend_init</function> handler is passed in the call to
3246 <function>statserv_start</function>.
3249 This handler is also called when operating in SRU mode - when
3250 a connection has been made (even though SRU does not offer
3254 Unlike previous versions of YAZ, the <function>bend_init</function> also
3255 serves as a handler that defines the Z39.50 services that the backend
3256 wish to support. Pointers to <emphasis>all</emphasis> service handlers,
3257 including search - and fetch must be specified here in this handler.
3260 The request - and result structures are defined as
3263 typedef struct bend_initrequest
3265 /** \brief user/name/password to be read */
3266 Z_IdAuthentication *auth;
3267 /** \brief encoding stream (for results) */
3269 /** \brief printing stream */
3271 /** \brief decoding stream (use stream for results) */
3273 /** \brief reference ID */
3274 Z_ReferenceId *referenceId;
3275 /** \brief peer address of client */
3278 /** \brief character set and language negotiation
3280 see include/yaz/z-charneg.h
3282 Z_CharSetandLanguageNegotiation *charneg_request;
3284 /** \brief character negotiation response */
3285 Z_External *charneg_response;
3287 /** \brief character set (encoding) for query terms
3289 This is NULL by default. It should be set to the native character
3290 set that the backend assumes for query terms */
3291 char *query_charset;
3293 /** \brief whehter query_charset also applies to recors
3295 Is 0 (No) by default. Set to 1 (yes) if records is in the same
3296 character set as queries. If in doubt, use 0 (No).
3298 int records_in_same_charset;
3300 char *implementation_id;
3301 char *implementation_name;
3302 char *implementation_version;
3304 /** \brief Z39.50 sort handler */
3305 int (*bend_sort)(void *handle, bend_sort_rr *rr);
3306 /** \brief SRU/Z39.50 search handler */
3307 int (*bend_search)(void *handle, bend_search_rr *rr);
3308 /** \brief SRU/Z39.50 fetch handler */
3309 int (*bend_fetch)(void *handle, bend_fetch_rr *rr);
3310 /** \brief SRU/Z39.50 present handler */
3311 int (*bend_present)(void *handle, bend_present_rr *rr);
3312 /** \brief Z39.50 extended services handler */
3313 int (*bend_esrequest) (void *handle, bend_esrequest_rr *rr);
3314 /** \brief Z39.50 delete result set handler */
3315 int (*bend_delete)(void *handle, bend_delete_rr *rr);
3316 /** \brief Z39.50 scan handler */
3317 int (*bend_scan)(void *handle, bend_scan_rr *rr);
3318 /** \brief Z39.50 segment facility handler */
3319 int (*bend_segment)(void *handle, bend_segment_rr *rr);
3320 /** \brief SRU explain handler */
3321 int (*bend_explain)(void *handle, bend_explain_rr *rr);
3322 /** \brief SRU scan handler */
3323 int (*bend_srw_scan)(void *handle, bend_scan_rr *rr);
3324 /** \brief SRU record update handler */
3325 int (*bend_srw_update)(void *handle, bend_update_rr *rr);
3327 /** \brief whether named result sets are supported (0=disable, 1=enable) */
3328 int named_result_sets;
3331 typedef struct bend_initresult
3333 int errcode; /* 0==OK */
3334 char *errstring; /* system error string or NULL */
3335 void *handle; /* private handle to the backend module */
3339 In general, the server frontend expects that the
3340 <literal>bend_*result</literal> pointer that you return is valid at
3341 least until the next call to a <literal>bend_* function</literal>.
3342 This applies to all of the functions described herein. The parameter
3343 structure passed to you in the call belongs to the server frontend, and
3344 you should not make assumptions about its contents after the current
3345 function call has completed. In other words, if you want to retain any
3346 of the contents of a request structure, you should copy them.
3349 The <literal>errcode</literal> should be zero if the initialization of
3350 the backend went well. Any other value will be interpreted as an error.
3351 The <literal>errstring</literal> isn't used in the current version, but
3352 one option would be to stick it in the initResponse as a VisibleString.
3353 The <literal>handle</literal> is the most important parameter. It should
3354 be set to some value that uniquely identifies the current session to
3355 the backend implementation. It is used by the frontend server in any
3356 future calls to a backend function.
3357 The typical use is to set it to point to a dynamically allocated state
3358 structure that is private to your backend module.
3361 The <literal>auth</literal> member holds the authentication information
3362 part of the Z39.50 Initialize Request. Interpret this if your serves
3363 requires authentication.
3366 The members <literal>peer_name</literal>,
3367 <literal>implementation_id</literal>,
3368 <literal>implementation_name</literal> and
3369 <literal>implementation_version</literal> holds
3370 DNS of client, ID of implementor, name
3371 of client (Z39.50) implementation - and version.
3374 The <literal>bend_</literal> - members are set to NULL when
3375 <function>bend_init</function> is called. Modify the pointers by
3376 setting them to point to backend functions.
3379 <sect2 id="server.search.retrieve">
3380 <title>Search and Retrieve</title>
3382 We now describe the handlers that are required to support search -
3383 and retrieve. You must support two functions - one for search - and one
3384 for fetch (retrieval of one record). If desirable you can provide a
3385 third handler which is called when a present request is received which
3386 allows you to optimize retrieval of multiple-records.
3389 int (*bend_search) (void *handle, bend_search_rr *rr);
3392 char *setname; /* name to give to this set */
3393 int replace_set; /* replace set, if it already exists */
3394 int num_bases; /* number of databases in list */
3395 char **basenames; /* databases to search */
3396 Z_ReferenceId *referenceId;/* reference ID */
3397 Z_Query *query; /* query structure */
3398 ODR stream; /* encode stream */
3399 ODR decode; /* decode stream */
3400 ODR print; /* print stream */
3402 bend_request request;
3403 bend_association association;
3405 int hits; /* number of hits */
3406 int errcode; /* 0==OK */
3407 char *errstring; /* system error string or NULL */
3408 Z_OtherInformation *search_info; /* additional search info */
3409 char *srw_sortKeys; /* holds SRU/SRW sortKeys info */
3410 char *srw_setname; /* holds SRU/SRW generated resultsetID */
3411 int *srw_setnameIdleTime; /* holds SRU/SRW life-time */
3412 int estimated_hit_count; /* if hit count is estimated */
3413 int partial_resultset; /* if result set is partial */
3417 The <function>bend_search</function> handler is a fairly close
3418 approximation of a protocol Z39.50 Search Request - and Response PDUs
3419 The <literal>setname</literal> is the resultSetName from the protocol.
3420 You are required to establish a mapping between the set name and whatever
3421 your backend database likes to use.
3422 Similarly, the <literal>replace_set</literal> is a boolean value
3423 corresponding to the resultSetIndicator field in the protocol.
3424 <literal>num_bases/basenames</literal> is a length of/array of character
3425 pointers to the database names provided by the client.
3426 The <literal>query</literal> is the full query structure as defined in
3427 the protocol ASN.1 specification.
3428 It can be either of the possible query types, and it's up to you to
3429 determine if you can handle the provided query type.
3430 Rather than reproduce the C interface here, we'll refer you to the
3431 structure definitions in the file
3432 <filename>include/yaz/z-core.h</filename>. If you want to look at the
3433 attributeSetId OID of the RPN query, you can either match it against
3434 your own internal tables, or you can use the <link linkend="tools.oid">
3438 The structure contains a number of hits, and an
3439 <literal>errcode/errstring</literal> pair. If an error occurs
3440 during the search, or if you're unhappy with the request, you should
3441 set the errcode to a value from the BIB-1 diagnostic set. The value
3442 will then be returned to the user in a nonsurrogate diagnostic record
3443 in the response. The <literal>errstring</literal>, if provided, will
3444 go in the addinfo field. Look at the protocol definition for the
3445 defined error codes, and the suggested uses of the addinfo field.
3448 The <function>bend_search</function> handler is also called when
3449 the frontend server receives a SRU SearchRetrieveRequest.
3450 For SRU, a CQL query is usually provided by the client.
3451 The CQL query is available as part of <literal>Z_Query</literal>
3452 structure (note that CQL is now part of Z39.50 via an external).
3453 To support CQL in existing implementations that only do Type-1,
3454 we refer to the CQL-to-PQF tool described
3455 <link linkend="cql.to.pqf">here</link>.
3458 To maintain backwards compatibility, the frontend server
3459 of yaz always assume that error codes are BIB-1 diagnostics.
3460 For SRU operation, a Bib-1 diagnostic code is mapped to
3464 int (*bend_fetch) (void *handle, bend_fetch_rr *rr);
3466 typedef struct bend_fetch_rr {
3467 char *setname; /* set name */
3468 int number; /* record number */
3469 Z_ReferenceId *referenceId;/* reference ID */
3470 Odr_oid *request_format; /* format, transfer syntax (OID) */
3471 Z_RecordComposition *comp; /* Formatting instructions */
3472 ODR stream; /* encoding stream - memory source if req */
3473 ODR print; /* printing stream */
3475 char *basename; /* name of database that provided record */
3476 int len; /* length of record or -1 if structured */
3477 char *record; /* record */
3478 int last_in_set; /* is it? */
3479 Odr_oid *output_format; /* response format/syntax (OID) */
3480 int errcode; /* 0==success */
3481 char *errstring; /* system error string or NULL */
3482 int surrogate_flag; /* surrogate diagnostic */
3483 char *schema; /* string record schema input/output */
3487 The frontend server calls the <function>bend_fetch</function> handler
3488 when it needs database records to fulfill a Z39.50 Search Request, a
3489 Z39.50 Present Request or a SRU SearchRetrieveRequest.
3490 The <literal>setname</literal> is simply the name of the result set
3491 that holds the reference to the desired record.
3492 The <literal>number</literal> is the offset into the set (with 1
3493 being the first record in the set). The <literal>format</literal> field
3494 is the record format requested by the client (See
3495 <xref linkend="tools.oid"/>).
3496 A value of NULL for <literal>format</literal> indicates that the
3497 client did not request a specific format.
3498 The <literal>stream</literal> argument is an &odr; stream which
3499 should be used for allocating space for structured data records.
3500 The stream will be reset when all records have been assembled, and
3501 the response package has been transmitted.
3502 For unstructured data, the backend is responsible for maintaining a
3503 static or dynamic buffer for the record between calls.
3506 If a SRU SearchRetrieveRequest is received by the frontend server,
3507 the <literal>referenceId</literal> is NULL and the
3508 <literal>format</literal> (transfer syntax) is the OID for XML.
3509 The schema for SRU is stored in both the
3510 <literal>Z_RecordComposition</literal>
3511 structure and <literal>schema</literal> (simple string).
3514 In the structure, the <literal>basename</literal> is the name of the
3515 database that holds the
3516 record. <literal>len</literal> is the length of the record returned, in
3517 bytes, and <literal>record</literal> is a pointer to the record.
3518 <literal>last_in_set</literal> should be nonzero only if the record
3519 returned is the last one in the given result set.
3520 <literal>errcode</literal> and <literal>errstring</literal>, if
3521 given, will be interpreted as a global error pertaining to the
3522 set, and will be returned in a non-surrogate-diagnostic.
3523 If you wish to return the error as a surrogate-diagnostic
3524 (local error) you can do this by setting
3525 <literal>surrogate_flag</literal> to 1 also.
3528 If the <literal>len</literal> field has the value -1, then
3529 <literal>record</literal> is assumed to point to a constructed data
3530 type. The <literal>format</literal> field will be used to determine
3531 which encoder should be used to serialize the data.
3535 If your backend generates structured records, it should use
3536 <function>odr_malloc()</function> on the provided stream for allocating
3537 data: This allows the frontend server to keep track of the record sizes.
3541 The <literal>format</literal> field is mapped to an object identifier
3542 in the direct reference of the resulting EXTERNAL representation
3547 The current version of &yaz; only supports the direct reference mode.
3551 int (*bend_present) (void *handle, bend_present_rr *rr);
3554 char *setname; /* set name */
3556 int number; /* record number */
3557 Odr_oid *format; /* format, transfer syntax (OID) */
3558 Z_ReferenceId *referenceId;/* reference ID */
3559 Z_RecordComposition *comp; /* Formatting instructions */
3560 ODR stream; /* encoding stream - memory source if required */
3561 ODR print; /* printing stream */
3562 bend_request request;
3563 bend_association association;
3565 int hits; /* number of hits */
3566 int errcode; /* 0==OK */
3567 char *errstring; /* system error string or NULL */
3571 The <function>bend_present</function> handler is called when
3572 the server receives a Z39.50 Present Request.
3573 The <literal>setname</literal>,
3574 <literal>start</literal> and <literal>number</literal> is the
3575 name of the result set - start position - and number of records to
3576 be retrieved respectively. <literal>format</literal> and
3577 <literal>comp</literal> is the preferred transfer syntax and element
3578 specifications of the present request.
3581 Note that this is handler serves as a supplement for
3582 <function>bend_fetch</function> and need not to be defined in order to
3583 support search - and retrieve.
3586 <sect2 id="server.delete">
3587 <title>Delete</title>
3589 For back-ends that supports delete of a result set only one handler
3593 int (*bend_delete)(void *handle, bend_delete_rr *rr);
3595 typedef struct bend_delete_rr {
3599 Z_ReferenceId *referenceId;
3600 int delete_status; /* status for the whole operation */
3601 int *statuses; /* status each set - indexed as setnames */
3608 The delete set function definition is rather primitive, mostly because
3609 we have had no practical need for it as of yet. If someone wants
3610 to provide a full delete service, we'd be happy to add the
3611 extra parameters that are required. Are there clients out there
3612 that will actually delete sets they no longer need?
3616 <sect2 id="server.scan">
3619 For servers that wish to offer the scan service one handler
3623 int (*bend_scan)(void *handle, bend_scan_rr *rr);
3626 BEND_SCAN_SUCCESS, /* ok */
3627 BEND_SCAN_PARTIAL /* not all entries could be found */
3630 typedef struct bend_scan_rr {
3631 int num_bases; /* number of elements in databaselist */
3632 char **basenames; /* databases to search */
3633 Odr_oid *attributeset;
3634 Z_ReferenceId *referenceId; /* reference ID */
3635 Z_AttributesPlusTerm *term;
3636 ODR stream; /* encoding stream - memory source if required */
3637 ODR print; /* printing stream */
3639 int *step_size; /* step size */
3640 int term_position; /* desired index of term in result list/returned */
3641 int num_entries; /* number of entries requested/returned */
3643 /* scan term entries. The called handler does not have
3644 to allocate this. Size of entries is num_entries (see above) */
3645 struct scan_entry *entries;
3646 bend_scan_status status;
3649 char *scanClause; /* CQL scan clause */
3650 char *setname; /* Scan in result set (NULL if omitted) */
3654 This backend server handles both Z39.50 scan
3655 and SRU scan. In order for a handler to distinguish between SRU (CQL) scan
3656 Z39.50 Scan , it must check for a non-NULL value of
3657 <literal>scanClause</literal>.
3661 if designed today, it would be a choice using a union or similar,
3662 but that would break binary compatibility with existing servers.
3667 <sect1 id="server.invocation">
3668 <title>Application Invocation</title>
3670 The finished application has the following
3671 invocation syntax (by way of <function>statserv_main()</function>):
3679 A listener specification consists of a transport mode followed by a
3680 colon (:) followed by a listener address. The transport mode is
3681 either <literal>tcp</literal>, <literal>unix:</literal> or
3682 <literal>ssl</literal>.
3685 For TCP and SSL, an address has the form
3688 hostname | IP-number [: portnumber]
3691 The port number defaults to 210 (standard Z39.50 port).
3694 For UNIX, the address is the filename of socket.
3697 For TCP/IP and SSL, the special hostnames <literal>@</literal>,
3698 maps to <literal>IN6ADDR_ANY_INIT</literal> with
3699 IPV4 binding as well (bindv6only=0),
3700 The special hostname <literal>@4</literal> binds to
3701 <literal>INADDR_ANY</literal> (IPV4 only listener).
3702 The special hostname <literal>@6</literal> binds to
3703 <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
3705 <example id="server.example.running.unix">
3706 <title>Running the GFS on Unix</title>
3708 Assuming the server application <replaceable>appname</replaceable> is
3709 started as root, the following will make it listen on port 210.
3710 The server will change identity to <literal>nobody</literal>
3711 and write its log to <filename>/var/log/app.log</filename>.
3713 application -l /var/log/app.log -u nobody tcp:@:210
3717 The server will accept Z39.50 requests and offer SRU service on port 210.
3720 <example id="server.example.apache.sru">
3721 <title>Setting up Apache as SRU Frontend</title>
3723 If you use <ulink url="&url.apache;">Apache</ulink>
3724 as your public web server and want to offer HTTP port 80
3725 access to the YAZ server on 210, you can use the
3726 <ulink url="&url.apache.directive.proxypass;">
3727 <literal>ProxyPass</literal></ulink>
3729 If you have virtual host
3730 <literal>srw.mydomain</literal> you can use the following directives
3731 in Apache's httpd.conf:
3734 ErrorLog /home/srw/logs/error_log
3735 TransferLog /home/srw/logs/access_log
3736 ProxyPass / http://srw.mydomain:210/
3741 The above for the Apache 1.3 series.
3744 <example id="server.example.local.access">
3745 <title>Running a server with local access only</title>
3747 Servers that is only being accessed from the local host should listen
3748 on UNIX file socket rather than a Internet socket. To listen on
3749 <filename>/tmp/mysocket</filename> start the server as follows:
3751 application unix:/tmp/mysocket
3756 <sect1 id="server.vhosts">
3757 <title>GFS Configuration and Virtual Hosts</title>
3762 <title>The Z39.50 ASN.1 Module</title>
3763 <sect1 id="asn.introduction">
3764 <title>Introduction</title>
3766 The &asn; module provides you with a set of C struct definitions for the
3767 various PDUs of the Z39.50 protocol, as well as for the complex types
3768 appearing within the PDUs. For the primitive data types, the C
3769 representation often takes the form of an ordinary C language type,
3770 such as <literal>Odr_int</literal> which is equivalent to an integral
3771 C integer. For ASN.1 constructs that have no direct
3772 representation in C, such as general octet strings and bit strings,
3773 the &odr; module (see section <link linkend="odr">The ODR Module</link>)
3774 provides auxiliary definitions.
3777 The &asn; module is located in sub directory <filename>z39.50</filename>.
3778 There you'll find C files that implements encoders and decoders for the
3779 Z39.50 types. You'll also find the protocol definitions:
3780 <filename>z3950v3.asn</filename>, <filename>esupdate.asn</filename>,
3784 <sect1 id="asn.preparing">
3785 <title>Preparing PDUs</title>
3787 A structure representing a complex ASN.1 type doesn't in itself contain the
3788 members of that type. Instead, the structure contains
3789 <emphasis>pointers</emphasis> to the members of the type.
3790 This is necessary, in part, to allow a mechanism for specifying which
3791 of the optional structure (SEQUENCE) members are present, and which
3792 are not. It follows that you will need to somehow provide space for
3793 the individual members of the structure, and set the pointers to
3794 refer to the members.
3797 The conversion routines don't care how you allocate and maintain your
3798 C structures - they just follow the pointers that you provide.
3799 Depending on the complexity of your application, and your personal
3800 taste, there are at least three different approaches that you may take
3801 when you allocate the structures.
3804 You can use static or automatic local variables in the function that
3805 prepares the PDU. This is a simple approach, and it provides the most
3806 efficient form of memory management. While it works well for flat
3807 PDUs like the InitReqest, it will generally not be sufficient for say,
3808 the generation of an arbitrarily complex RPN query structure.
3811 You can individually create the structure and its members using the
3812 <function>malloc(2)</function> function. If you want to ensure that
3813 the data is freed when it is no longer needed, you will have to
3814 define a function that individually releases each member of a
3815 structure before freeing the structure itself.
3818 You can use the <function>odr_malloc()</function> function (see
3819 <xref linkend="odr.use"/> for details). When you use
3820 <function>odr_malloc()</function>, you can release all of the
3821 allocated data in a single operation, independent of any pointers and
3822 relations between the data. <function>odr_malloc()</function> is based on a
3823 "nibble-memory"
3824 scheme, in which large portions of memory are allocated, and then
3825 gradually handed out with each call to <function>odr_malloc()</function>.
3826 The next time you call <function>odr_reset()</function>, all of the
3827 memory allocated since the last call is recycled for future use (actually,
3828 it is placed on a free-list).
3831 You can combine all of the methods described here. This will often be
3832 the most practical approach. For instance, you might use
3833 <function>odr_malloc()</function> to allocate an entire structure and
3834 some of its elements, while you leave other elements pointing to global
3835 or per-session default variables.
3838 The &asn; module provides an important aid in creating new PDUs. For
3839 each of the PDU types (say, <function>Z_InitRequest</function>), a
3840 function is provided that allocates and initializes an instance of
3841 that PDU type for you. In the case of the InitRequest, the function is
3842 simply named <function>zget_InitRequest()</function>, and it sets up
3843 reasonable default value for all of the mandatory members. The optional
3844 members are generally initialized to null pointers. This last aspect
3845 is very important: it ensures that if the PDU definitions are
3846 extended after you finish your implementation (to accommodate
3847 new versions of the protocol, say), you won't get into trouble with
3848 uninitialized pointers in your structures. The functions use
3849 <function>odr_malloc()</function> to
3850 allocate the PDUs and its members, so you can free everything again with a
3851 single call to <function>odr_reset()</function>. We strongly recommend
3852 that you use the <literal>zget_*</literal>
3853 functions whenever you are preparing a PDU (in a C++ API, the
3854 <literal>zget_</literal>
3855 functions would probably be promoted to constructors for the
3859 The prototype for the individual PDU types generally look like this:
3862 Z_<type> *zget_<type>(ODR o);
3868 Z_InitRequest *zget_InitRequest(ODR o);
3871 The &odr; handle should generally be your encoding stream, but it
3875 As well as the individual PDU functions, a function
3876 <function>zget_APDU()</function> is provided, which allocates
3877 a top-level Z-APDU of the type requested:
3880 Z_APDU *zget_APDU(ODR o, int which);
3883 The <varname>which</varname> parameter is (of course) the discriminator
3884 belonging to the <varname>Z_APDU</varname> <literal>CHOICE</literal> type.
3885 All of the interface described here is provided by the &asn; module, and
3886 you access it through the <filename>proto.h</filename> header file.
3889 <sect1 id="asn.external">
3890 <title>EXTERNAL Data</title>
3892 In order to achieve extensibility and adaptability to different
3893 application domains, the new version of the protocol defines many
3894 structures outside of the main ASN.1 specification, referencing them
3895 through ASN.1 EXTERNAL constructs. To simplify the construction and
3896 access to the externally referenced data, the &asn; module defines a
3897 specialized version of the EXTERNAL construct, called
3898 <literal>Z_External</literal>.It is defined thus:
3901 typedef struct Z_External
3903 Odr_oid *direct_reference;
3904 int *indirect_reference;
3909 Z_External_single = 0,
3911 Z_External_arbitrary,
3913 /* Specific types */
3915 Z_External_explainRecord,
3916 Z_External_resourceReport1,
3917 Z_External_resourceReport2
3925 Odr_any *single_ASN1_type;
3926 Odr_oct *octet_aligned;
3927 Odr_bitmask *arbitrary;
3929 /* Specific types */
3931 Z_ExplainRecord *explainRecord;
3932 Z_ResourceReport1 *resourceReport1;
3933 Z_ResourceReport2 *resourceReport2;
3941 When decoding, the &asn; module will attempt to determine which
3942 syntax describes the data by looking at the reference fields
3943 (currently only the direct-reference). For ASN.1 structured data, you
3944 need only consult the <literal>which</literal> field to determine the
3945 type of data. You can the access the data directly through the union.
3946 When constructing data for encoding, you set the union pointer to point
3947 to the data, and set the <literal>which</literal> field accordingly.
3948 Remember also to set the direct (or indirect) reference to the correct
3949 OID for the data type.
3950 For non-ASN.1 data such as MARC records, use the
3951 <literal>octet_aligned</literal> arm of the union.
3954 Some servers return ASN.1 structured data values (eg. database
3955 records) as BER-encoded records placed in the
3956 <literal>octet-aligned</literal> branch of the EXTERNAL CHOICE.
3957 The ASN-module will <emphasis>not</emphasis> automatically decode
3958 these records. To help you decode the records in the application, the
3962 Z_ext_typeent *z_ext_gettypebyref(const oid *oid);
3965 Can be used to retrieve information about the known, external data
3966 types. The function return a pointer to a static area, or NULL, if no
3967 match for the given direct reference is found. The
3968 <literal>Z_ext_typeent</literal>
3972 typedef struct Z_ext_typeent
3974 int oid[OID_SIZE]; /* the direct-reference OID. */
3975 int what; /* discriminator value for the external CHOICE */
3976 Odr_fun fun; /* decoder function */
3980 The <literal>what</literal> member contains the
3981 <literal>Z_External</literal> union discriminator value for the
3982 given type: For the SUTRS record syntax, the value would be
3983 <literal>Z_External_sutrs</literal>.
3984 The <literal>fun</literal> member contains a pointer to the
3985 function which encodes/decodes the given type. Again, for the SUTRS
3986 record syntax, the value of <literal>fun</literal> would be
3987 <literal>z_SUTRS</literal> (a function pointer).
3990 If you receive an EXTERNAL which contains an octet-string value that
3991 you suspect of being an ASN.1-structured data value, you can use
3992 <literal>z_ext_gettypebyref</literal> to look for the provided
3994 If the return value is different from NULL, you can use the provided
3995 function to decode the BER string (see <xref linkend="odr.use"/>
3999 If you want to <emphasis>send</emphasis> EXTERNALs containing
4000 ASN.1-structured values in the occtet-aligned branch of the CHOICE, this
4001 is possible too. However, on the encoding phase, it requires a somewhat
4002 involved juggling around of the various buffers involved.
4005 If you need to add new, externally defined data types, you must update
4006 the struct above, in the source file <filename>prt-ext.h</filename>, as
4007 well as the encoder/decoder in the file <filename>prt-ext.c</filename>.
4008 When changing the latter, remember to update both the
4009 <literal>arm</literal> arrary and the list
4010 <literal>type_table</literal>, which drives the CHOICE biasing that
4011 is necessary to tell the different, structured types apart
4016 Eventually, the EXTERNAL processing will most likely
4017 automatically insert the correct OIDs or indirect-refs. First,
4018 however, we need to determine how application-context management
4019 (specifically the presentation-context-list) should fit into the
4024 <sect1 id="asn.pdu">
4025 <title>PDU Contents Table</title>
4027 We include, for reference, a listing of the fields of each top-level
4028 PDU, as well as their default settings.
4030 <table frame="top" id="asn.default.initialize.request">
4031 <title>Default settings for PDU Initialize Request</title>
4033 <colspec colwidth="7*" colname="field"></colspec>
4034 <colspec colwidth="5*" colname="type"></colspec>
4035 <colspec colwidth="7*" colname="value"></colspec>
4038 <entry>Field</entry>
4040 <entry>Default Value</entry>
4045 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4048 protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
4051 options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
4054 preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
4057 maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
4060 idAuthentication</entry><entry>Z_IdAuthentication</entry><entry>NULL
4063 implementationId</entry><entry>char*</entry><entry>"81"
4066 implementationName</entry><entry>char*</entry><entry>"YAZ"
4069 implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
4072 userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
4075 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4080 <table frame="top" id="asn.default.initialize.response">
4081 <title>Default settings for PDU Initialize Response</title>
4083 <colspec colwidth="7*" colname="field"></colspec>
4084 <colspec colwidth="5*" colname="type"></colspec>
4085 <colspec colwidth="7*" colname="value"></colspec>
4088 <entry>Field</entry>
4090 <entry>Default Value</entry>
4095 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4098 protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
4101 options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
4104 preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
4107 maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
4110 result</entry><entry>Odr_bool</entry><entry>TRUE
4113 implementationId</entry><entry>char*</entry><entry>"id)"
4116 implementationName</entry><entry>char*</entry><entry>"YAZ"
4119 implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
4122 userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
4125 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4130 <table frame="top" id="asn.default.search.request">
4131 <title>Default settings for PDU Search Request</title>
4133 <colspec colwidth="7*" colname="field"></colspec>
4134 <colspec colwidth="5*" colname="type"></colspec>
4135 <colspec colwidth="7*" colname="value"></colspec>
4138 <entry>Field</entry>
4140 <entry>Default Value</entry>
4145 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4148 smallSetUpperBound</entry><entry>Odr_int</entry><entry>0
4151 largeSetLowerBound</entry><entry>Odr_int</entry><entry>1
4154 mediumSetPresentNumber</entry><entry>Odr_int</entry><entry>0
4157 replaceIndicator</entry><entry>Odr_bool</entry><entry>TRUE
4160 resultSetName</entry><entry>char *</entry><entry>"default"
4163 num_databaseNames</entry><entry>Odr_int</entry><entry>0
4166 databaseNames</entry><entry>char **</entry><entry>NULL
4169 smallSetElementSetNames</entry><entry>Z_ElementSetNames
4173 mediumSetElementSetNames</entry><entry>Z_ElementSetNames
4177 preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
4180 query</entry><entry>Z_Query</entry><entry>NULL
4183 additionalSearchInfo</entry><entry>Z_OtherInformation
4187 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4192 <table frame="top" id="asn.default.search.response">
4193 <title>Default settings for PDU Search Response</title>
4195 <colspec colwidth="7*" colname="field"></colspec>
4196 <colspec colwidth="5*" colname="type"></colspec>
4197 <colspec colwidth="7*" colname="value"></colspec>
4200 <entry>Field</entry>
4202 <entry>Default Value</entry>
4207 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4210 resultCount</entry><entry>Odr_int</entry><entry>0
4213 numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
4216 nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
4219 searchStatus</entry><entry>Odr_bool</entry><entry>TRUE
4222 resultSetStatus</entry><entry>Odr_int</entry><entry>NULL
4225 presentStatus</entry><entry>Odr_int</entry><entry>NULL
4228 records</entry><entry>Z_Records</entry><entry>NULL
4231 additionalSearchInfo</entry>
4232 <entry>Z_OtherInformation</entry><entry>NULL
4235 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4240 <table frame="top" id="asn.default.present.request">
4241 <title>Default settings for PDU Present Request</title>
4243 <colspec colwidth="7*" colname="field"></colspec>
4244 <colspec colwidth="5*" colname="type"></colspec>
4245 <colspec colwidth="7*" colname="value"></colspec>
4248 <entry>Field</entry>
4250 <entry>Default Value</entry>
4255 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4258 resultSetId</entry><entry>char*</entry><entry>"default"
4261 resultSetStartPoint</entry><entry>Odr_int</entry><entry>1
4264 numberOfRecordsRequested</entry><entry>Odr_int</entry><entry>10
4267 num_ranges</entry><entry>Odr_int</entry><entry>0
4270 additionalRanges</entry><entry>Z_Range</entry><entry>NULL
4273 recordComposition</entry><entry>Z_RecordComposition</entry><entry>NULL
4276 preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
4279 maxSegmentCount</entry><entry>Odr_int</entry><entry>NULL
4282 maxRecordSize</entry><entry>Odr_int</entry><entry>NULL
4285 maxSegmentSize</entry><entry>Odr_int</entry><entry>NULL
4288 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4293 <table frame="top" id="asn.default.present.response">
4294 <title>Default settings for PDU Present Response</title>
4296 <colspec colwidth="7*" colname="field"></colspec>
4297 <colspec colwidth="5*" colname="type"></colspec>
4298 <colspec colwidth="7*" colname="value"></colspec>
4301 <entry>Field</entry>
4303 <entry>Default Value</entry>
4308 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4311 numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
4314 nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
4317 presentStatus</entry><entry>Odr_int</entry><entry>Z_PresentStatus_success
4320 records</entry><entry>Z_Records</entry><entry>NULL
4323 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4328 <table frame="top" id="asn.default.delete.result.set.request">
4329 <title>Default settings for Delete Result Set Request</title>
4331 <colspec colwidth="7*" colname="field"></colspec>
4332 <colspec colwidth="5*" colname="type"></colspec>
4333 <colspec colwidth="7*" colname="value"></colspec>
4336 <entry>Field</entry>
4338 <entry>Default Value</entry>
4342 <row><entry>referenceId
4343 </entry><entry>Z_ReferenceId</entry><entry>NULL
4346 deleteFunction</entry><entry>Odr_int</entry><entry>Z_DeleteResultSetRequest_list
4349 num_ids</entry><entry>Odr_int</entry><entry>0
4352 resultSetList</entry><entry>char**</entry><entry>NULL
4355 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4360 <table frame="top" id="asn.default.delete.result.set.response">
4361 <title>Default settings for Delete Result Set Response</title>
4363 <colspec colwidth="7*" colname="field"></colspec>
4364 <colspec colwidth="5*" colname="type"></colspec>
4365 <colspec colwidth="7*" colname="value"></colspec>
4368 <entry>Field</entry>
4370 <entry>Default Value</entry>
4375 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4378 deleteOperationStatus</entry><entry>Odr_int</entry>
4379 <entry>Z_DeleteStatus_success</entry></row>
4381 num_statuses</entry><entry>Odr_int</entry><entry>0
4384 deleteListStatuses</entry><entry>Z_ListStatus**</entry><entry>NULL
4387 numberNotDeleted</entry><entry>Odr_int</entry><entry>NULL
4390 num_bulkStatuses</entry><entry>Odr_int</entry><entry>0
4393 bulkStatuses</entry><entry>Z_ListStatus</entry><entry>NUL
4396 deleteMessage</entry><entry>char*</entry><entry>NULL
4399 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4404 <table frame="top" id="asn.default.scan.request">
4405 <title>Default settings for Scan Request</title>
4407 <colspec colwidth="7*" colname="field"></colspec>
4408 <colspec colwidth="5*" colname="type"></colspec>
4409 <colspec colwidth="7*" colname="value"></colspec>
4412 <entry>Field</entry>
4414 <entry>Default Value</entry>
4419 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4422 num_databaseNames</entry><entry>Odr_int</entry><entry>0
4425 databaseNames</entry><entry>char**</entry><entry>NULL
4428 attributeSet</entry><entry>Odr_oid</entry><entry>NULL
4431 termListAndStartPoint</entry><entry>Z_AttributesPlus...
4432 </entry><entry>NULL</entry></row>
4434 stepSize</entry><entry>Odr_int</entry><entry>NULL
4437 numberOfTermsRequested</entry><entry>Odr_int</entry><entry>20
4440 preferredPositionInResponse</entry><entry>Odr_int</entry><entry>NULL
4443 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4448 <table frame="top" id="asn.default.scan.response">
4449 <title>Default settings for Scan Response</title>
4451 <colspec colwidth="7*" colname="field"></colspec>
4452 <colspec colwidth="5*" colname="type"></colspec>
4453 <colspec colwidth="7*" colname="value"></colspec>
4456 <entry>Field</entry>
4458 <entry>Default Value</entry>
4463 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4466 stepSize</entry><entry>Odr_int</entry><entry>NULL
4469 scanStatus</entry><entry>Odr_int</entry><entry>Z_Scan_success
4472 numberOfEntriesReturned</entry><entry>Odr_int</entry><entry>0
4475 positionOfTerm</entry><entry>Odr_int</entry><entry>NULL
4478 entries</entry><entry>Z_ListEntris</entry><entry>NULL
4481 attributeSet</entry><entry>Odr_oid</entry><entry>NULL
4484 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4489 <table frame="top" id="asn.default.trigger.resource.control.request">
4490 <title>Default settings for Trigger Resource Control Request</title>
4492 <colspec colwidth="7*" colname="field"></colspec>
4493 <colspec colwidth="5*" colname="type"></colspec>
4494 <colspec colwidth="7*" colname="value"></colspec>
4497 <entry>Field</entry>
4499 <entry>Default Value</entry>
4504 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4507 requestedAction</entry><entry>Odr_int</entry><entry>
4508 Z_TriggerResourceCtrl_resou..
4511 prefResourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
4514 resultSetWanted</entry><entry>Odr_bool</entry><entry>NULL
4517 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4522 <table frame="top" id="asn.default.resource.control.request">
4523 <title>Default settings for Resource Control Request</title>
4525 <colspec colwidth="7*" colname="field"></colspec>
4526 <colspec colwidth="5*" colname="type"></colspec>
4527 <colspec colwidth="7*" colname="value"></colspec>
4530 <entry>Field</entry>
4532 <entry>Default Value</entry>
4537 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4540 suspendedFlag</entry><entry>Odr_bool</entry><entry>NULL
4543 resourceReport</entry><entry>Z_External</entry><entry>NULL
4546 partialResultsAvailable</entry><entry>Odr_int</entry><entry>NULL
4549 responseRequired</entry><entry>Odr_bool</entry><entry>FALSE
4552 triggeredRequestFlag</entry><entry>Odr_bool</entry><entry>NULL
4555 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4560 <table frame="top" id="asn.default.resource.control.response">
4561 <title>Default settings for Resource Control Response</title>
4563 <colspec colwidth="7*" colname="field"></colspec>
4564 <colspec colwidth="5*" colname="type"></colspec>
4565 <colspec colwidth="7*" colname="value"></colspec>
4568 <entry>Field</entry>
4570 <entry>Default Value</entry>
4575 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4578 continueFlag</entry><entry>bool_t</entry><entry>TRUE
4581 resultSetWanted</entry><entry>bool_t</entry><entry>NULL
4584 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4589 <table frame="top" id="asn.default.access.control.request">
4590 <title>Default settings for Access Control Request</title>
4592 <colspec colwidth="7*" colname="field"></colspec>
4593 <colspec colwidth="5*" colname="type"></colspec>
4594 <colspec colwidth="7*" colname="value"></colspec>
4597 <entry>Field</entry>
4599 <entry>Default Value</entry>
4604 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4607 which</entry><entry>enum</entry><entry>Z_AccessRequest_simpleForm;
4610 u</entry><entry>union</entry><entry>NULL
4613 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4618 <table frame="top" id="asn.default.access.control.response">
4619 <title>Default settings for Access Control Response</title>
4621 <colspec colwidth="7*" colname="field"></colspec>
4622 <colspec colwidth="5*" colname="type"></colspec>
4623 <colspec colwidth="7*" colname="value"></colspec>
4626 <entry>Field</entry>
4628 <entry>Default Value</entry>
4633 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4636 which</entry><entry>enum</entry><entry>Z_AccessResponse_simpleForm
4639 u</entry><entry>union</entry><entry>NULL
4642 diagnostic</entry><entry>Z_DiagRec</entry><entry>NULL
4645 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4650 <table frame="top" id="asn.default.segment">
4651 <title>Default settings for Segment</title>
4653 <colspec colwidth="7*" colname="field"></colspec>
4654 <colspec colwidth="5*" colname="type"></colspec>
4655 <colspec colwidth="7*" colname="value"></colspec>
4658 <entry>Field</entry>
4660 <entry>Default Value</entry>
4665 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4668 numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>value=0
4671 num_segmentRecords</entry><entry>Odr_int</entry><entry>0
4674 segmentRecords</entry><entry>Z_NamePlusRecord</entry><entry>NULL
4676 <row><entry>otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4681 <table frame="top" id="asn.default.close">
4682 <title>Default settings for Close</title>
4684 <colspec colwidth="7*" colname="field"></colspec>
4685 <colspec colwidth="5*" colname="type"></colspec>
4686 <colspec colwidth="7*" colname="value"></colspec>
4689 <entry>Field</entry>
4691 <entry>Default Value</entry>
4696 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
4699 closeReason</entry><entry>Odr_int</entry><entry>Z_Close_finished
4702 diagnosticInformation</entry><entry>char*</entry><entry>NULL
4705 resourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
4708 resourceFormat</entry><entry>Z_External</entry><entry>NULL
4711 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
4719 <title>SOAP and SRU</title>
4720 <sect1 id="soap.introduction">
4721 <title>Introduction</title>
4723 &yaz; uses a very simple implementation of
4724 <ulink url="&url.soap;">SOAP</ulink> that only,
4725 currenly, supports what is sufficient to offer SRU SOAP functionality.
4726 The implementation uses the
4727 <ulink url="&url.libxml2.api.tree;">tree API</ulink> of
4728 libxml2 to encode and decode SOAP packages.
4731 Like the Z39.50 ASN.1 module, the &yaz; SRU implementation uses
4732 simple C structs to represent SOAP packages as well as
4736 <sect1 id="soap.http">
4739 &yaz; only offers HTTP as transport carrier for SOAP, but it is
4740 relatively easy to change that.
4743 The following definition of <literal>Z_GDU</literal> (Generic Data
4744 Unit) allows for both HTTP and Z39.50 in one packet.
4747 #include <yaz/zgdu.h>
4749 #define Z_GDU_Z3950 1
4750 #define Z_GDU_HTTP_Request 2
4751 #define Z_GDU_HTTP_Response 3
4756 Z_HTTP_Request *HTTP_Request;
4757 Z_HTTP_Response *HTTP_Response;
4762 The corresponding Z_GDU encoder/decoder is <function>z_GDU</function>.
4763 The <literal>z3950</literal> is any of the known BER encoded Z39.50
4765 <literal>HTTP_Request</literal> and <literal>HTTP_Response</literal>
4766 is the HTTP Request and Response respectively.
4769 <sect1 id="soap.xml">
4770 <title>SOAP Packages</title>
4772 Every SOAP package in &yaz; is represented as follows:
4774 #include <yaz/soap.h>
4788 #define Z_SOAP_fault 1
4789 #define Z_SOAP_generic 2
4790 #define Z_SOAP_error 3
4794 Z_SOAP_Fault *fault;
4795 Z_SOAP_Generic *generic;
4796 Z_SOAP_Fault *soap_error;
4803 The <literal>fault</literal> and <literal>soap_error</literal>
4804 arms represent both a SOAP fault - struct
4805 <literal>Z_SOAP_Fault</literal>. Any other generic
4806 (valid) package is represented by <literal>Z_SOAP_Generic</literal>.
4809 The <literal>ns</literal> as part of <literal>Z_SOAP</literal>
4810 is the namespace for SOAP itself and reflects the SOAP
4811 version. For version 1.1 it is
4812 <literal>http://schemas.xmlsoap.org/soap/envelope/</literal>,
4813 for version 1.2 it is
4814 <literal>http://www.w3.org/2001/06/soap-envelope</literal>.
4817 int z_soap_codec(ODR o, Z_SOAP **pp,
4818 char **content_buf, int *content_len,
4819 Z_SOAP_Handler *handlers);
4822 The <literal>content_buf</literal> and <literal>content_len</literal>
4823 is XML buffer and length of buffer respectively.
4826 The <literal>handlers</literal> is a list of SOAP codec
4827 handlers - one handler for each service namespace. For SRU SOAP, the
4828 namespace would be <literal>http://www.loc.gov/zing/srw/v1.0/</literal>.
4831 When decoding, the <function>z_soap_codec</function>
4832 inspects the XML content
4833 and tries to match one of the services namespaces of the
4834 supplied handlers. If there is a match a handler function
4835 is invoked which decodes that particular SOAP package.
4836 If successful, the returned <literal>Z_SOAP</literal> package will be
4837 of type <literal>Z_SOAP_Generic</literal>.
4838 Member <literal>no</literal> is
4839 set the offset of handler that matched; <literal>ns</literal>
4840 is set to namespace of matching handler; the void pointer
4841 <literal>p</literal> is set to the C data structure assocatiated
4845 When a NULL namespace is met (member <literal>ns</literal> bwlow),
4846 that specifies end-of-list.
4849 Each handler is defined as follows:
4857 The <literal>ns</literal> is namespace of service associated with
4858 handler <literal>f</literal>. <literal>client_data</literal>
4859 is user-defined data which is passed to handler.
4862 The prototype for a SOAP service handler is:
4864 int handler(ODR o, void * ptr, void **handler_data,
4865 void *client_data, const char *ns);
4867 The <parameter>o</parameter> specifies the mode (decode/encode)
4868 as usual. The second argument, <parameter>ptr</parameter>,
4869 is a libxml2 tree node pointer (<literal>xmlNodePtr</literal>)
4870 and is a pointer to the <literal>Body</literal> element
4871 of the SOAP package. The <parameter>handler_data</parameter>
4872 is an opaque pointer to a C definitions associated with the
4873 SOAP service. <parameter>client_data</parameter> is the pointer
4874 which was set as part of the <literal>Z_SOAP_handler</literal>.
4875 Finally, <parameter>ns</parameter> the service namespace.
4878 <sect1 id="soap.srw">
4881 SRU SOAP is just one implementation of a SOAP handler as described
4882 in the previous section.
4883 The encoder/decoder handler for SRU is defined as
4886 #include <yaz/srw.h>
4888 int yaz_srw_codec(ODR o, void * pptr,
4889 Z_SRW_GDU **handler_data,
4890 void *client_data, const char *ns);
4892 Here, <literal>Z_SRW_GDU</literal> is either
4893 searchRetrieveRequest or a searchRetrieveResponse.
4897 The xQuery and xSortKeys are not handled yet by
4898 the SRW implementation of &yaz;. Explain is also missing.
4899 Future versions of &yaz; will include these features.
4903 The definition of searchRetrieveRequest is:
4907 #define Z_SRW_query_type_cql 1
4908 #define Z_SRW_query_type_xcql 2
4909 #define Z_SRW_query_type_pqf 3
4917 #define Z_SRW_sort_type_none 1
4918 #define Z_SRW_sort_type_sort 2
4919 #define Z_SRW_sort_type_xSort 3
4927 int *maximumRecords;
4929 char *recordPacking;
4931 } Z_SRW_searchRetrieveRequest;
4933 Please observe that data of type xsd:string is represented
4934 as a char pointer (<literal>char *</literal>). A null pointer
4935 means that the element is absent.
4936 Data of type xsd:integer is representd as a pointer to
4937 an int (<literal>int *</literal>). Again, a null pointer
4938 us used for absent elements.
4941 The SearchRetrieveResponse has the following definition.
4944 int * numberOfRecords;
4946 int * resultSetIdleTime;
4948 Z_SRW_record *records;
4951 Z_SRW_diagnostic *diagnostics;
4952 int num_diagnostics;
4953 int *nextRecordPosition;
4954 } Z_SRW_searchRetrieveResponse;
4956 The <literal>num_records</literal> and <literal>num_diagnostics</literal>
4957 is number of returned records and diagnostics respectively and also
4958 correspond to the "size of" arrays <literal>records</literal>
4959 and <literal>diagnostics</literal>.
4962 A retrieval record is defined as follows:
4966 char *recordData_buf;
4968 int *recordPosition;
4971 The record data is defined as a buffer of some length so that
4972 data can be of any type. SRW 1.0 currenly doesn't allow for this
4973 (only XML), but future versions might do.
4976 And, a diagnostic as:
4986 <chapter id="tools">
4987 <title>Supporting Tools</title>
4989 In support of the service API - primarily the ASN module, which
4990 provides the pro-grammatic interface to the Z39.50 APDUs, &yaz; contains
4991 a collection of tools that support the development of applications.
4993 <sect1 id="tools.query">
4994 <title>Query Syntax Parsers</title>
4996 Since the type-1 (RPN) query structure has no direct, useful string
4997 representation, every origin application needs to provide some form of
4998 mapping from a local query notation or representation to a
4999 <token>Z_RPNQuery</token> structure. Some programmers will prefer to
5000 construct the query manually, perhaps using
5001 <function>odr_malloc()</function> to simplify memory management.
5002 The &yaz; distribution includes three separate, query-generating tools
5003 that may be of use to you.
5006 <title>Prefix Query Format</title>
5008 Since RPN or reverse polish notation is really just a fancy way of
5009 describing a suffix notation format (operator follows operands), it
5010 would seem that the confusion is total when we now introduce a prefix
5011 notation for RPN. The reason is one of simple laziness - it's somewhat
5012 simpler to interpret a prefix format, and this utility was designed
5013 for maximum simplicity, to provide a baseline representation for use
5014 in simple test applications and scripting environments (like Tcl). The
5015 demonstration client included with YAZ uses the PQF.
5019 The PQF have been adopted by other parties developing Z39.50
5020 software. It is often referred to as Prefix Query Notation
5025 The PQF is defined by the pquery module in the YAZ library.
5026 There are two sets of function that have similar behavior. First
5027 set operates on a PQF parser handle, second set doesn't. First set
5028 set of functions are more flexible than the second set. Second set
5029 is obsolete and is only provided to ensure backwards compatibility.
5032 First set of functions all operate on a PQF parser handle:
5035 #include <yaz/pquery.h>
5037 YAZ_PQF_Parser yaz_pqf_create(void);
5039 void yaz_pqf_destroy(YAZ_PQF_Parser p);
5041 Z_RPNQuery *yaz_pqf_parse(YAZ_PQF_Parser p, ODR o, const char *qbuf);
5043 Z_AttributesPlusTerm *yaz_pqf_scan(YAZ_PQF_Parser p, ODR o,
5044 Odr_oid **attributeSetId, const char *qbuf);
5046 int yaz_pqf_error(YAZ_PQF_Parser p, const char **msg, size_t *off);
5049 A PQF parser is created and destructed by functions
5050 <function>yaz_pqf_create</function> and
5051 <function>yaz_pqf_destroy</function> respectively.
5052 Function <function>yaz_pqf_parse</function> parses query given
5053 by string <literal>qbuf</literal>. If parsing was successful,
5054 a Z39.50 RPN Query is returned which is created using ODR stream
5055 <literal>o</literal>. If parsing failed, a NULL pointer is
5057 Function <function>yaz_pqf_scan</function> takes a scan query in
5058 <literal>qbuf</literal>. If parsing was successful, the function
5059 returns attributes plus term pointer and modifies
5060 <literal>attributeSetId</literal> to hold attribute set for the
5061 scan request - both allocated using ODR stream <literal>o</literal>.
5062 If parsing failed, yaz_pqf_scan returns a NULL pointer.
5063 Error information for bad queries can be obtained by a call to
5064 <function>yaz_pqf_error</function> which returns an error code and
5065 modifies <literal>*msg</literal> to point to an error description,
5066 and modifies <literal>*off</literal> to the offset within last
5067 query were parsing failed.
5070 The second set of functions are declared as follows:
5073 #include <yaz/pquery.h>
5075 Z_RPNQuery *p_query_rpn(ODR o, oid_proto proto, const char *qbuf);
5077 Z_AttributesPlusTerm *p_query_scan(ODR o, oid_proto proto,
5078 Odr_oid **attributeSetP, const char *qbuf);
5080 int p_query_attset(const char *arg);
5083 The function <function>p_query_rpn()</function> takes as arguments an
5084 &odr; stream (see section <link linkend="odr">The ODR Module</link>)
5085 to provide a memory source (the structure created is released on
5086 the next call to <function>odr_reset()</function> on the stream), a
5087 protocol identifier (one of the constants <token>PROTO_Z3950</token> and
5088 <token>PROTO_SR</token>), an attribute set reference, and
5089 finally a null-terminated string holding the query string.
5092 If the parse went well, <function>p_query_rpn()</function> returns a
5093 pointer to a <literal>Z_RPNQuery</literal> structure which can be
5094 placed directly into a <literal>Z_SearchRequest</literal>.
5095 If parsing failed, due to syntax error, a NULL pointer is returned.
5098 The <literal>p_query_attset</literal> specifies which attribute set
5099 to use if the query doesn't specify one by the
5100 <literal>@attrset</literal> operator.
5101 The <literal>p_query_attset</literal> returns 0 if the argument is a
5102 valid attribute set specifier; otherwise the function returns -1.
5105 The grammar of the PQF is as follows:
5108 query ::= top-set query-struct.
5110 top-set ::= [ '@attrset' string ]
5112 query-struct ::= attr-spec | simple | complex | '@term' term-type query
5114 attr-spec ::= '@attr' [ string ] string query-struct
5116 complex ::= operator query-struct query-struct.
5118 operator ::= '@and' | '@or' | '@not' | '@prox' proximity.
5120 simple ::= result-set | term.
5122 result-set ::= '@set' string.
5126 proximity ::= exclusion distance ordered relation which-code unit-code.
5128 exclusion ::= '1' | '0' | 'void'.
5130 distance ::= integer.
5132 ordered ::= '1' | '0'.
5134 relation ::= integer.
5136 which-code ::= 'known' | 'private' | integer.
5138 unit-code ::= integer.
5140 term-type ::= 'general' | 'numeric' | 'string' | 'oid' | 'datetime' | 'null'.
5143 You will note that the syntax above is a fairly faithful
5144 representation of RPN, except for the Attribute, which has been
5145 moved a step away from the term, allowing you to associate one or more
5146 attributes with an entire query structure. The parser will
5147 automatically apply the given attributes to each term as required.
5150 The @attr operator is followed by an attribute specification
5151 (<literal>attr-spec</literal> above). The specification consists
5152 of an optional attribute set, an attribute type-value pair and
5153 a sub-query. The attribute type-value pair is packed in one string:
5154 an attribute type, an equals sign, and an attribute value, like this:
5155 <literal>@attr 1=1003</literal>.
5156 The type is always an integer but the value may be either an
5157 integer or a string (if it doesn't start with a digit character).
5158 A string attribute-value is encoded as a Type-1 ``complex''
5159 attribute with the list of values containing the single string
5160 specified, and including no semantic indicators.
5163 Version 3 of the Z39.50 specification defines various encoding of terms.
5164 Use <literal>@term </literal> <replaceable>type</replaceable>
5165 <replaceable>string</replaceable>,
5166 where type is one of: <literal>general</literal>,
5167 <literal>numeric</literal> or <literal>string</literal>
5168 (for InternationalString).
5169 If no term type has been given, the <literal>general</literal> form
5170 is used. This is the only encoding allowed in both versions 2 and 3
5171 of the Z39.50 standard.
5173 <sect3 id="PQF-prox">
5174 <title>Using Proximity Operators with PQF</title>
5177 This is an advanced topic, describing how to construct
5178 queries that make very specific requirements on the
5179 relative location of their operands.
5180 You may wish to skip this section and go straight to
5181 <link linkend="pqf-examples">the example PQF queries</link>.
5186 Most Z39.50 servers do not support proximity searching, or
5187 support only a small subset of the full functionality that
5188 can be expressed using the PQF proximity operator. Be
5189 aware that the ability to <emphasis>express</emphasis> a
5190 query in PQF is no guarantee that any given server will
5191 be able to <emphasis>execute</emphasis> it.
5197 The proximity operator <literal>@prox</literal> is a special
5198 and more restrictive version of the conjunction operator
5199 <literal>@and</literal>. Its semantics are described in
5200 section 3.7.2 (Proximity) of Z39.50 the standard itself, which
5201 can be read on-line at
5202 <ulink url="&url.z39.50.proximity;"/>
5205 In PQF, the proximity operation is represented by a sequence
5208 @prox <replaceable>exclusion</replaceable> <replaceable>distance</replaceable> <replaceable>ordered</replaceable> <replaceable>relation</replaceable> <replaceable>which-code</replaceable> <replaceable>unit-code</replaceable>
5210 in which the meanings of the parameters are as described in in
5211 the standard, and they can take the following values:
5214 <formalpara><title>exclusion</title>
5216 0 = false (i.e. the proximity condition specified by the
5217 remaining parameters must be satisfied) or
5218 1 = true (the proximity condition specified by the
5219 remaining parameters must <emphasis>not</emphasis> be
5225 <formalpara><title>distance</title><para>
5226 An integer specifying the difference between the locations
5227 of the operands: e.g. two adjacent words would have
5228 distance=1 since their locations differ by one unit.
5230 </formalpara></listitem>
5232 <formalpara><title>ordered</title><para>
5233 1 = ordered (the operands must occur in the order the
5234 query specifies them) or
5235 0 = unordered (they may appear in either order).
5240 <formalpara><title>relation</title><para>
5241 Recognised values are
5243 2 (lessThanOrEqual),
5245 4 (greaterThanOrEqual),
5252 <formalpara><title>which-code</title><para>
5253 <literal>known</literal>
5255 <literal>k</literal>
5256 (the unit-code parameter is taken from the well-known list
5257 of alternatives described in below) or
5258 <literal>private</literal>
5260 <literal>p</literal>
5261 (the unit-code paramater has semantics specific to an
5262 out-of-band agreement such as a profile).
5267 <formalpara><title>unit-code</title><para>
5268 If the which-code parameter is <literal>known</literal>
5269 then the recognised values are
5279 10 (elementType) and
5281 If which-code is <literal>private</literal> then the
5282 acceptable values are determined by the profile.
5287 (The numeric values of the relation and well-known unit-code
5288 parameters are taken straight from
5289 <ulink url="&url.z39.50.proximity.asn1;"
5290 >the ASN.1</ulink> of the proximity structure in the standard.)
5293 <sect3 id="pqf-examples">
5294 <title>PQF queries</title>
5295 <example id="example.pqf.simple.terms">
5296 <title>PQF queries using simple terms</title>
5305 <example id="pqf.example.pqf.boolean.operators">
5306 <title>PQF boolean operators</title>
5309 @or "dylan" "zimmerman"
5311 @and @or dylan zimmerman when
5313 @and when @or dylan zimmerman
5317 <example id="example.pqf.result.sets">
5318 <title>PQF references to result sets</title>
5323 @and @set seta @set setb
5327 <example id="example.pqf.attributes">
5328 <title>Attributes for terms</title>
5333 @attr 1=4 @attr 4=1 "self portrait"
5335 @attrset exp1 @attr 1=1 CategoryList
5337 @attr gils 1=2008 Copenhagen
5339 @attr 1=/book/title computer
5343 <example id="example.pqf.proximity">
5344 <title>PQF Proximity queries</title>
5347 @prox 0 3 1 2 k 2 dylan zimmerman
5349 Here the parameters 0, 3, 1, 2, k and 2 represent exclusion,
5350 distance, ordered, relation, which-code and unit-code, in that
5354 <para>exclusion = 0: the proximity condition must hold</para>
5357 <para>distance = 3: the terms must be three units apart</para>
5361 ordered = 1: they must occur in the order they are specified
5366 relation = 2: lessThanOrEqual (to the distance of 3 units)
5371 which-code is ``known'', so the standard unit-codes are used
5375 <para>unit-code = 2: word.</para>
5378 So the whole proximity query means that the words
5379 <literal>dylan</literal> and <literal>zimmerman</literal> must
5380 both occur in the record, in that order, differing in position
5381 by three or fewer words (i.e. with two or fewer words between
5382 them.) The query would find ``Bob Dylan, aka. Robert
5383 Zimmerman'', but not ``Bob Dylan, born as Robert Zimmerman''
5384 since the distance in this case is four.
5387 <example id="example.pqf.search.term.type">
5388 <title>PQF specification of search term type</title>
5391 @term string "a UTF-8 string, maybe?"
5395 <example id="example.pqf.mixed.queries">
5396 <title>PQF mixed queries</title>
5399 @or @and bob dylan @set Result-1
5401 @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
5403 @and @attr 2=4 @attr gils 1=2038 -114 @attr 2=2 @attr gils 1=2039 -109
5405 The last of these examples is a spatial search: in
5406 <ulink url="http://www.gils.net/prof_v2.html#sec_7_4"
5407 >the GILS attribute set</ulink>,
5409 2038 indicates West Bounding Coordinate and
5410 2030 indicates East Bounding Coordinate,
5411 so the query is for areas extending from -114 degrees
5412 to no more than -109 degrees.
5417 <sect2 id="CCL"><title>CCL</title>
5419 Not all users enjoy typing in prefix query structures and numerical
5420 attribute values, even in a minimalistic test client. In the library
5421 world, the more intuitive Common Command Language - CCL (ISO 8777)
5422 has enjoyed some popularity - especially before the widespread
5423 availability of graphical interfaces. It is still useful in
5424 applications where you for some reason or other need to provide a
5425 symbolic language for expressing boolean query structures.
5427 <sect3 id="ccl.syntax">
5428 <title>CCL Syntax</title>
5430 The CCL parser obeys the following grammar for the FIND argument.
5431 The syntax is annotated by in the lines prefixed by
5432 <literal>--</literal>.
5435 CCL-Find ::= CCL-Find Op Elements
5438 Op ::= "and" | "or" | "not"
5439 -- The above means that Elements are separated by boolean operators.
5441 Elements ::= '(' CCL-Find ')'
5444 | Qualifiers Relation Terms
5445 | Qualifiers Relation '(' CCL-Find ')'
5446 | Qualifiers '=' string '-' string
5447 -- Elements is either a recursive definition, a result set reference, a
5448 -- list of terms, qualifiers followed by terms, qualifiers followed
5449 -- by a recursive definition or qualifiers in a range (lower - upper).
5451 Set ::= 'set' = string
5452 -- Reference to a result set
5454 Terms ::= Terms Prox Term
5456 -- Proximity of terms.
5458 Term ::= Term string
5460 -- This basically means that a term may include a blank
5462 Qualifiers ::= Qualifiers ',' string
5464 -- Qualifiers is a list of strings separated by comma
5466 Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
5467 -- Relational operators. This really doesn't follow the ISO8777
5471 -- Proximity operator
5474 <example id="example.ccl.queries">
5475 <title>CCL queries</title>
5477 The following queries are all valid:
5488 (dylan and bob) or set=1
5497 Assuming that the qualifiers <literal>ti</literal>,
5498 <literal>au</literal>
5499 and <literal>date</literal> are defined we may use:
5504 au=(bob dylan and slow train coming)
5506 date>1980 and (ti=((self portrait)))
5510 <sect3 id="ccl.qualifiers">
5511 <title>CCL Qualifiers</title>
5513 Qualifiers are used to direct the search to a particular searchable
5514 index, such as title (ti) and author indexes (au). The CCL standard
5515 itself doesn't specify a particular set of qualifiers, but it does
5516 suggest a few short-hand notations. You can customize the CCL parser
5517 to support a particular set of qualifiers to reflect the current target
5518 profile. Traditionally, a qualifier would map to a particular
5519 use-attribute within the BIB-1 attribute set. It is also
5520 possible to set other attributes, such as the structure
5524 A CCL profile is a set of predefined CCL qualifiers that may be
5525 read from a file or set in the CCL API.
5526 The YAZ client reads its CCL qualifiers from a file named
5527 <filename>default.bib</filename>. There are four types of
5528 lines in a CCL profile: qualifier specification,
5529 qualifier alias, comments and directives.
5531 <sect4 id="ccl.qualifier.specification">
5532 <title>Qualifier specification</title>
5534 A qualifier specification is of the form:
5537 <replaceable>qualifier-name</replaceable>
5538 [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable>
5539 [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable> ...
5542 where <replaceable>qualifier-name</replaceable> is the name of the
5543 qualifier to be used (eg. <literal>ti</literal>),
5544 <replaceable>type</replaceable> is attribute type in the attribute
5545 set (Bib-1 is used if no attribute set is given) and
5546 <replaceable>val</replaceable> is attribute value.
5547 The <replaceable>type</replaceable> can be specified as an
5548 integer or as it be specified either as a single-letter:
5549 <literal>u</literal> for use,
5550 <literal>r</literal> for relation,<literal>p</literal> for position,
5551 <literal>s</literal> for structure,<literal>t</literal> for truncation
5552 or <literal>c</literal> for completeness.
5553 The attributes for the special qualifier name <literal>term</literal>
5554 are used when no CCL qualifier is given in a query.
5555 <table id="ccl.common.bib1.attributes">
5556 <title>Common Bib-1 attributes</title>
5558 <colspec colwidth="2*" colname="type"></colspec>
5559 <colspec colwidth="9*" colname="description"></colspec>
5563 <entry>Description</entry>
5568 <entry><literal>u=</literal><replaceable>value</replaceable></entry>
5570 Use attribute (1). Common use attributes are
5571 1 Personal-name, 4 Title, 7 ISBN, 8 ISSN, 30 Date,
5572 62 Subject, 1003 Author), 1016 Any. Specify value
5577 <entry><literal>r=</literal><replaceable>value</replaceable></entry>
5579 Relation attribute (2). Common values are
5580 1 <, 2 <=, 3 =, 4 >=, 5 >, 6 <>,
5581 100 phonetic, 101 stem, 102 relevance, 103 always matches.
5585 <entry><literal>p=</literal><replaceable>value</replaceable></entry>
5587 Position attribute (3). Values: 1 first in field, 2
5588 first in any subfield, 3 any position in field.
5592 <entry><literal>s=</literal><replaceable>value</replaceable></entry>
5594 Structure attribute (4). Values: 1 phrase, 2 word,
5595 3 key, 4 year, 5 date, 6 word list, 100 date (un),
5596 101 name (norm), 102 name (un), 103 structure, 104 urx,
5597 105 free-form-text, 106 document-text, 107 local-number,
5598 108 string, 109 numeric string.
5602 <entry><literal>t=</literal><replaceable>value</replaceable></entry>
5604 Truncation attribute (5). Values: 1 right, 2 left,
5605 3 left& right, 100 none, 101 process #, 102 regular-1,
5606 103 regular-2, 104 CCL.
5610 <entry><literal>c=</literal><replaceable>value</replaceable></entry>
5612 Completeness attribute (6). Values: 1 incomplete subfield,
5613 2 complete subfield, 3 complete field.
5621 Refer to <xref linkend="bib1"/> or the complete
5622 <ulink url="&url.z39.50.attset.bib1;">list of Bib-1 attributes</ulink>
5625 It is also possible to specify non-numeric attribute values,
5626 which are used in combination with certain types.
5627 The special combinations are:
5628 <table id="ccl.special.attribute.combos">
5629 <title>Special attribute combos</title>
5631 <colspec colwidth="2*" colname="name"></colspec>
5632 <colspec colwidth="9*" colname="description"></colspec>
5636 <entry>Description</entry>
5641 <entry><literal>s=pw</literal></entry>
5643 The structure is set to either word or phrase depending
5644 on the number of tokens in a term (phrase-word).
5648 <entry><literal>s=al</literal></entry>
5650 Each token in the term is ANDed. (and-list).
5651 This does not set the structure at all.
5654 <row><entry><literal>s=ol</literal></entry>
5656 Each token in the term is ORed. (or-list).
5657 This does not set the structure at all.
5660 <row><entry><literal>s=ag</literal></entry>
5662 Tokens that appears as phrases (with blank in them) gets
5663 structure phrase attached (4=1). Tokens that appear to be words
5664 gets structure word attached (4=2). Phrases and words are
5665 ANDed. This is a variant of s=al and s=pw, with the main
5666 difference that words are not split (with operator AND)
5667 but instead kept in one RPN token. This facility appeared
5671 <row><entry><literal>r=o</literal></entry>
5673 Allows ranges and the operators greather-than, less-than, ...
5675 This sets Bib-1 relation attribute accordingly (relation
5676 ordered). A query construct is only treated as a range if
5677 dash is used and that is surrounded by white-space. So
5678 <literal>-1980</literal> is treated as term
5679 <literal>"-1980"</literal> not <literal><= 1980</literal>.
5680 If <literal>- 1980</literal> is used, however, that is
5684 <row><entry><literal>r=r</literal></entry>
5686 Similar to <literal>r=o</literal> but assumes that terms
5687 are non-negative (not prefixed with <literal>-</literal>).
5688 Thus, a dash will always be treated as a range.
5689 The construct <literal>1980-1990</literal> is
5690 treated as a range with <literal>r=r</literal> but as a
5691 single term <literal>"1980-1990"</literal> with
5692 <literal>r=o</literal>. The special attribute
5693 <literal>r=r</literal> is available in YAZ 2.0.24 or later.
5696 <row><entry><literal>t=l</literal></entry>
5698 Allows term to be left-truncated.
5699 If term is of the form <literal>?x</literal>, the resulting
5700 Type-1 term is <literal>x</literal> and truncation is left.
5703 <row><entry><literal>t=r</literal></entry>
5705 Allows term to be right-truncated.
5706 If term is of the form <literal>x?</literal>, the resulting
5707 Type-1 term is <literal>x</literal> and truncation is right.
5710 <row><entry><literal>t=n</literal></entry>
5712 If term is does not include <literal>?</literal>, the
5713 truncation attribute is set to none (100).
5716 <row><entry><literal>t=b</literal></entry>
5718 Allows term to be both left&right truncated.
5719 If term is of the form <literal>?x?</literal>, the
5720 resulting term is <literal>x</literal> and trunctation is
5721 set to both left&right.
5724 <row><entry><literal>t=x</literal></entry>
5726 Allows masking anywhere in a term, thus fully supporting
5727 # (mask one character) and ? (zero or more of any).
5728 If masking is used, trunction is set to 102 (regexp-1 in term)
5729 and the term is converted accordingly to a regular expression.
5732 <row><entry><literal>t=z</literal></entry>
5734 Allows masking anywhere in a term, thus fully supporting
5735 # (mask one character) and ? (zero or more of any).
5736 If masking is used, trunction is set to 104 (Z39.58 in term)
5737 and the term is converted accordingly to Z39.58 masking term -
5738 actually the same truncation as CCL itself.
5745 <example id="example.ccl.profile">
5746 <title>CCL profile</title>
5748 Consider the following definition:
5758 <literal>ti</literal> and <literal>au</literal> both set
5759 structure attribute to phrase (s=1).
5760 <literal>ti</literal>
5761 sets the use-attribute to 4. <literal>au</literal> sets the
5763 When no qualifiers are used in the query the structure-attribute is
5764 set to free-form-text (105) (rule for <literal>term</literal>).
5765 The <literal>date</literal> sets the relation attribute to
5766 the relation used in the CCL query and sets the use attribute
5770 You can combine attributes. To Search for "ranked title" you
5773 ti,ranked=knuth computer
5775 which will set relation=ranked, use=title, structure=phrase.
5782 is a valid query. But
5790 <sect4 id="ccl.qualifier.alias">
5791 <title>Qualifier alias</title>
5793 A qualifier alias is of the form:
5796 <replaceable>q</replaceable>
5797 <replaceable>q1</replaceable> <replaceable>q2</replaceable> ..
5800 which declares <replaceable>q</replaceable> to
5801 be an alias for <replaceable>q1</replaceable>,
5802 <replaceable>q2</replaceable>... such that the CCL
5803 query <replaceable>q=x</replaceable> is equivalent to
5804 <replaceable>q1=x or q2=x or ...</replaceable>.
5807 <sect4 id="ccl.comments">
5808 <title>Comments</title>
5810 Lines with white space or lines that begin with
5811 character <literal>#</literal> are treated as comments.
5814 <sect4 id="ccl.directives">
5815 <title>Directives</title>
5817 Directive specifications takes the form
5819 <para><literal>@</literal><replaceable>directive</replaceable> <replaceable>value</replaceable>
5821 <table id="ccl.directives.table">
5822 <title>CCL directives</title>
5824 <colspec colwidth="2*" colname="name"></colspec>
5825 <colspec colwidth="8*" colname="description"></colspec>
5826 <colspec colwidth="1*" colname="default"></colspec>
5830 <entry>Description</entry>
5831 <entry>Default</entry>
5836 <entry>truncation</entry>
5837 <entry>Truncation character</entry>
5838 <entry><literal>?</literal></entry>
5842 <entry>Masking character. Requires YAZ 4.2.58 or later</entry>
5843 <entry><literal>#</literal></entry>
5846 <entry>field</entry>
5847 <entry>Specifies how multiple fields are to be
5848 combined. There are two modes: <literal>or</literal>:
5849 multiple qualifier fields are ORed,
5850 <literal>merge</literal>: attributes for the qualifier
5851 fields are merged and assigned to one term.
5853 <entry><literal>merge</literal></entry>
5857 <entry>Specifies if CCL operators and qualifiers should be
5858 compared with case sensitivity or not. Specify 1 for
5859 case sensitive; 0 for case insensitive.</entry>
5860 <entry><literal>1</literal></entry>
5864 <entry>Specifies token for CCL operator AND.</entry>
5865 <entry><literal>and</literal></entry>
5869 <entry>Specifies token for CCL operator OR.</entry>
5870 <entry><literal>or</literal></entry>
5874 <entry>Specifies token for CCL operator NOT.</entry>
5875 <entry><literal>not</literal></entry>
5879 <entry>Specifies token for CCL operator SET.</entry>
5880 <entry><literal>set</literal></entry>
5887 <sect3 id="ccl.api">
5888 <title>CCL API</title>
5890 All public definitions can be found in the header file
5891 <filename>ccl.h</filename>. A profile identifier is of type
5892 <literal>CCL_bibset</literal>. A profile must be created with the call
5893 to the function <function>ccl_qual_mk</function> which returns a profile
5894 handle of type <literal>CCL_bibset</literal>.
5897 To read a file containing qualifier definitions the function
5898 <function>ccl_qual_file</function> may be convenient. This function
5899 takes an already opened <literal>FILE</literal> handle pointer as
5900 argument along with a <literal>CCL_bibset</literal> handle.
5903 To parse a simple string with a FIND query use the function
5906 struct ccl_rpn_node *ccl_find_str(CCL_bibset bibset, const char *str,
5907 int *error, int *pos);
5910 which takes the CCL profile (<literal>bibset</literal>) and query
5911 (<literal>str</literal>) as input. Upon successful completion the RPN
5912 tree is returned. If an error occur, such as a syntax error, the integer
5913 pointed to by <literal>error</literal> holds the error code and
5914 <literal>pos</literal> holds the offset inside query string in which
5918 An English representation of the error may be obtained by calling
5919 the <literal>ccl_err_msg</literal> function. The error codes are
5920 listed in <filename>ccl.h</filename>.
5923 To convert the CCL RPN tree (type
5924 <literal>struct ccl_rpn_node *</literal>)
5925 to the Z_RPNQuery of YAZ the function <function>ccl_rpn_query</function>
5926 must be used. This function which is part of YAZ is implemented in
5927 <filename>yaz-ccl.c</filename>.
5928 After calling this function the CCL RPN tree is probably no longer
5929 needed. The <literal>ccl_rpn_delete</literal> destroys the CCL RPN tree.
5932 A CCL profile may be destroyed by calling the
5933 <function>ccl_qual_rm</function> function.
5936 The token names for the CCL operators may be changed by setting the
5937 globals (all type <literal>char *</literal>)
5938 <literal>ccl_token_and</literal>, <literal>ccl_token_or</literal>,
5939 <literal>ccl_token_not</literal> and <literal>ccl_token_set</literal>.
5940 An operator may have aliases, i.e. there may be more than one name for
5941 the operator. To do this, separate each alias with a space character.
5948 <ulink url="&url.cql;">CQL</ulink>
5949 - Common Query Language - was defined for the
5950 <ulink url="&url.sru;">SRU</ulink> protocol.
5951 In many ways CQL has a similar syntax to CCL.
5952 The objective of CQL is different. Where CCL aims to be
5953 an end-user language, CQL is <emphasis>the</emphasis> protocol
5954 query language for SRU.
5958 If you are new to CQL, read the
5959 <ulink url="&url.cql.intro;">Gentle Introduction</ulink>.
5963 The CQL parser in &yaz; provides the following:
5967 It parses and validates a CQL query.
5972 It generates a C structure that allows you to convert
5973 a CQL query to some other query language, such as SQL.
5978 The parser converts a valid CQL query to PQF, thus providing a
5979 way to use CQL for both SRU servers and Z39.50 targets at the
5985 The parser converts CQL to XCQL.
5986 XCQL is an XML representation of CQL.
5987 XCQL is part of the SRU specification. However, since SRU
5988 supports CQL only, we don't expect XCQL to be widely used.
5989 Furthermore, CQL has the advantage over XCQL that it is
5995 <sect3 id="cql.parsing">
5996 <title>CQL parsing</title>
5998 A CQL parser is represented by the <literal>CQL_parser</literal>
5999 handle. Its contents should be considered &yaz; internal (private).
6001 #include <yaz/cql.h>
6003 typedef struct cql_parser *CQL_parser;
6005 CQL_parser cql_parser_create(void);
6006 void cql_parser_destroy(CQL_parser cp);
6008 A parser is created by <function>cql_parser_create</function> and
6009 is destroyed by <function>cql_parser_destroy</function>.
6012 To parse a CQL query string, the following function
6015 int cql_parser_string(CQL_parser cp, const char *str);
6017 A CQL query is parsed by the <function>cql_parser_string</function>
6018 which takes a query <parameter>str</parameter>.
6019 If the query was valid (no syntax errors), then zero is returned;
6020 otherwise -1 is returned to indicate a syntax error.
6024 int cql_parser_stream(CQL_parser cp,
6025 int (*getbyte)(void *client_data),
6026 void (*ungetbyte)(int b, void *client_data),
6029 int cql_parser_stdio(CQL_parser cp, FILE *f);
6031 The functions <function>cql_parser_stream</function> and
6032 <function>cql_parser_stdio</function> parses a CQL query
6033 - just like <function>cql_parser_string</function>.
6034 The only difference is that the CQL query can be
6035 fed to the parser in different ways.
6036 The <function>cql_parser_stream</function> uses a generic
6037 byte stream as input. The <function>cql_parser_stdio</function>
6038 uses a <literal>FILE</literal> handle which is opened for reading.
6041 <sect3 id="cql.tree">
6042 <title>CQL tree</title>
6044 The the query string is valid, the CQL parser
6045 generates a tree representing the structure of the
6050 struct cql_node *cql_parser_result(CQL_parser cp);
6052 <function>cql_parser_result</function> returns the
6053 a pointer to the root node of the resulting tree.
6056 Each node in a CQL tree is represented by a
6057 <literal>struct cql_node</literal>.
6058 It is defined as follows:
6060 #define CQL_NODE_ST 1
6061 #define CQL_NODE_BOOL 2
6062 #define CQL_NODE_SORT 3
6072 struct cql_node *modifiers;
6076 struct cql_node *left;
6077 struct cql_node *right;
6078 struct cql_node *modifiers;
6082 struct cql_node *next;
6083 struct cql_node *modifiers;
6084 struct cql_node *search;
6089 There are three node types: search term (ST), boolean (BOOL)
6091 A modifier is treated as a search term too.
6094 The search term node has five members:
6098 <literal>index</literal>: index for search term.
6099 If an index is unspecified for a search term,
6100 <literal>index</literal> will be NULL.
6105 <literal>index_uri</literal>: index URi for search term
6106 or NULL if none could be resolved for the index.
6111 <literal>term</literal>: the search term itself.
6116 <literal>relation</literal>: relation for search term.
6121 <literal>relation_uri</literal>: relation URI for search term.
6126 <literal>modifiers</literal>: relation modifiers for search
6127 term. The <literal>modifiers</literal> list itself of cql_nodes
6128 each of type <literal>ST</literal>.
6134 The boolean node represents <literal>and</literal>,
6135 <literal>or</literal>, <literal>not</literal> +
6140 <literal>left</literal> and <literal>right</literal>: left
6141 - and right operand respectively.
6146 <literal>modifiers</literal>: proximity arguments.
6152 The sort node represents both the SORTBY clause.
6155 <sect3 id="cql.to.pqf">
6156 <title>CQL to PQF conversion</title>
6158 Conversion to PQF (and Z39.50 RPN) is tricky by the fact
6159 that the resulting RPN depends on the Z39.50 target
6160 capabilities (combinations of supported attributes).
6161 In addition, the CQL and SRU operates on index prefixes
6162 (URI or strings), whereas the RPN uses Object Identifiers
6166 The CQL library of &yaz; defines a <literal>cql_transform_t</literal>
6167 type. It represents a particular mapping between CQL and RPN.
6168 This handle is created and destroyed by the functions:
6170 cql_transform_t cql_transform_open_FILE (FILE *f);
6171 cql_transform_t cql_transform_open_fname(const char *fname);
6172 void cql_transform_close(cql_transform_t ct);
6174 The first two functions create a tranformation handle from
6175 either an already open FILE or from a filename respectively.
6178 The handle is destroyed by <function>cql_transform_close</function>
6179 in which case no further reference of the handle is allowed.
6182 When a <literal>cql_transform_t</literal> handle has been created
6183 you can convert to RPN.
6185 int cql_transform_buf(cql_transform_t ct,
6186 struct cql_node *cn, char *out, int max);
6188 This function converts the CQL tree <literal>cn</literal>
6189 using handle <literal>ct</literal>.
6190 For the resulting PQF, you supply a buffer <literal>out</literal>
6191 which must be able to hold at at least <literal>max</literal>
6195 If conversion failed, <function>cql_transform_buf</function>
6196 returns a non-zero SRU error code; otherwise zero is returned
6197 (conversion successful). The meanings of the numeric error
6198 codes are listed in the SRU specification somewhere (no
6199 direct link anymore).
6202 If conversion fails, more information can be obtained by calling
6204 int cql_transform_error(cql_transform_t ct, char **addinfop);
6206 This function returns the most recently returned numeric
6207 error-code and sets the string-pointer at
6208 <literal>*addinfop</literal> to point to a string containing
6209 additional information about the error that occurred: for
6210 example, if the error code is 15 (``Illegal or unsupported context
6211 set''), the additional information is the name of the requested
6212 context set that was not recognised.
6215 The SRU error-codes may be translated into brief human-readable
6216 error messages using
6218 const char *cql_strerror(int code);
6222 If you wish to be able to produce a PQF result in a different
6223 way, there are two alternatives.
6225 void cql_transform_pr(cql_transform_t ct,
6226 struct cql_node *cn,
6227 void (*pr)(const char *buf, void *client_data),
6230 int cql_transform_FILE(cql_transform_t ct,
6231 struct cql_node *cn, FILE *f);
6233 The former function produces output to a user-defined
6234 output stream. The latter writes the result to an already
6235 open <literal>FILE</literal>.
6238 <sect3 id="cql.to.rpn">
6239 <title>Specification of CQL to RPN mappings</title>
6241 The file supplied to functions
6242 <function>cql_transform_open_FILE</function>,
6243 <function>cql_transform_open_fname</function> follows
6244 a structure found in many Unix utilities.
6245 It consists of mapping specifications - one per line.
6246 Lines starting with <literal>#</literal> are ignored (comments).
6249 Each line is of the form
6251 <replaceable>CQL pattern</replaceable><literal> = </literal> <replaceable> RPN equivalent</replaceable>
6255 An RPN pattern is a simple attribute list. Each attribute pair
6258 [<replaceable>set</replaceable>] <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>
6260 The attribute <replaceable>set</replaceable> is optional.
6261 The <replaceable>type</replaceable> is the attribute type,
6262 <replaceable>value</replaceable> the attribute value.
6265 The character <literal>*</literal> (asterisk) has special meaning
6266 when used in the RPN pattern.
6267 Each occurrence of <literal>*</literal> is substituted with the
6268 CQL matching name (index, relation, qualifier etc).
6269 This facility can be used to copy a CQL name verbatim to the RPN result.
6272 The following CQL patterns are recognized:
6276 <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
6280 This pattern is invoked when a CQL index, such as
6281 dc.title is converted. <replaceable>set</replaceable>
6282 and <replaceable>name</replaceable> are the context set and index
6284 Typically, the RPN specifies an equivalent use attribute.
6287 For terms not bound by an index the pattern
6288 <literal>index.cql.serverChoice</literal> is used.
6289 Here, the prefix <literal>cql</literal> is defined as
6290 <literal>http://www.loc.gov/zing/cql/cql-indexes/v1.0/</literal>.
6291 If this pattern is not defined, the mapping will fail.
6295 <literal>index.</literal><replaceable>set</replaceable><literal>.*</literal>
6296 is used when no other index pattern is matched.
6302 <literal>qualifier.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
6307 For backwards compatibility, this is recognised as a synonym of
6308 <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
6314 <literal>relation.</literal><replaceable>relation</replaceable>
6318 This pattern specifies how a CQL relation is mapped to RPN.
6319 <replaceable>pattern</replaceable> is name of relation
6320 operator. Since <literal>=</literal> is used as
6321 separator between CQL pattern and RPN, CQL relations
6322 including <literal>=</literal> cannot be
6323 used directly. To avoid a conflict, the names
6324 <literal>ge</literal>,
6325 <literal>eq</literal>,
6326 <literal>le</literal>,
6327 must be used for CQL operators, greater-than-or-equal,
6328 equal, less-than-or-equal respectively.
6329 The RPN pattern is supposed to include a relation attribute.
6332 For terms not bound by a relation, the pattern
6333 <literal>relation.scr</literal> is used. If the pattern
6334 is not defined, the mapping will fail.
6337 The special pattern, <literal>relation.*</literal> is used
6338 when no other relation pattern is matched.
6344 <literal>relationModifier.</literal><replaceable>mod</replaceable>
6348 This pattern specifies how a CQL relation modifier is mapped to RPN.
6349 The RPN pattern is usually a relation attribute.
6355 <literal>structure.</literal><replaceable>type</replaceable>
6359 This pattern specifies how a CQL structure is mapped to RPN.
6360 Note that this CQL pattern is somewhat to similar to
6361 CQL pattern <literal>relation</literal>.
6362 The <replaceable>type</replaceable> is a CQL relation.
6365 The pattern, <literal>structure.*</literal> is used
6366 when no other structure pattern is matched.
6367 Usually, the RPN equivalent specifies a structure attribute.
6373 <literal>position.</literal><replaceable>type</replaceable>
6377 This pattern specifies how the anchor (position) of
6378 CQL is mapped to RPN.
6379 The <replaceable>type</replaceable> is one
6380 of <literal>first</literal>, <literal>any</literal>,
6381 <literal>last</literal>, <literal>firstAndLast</literal>.
6384 The pattern, <literal>position.*</literal> is used
6385 when no other position pattern is matched.
6391 <literal>set.</literal><replaceable>prefix</replaceable>
6395 This specification defines a CQL context set for a given prefix.
6396 The value on the right hand side is the URI for the set -
6397 <emphasis>not</emphasis> RPN. All prefixes used in
6398 index patterns must be defined this way.
6404 <literal>set</literal>
6408 This specification defines a default CQL context set for index names.
6409 The value on the right hand side is the URI for the set.
6415 <example id="example.cql.to.rpn.mapping">
6416 <title>CQL to RPN mapping file</title>
6418 This simple file defines two context sets, three indexes and three
6419 relations, a position pattern and a default structure.
6421 <programlisting><![CDATA[
6422 set.cql = http://www.loc.gov/zing/cql/context-sets/cql/v1.1/
6423 set.dc = http://www.loc.gov/zing/cql/dc-indexes/v1.0/
6425 index.cql.serverChoice = 1=1016
6426 index.dc.title = 1=4
6427 index.dc.subject = 1=21
6433 position.any = 3=3 6=1
6439 With the mappings above, the CQL query
6443 is converted to the PQF:
6445 @attr 1=1016 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "computer"
6447 by rules <literal>index.cql.serverChoice</literal>,
6448 <literal>relation.scr</literal>, <literal>structure.*</literal>,
6449 <literal>position.any</literal>.
6456 is rejected, since <literal>position.right</literal> is
6462 >my = "http://www.loc.gov/zing/cql/dc-indexes/v1.0/" my.title = x
6466 @attr 1=4 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "x"
6470 <example id="example.cql.to.rpn.string">
6471 <title>CQL to RPN string attributes</title>
6473 In this example we allow any index to be passed to RPN as
6476 <programlisting><![CDATA[
6477 # Identifiers for prefixes used in this file. (index.*)
6478 set.cql = info:srw/cql-context-set/1/cql-v1.1
6479 set.rpn = http://bogus/rpn
6480 set = http://bogus/rpn
6482 # The default index when none is specified by the query
6483 index.cql.serverChoice = 1=any
6492 The <literal>http://bogus/rpn</literal> context set is also the default
6493 so we can make queries such as
6497 which is converted to
6499 @attr 2=3 @attr 4=1 @attr 3=3 @attr 1=title "a"
6503 <example id="example.cql.to.rpn.bathprofile">
6504 <title>CQL to RPN using Bath Profile</title>
6506 The file <filename>etc/pqf.properties</filename> has mappings from
6507 the Bath Profile and Dublin Core to RPN.
6508 If YAZ is installed as a package it's usually located
6509 in <filename>/usr/share/yaz/etc</filename> and part of the
6510 development package, such as <literal>libyaz-dev</literal>.
6514 <sect3 id="cql.xcql">
6515 <title>CQL to XCQL conversion</title>
6517 Conversion from CQL to XCQL is trivial and does not
6518 require a mapping to be defined.
6519 There three functions to choose from depending on the
6520 way you wish to store the resulting output (XML buffer
6523 int cql_to_xml_buf(struct cql_node *cn, char *out, int max);
6524 void cql_to_xml(struct cql_node *cn,
6525 void (*pr)(const char *buf, void *client_data),
6527 void cql_to_xml_stdio(struct cql_node *cn, FILE *f);
6529 Function <function>cql_to_xml_buf</function> converts
6530 to XCQL and stores result in a user supplied buffer of a given
6534 <function>cql_to_xml</function> writes the result in
6535 a user defined output stream.
6536 <function>cql_to_xml_stdio</function> writes to a
6540 <sect3 id="rpn.to.cql">
6541 <title>PQF to CQL conversion</title>
6543 Conversion from PQF to CQL is offered by the two functions shown
6544 below. The former uses a generic stream for result. The latter
6545 puts result in a WRBUF (string container).
6547 #include <yaz/rpn2cql.h>
6549 int cql_transform_rpn2cql_stream(cql_transform_t ct,
6550 void (*pr)(const char *buf, void *client_data),
6554 int cql_transform_rpn2cql_wrbuf(cql_transform_t ct,
6558 The configuration is the same as used in CQL to PQF conversions.
6563 <sect1 id="tools.oid">
6564 <title>Object Identifiers</title>
6566 The basic YAZ representation of an OID is an array of integers,
6567 terminated with the value -1. This integer is of type
6568 <literal>Odr_oid</literal>.
6571 Fundamental OID operations and the type <literal>Odr_oid</literal>
6572 are defined in <filename>yaz/oid_util.h</filename>.
6575 An OID can either be declared as a automatic variable or it can
6576 allocated using the memory utilities or ODR/NMEM. It's
6577 guaranteed that an OID can fit in <literal>OID_SIZE</literal> integers.
6579 <example id="tools.oid.bib1.1"><title>Create OID on stack</title>
6581 We can create an OID for the Bib-1 attribute set with:
6583 Odr_oid bib1[OID_SIZE];
6595 And OID may also be filled from a string-based representation using
6596 dots (.). This is achieved by function
6598 int oid_dotstring_to_oid(const char *name, Odr_oid *oid);
6600 This functions returns 0 if name could be converted; -1 otherwise.
6602 <example id="tools.oid.bib1.2"><title>Using oid_oiddotstring_to_oid</title>
6604 We can fill the Bib-1 attribute set OID easier with:
6606 Odr_oid bib1[OID_SIZE];
6607 oid_oiddotstring_to_oid("1.2.840.10003.3.1", bib1);
6612 We can also allocate an OID dynamically on a ODR stream with:
6614 Odr_oid *odr_getoidbystr(ODR o, const char *str);
6616 This creates an OID from string-based representation using dots.
6617 This function take an &odr; stream as parameter. This stream is used to
6618 allocate memory for the data elements, which is released on a
6619 subsequent call to <function>odr_reset()</function> on that stream.
6621 <example id="tools.oid.bib1.3">
6622 <title>Using odr_getoidbystr</title>
6624 We can create a OID for the Bib-1 attribute set with:
6626 Odr_oid *bib1 = odr_getoidbystr(odr, "1.2.840.10003.3.1");
6633 char *oid_oid_to_dotstring(const Odr_oid *oid, char *oidbuf)
6635 does the reverse of <function>oid_oiddotstring_to_oid</function>. It
6636 converts an OID to the string-based representation using dots.
6637 The supplied char buffer <literal>oidbuf</literal> holds the resulting
6638 string and must be at least <literal>OID_STR_MAX</literal> in size.
6641 OIDs can be copied with <function>oid_oidcpy</function> which takes
6642 two OID lists as arguments. Alternativly, an OID copy can be allocated
6643 on a ODR stream with:
6645 Odr_oid *odr_oiddup(ODR odr, const Odr_oid *o);
6649 OIDs can be compared with <function>oid_oidcmp</function> which returns
6650 zero if the two OIDs provided are identical; non-zero otherwise.
6652 <sect2 id="tools.oid.database">
6653 <title>OID database</title>
6655 From YAZ version 3 and later, the oident system has been replaced
6656 by an OID database. OID database is a misnomer .. the old odient
6657 system was also a database.
6660 The OID database is really just a map between named Object Identifiers
6661 (string) and their OID raw equivalents. Most operations either
6662 convert from string to OID or other way around.
6665 Unfortunately, whenever we supply a string we must also specify the
6666 <emphasis>OID class</emphasis>. The class is necessary because some
6667 strings correspond to multiple OIDs. An example of such a string is
6668 <literal>Bib-1</literal> which may either be an attribute-set
6669 or a diagnostic-set.
6672 Applications using the YAZ database should include
6673 <filename>yaz/oid_db.h</filename>.
6676 A YAZ database handle is of type <literal>yaz_oid_db_t</literal>.
6677 Actually that's a pointer. You need not think deal with that.
6678 YAZ has a built-in database which can be considered "constant" for
6680 We can get hold that by using function <function>yaz_oid_std</function>.
6683 All functions with prefix <function>yaz_string_to_oid</function>
6684 converts from class + string to OID. We have variants of this
6685 operation due to different memory allocation strategies.
6688 All functions with prefix
6689 <function>yaz_oid_to_string</function> converts from OID to string
6692 <example id="tools.oid.bib1.4">
6693 <title>Create OID with YAZ DB</title>
6695 We can create an OID for the Bib-1 attribute set on the ODR stream
6699 yaz_string_to_oid_odr(yaz_oid_std(), CLASS_ATTSET, "Bib-1", odr);
6701 This is more complex than using <function>odr_getoidbystr</function>.
6702 You would only use <function>yaz_string_to_oid_odr</function> when the
6703 string (here Bib-1) is supplied by a user or configuration.
6707 <sect2 id="tools.oid.std">
6708 <title>Standard OIDs</title>
6710 All the object identifers in the standard OID database as returned
6711 by <function>yaz_oid_std</function> can referenced directly in a
6712 program as a constant OID.
6713 Each constant OID is prefixed with <literal>yaz_oid_</literal> -
6714 followed by OID class (lowercase) - then by OID name (normalized and
6718 See <xref linkend="list-oids"/> for list of all object identifiers
6720 These are declared in <filename>yaz/oid_std.h</filename> but are
6721 included by <filename>yaz/oid_db.h</filename> as well.
6723 <example id="tools.oid.bib1.5">
6724 <title>Use a built-in OID</title>
6726 We can allocate our own OID filled with the constant OID for
6729 Odr_oid *bib1 = odr_oiddup(o, yaz_oid_attset_bib1);
6735 <sect1 id="tools.nmem">
6736 <title>Nibble Memory</title>
6738 Sometimes when you need to allocate and construct a large,
6739 interconnected complex of structures, it can be a bit of a pain to
6740 release the associated memory again. For the structures describing the
6741 Z39.50 PDUs and related structures, it is convenient to use the
6742 memory-management system of the &odr; subsystem (see
6743 <xref linkend="odr.use"/>). However, in some circumstances
6744 where you might otherwise benefit from using a simple nibble memory
6745 management system, it may be impractical to use
6746 <function>odr_malloc()</function> and <function>odr_reset()</function>.
6747 For this purpose, the memory manager which also supports the &odr;
6748 streams is made available in the NMEM module. The external interface
6749 to this module is given in the <filename>nmem.h</filename> file.
6752 The following prototypes are given:
6755 NMEM nmem_create(void);
6756 void nmem_destroy(NMEM n);
6757 void *nmem_malloc(NMEM n, size_t size);
6758 void nmem_reset(NMEM n);
6759 size_t nmem_total(NMEM n);
6760 void nmem_init(void);
6761 void nmem_exit(void);
6764 The <function>nmem_create()</function> function returns a pointer to a
6765 memory control handle, which can be released again by
6766 <function>nmem_destroy()</function> when no longer needed.
6767 The function <function>nmem_malloc()</function> allocates a block of
6768 memory of the requested size. A call to <function>nmem_reset()</function>
6769 or <function>nmem_destroy()</function> will release all memory allocated
6770 on the handle since it was created (or since the last call to
6771 <function>nmem_reset()</function>. The function
6772 <function>nmem_total()</function> returns the number of bytes currently
6773 allocated on the handle.
6776 The nibble memory pool is shared amongst threads. POSIX
6777 mutex'es and WIN32 Critical sections are introduced to keep the
6778 module thread safe. Function <function>nmem_init()</function>
6779 initializes the nibble memory library and it is called automatically
6780 the first time the <literal>YAZ.DLL</literal> is loaded. &yaz; uses
6781 function <function>DllMain</function> to achieve this. You should
6782 <emphasis>not</emphasis> call <function>nmem_init</function> or
6783 <function>nmem_exit</function> unless you're absolute sure what
6784 you're doing. Note that in previous &yaz; versions you'd have to call
6785 <function>nmem_init</function> yourself.
6788 <sect1 id="tools.log">
6791 &yaz; has evolved a fairly complex log system which should be useful both
6792 for debugging &yaz; itself, debugging applications that use &yaz;, and for
6793 production use of those applications.
6796 The log functions are declared in header <filename>yaz/log.h</filename>
6797 and implemented in <filename>src/log.c</filename>.
6798 Due to name clash with syslog and some math utilities the logging
6799 interface has been modified as of YAZ 2.0.29. The obsolete interface
6800 is still available if in header file <filename>yaz/log.h</filename>.
6801 The key points of the interface are:
6804 void yaz_log(int level, const char *fmt, ...)
6805 void yaz_log_init(int level, const char *prefix, const char *name);
6806 void yaz_log_init_file(const char *fname);
6807 void yaz_log_init_level(int level);
6808 void yaz_log_init_prefix(const char *prefix);
6809 void yaz_log_time_format(const char *fmt);
6810 void yaz_log_init_max_size(int mx);
6812 int yaz_log_mask_str(const char *str);
6813 int yaz_log_module_level(const char *name);
6816 The reason for the whole log module is the <function>yaz_log</function>
6817 function. It takes a bitmask indicating the log levels, a
6818 <literal>printf</literal>-like format string, and a variable number of
6822 The <literal>log level</literal> is a bit mask, that says on which level(s)
6823 the log entry should be made, and optionally set some behaviour of the
6824 logging. In the most simple cases, it can be one of <literal>YLOG_FATAL,
6825 YLOG_DEBUG, YLOG_WARN, YLOG_LOG</literal>. Those can be combined with bits
6826 that modify the way the log entry is written:<literal>YLOG_ERRNO,
6827 YLOG_NOTIME, YLOG_FLUSH</literal>.
6828 Most of the rest of the bits are deprecated, and should not be used. Use
6829 the dynamic log levels instead.
6832 Applications that use &yaz;, should not use the LOG_LOG for ordinary
6833 messages, but should make use of the dynamic loglevel system. This consists
6834 of two parts, defining the loglevel and checking it.
6837 To define the log levels, the (main) program should pass a string to
6838 <function>yaz_log_mask_str</function> to define which log levels are to be
6839 logged. This string should be a comma-separated list of log level names,
6840 and can contain both hard-coded names and dynamic ones. The log level
6841 calculation starts with <literal>YLOG_DEFAULT_LEVEL</literal> and adds a bit
6842 for each word it meets, unless the word starts with a '-', in which case it
6843 clears the bit. If the string <literal>'none'</literal> is found,
6844 all bits are cleared. Typically this string comes from the command-line,
6845 often identified by <literal>-v</literal>. The
6846 <function>yaz_log_mask_str</function> returns a log level that should be
6847 passed to <function>yaz_log_init_level</function> for it to take effect.
6850 Each module should check what log bits it should be used, by calling
6851 <function>yaz_log_module_level</function> with a suitable name for the
6852 module. The name is cleared from a preceding path and an extension, if any,
6853 so it is quite possible to use <literal>__FILE__</literal> for it. If the
6854 name has been passed to <function>yaz_log_mask_str</function>, the routine
6855 returns a non-zero bitmask, which should then be used in consequent calls
6856 to yaz_log. (It can also be tested, so as to avoid unnecessary calls to
6857 yaz_log, in time-critical places, or when the log entry would take time
6861 Yaz uses the following dynamic log levels:
6862 <literal>server, session, request, requestdetail</literal> for the server
6864 <literal>zoom</literal> for the zoom client api.
6865 <literal>ztest</literal> for the simple test server.
6866 <literal>malloc, nmem, odr, eventl</literal> for internal
6867 debugging of yaz itself.
6868 Of course, any program using yaz is welcome to define as many new
6872 By default the log is written to stderr, but this can be changed by a call
6873 to <function>yaz_log_init_file</function> or
6874 <function>yaz_log_init</function>. If the log is directed to a file, the
6875 file size is checked at every write, and if it exceeds the limit given in
6876 <function>yaz_log_init_max_size</function>, the log is rotated. The
6877 rotation keeps one old version (with a <literal>.1</literal> appended to
6878 the name). The size defaults to 1GB. Setting it to zero will disable the
6882 A typical yaz-log looks like this
6883 13:23:14-23/11 yaz-ztest(1) [session] Starting session from tcp:127.0.0.1 (pid=30968)
6884 13:23:14-23/11 yaz-ztest(1) [request] Init from 'YAZ' (81) (ver 2.0.28) OK
6885 13:23:17-23/11 yaz-ztest(1) [request] Search Z: @attrset Bib-1 foo OK:7 hits
6886 13:23:22-23/11 yaz-ztest(1) [request] Present: [1] 2+2 OK 2 records returned
6887 13:24:13-23/11 yaz-ztest(1) [request] Close OK
6890 The log entries start with a time stamp. This can be omitted by setting the
6891 <literal>YLOG_NOTIME</literal> bit in the loglevel. This way automatic tests
6892 can be hoped to produce identical log files, that are easy to diff. The
6893 format of the time stamp can be set with
6894 <function>yaz_log_time_format</function>, which takes a format string just
6895 like <function>strftime</function>.
6898 Next in a log line comes the prefix, often the name of the program. For
6899 yaz-based servers, it can also contain the session number. Then
6900 comes one or more logbits in square brackets, depending on the logging
6901 level set by <function>yaz_log_init_level</function> and the loglevel
6902 passed to <function>yaz_log_init_level</function>. Finally comes the format
6903 string and additional values passed to <function>yaz_log</function>
6906 The log level <literal>YLOG_LOGLVL</literal>, enabled by the string
6907 <literal>loglevel</literal>, will log all the log-level affecting
6908 operations. This can come in handy if you need to know what other log
6909 levels would be useful. Grep the logfile for <literal>[loglevel]</literal>.
6912 The log system is almost independent of the rest of &yaz;, the only
6913 important dependence is of <filename>nmem</filename>, and that only for
6914 using the semaphore definition there.
6917 The dynamic log levels and log rotation were introduced in &yaz; 2.0.28. At
6918 the same time, the log bit names were changed from
6919 <literal>LOG_something</literal> to <literal>YLOG_something</literal>,
6920 to avoid collision with <filename>syslog.h</filename>.
6926 YAZ provides a fast utility for working with MARC records.
6927 Early versions of the MARC utility only allowed decoding of ISO2709.
6928 Today the utility may both encode - and decode to a varity of formats.
6931 #include <yaz/marcdisp.h>
6933 /* create handler */
6934 yaz_marc_t yaz_marc_create(void);
6936 void yaz_marc_destroy(yaz_marc_t mt);
6938 /* set XML mode YAZ_MARC_LINE, YAZ_MARC_SIMPLEXML, ... */
6939 void yaz_marc_xml(yaz_marc_t mt, int xmlmode);
6940 #define YAZ_MARC_LINE 0
6941 #define YAZ_MARC_SIMPLEXML 1
6942 #define YAZ_MARC_OAIMARC 2
6943 #define YAZ_MARC_MARCXML 3
6944 #define YAZ_MARC_ISO2709 4
6945 #define YAZ_MARC_XCHANGE 5
6946 #define YAZ_MARC_CHECK 6
6947 #define YAZ_MARC_TURBOMARC 7
6948 #define YAZ_MARC_JSON 8
6950 /* supply iconv handle for character set conversion .. */
6951 void yaz_marc_iconv(yaz_marc_t mt, yaz_iconv_t cd);
6953 /* set debug level, 0=none, 1=more, 2=even more, .. */
6954 void yaz_marc_debug(yaz_marc_t mt, int level);
6956 /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
6957 On success, result in *result with size *rsize. */
6958 int yaz_marc_decode_buf(yaz_marc_t mt, const char *buf, int bsize,
6959 const char **result, size_t *rsize);
6961 /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
6962 On success, result in WRBUF */
6963 int yaz_marc_decode_wrbuf(yaz_marc_t mt, const char *buf,
6964 int bsize, WRBUF wrbuf);
6969 The synopsis is just a basic subset of all functionality. Refer
6970 to the actual header file <filename>marcdisp.h</filename> for
6975 A MARC conversion handle must be created by using
6976 <function>yaz_marc_create</function> and destroyed
6977 by calling <function>yaz_marc_destroy</function>.
6980 All other function operate on a <literal>yaz_marc_t</literal> handle.
6981 The output is specified by a call to <function>yaz_marc_xml</function>.
6982 The <literal>xmlmode</literal> must be one of
6985 <term>YAZ_MARC_LINE</term>
6988 A simple line-by-line format suitable for display but not
6989 recommend for further (machine) processing.
6994 <term>YAZ_MARC_MARCXML</term>
6997 <ulink url="&url.marcxml;">MARCXML</ulink>.
7002 <term>YAZ_MARC_ISO2709</term>
7005 ISO2709 (sometimes just referred to as "MARC").
7010 <term>YAZ_MARC_XCHANGE</term>
7013 <ulink url="&url.marcxchange;">MarcXchange</ulink>.
7018 <term>YAZ_MARC_CHECK</term>
7021 Pseudo format for validation only. Does not generate
7022 any real output except diagnostics.
7027 <term>YAZ_MARC_TURBOMARC</term>
7030 XML format with same semantics as MARCXML but more compact
7031 and geared towards fast processing with XSLT. Refer to
7032 <xref linkend="tools.turbomarc"/> for more information.
7037 <term>YAZ_MARC_JSON</term>
7040 <ulink url="&url.marc_in_json;">MARC-in_JSON</ulink> format.
7047 The actual conversion functions are
7048 <function>yaz_marc_decode_buf</function> and
7049 <function>yaz_marc_decode_wrbuf</function> which decodes and encodes
7050 a MARC record. The former function operates on simple buffers, the
7051 stores the resulting record in a WRBUF handle (WRBUF is a simple string
7054 <example id="example.marc.display">
7055 <title>Display of MARC record</title>
7057 The following program snippet illustrates how the MARC API may
7058 be used to convert a MARC record to the line-by-line format:
7059 <programlisting><![CDATA[
7060 void print_marc(const char *marc_buf, int marc_buf_size)
7062 char *result; /* for result buf */
7063 size_t result_len; /* for size of result */
7064 yaz_marc_t mt = yaz_marc_create();
7065 yaz_marc_xml(mt, YAZ_MARC_LINE);
7066 yaz_marc_decode_buf(mt, marc_buf, marc_buf_size,
7067 &result, &result_len);
7068 fwrite(result, result_len, 1, stdout);
7069 yaz_marc_destroy(mt); /* note that result is now freed... */
7075 <sect2 id="tools.turbomarc">
7076 <title>TurboMARC</title>
7078 TurboMARC is yet another XML encoding of a MARC record. The format
7079 was designed for fast processing with XSLT.
7083 Pazpar2 uses XSLT to convert an XML encoded MARC record to an internal
7084 representation. This conversion mostly check the tag of a MARC field
7085 to determine the basic rules in the conversion. This check is
7086 costly when that is tag is encoded as an attribute in MARCXML.
7087 By having the tag value as the element instead, makes processing
7088 many times faster (at least for Libxslt).
7091 TurboMARC is encoded as follows:
7095 Record elements is part of namespace
7096 "<literal>http://www.indexdata.com/turbomarc</literal>".
7101 A record is enclosed in element <literal>r</literal>.
7106 A collection of records is enclosed in element
7107 <literal>collection</literal>.
7112 The leader is encoded as element <literal>l</literal> with the
7113 leader content as its (text) value.
7118 A control field is encoded as element <literal>c</literal> concatenated
7119 with the tag value of the control field if the tag value
7120 matches the regular expression <literal>[a-zA-Z0-9]*</literal>.
7121 If the tag value do not match the regular expression
7122 <literal>[a-zA-Z0-9]*</literal> the control field is encoded
7123 as element <literal>c</literal> and attribute <literal>code</literal>
7124 will hold the tag value.
7125 This rule ensure that in the rare cases where a tag value might
7126 result in a non-wellformed XML YAZ encode it as a coded attribute
7130 The control field content is the the text value of this element.
7131 Indicators are encoded as attribute names
7132 <literal>i1</literal>, <literal>i2</literal>, etc.. and
7133 corresponding values for each indicator.
7138 A data field is encoded as element <literal>d</literal> concatenated
7139 with the tag value of the data field or using the attribute
7140 <literal>code</literal> as described in the rules for control fields.
7141 The children of the data field element is subfield elements.
7142 Each subfield element is encoded as <literal>s</literal>
7143 concatenated with the sub field code.
7144 The text of the subfield element is the contents of the subfield.
7145 Indicators are encoded as attributes for the data field element similar
7146 to the encoding for control fields.
7153 <sect1 id="tools.retrieval">
7154 <title>Retrieval Facility</title>
7156 YAZ version 2.1.20 or later includes a Retrieval facility tool
7157 which allows a SRU/Z39.50 to describe itself and perform record
7158 conversions. The idea is the following:
7162 An SRU/Z39.50 client sends a retrieval request which includes
7163 a combination of the following parameters: syntax (format),
7164 schema (or element set name).
7169 The retrieval facility is invoked with parameters in a
7170 server/proxy. The retrieval facility matches the parameters a set of
7171 "supported" retrieval types.
7172 If there is no match, the retrieval signals an error
7173 (syntax and / or schema not supported).
7178 For a successful match, the backend is invoked with the same
7179 or altered retrieval parameters (syntax, schema). If
7180 a record is received from the backend, it is converted to the
7181 frontend name / syntax.
7186 The resulting record is sent back the client and tagged with
7187 the frontend syntax / schema.
7193 The Retrieval facility is driven by an XML configuration. The
7194 configuration is neither Z39.50 ZeeRex or SRU ZeeRex. But it
7195 should be easy to generate both of them from the XML configuration.
7196 (unfortunately the two versions
7197 of ZeeRex differ substantially in this regard).
7199 <sect2 id="tools.retrieval.format">
7200 <title>Retrieval XML format</title>
7202 All elements should be covered by namespace
7203 <literal>http://indexdata.com/yaz</literal> .
7204 The root element node must be <literal>retrievalinfo</literal>.
7207 The <literal>retrievalinfo</literal> must include one or
7208 more <literal>retrieval</literal> elements. Each
7209 <literal>retrieval</literal> defines specific combination of
7210 syntax, name and identifier supported by this retrieval service.
7213 The <literal>retrieval</literal> element may include any of the
7214 following attributes:
7216 <varlistentry><term><literal>syntax</literal> (REQUIRED)</term>
7219 Defines the record syntax. Possible values is any
7220 of the names defined in YAZ' OID database or a raw
7225 <varlistentry><term><literal>name</literal> (OPTIONAL)</term>
7228 Defines the name of the retrieval format. This can be
7229 any string. For SRU, the value, is equivalent to schema (short-hand);
7230 for Z39.50 it's equivalent to simple element set name.
7231 For YAZ 3.0.24 and later this name may be specified as a glob
7232 expression with operators
7233 <literal>*</literal> and <literal>?</literal>.
7237 <varlistentry><term><literal>identifier</literal> (OPTIONAL)</term>
7240 Defines the URI schema name of the retrieval format. This can be
7241 any string. For SRU, the value, is equivalent to URI schema.
7242 For Z39.50, there is no equivalent.
7249 The <literal>retrieval</literal> may include one
7250 <literal>backend</literal> element. If a <literal>backend</literal>
7251 element is given, it specifies how the records are retrieved by
7252 some backend and how the records are converted from the backend to
7256 The attributes, <literal>name</literal> and <literal>syntax</literal>
7257 may be specified for the <literal>backend</literal> element. These
7258 semantics of these attributes is equivalent to those for the
7259 <literal>retrieval</literal>. However, these values are passed to
7263 The <literal>backend</literal> element may includes one or more
7264 conversion instructions (as children elements). The supported
7267 <varlistentry><term><literal>marc</literal></term>
7270 The <literal>marc</literal> element specifies a conversion
7271 to - and from ISO2709 encoded MARC and
7272 <ulink url="&url.marcxml;">&acro.marcxml;</ulink>/MarcXchange.
7273 The following attributes may be specified:
7276 <term><literal>inputformat</literal> (REQUIRED)</term>
7279 Format of input. Supported values are
7280 <literal>marc</literal> (for ISO2709), <literal>xml</literal>
7281 (MARCXML/MarcXchange) and <literal>json</literal>
7282 (<ulink url="&url.marc_in_json;">MARC-in_JSON</ulink>).
7287 <term><literal>outputformat</literal> (REQUIRED)</term>
7290 Format of output. Supported values are
7291 <literal>line</literal> (MARC line format);
7292 <literal>marcxml</literal> (for MARCXML),
7293 <literal>marc</literal> (ISO2709),
7294 <literal>marcxhcange</literal> (for MarcXchange),
7295 or <literal>json</literal>
7296 (<ulink url="&url.marc_in_json;">MARC-in_JSON </ulink>).
7301 <term><literal>inputcharset</literal> (OPTIONAL)</term>
7304 Encoding of input. For XML input formats, this need not
7305 be given, but for ISO2709 based inputformats, this should
7306 be set to the encoding used. For MARC21 records, a common
7307 inputcharset value would be <literal>marc-8</literal>.
7312 <term><literal>outputcharset</literal> (OPTIONAL)</term>
7315 Encoding of output. If outputformat is XML based, it is
7316 strongly recommened to use <literal>utf-8</literal>.
7325 <term><literal>xslt</literal></term>
7328 The <literal>xslt</literal> element specifies a conversion
7329 via &acro.xslt;. The following attributes may be specified:
7331 <varlistentry><term><literal>stylesheet</literal> (REQUIRED)</term>
7343 <term><literal>solrmarc</literal></term>
7346 The <literal>solrmarc</literal> decodes solrmarc records.
7347 It assumes that the input is pure solrmarc text (no escaping)
7348 and will convert all sequences of the form #XX; to a single
7349 character of the hexadecimal value as given by XX. The output,
7350 presumably, is a valid ISO2709 buffer.
7353 This conversion is available in YAZ 5.0.21 and later.
7360 <sect2 id="tools.retrieval.examples">
7361 <title>Retrieval Facility Examples</title>
7362 <example id="tools.retrieval.marc21">
7363 <title>MARC21 backend</title>
7365 A typical way to use the retrieval facility is to enable XML
7366 for servers that only supports ISO2709 encoded MARC21 records.
7368 <programlisting><![CDATA[
7370 <retrieval syntax="usmarc" name="F"/>
7371 <retrieval syntax="usmarc" name="B"/>
7372 <retrieval syntax="xml" name="marcxml"
7373 identifier="info:srw/schema/1/marcxml-v1.1">
7374 <backend syntax="usmarc" name="F">
7375 <marc inputformat="marc" outputformat="marcxml"
7376 inputcharset="marc-8"/>
7379 <retrieval syntax="xml" name="dc">
7380 <backend syntax="usmarc" name="F">
7381 <marc inputformat="marc" outputformat="marcxml"
7382 inputcharset="marc-8"/>
7383 <xslt stylesheet="MARC21slim2DC.xsl"/>
7390 This means that our frontend supports:
7394 MARC21 F(ull) records.
7399 MARC21 B(rief) records.
7409 Dublin core records.
7415 <example id="tools.retrieval.marcxml">
7416 <title>MARCXML backend</title>
7418 SRW/SRU and Solr backends returns records in XML.
7419 If they return MARCXML or MarcXchange, the retrieval module
7420 can convert those into ISO2709 formats, most commonly USMARC
7422 In this example, the backend returns MARCXML for schema="marcxml".
7424 <programlisting><![CDATA[
7426 <retrieval syntax="usmarc">
7427 <backend syntax="xml" name="marcxml">
7428 <marc inputformat="xml" outputformat="marc"
7429 outputcharset="marc-8"/>
7432 <retrieval syntax="xml" name="marcxml"
7433 identifier="info:srw/schema/1/marcxml-v1.1"/>
7434 <retrieval syntax="xml" name="dc">
7435 <backend syntax="xml" name="marcxml">
7436 <xslt stylesheet="MARC21slim2DC.xsl"/>
7443 This means that our frontend supports:
7447 MARC21 records (any element set name) in MARC-8 encoding.
7452 MARCXML records for element-set=marcxml
7457 Dublin core records for element-set=dc.
7464 <sect2 id="tools.retrieval.api">
7467 It should be easy to use the retrieval systems from applications. Refer
7469 <filename>yaz/retrieval.h</filename> and
7470 <filename>yaz/record_conv.h</filename>.
7474 <sect1 id="sorting">
7475 <title>Sorting</title>
7477 This chapter describes sorting and how it is supported in YAZ.
7478 Sorting applies to a result-set.
7480 <ulink url="http://www.loc.gov/z3950/agency/markup/05.html#3.2.7">
7481 Z39.50 sorting facility
7483 takes one or more input result-sets
7484 and one result-set as output. The most simple case is that
7485 the input-set is the same as the output-set.
7488 Z39.50 sorting has a separate APDU (service) that is, thus, performed
7489 following a search (two phases).
7492 In SRU/Solr, however, the model is different. Here, sorting is specified
7493 during the the search operation. Note, however, that SRU might
7494 perform sort as separate search, by referring to an existing result-set
7495 in the query (result-set reference).
7498 <title>Using the Z39.50 sort service</title>
7500 yaz-client and the ZOOM API supports the Z39.50 sort facility. In any
7501 case the sort sequence or sort critiera is using a string notation.
7502 This notation is a one-line notation suitable for being manually
7503 entered or generated and allows for easy logging (one liner).
7504 For the ZOOM API, the sort is specified in the call to ZOOM_query_sortby
7505 function. For yaz-client the sort is performed and specified using
7506 the sort and sort+ commands. For description of the sort criteria notation
7507 refer to the <link linkend="sortspec">sort command</link> in the
7511 The ZOOM API might choose one of several sort strategies for
7512 sorting. Refer to <xref linkend="zoom-sort-strategy"/>.
7516 <title>Type-7 sort</title>
7518 Type-7 sort is an extension to the Bib-1 based RPN query where the
7519 sort specification is embedded as an Attribute-Plus-Term.
7522 The objectives for introducing Type-7 sorting is that it allows
7523 a client to perform sorting even if it does not implement/support
7524 Z39.50 sort. Virtually all Z39.50 client software supports
7525 RPN queries. It also may improve performance because the sort
7526 critieria is specified along with the search query.
7529 The sort is triggered by the presence of type 7 and the value of type 7
7531 <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortKeySpec">
7534 The value for type 7 is 1 for ascending and 2 for descending.
7536 <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortElement">
7539 only the generic part is handled. If generic sortKey is of type
7540 sortField, then attribute type 1 is present and the value is
7541 sortField (InternationalString). If generic sortKey is of type
7542 sortAttributes, then the attributes in list is used . generic sortKey
7543 of type elementSpec is not supported.
7546 The term in the sorting Attribute-Plus-Term combo should hold
7547 an integer. The value is 0 for primary sorting criteria, 1 for second
7553 <title>Facets</title>
7555 YAZ supports facets for in Solr, SRU 2.0 and Z39.50 protocols.
7558 Like Type-1/RPN, YAZ supports a string notation for specifying
7559 facets. For the API this is performed by
7560 <function>yaz_pqf_parse_facet_list</function>.
7563 For ZOOM C the facets are given by option "facets"
7564 For yaz-client it is used for the facets command.
7567 The grammar of this specification is as follows:
7569 facet-spec ::= facet-list
7571 facet-list ::= facet-list ',' attr-spec | attr-spec
7573 attr-spec ::= attr-spec '@attr' string | '@attr' string
7576 The notation is inspired by PQF. The string following '@attr'
7577 may not include blanks and is of the form
7578 <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>,
7579 where <replaceable>type</replaceable> is an integer and
7580 <replaceable>value</replaceable> is a string or an integer.
7583 The Facets specification is not Bib-1. The following types apply:
7585 <table id="facet.attributes">
7586 <title>Facet attributes</title>
7588 <colspec colwidth="2*" colname="type"></colspec>
7589 <colspec colwidth="9*" colname="description"></colspec>
7593 <entry>Description</entry>
7600 Field-name. This is often a string, eg "Author", "Year", etc.
7606 Sort order. Value should be an integer.
7607 Value 0: count descending (frequency). Value 1: alpha ascending.
7613 Number of terms requested.
7628 <title>The ODR Module</title>
7629 <sect1 id="odr.introduction">
7630 <title>Introduction</title>
7632 &odr; is the BER-encoding/decoding subsystem of &yaz;. Care as been taken
7633 to isolate &odr; from the rest of the package - specifically from the
7634 transport interface. &odr; may be used in any context where basic
7635 ASN.1/BER representations are used.
7638 If you are only interested in writing a Z39.50 implementation based on
7639 the PDUs that are already provided with &yaz;, you only need to concern
7640 yourself with the section on managing ODR streams
7641 (<xref linkend="odr.use"/>). Only if you need to
7642 implement ASN.1 beyond that which has been provided, should you
7643 worry about the second half of the documentation
7644 (<xref linkend="odr.programming"/>).
7645 If you use one of the higher-level interfaces, you can skip this
7649 This is important, so we'll repeat it for emphasis: <emphasis>You do
7650 not need to read <xref linkend="odr.programming"/>
7651 to implement Z39.50 with &yaz;.</emphasis>
7654 If you need a part of the protocol that isn't already in &yaz;, you
7655 should contact the authors before going to work on it yourself: We
7656 might already be working on it. Conversely, if you implement a useful
7657 part of the protocol before us, we'd be happy to include it in a
7661 <sect1 id="odr.use">
7662 <title>Using ODR</title>
7663 <sect2 id="odr.streams">
7664 <title>ODR Streams</title>
7666 Conceptually, the ODR stream is the source of encoded data in the
7667 decoding mode; when encoding, it is the receptacle for the encoded
7668 data. Before you can use an ODR stream it must be allocated. This is
7669 done with the function
7672 ODR odr_createmem(int direction);
7675 The <function>odr_createmem()</function> function takes as argument one
7676 of three manifest constants: <literal>ODR_ENCODE</literal>,
7677 <literal>ODR_DECODE</literal>, or <literal>ODR_PRINT</literal>.
7678 An &odr; stream can be in only one mode - it is not possible to change
7679 its mode once it's selected. Typically, your program will allocate
7680 at least two ODR streams - one for decoding, and one for encoding.
7683 When you're done with the stream, you can use
7686 void odr_destroy(ODR o);
7689 to release the resources allocated for the stream.
7692 <sect2 id="odr.memory.management">
7693 <title id="memory">Memory Management</title>
7695 Two forms of memory management take place in the &odr; system. The first
7696 one, which has to do with allocating little bits of memory (sometimes
7697 quite large bits of memory, actually) when a protocol package is
7698 decoded, and turned into a complex of interlinked structures. This
7699 section deals with this system, and how you can use it for your own
7700 purposes. The next section deals with the memory management which is
7701 required when encoding data - to make sure that a large enough buffer is
7702 available to hold the fully encoded PDU.
7705 The &odr; module has its own memory management system, which is
7706 used whenever memory is required. Specifically, it is used to allocate
7707 space for data when decoding incoming PDUs. You can use the memory
7708 system for your own purposes, by using the function
7711 void *odr_malloc(ODR o, size_t size);
7714 You can't use the normal <function>free(2)</function> routine to free
7715 memory allocated by this function, and &odr; doesn't provide a parallel
7716 function. Instead, you can call
7719 void odr_reset(ODR o);
7722 when you are done with the
7723 memory: Everything allocated since the last call to
7724 <function>odr_reset()</function> is released.
7725 The <function>odr_reset()</function> call is also required to clear
7726 up an error condition on a stream.
7732 size_t odr_total(ODR o);
7735 returns the number of bytes allocated on the stream since the last call to
7736 <function>odr_reset()</function>.
7739 The memory subsystem of &odr; is fairly efficient at allocating and
7740 releasing little bits of memory. Rather than managing the individual,
7741 small bits of space, the system maintains a free-list of larger chunks
7742 of memory, which are handed out in small bits. This scheme is
7743 generally known as a <emphasis>nibble memory</emphasis> system.
7744 It is very useful for maintaining short-lived constructions such
7748 If you want to retain a bit of memory beyond the next call to
7749 <function>odr_reset()</function>, you can use the function
7752 ODR_MEM odr_extract_mem(ODR o);
7755 This function will give you control of the memory recently allocated
7756 on the ODR stream. The memory will live (past calls to
7757 <function>odr_reset()</function>), until you call the function
7760 void odr_release_mem(ODR_MEM p);
7763 The opaque <literal>ODR_MEM</literal> handle has no other purpose than
7764 referencing the memory block for you until you want to release it.
7767 You can use <function>odr_extract_mem()</function> repeatedly between
7768 allocating data, to retain individual control of separate chunks of data.
7771 <sect2 id="odr.encoding.and.decoding">
7772 <title>Encoding and Decoding Data</title>
7774 When encoding data, the ODR stream will write the encoded octet string
7775 in an internal buffer. To retrieve the data, use the function
7778 char *odr_getbuf(ODR o, int *len, int *size);
7781 The integer pointed to by len is set to the length of the encoded
7782 data, and a pointer to that data is returned. <literal>*size</literal>
7783 is set to the size of the buffer (unless <literal>size</literal> is null,
7784 signaling that you are not interested in the size). The next call to
7785 a primitive function using the same &odr; stream will overwrite the
7786 data, unless a different buffer has been supplied using the call
7789 void odr_setbuf(ODR o, char *buf, int len, int can_grow);
7792 which sets the encoding (or decoding) buffer used by
7793 <literal>o</literal> to <literal>buf</literal>, using the length
7794 <literal>len</literal>.
7795 Before a call to an encoding function, you can use
7796 <function>odr_setbuf()</function> to provide the stream with an encoding
7797 buffer of sufficient size (length). The <literal>can_grow</literal>
7798 parameter tells the encoding &odr; stream whether it is allowed to use
7799 <function>realloc(2)</function> to increase the size of the buffer when
7800 necessary. The default condition of a new encoding stream is equivalent
7801 to the results of calling
7804 odr_setbuf(stream, 0, 0, 1);
7807 In this case, the stream will allocate and reallocate memory as
7808 necessary. The stream reallocates memory by repeatedly doubling the
7809 size of the buffer - the result is that the buffer will typically
7810 reach its maximum, working size with only a small number of reallocation
7811 operations. The memory is freed by the stream when the latter is destroyed,
7812 unless it was assigned by the user with the <literal>can_grow</literal>
7813 parameter set to zero (in this case, you are expected to retain
7814 control of the memory yourself).
7817 To assume full control of an encoded buffer, you must first call
7818 <function>odr_getbuf()</function> to fetch the buffer and its length.
7819 Next, you should call <function>odr_setbuf()</function> to provide a
7820 different buffer (or a null pointer) to the stream. In the simplest
7821 case, you will reuse the same buffer over and over again, and you
7822 will just need to call <function>odr_getbuf()</function> after each
7823 encoding operation to get the length and address of the buffer.
7824 Note that the stream may reallocate the buffer during an encoding
7825 operation, so it is necessary to retrieve the correct address after
7826 each encoding operation.
7829 It is important to realize that the ODR stream will not release this
7830 memory when you call <function>odr_reset()</function>: It will
7831 merely update its internal pointers to prepare for the encoding of a
7833 When the stream is released by the <function>odr_destroy()</function>
7834 function, the memory given to it by <function>odr_setbuf</function> will
7835 be released <emphasis>only</emphasis> if the <literal>can_grow</literal>
7836 parameter to <function>odr_setbuf()</function> was nonzero. The
7837 <literal>can_grow</literal> parameter, in other words, is a way of
7838 signaling who is to own the buffer, you or the ODR stream. If you never call
7839 <function>odr_setbuf()</function> on your encoding stream, which is
7840 typically the case, the buffer allocated by the stream will belong to
7841 the stream by default.
7844 When you wish to decode data, you should first call
7845 <function>odr_setbuf()</function>, to tell the decoding stream
7846 where to find the encoded data, and how long the buffer is
7847 (the <literal>can_grow</literal> parameter is ignored by a decoding
7848 stream). After this, you can call the function corresponding to the
7849 data you wish to decode (eg, <function>odr_integer()</function> odr
7850 <function>z_APDU()</function>).
7852 <example id="example.odr.encoding.and.decoding.functions">
7853 <title>Encoding and decoding functions</title>
7855 int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
7857 int z_APDU(ODR o, Z_APDU **p, int optional, const char *name);
7861 If the data is absent (or doesn't match the tag corresponding to
7862 the type), the return value will be either 0 or 1 depending on the
7863 <literal>optional</literal> flag. If <literal>optional</literal>
7864 is 0 and the data is absent, an error flag will be raised in the
7865 stream, and you'll need to call <function>odr_reset()</function> before
7866 you can use the stream again. If <literal>optional</literal> is
7867 nonzero, the pointer <emphasis>pointed</emphasis> to/ by
7868 <literal>p</literal> will be set to the null value, and the function
7870 The <literal>name</literal> argument is used to pretty-print the
7871 tag in question. It may be set to <literal>NULL</literal> if
7872 pretty-printing is not desired.
7875 If the data value is found where it's expected, the pointer
7876 <emphasis>pointed to</emphasis> by the <literal>p</literal> argument
7877 will be set to point to the decoded type.
7878 The space for the type will be allocated and owned by the &odr;
7879 stream, and it will live until you call
7880 <function>odr_reset()</function> on the stream. You cannot use
7881 <function>free(2)</function> to release the memory.
7882 You can decode several data elements (by repeated calls to
7883 <function>odr_setbuf()</function> and your decoding function), and
7884 new memory will be allocated each time. When you do call
7885 <function>odr_reset()</function>, everything decoded since the
7886 last call to <function>odr_reset()</function> will be released.
7888 <example id="example.odr.encoding.of.integer">
7889 <title>Encoding and decoding of an integer</title>
7891 The use of the double indirection can be a little confusing at first
7892 (its purpose will become clear later on, hopefully),
7893 so an example is in order. We'll encode an integer value, and
7894 immediately decode it again using a different stream. A useless, but
7895 informative operation.
7897 <programlisting><![CDATA[
7898 void do_nothing_useful(Odr_int value)
7901 Odr_int *valp, *resvalp;
7905 /* allocate streams */
7906 if (!(encode = odr_createmem(ODR_ENCODE)))
7908 if (!(decode = odr_createmem(ODR_DECODE)))
7912 if (odr_integer(encode, &valp, 0, 0) == 0)
7914 printf("encoding went bad\n");
7917 bufferp = odr_getbuf(encode, &len, 0);
7918 printf("length of encoded data is %d\n", len);
7920 /* now let's decode the thing again */
7921 odr_setbuf(decode, bufferp, len, 0);
7922 if (odr_integer(decode, &resvalp, 0, 0) == 0)
7924 printf("decoding went bad\n");
7927 /* ODR_INT_PRINTF format for printf (such as %d) */
7928 printf("the value is " ODR_INT_PRINTF "\n", *resvalp);
7931 odr_destroy(encode);
7932 odr_destroy(decode);
7937 This looks like a lot of work, offhand. In practice, the &odr; streams
7938 will typically be allocated once, in the beginning of your program
7939 (or at the beginning of a new network session), and the encoding
7940 and decoding will only take place in a few, isolated places in your
7941 program, so the overhead is quite manageable.
7945 <sect2 id="odr.printing">
7946 <title>Printing</title>
7948 When an ODR stream is created of type <literal>ODR_PRINT</literal>
7949 the ODR module will print the contents of a PDU in a readable format.
7950 By default output is written to the <literal>stderr</literal> stream.
7951 This behavior can be changed, however, by calling the function
7953 odr_setprint(ODR o, FILE *file);
7955 before encoders or decoders are being invoked.
7956 It is also possible to direct the output to a buffer (of indeed
7957 another file), by using the more generic mechanism:
7959 void odr_set_stream(ODR o, void *handle,
7960 void (*stream_write)(ODR o, void *handle, int type,
7961 const char *buf, int len),
7962 void (*stream_close)(void *handle));
7964 Here the user provides an opaque handle and two handlers,
7965 <replaceable>stream_write</replaceable> for writing,
7966 and <replaceable>stream_close</replaceable> which is supposed
7967 to close/free resources associated with handle.
7968 The <replaceable>stream_close</replaceable> handler is optional and
7969 if NULL for the function is provided, it will not be invoked.
7970 The <replaceable>stream_write</replaceable> takes the ODR handle
7971 as parameter, the user defined handle, a type
7972 <literal>ODR_OCTETSTRING</literal>, <literal>ODR_VISIBLESTRING</literal>
7973 which indicates the type of contents is being written.
7976 Another utility useful for diagnostics (error handling) or as
7977 part of the printing facilities is:
7979 const char **odr_get_element_path(ODR o);
7981 which returns a list of current elements that ODR deals with at the
7982 moment. For the returned array, say <literal>ar</literal>,
7983 <literal>ar[0]</literal> is the top level element,
7984 <literal>ar[n]</literal> is the last. The last element has the
7985 property that <literal>ar[n+1] == NULL</literal>.
7987 <example id="example.odr.element.path.record">
7988 <title>Element Path for record</title>
7990 For a database record part of a PresentResponse the
7991 array returned by <function>odr_get_element</function>
7992 is <literal>presentResponse</literal>, <literal>databaseOrSurDiagnostics</literal>, <literal>?</literal>, <literal>record</literal>, <literal>?</literal>, <literal>databaseRecord</literal> . The question mark appears due to
7993 unnamed constructions.
7997 <sect2 id="odr.diagnostics">
7998 <title>Diagnostics</title>
8000 The encoding/decoding functions all return 0 when an error occurs.
8001 Until you call <function>odr_reset()</function>, you cannot use the
8002 stream again, and any function called will immediately return 0.
8005 To provide information to the programmer or administrator, the function
8008 void odr_perror(ODR o, char *message);
8011 is provided, which prints the <literal>message</literal> argument to
8012 <literal>stderr</literal> along with an error message from the stream.
8015 You can also use the function
8018 int odr_geterror(ODR o);
8021 to get the current error number from the screen. The number will be
8022 one of these constants:
8024 <table frame="top" id="odr.error.codes">
8025 <title>ODR Error codes</title>
8030 <entry>Description</entry>
8035 <entry>OMEMORY</entry><entry>Memory allocation failed.</entry>
8038 <entry>OSYSERR</entry><entry>A system- or library call has failed.
8039 The standard diagnostic variable <literal>errno</literal> should be
8040 examined to determine the actual error.</entry>
8043 <entry>OSPACE</entry><entry>No more space for encoding.
8044 This will only occur when the user has explicitly provided a
8045 buffer for an encoding stream without allowing the system to
8046 allocate more space.</entry>
8049 <entry>OREQUIRED</entry><entry>This is a common protocol error; A
8050 required data element was missing during encoding or decoding.</entry>
8053 <entry>OUNEXPECTED</entry><entry>An unexpected data element was
8054 found during decoding.</entry>
8057 <entry>OOTHER</entry><entry>Other error. This is typically an
8058 indication of misuse of the &odr; system by the programmer, and also
8059 that the diagnostic system isn't as good as it should be, yet.</entry>
8065 The character string array
8071 can be indexed by the error code to obtain a human-readable
8072 representation of the problem.
8075 <sect2 id="odr.summary.and.synopsis">
8076 <title>Summary and Synopsis</title>
8078 #include <yaz/odr.h>
8080 ODR odr_createmem(int direction);
8082 void odr_destroy(ODR o);
8084 void odr_reset(ODR o);
8086 char *odr_getbuf(ODR o, int *len, int *size);
8088 void odr_setbuf(ODR o, char *buf, int len, int can_grow);
8090 void *odr_malloc(ODR o, int size);
8092 NMEM odr_extract_mem(ODR o);
8094 int odr_geterror(ODR o);
8096 void odr_perror(ODR o, const char *message);
8098 extern char *odr_errlist[];
8102 <sect1 id="odr.programming">
8103 <title>Programming with ODR</title>
8105 The API of &odr; is designed to reflect the structure of ASN.1, rather
8106 than BER itself. Future releases may be able to represent data in
8107 other external forms.
8111 There is an ASN.1 tutorial available at
8112 <ulink url="&url.asn.1.tutorial;">this site</ulink>.
8113 This site also has standards for ASN.1 (X.680) and BER (X.690)
8114 <ulink url="&url.asn.1.standards;">online</ulink>.
8118 The ODR interface is based loosely on that of the Sun Microsystems
8120 Specifically, each function which corresponds to an ASN.1 primitive
8121 type has a dual function. Depending on the settings of the ODR
8122 stream which is supplied as a parameter, the function may be used
8123 either to encode or decode data. The functions that can be built
8124 using these primitive functions, to represent more complex data types,
8125 share this quality. The result is that you only have to enter the
8126 definition for a type once - and you have the functionality of encoding,
8127 decoding (and pretty-printing) all in one unit.
8128 The resulting C source code is quite compact, and is a pretty
8129 straightforward representation of the source ASN.1 specification.
8132 In many cases, the model of the XDR functions works quite well in this
8134 In others, it is less elegant. Most of the hassle comes from the optional
8135 SEQUENCE members which don't exist in XDR.
8137 <sect2 id="odr.primitive.asn1.types">
8138 <title>The Primitive ASN.1 Types</title>
8140 ASN.1 defines a number of primitive types (many of which correspond
8141 roughly to primitive types in structured programming languages, such as C).
8143 <sect3 id="odr.integer">
8144 <title>INTEGER</title>
8146 The &odr; function for encoding or decoding (or printing) the ASN.1
8147 INTEGER type looks like this:
8150 int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
8153 The <literal>Odr_int</literal> is just a simple integer.
8156 This form is typical of the primitive &odr; functions. They are named
8157 after the type of data that they encode or decode. They take an &odr;
8158 stream, an indirect reference to the type in question, and an
8159 <literal>optional</literal> flag (corresponding to the OPTIONAL keyword
8160 of ASN.1) as parameters. They all return an integer value of either one
8162 When you use the primitive functions to construct encoders for complex
8163 types of your own, you should follow this model as well. This
8164 ensures that your new types can be reused as elements in yet more
8168 The <literal>o</literal> parameter should obviously refer to a properly
8169 initialized &odr; stream of the right type (encoding/decoding/printing)
8170 for the operation that you wish to perform.
8173 When encoding or printing, the function first looks at
8174 <literal>* p</literal>. If <literal>* p</literal> (the pointer pointed
8175 to by <literal>p</literal>) is a null pointer, this is taken to mean that
8176 the data element is absent. If the <literal>optional</literal> parameter
8177 is nonzero, the function will return one (signifying success) without
8178 any further processing. If the <literal>optional</literal> is zero, an
8179 internal error flag is set in the &odr; stream, and the function will
8180 return 0. No further operations can be carried out on the stream without
8181 a call to the function <function>odr_reset()</function>.
8184 If <literal>*p</literal> is not a null pointer, it is expected to
8185 point to an instance of the data type. The data will be subjected to
8186 the encoding rules, and the result will be placed in the buffer held
8187 by the &odr; stream.
8190 The other ASN.1 primitives have similar functions that operate in
8194 <sect3 id="odr.boolean">
8195 <title>BOOLEAN</title>
8197 int odr_bool(ODR o, Odr_bool **p, int optional, const char *name);
8200 <sect3 id="odr.real">
8206 <sect3 id="odr.null">
8209 int odr_null(ODR o, Odr_null **p, int optional, const char *name);
8212 In this case, the value of **p is not important. If <literal>*p</literal>
8213 is different from the null pointer, the null value is present, otherwise
8217 <sect3 id="odr.octet.string">
8218 <title>OCTET STRING</title>
8220 typedef struct odr_oct
8226 int odr_octetstring(ODR o, Odr_oct **p, int optional,
8230 The <literal>buf</literal> field should point to the character array
8231 that holds the octetstring. The <literal>len</literal> field holds the
8233 The character array need not be null terminated.
8236 To make things a little easier, an alternative is given for string
8237 types that are not expected to contain embedded NULL characters (eg.
8241 int odr_cstring(ODR o, char **p, int optional, const char *name);
8244 Which encoded or decodes between OCTETSTRING representations and
8245 null-terminates C strings.
8248 Functions are provided for the derived string types, eg:
8251 int odr_visiblestring(ODR o, char **p, int optional,
8255 <sect3 id="odr.bit.string">
8256 <title>BIT STRING</title>
8258 int odr_bitstring(ODR o, Odr_bitmask **p, int optional,
8262 The opaque type <literal>Odr_bitmask</literal> is only suitable for
8263 holding relatively brief bit strings, eg. for options fields, etc.
8264 The constant <literal>ODR_BITMASK_SIZE</literal> multiplied by 8
8265 gives the maximum possible number of bits.
8268 A set of macros are provided for manipulating the
8269 <literal>Odr_bitmask</literal> type:
8272 void ODR_MASK_ZERO(Odr_bitmask *b);
8274 void ODR_MASK_SET(Odr_bitmask *b, int bitno);
8276 void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno);
8278 int ODR_MASK_GET(Odr_bitmask *b, int bitno);
8281 The functions are modeled after the manipulation functions that
8282 accompany the <literal>fd_set</literal> type used by the
8283 <function>select(2)</function> call.
8284 <literal>ODR_MASK_ZERO</literal> should always be called first on a
8285 new bitmask, to initialize the bits to zero.
8288 <sect3 id="odr.object.identifier">
8289 <title>OBJECT IDENTIFIER</title>
8291 int odr_oid(ODR o, Odr_oid **p, int optional, const char *name);
8294 The C OID representation is simply an array of integers, terminated by
8295 the value -1 (the <literal>Odr_oid</literal> type is synonymous with
8296 the <literal>short</literal> type).
8297 We suggest that you use the OID database module (see
8298 <xref linkend="tools.oid.database"/>) to handle object identifiers
8299 in your application.
8303 <sect2 id="odr.tagging.primitive.types">
8304 <title>Tagging Primitive Types</title>
8306 The simplest way of tagging a type is to use the
8307 <function>odr_implicit_tag()</function> or
8308 <function>odr_explicit_tag()</function> macros:
8311 int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag,
8312 int optional, const char *name);
8314 int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag,
8315 int optional, const char *name);
8318 To create a type derived from the integer type by implicit tagging, you
8322 MyInt ::= [210] IMPLICIT INTEGER
8325 In the &odr; system, this would be written like:
8328 int myInt(ODR o, Odr_int **p, int optional, const char *name)
8330 return odr_implicit_tag(o, odr_integer, p,
8331 ODR_CONTEXT, 210, optional, name);
8335 The function <function>myInt()</function> can then be used like any of
8336 the primitive functions provided by &odr;. Note that the behavior of
8337 <function>odr_explicit_tag()</function>
8338 and <function>odr_implicit_tag()</function> macros
8339 act exactly the same as the functions they are applied to - they
8340 respond to error conditions, etc, in the same manner - they
8341 simply have three extra parameters. The class parameter may
8342 take one of the values: <literal>ODR_CONTEXT</literal>,
8343 <literal>ODR_PRIVATE</literal>, <literal>ODR_UNIVERSAL</literal>, or
8344 <literal>/ODR_APPLICATION</literal>.
8347 <sect2 id="odr.constructed.types">
8348 <title>Constructed Types</title>
8350 Constructed types are created by combining primitive types. The
8351 &odr; system only implements the SEQUENCE and SEQUENCE OF constructions
8352 (although adding the rest of the container types should be simple
8353 enough, if the need arises).
8356 For implementing SEQUENCEs, the functions
8359 int odr_sequence_begin(ODR o, void *p, int size, const char *name);
8360 int odr_sequence_end(ODR o);
8366 The <function>odr_sequence_begin()</function> function should be
8367 called in the beginning of a function that implements a SEQUENCE type.
8368 Its parameters are the &odr; stream, a pointer (to a pointer to the type
8369 you're implementing), and the <literal>size</literal> of the type
8370 (typically a C structure). On encoding, it returns 1 if
8371 <literal>* p</literal> is a null pointer. The <literal>size</literal>
8372 parameter is ignored. On decoding, it returns 1 if the type is found in
8373 the data stream. <literal>size</literal> bytes of memory are allocated,
8374 and <literal>*p</literal> is set to point to this space.
8375 <function>odr_sequence_end()</function> is called at the end of the
8376 complex function. Assume that a type is defined like this:
8379 MySequence ::= SEQUENCE {
8381 boolval BOOLEAN OPTIONAL
8385 The corresponding &odr; encoder/decoder function and the associated data
8386 structures could be written like this:
8389 typedef struct MySequence
8395 int mySequence(ODR o, MySequence **p, int optional, const char *name)
8397 if (odr_sequence_begin(o, p, sizeof(**p), name) == 0)
8398 return optional && odr_ok(o);
8400 odr_integer(o, &(*p)->intval, 0, "intval") &&
8401 odr_bool(o, &(*p)->boolval, 1, "boolval") &&
8402 odr_sequence_end(o);
8406 Note the 1 in the call to <function>odr_bool()</function>, to mark
8407 that the sequence member is optional.
8408 If either of the member types had been tagged, the macros
8409 <function>odr_implicit_tag()</function> or
8410 <function>odr_explicit_tag()</function>
8411 could have been used.
8412 The new function can be used exactly like the standard functions provided
8413 with &odr;. It will encode, decode or pretty-print a data value of the
8414 <literal>MySequence</literal> type. We like to name types with an
8415 initial capital, as done in ASN.1 definitions, and to name the
8416 corresponding function with the first character of the name in lower case.
8417 You could, of course, name your structures, types, and functions any way
8418 you please - as long as you're consistent, and your code is easily readable.
8419 <literal>odr_ok</literal> is just that - a predicate that returns the
8420 state of the stream. It is used to ensure that the behavior of the new
8421 type is compatible with the interface of the primitive types.
8424 <sect2 id="odr.tagging.constructed.types">
8425 <title>Tagging Constructed Types</title>
8428 See <xref linkend="odr.tagging.primitive.types"/> for information
8429 on how to tag the primitive types, as well as types that are
8433 <sect3 id="odr.implicit.tagging">
8434 <title>Implicit Tagging</title>
8436 Assume the type above had been defined as
8439 MySequence ::= [10] IMPLICIT SEQUENCE {
8441 boolval BOOLEAN OPTIONAL
8445 You would implement this in &odr; by calling the function
8448 int odr_implicit_settag(ODR o, int class, int tag);
8451 which overrides the tag of the type immediately following it. The
8452 macro <function>odr_implicit_tag()</function> works by calling
8453 <function>odr_implicit_settag()</function> immediately
8454 before calling the function pointer argument.
8455 Your type function could look like this:
8458 int mySequence(ODR o, MySequence **p, int optional, const char *name)
8460 if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 ||
8461 odr_sequence_begin(o, p, sizeof(**p), name) == 0)
8462 return optional && odr_ok(o);
8464 odr_integer(o, &(*p)->intval, 0, "intval") &&
8465 odr_bool(o, &(*p)->boolval, 1, "boolval") &&
8466 odr_sequence_end(o);
8470 The definition of the structure <literal>MySequence</literal> would be
8474 <sect3 id="odr.explicit.tagging">
8475 <title>Explicit Tagging</title>
8477 Explicit tagging of constructed types is a little more complicated,
8478 since you are in effect adding a level of construction to the data.
8481 Assume the definition:
8484 MySequence ::= [10] IMPLICIT SEQUENCE {
8486 boolval BOOLEAN OPTIONAL
8490 Since the new type has an extra level of construction, two new functions
8491 are needed to encapsulate the base type:
8494 int odr_constructed_begin(ODR o, void *p, int class, int tag,
8497 int odr_constructed_end(ODR o);
8500 Assume that the IMPLICIT in the type definition above were replaced
8501 with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which
8502 would be equivalent). The structure definition would look the same,
8503 but the function would look like this:
8506 int mySequence(ODR o, MySequence **p, int optional, const char *name)
8508 if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0)
8509 return optional && odr_ok(o);
8510 if (o->direction == ODR_DECODE)
8511 *p = odr_malloc(o, sizeof(**p));
8512 if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0)
8514 *p = 0; /* this is almost certainly a protocol error */
8518 odr_integer(o, &(*p)->intval, 0, "intval") &&
8519 odr_bool(o, &(*p)->boolval, 1, "boolval") &&
8520 odr_sequence_end(o) &&
8521 odr_constructed_end(o);
8525 Notice that the interface here gets kind of nasty. The reason is
8526 simple: Explicitly tagged, constructed types are fairly rare in
8527 the protocols that we care about, so the
8528 esthetic annoyance (not to mention the dangers of a cluttered
8529 interface) is less than the time that would be required to develop a
8530 better interface. Nevertheless, it is far from satisfying, and it's a
8531 point that will be worked on in the future. One option for you would
8532 be to simply apply the <function>odr_explicit_tag()</function> macro to
8533 the first function, and not
8534 have to worry about <function>odr_constructed_*</function> yourself.
8535 Incidentally, as you might have guessed, the
8536 <function>odr_sequence_</function> functions are themselves
8537 implemented using the <function>/odr_constructed_</function> functions.
8541 <sect2 id="odr.sequence.of">
8542 <title>SEQUENCE OF</title>
8544 To handle sequences (arrays) of a specific type, the function
8547 int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional),
8548 void *p, int *num, const char *name);
8551 The <literal>fun</literal> parameter is a pointer to the decoder/encoder
8552 function of the type. <literal>p</literal> is a pointer to an array of
8553 pointers to your type. <literal>num</literal> is the number of elements
8560 MyArray ::= SEQUENCE OF INTEGER
8563 The C representation might be
8566 typedef struct MyArray
8573 And the function might look like
8576 int myArray(ODR o, MyArray **p, int optional, const char *name)
8578 if (o->direction == ODR_DECODE)
8579 *p = odr_malloc(o, sizeof(**p));
8580 if (odr_sequence_of(o, odr_integer, &(*p)->elements,
8581 &(*p)->num_elements, name))
8584 return optional && odr_ok(o);
8588 <sect2 id="odr.choice.types">
8589 <title>CHOICE Types</title>
8591 The choice type is used fairly often in some ASN.1 definitions, so
8592 some work has gone into streamlining its interface.
8595 CHOICE types are handled by the function:
8598 int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp,
8602 The <literal>arm</literal> array is used to describe each of the possible
8603 types that the CHOICE type may assume. Internally in your application,
8604 the CHOICE type is represented as a discriminated union. That is, a
8605 C union accompanied by an integer (or enum) identifying the active
8607 <literal>whichp</literal> is a pointer to the union discriminator.
8608 When encoding, it is examined to determine the current type.
8609 When decoding, it is set to reference the type that was found in
8613 The Odr_arm type is defined thus:
8616 typedef struct odr_arm
8627 The interpretation of the fields are:
8631 <term>tagmode</term>
8632 <listitem><para>Either <literal>ODR_IMPLICIT</literal>,
8633 <literal>ODR_EXPLICIT</literal>, or <literal>ODR_NONE</literal> (-1)
8634 to mark no tagging.</para></listitem>
8638 <listitem><para>The value of the discriminator that corresponds to
8639 this CHOICE element. Typically, it will be a #defined constant, or
8640 an enum member.</para></listitem>
8644 <listitem><para>A pointer to a function that implements the type of
8645 the CHOICE member. It may be either a standard &odr; type or a type
8646 defined by yourself.</para></listitem>
8650 <listitem><para>Name of tag.</para></listitem>
8654 A handy way to prepare the array for use by the
8655 <function>odr_choice()</function> function is to
8656 define it as a static, initialized array in the beginning of your
8657 decoding/encoding function. Assume the type definition:
8660 MyChoice ::= CHOICE {
8662 tagged [99] IMPLICIT INTEGER,
8667 Your C type might look like
8670 typedef struct MyChoice
8687 And your function could look like this:
8690 int myChoice(ODR o, MyChoice **p, int optional, const char *name)
8692 static Odr_arm arm[] =
8694 {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"},
8695 {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer,
8697 {-1, -1, -1, MyChoice_other, odr_boolean, "other"},
8701 if (o->direction == ODR_DECODE)
8702 *p = odr_malloc(o, sizeof(**p);
8704 return optional && odr_ok(o);
8706 if (odr_choice(o, arm, &(*p)->u, &(*p)->which), name)
8709 return optional && odr_ok(o);
8713 In some cases (say, a non-optional choice which is a member of a
8714 sequence), you can "embed" the union and its discriminator in the
8715 structure belonging to the enclosing type, and you won't need to
8716 fiddle with memory allocation to create a separate structure to
8717 wrap the discriminator and union.
8720 The corresponding function is somewhat nicer in the Sun XDR interface.
8721 Most of the complexity of this interface comes from the possibility of
8722 declaring sequence elements (including CHOICEs) optional.
8725 The ASN.1 specifications naturally requires that each member of a
8726 CHOICE have a distinct tag, so they can be told apart on decoding.
8727 Sometimes it can be useful to define a CHOICE that has multiple types
8728 that share the same tag. You'll need some other mechanism, perhaps
8729 keyed to the context of the CHOICE type. In effect, we would like to
8730 introduce a level of context-sensitiveness to our ASN.1 specification.
8731 When encoding an internal representation, we have no problem, as long
8732 as each CHOICE member has a distinct discriminator value. For
8733 decoding, we need a way to tell the choice function to look for a
8734 specific arm of the table. The function
8737 void odr_choice_bias(ODR o, int what);
8740 provides this functionality. When called, it leaves a notice for the next
8741 call to <function>odr_choice()</function> to be called on the decoding
8742 stream <literal>o</literal> that only the <literal>arm</literal> entry with
8743 a <literal>which</literal> field equal to <literal>what</literal>
8747 The most important application (perhaps the only one, really) is in
8748 the definition of application-specific EXTERNAL encoders/decoders
8749 which will automatically decode an ANY member given the direct or
8754 <sect1 id="odr.debugging">
8755 <title>Debugging</title>
8757 The protocol modules are suffering somewhat from a lack of diagnostic
8758 tools at the moment. Specifically ways to pretty-print PDUs that
8759 aren't recognized by the system. We'll include something to this end
8760 in a not-too-distant release. In the meantime, what we do when we get
8761 packages we don't understand is to compile the ODR module with
8762 <literal>ODR_DEBUG</literal> defined. This causes the module to dump tracing
8763 information as it processes data units. With this output and the
8764 protocol specification (Z39.50), it is generally fairly easy to see
8769 <chapter id="comstack">
8770 <title>The COMSTACK Module</title>
8771 <sect1 id="comstack.synopsis">
8772 <title>Synopsis (blocking mode)</title>
8773 <programlisting><![CDATA[
8776 int size = 0, length_incoming;
8777 char server_address_str[] = "localhost:9999";
8778 void *server_address_ip;
8781 char *protocol_package = "GET / HTTP/1.0\r\n\r\n";
8782 int protocol_package_length = strlen(protocol_package);
8784 stack = cs_create(tcpip_type, 1, PROTO_HTTP);
8786 perror("cs_create"); /* use perror() here since we have no stack yet */
8790 server_address_ip = cs_straddr(stack, server_address_str);
8791 if (!server_address_ip) {
8792 fprintf(stderr, "cs_straddr: address could not be resolved\n");
8796 status = cs_connect(stack, server_address_ip);
8798 fprintf(stderr, "cs_connect: %s\n", cs_strerror(stack));
8802 status = cs_rcvconnect(stack);
8804 fprintf(stderr, "cs_rcvconnect: %s\n", cs_strerror(stack));
8808 status = cs_put(stack, protocol_package, protocol_package_length);
8810 fprintf(stderr, "cs_put: %s\n", cs_strerror(stack));
8814 /* Now get a response */
8815 length_incoming = cs_get(stack, &buf, &size);
8816 if (!length_incoming) {
8817 fprintf(stderr, "Connection closed\n");
8819 } else if (length_incoming < 0) {
8820 fprintf(stderr, "cs_get: %s\n", cs_strerror(stack));
8825 fwrite(buf, length_incoming, 1, stdout);
8836 <sect1 id="comstack.introduction">
8837 <title>Introduction</title>
8840 subsystem provides a transparent interface to different types of transport
8841 stacks for the exchange of BER-encoded data and HTTP packets.
8842 At present, the RFC1729 method (BER over TCP/IP), local UNIX socket and an
8843 experimental SSL stack are supported, but others may be added in time.
8844 The philosophy of the
8845 module is to provide a simple interface by hiding unused options and
8846 facilities of the underlying libraries. This is always done at the risk
8847 of losing generality, and it may prove that the interface will need
8852 There hasn't been interest in the XTImOSI stack for some years.
8853 Therefore, it is no longer supported.
8857 The interface is implemented in such a fashion that only the
8858 sub-layers constructed to the transport methods that you wish to
8859 use in your application are linked in.
8862 You will note that even though simplicity was a goal in the design,
8863 the interface is still orders of magnitudes more complex than the
8864 transport systems found in many other packages. One reason is that
8865 the interface needs to support the somewhat different requirements of
8866 the different lower-layer communications stacks; another important
8867 reason is that the interface seeks to provide a more or less
8868 industrial-strength approach to asynchronous event-handling.
8869 When no function is allowed to block, things get more complex -
8870 particularly on the server side.
8871 We urge you to have a look at the demonstration client and server
8872 provided with the package. They are meant to be easily readable and
8873 instructive, while still being at least moderately useful.
8876 <sect1 id="comstack.common">
8877 <title>Common Functions</title>
8878 <sect2 id="comstack.managing.endpoints">
8879 <title>Managing Endpoints</title>
8881 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
8884 Creates an instance of the protocol stack - a communications endpoint.
8885 The <literal>type</literal> parameter determines the mode
8886 of communication. At present the following values are supported:
8890 <term><literal>tcpip_type</literal></term>
8891 <listitem><para>TCP/IP (BER over TCP/IP or HTTP over TCP/IP)
8895 <term><literal>ssl_type</literal></term>
8896 <listitem><para>Secure Socket Layer (SSL). This COMSTACK
8897 is experimental and is not fully implemented. If
8898 HTTP is used, this effectively is HTTPS.
8902 <term><literal>unix_type</literal></term>
8903 <listitem><para>Unix socket (unix only). Local Transfer via
8904 file socket. See <citerefentry><refentrytitle>unix</refentrytitle>
8905 <manvolnum>7</manvolnum></citerefentry>.
8910 The <function>cs_create</function> function returns a null-pointer
8911 if a system error occurs.
8912 The <literal>blocking</literal> parameter should be one if
8913 you wish the association to operate in blocking mode, zero otherwise.
8914 The <literal>protocol</literal> field should be
8915 <literal>PROTO_Z3950</literal> or <literal>PROTO_HTTP</literal>.
8916 Protocol <literal>PROTO_SR</literal> is no longer supported.
8919 void cs_close(COMSTACK handle);
8922 Closes the connection (as elegantly as the lower layers will permit),
8923 and releases the resources pointed to by the
8924 <literal>handle</literal>
8926 <literal>handle</literal>
8927 should not be referenced again after this call.
8931 We really need a soft disconnect, don't we?
8935 <sect2 id="comstack.data.exchange">
8936 <title>Data Exchange</title>
8938 int cs_put(COMSTACK handle, char *buf, int len);
8941 Sends <literal>buf</literal> down the wire.
8942 In blocking mode, this function will return only when a full buffer has
8943 been written, or an error has occurred. In nonblocking mode, it's
8944 possible that the function will be unable to send the full buffer
8945 at once, which will be indicated by a return value of 1.
8946 The function will keep track of the number of octets already written; you
8947 should call it repeatedly with the same values of <literal>buf</literal>
8948 and <literal>len</literal>, until the buffer has been transmitted.
8949 When a full buffer has been sent, the function will return 0 for
8950 success. -1 indicates an error condition (see below).
8953 int cs_get(COMSTACK handle, char **buf, int *size);
8956 Receives a PDU or HTTP Response from the peer. Returns the number of
8958 In nonblocking mode, it is possible that not all of the packet can be
8959 read at once. In this case, the function returns 1. To simplify the
8960 interface, the function is
8961 responsible for managing the size of the buffer. It will be reallocated
8962 if necessary to contain large packages, and will sometimes be moved
8963 around internally by the subsystem when partial packages are read. Before
8965 <function>cs_get</function>
8966 for the fist time, the buffer can be initialized to the null pointer,
8967 and the length should also be set to 0 - cs_get will perform a
8968 <function>malloc(2)</function>
8969 on the buffer for you. When a full buffer has been read, the size of
8970 the package is returned (which will always be greater than 1). -1
8971 indicates an error condition.
8974 See also the <function>cs_more()</function> function below.
8977 int cs_more(COMSTACK handle);
8980 The <function>cs_more()</function> function should be used in conjunction
8981 with <function>cs_get</function> and
8982 <function>select(2)</function>.
8983 The <function>cs_get()</function> function will sometimes
8984 (notably in the TCP/IP mode) read more than a single protocol package
8985 off the network. When this happens, the extra package is stored
8986 by the subsystem. After calling <function>cs_get()</function>, and before
8987 waiting for more input, You should always call
8988 <function>cs_more()</function>
8989 to check if there's a full protocol package already read. If
8990 <function>cs_more()</function>
8992 <function>cs_get()</function>
8993 can be used to immediately fetch the new package. For the
8995 subsystem, the function should always return 0, but if you want your
8996 stuff to be protocol independent, you should use it.
9000 The <function>cs_more()</function>
9001 function is required because the RFC1729-method
9002 does not provide a way of separating individual PDUs, short of
9003 partially decoding the BER. Some other implementations will carefully
9004 nibble at the packet by calling
9005 <function>read(2)</function>
9006 several times. This was felt to be too inefficient (or at least
9007 clumsy) - hence the call for this extra function.
9011 int cs_look(COMSTACK handle);
9014 This function is useful when you're operating in nonblocking
9016 <function>select(2)</function>
9017 tells you there's something happening on the line. It returns one of
9018 the following values:
9022 <term>CS_NONE</term>
9024 No event is pending. The data found on the line was not a
9029 <term>CS_CONNECT</term>
9031 A response to your connect request has been received. Call
9032 <function>cs_rcvconnect</function>
9033 to process the event and to finalize the connection establishment.
9037 <term>CS_DISCON</term>
9039 The other side has closed the connection (or maybe sent a disconnect
9040 request - but do we care? Maybe later). Call
9041 <function>cs_close</function> to close your end of the association
9046 <term>CS_LISTEN</term>
9048 A connect request has been received.
9049 Call <function>cs_listen</function> to process the event.
9053 <term>CS_DATA</term>
9055 There's data to be found on the line.
9056 Call <function>cs_get</function> to get it.
9062 You should be aware that even if
9063 <function>cs_look()</function>
9064 tells you that there's an event event pending, the corresponding
9065 function may still return and tell you there was nothing to be found.
9066 This means that only part of a package was available for reading. The
9067 same event will show up again, when more data has arrived.
9071 int cs_fileno(COMSTACK h);
9074 Returns the file descriptor of the association. Use this when
9075 file-level operations on the endpoint are required
9076 (<function>select(2)</function> operations, specifically).
9080 <sect1 id="comstack.client">
9081 <title>Client Side</title>
9083 int cs_connect(COMSTACK handle, void *address);
9086 Initiate a connection with the target at <literal>address</literal>
9087 (more on addresses below). The function will return 0 on success, and 1 if
9088 the operation does not complete immediately (this will only
9089 happen on a nonblocking endpoint). In this case, use
9090 <function>cs_rcvconnect</function> to complete the operation,
9091 when <function>select(2)</function> or <function>poll(2)</function>
9092 reports input pending on the association.
9095 int cs_rcvconnect(COMSTACK handle);
9098 Complete a connect operation initiated by <function>cs_connect()</function>.
9099 It will return 0 on success; 1 if the operation has not yet completed (in
9100 this case, call the function again later); -1 if an error has occurred.
9103 <sect1 id="comstack.server">
9104 <title>Server Side</title>
9106 To establish a server under the <application>inetd</application>
9110 COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
9114 The <literal>socket</literal> parameter is an established socket (when
9115 your application is invoked from <application>inetd</application>, the
9116 socket will typically be 0.
9117 The following parameters are identical to the ones for
9118 <function>cs_create</function>.
9121 int cs_bind(COMSTACK handle, void *address, int mode)
9124 Binds a local address to the endpoint. Read about addresses below. The
9125 <literal>mode</literal> parameter should be either
9126 <literal>CS_CLIENT</literal> or <literal>CS_SERVER</literal>.
9129 int cs_listen(COMSTACK handle, char *addr, int *addrlen);
9132 Call this to process incoming events on an endpoint that has been
9133 bound in listening mode. It will return 0 to indicate that the connect
9134 request has been received, 1 to signal a partial reception, and -1 to
9135 indicate an error condition.
9138 COMSTACK cs_accept(COMSTACK handle);
9141 This finalizes the server-side association establishment, after
9142 cs_listen has completed successfully. It returns a new connection
9143 endpoint, which represents the new association. The application will
9144 typically wish to fork off a process to handle the association at this
9145 point, and continue listen for new connections on the old
9146 <literal>handle</literal>.
9149 You can use the call
9152 const char *cs_addrstr(COMSTACK);
9155 on an established connection to retrieve the host-name of the remote host.
9159 You may need to use this function with some care if your
9160 name server service is slow or unreliable
9164 <sect1 id="comstack.addresses">
9165 <title>Addresses</title>
9167 The low-level format of the addresses are different depending on the
9168 mode of communication you have chosen. A function is provided by each
9169 of the lower layers to map a user-friendly string-form address to the
9170 binary form required by the lower layers.
9173 void *cs_straddr(COMSTACK handle, const char *str);
9176 The format for TCP/IP and SSL addresses is:
9179 <host> [ ':' <portnum> ]
9182 The <literal>hostname</literal> can be either a domain name or an
9183 IP address. The port number, if omitted, defaults to 210.
9186 For TCP/IP and SSL, the special hostnames <literal>@</literal>,
9187 maps to <literal>IN6ADDR_ANY_INIT</literal> with
9188 IPV4 binding as well (bindv6only=0),
9189 The special hostname <literal>@4</literal> binds to
9190 <literal>INADDR_ANY</literal> (IPV4 only listener).
9191 The special hostname <literal>@6</literal> binds to
9192 <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
9195 For UNIX sockets, the format of an address is the socket filename.
9198 When a connection has been established, you can use
9201 const char *cs_addrstr(COMSTACK h);
9204 to retrieve the host name of the peer system. The function returns
9205 a pointer to a static area, which is overwritten on the next call
9209 A fairly recent addition to the &comstack; module is the utility
9213 COMSTACK cs_create_host (const char *str, int blocking, void **vp);
9216 which is just a wrapper for <function>cs_create</function> and
9217 <function>cs_straddr</function>. The <parameter>str</parameter>
9218 is similar to that described for <function>cs_straddr</function>
9219 but with a prefix denoting the &comstack; type. Prefixes supported
9220 are <literal>tcp:</literal>, <literal>unix:</literal> and
9221 <literal>ssl:</literal> for TCP/IP, UNIX and SSL respectively.
9222 If no prefix is given, then TCP/IP is used.
9223 The <parameter>blocking</parameter> is passed to
9224 function <function>cs_create</function>. The third parameter
9225 <parameter>vp</parameter> is a pointer to &comstack; stack type
9227 Parameter <parameter>vp</parameter> is reserved for future use.
9228 Set it to <literal>NULL</literal>.
9231 <sect1 id="comstack.ssl">
9235 void *cs_get_ssl(COMSTACK cs);
9237 Returns the SSL handle, <literal>SSL *</literal> for comstack. If comstack
9238 is not of type SSL, NULL is returned.
9242 int cs_set_ssl_ctx(COMSTACK cs, void *ctx);
9244 Sets SSL context for comstack. The parameter is expected to be of type
9245 <literal>SSL_CTX *</literal>. This function should be called just
9246 after comstack has been created (before connect, bind, etc).
9247 This function returns 1 for success; 0 for failure.
9251 int cs_set_ssl_certificate_file(COMSTACK cs, const char *fname);
9253 Sets SSL certificate for comstack as a PEM file. This function
9254 returns 1 for success; 0 for failure.
9258 int cs_get_ssl_peer_certificate_x509(COMSTACK cs, char **buf, int *len);
9260 This function returns the peer certificate. If successful,
9261 <literal>*buf</literal> and <literal>*len</literal> holds
9262 X509 buffer and length respectively. Buffer should be freed
9263 with <literal>xfree</literal>. This function returns 1 for success;
9267 <sect1 id="comstack.diagnostics">
9268 <title>Diagnostics</title>
9270 All functions return -1 if an error occurs. Typically, the functions
9271 will return 0 on success, but the data exchange functions
9272 (<function>cs_get</function>, <function>cs_put</function>,
9273 <function>cs_more</function>) follow special rules. Consult their
9277 The error code for the COMSTACK can be retrieved using C macro
9278 <function>cs_errno</function> which will return one
9279 of the error codes <literal>CSYSERR</literal>,
9280 <literal>CSOUTSTATE</literal>,
9281 <literal>CSNODATA</literal>, ...
9284 int cs_errno(COMSTACK handle);
9287 You can the textual representation of the error code
9288 by using <function>cs_errmsg</function> - which
9289 works like <function>strerror(3)</function>
9292 const char *cs_errmsg(int n);
9295 It is also possible to get straight to the textual represenataion
9296 without the error code by using
9297 <function>cs_strerror</function>.
9300 const char *cs_strerror(COMSTACK h);
9303 <sect1 id="comstack.summary">
9304 <title>Summary and Synopsis</title>
9306 #include <yaz/comstack.h>
9308 #include <yaz/tcpip.h> /* this is for TCP/IP and SSL support */
9309 #include <yaz/unix.h> /* this is for UNIX socket support */
9311 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
9313 COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
9315 COMSTACK cs_create_host(const char *str, int blocking,
9318 int cs_bind(COMSTACK handle, int mode);
9320 int cs_connect(COMSTACK handle, void *address);
9322 int cs_rcvconnect(COMSTACK handle);
9324 int cs_listen(COMSTACK handle);
9326 COMSTACK cs_accept(COMSTACK handle);
9328 int cs_put(COMSTACK handle, char *buf, int len);
9330 int cs_get(COMSTACK handle, char **buf, int *size);
9332 int cs_more(COMSTACK handle);
9334 void cs_close(COMSTACK handle);
9336 int cs_look(COMSTACK handle);
9338 void *cs_straddr(COMSTACK handle, const char *str);
9340 const char *cs_addrstr(COMSTACK h);
9345 <chapter id="future">
9346 <title>Future Directions</title>
9348 We have a new and better version of the front-end server on the drawing
9349 board. Resources and external commitments will govern when we'll be
9350 able to do something real with it. Features should include greater
9351 flexibility, greater support for access/resource control, and easy
9352 support for Explain (possibly with Zebra as an extra database engine).
9355 &yaz; is a BER toolkit and as such should support all protocols
9356 out there based on that. We'd like to see running ILL applications.
9357 It shouldn't be that hard. Another thing that would be interesting is
9358 LDAP. Maybe a generic framework for doing IR using both LDAP and
9359 Z39.50 transparently.
9362 The SOAP implementation is incomplete. In the future we hope
9363 to add more features to it. Perhaps make a WSDL/XML Schema compiler.
9364 The authors of libxml2 are already working on XML Schema / RelaxNG
9365 compilers so this may not be too hard.
9368 It would be neat to have a proper module mechanism for the Generic
9369 Frontend Server so that backend would be dynamically
9370 loaded (as shared objects / DLLs).
9373 Other than that, &yaz; generally moves in the directions which appear to
9374 make the most people happy (including ourselves, as prime users of the
9375 software). If there's something you'd like to see in here, then drop
9376 us a note and let's see what we can come up with.
9379 <reference id="reference">
9380 <title>Reference</title>
9381 <partintro id="reference-introduction">
9383 The material in this chapter is drawn directly from the individual
9389 <appendix id="list-oids">
9390 <title>List of Object Identifiers</title>
9392 These is a list of object identifiers that are built into YAZ.
9396 <appendix id="bib1-diagnostics">
9397 <title>Bib-1 diagnostics</title>
9399 List of Bib-1 diagnostics that are known to YAZ.
9403 <appendix id="sru-diagnostics">
9404 <title>SRU diagnostics</title>
9406 List of SRU diagnostics that are known to YAZ.
9410 <appendix id="license">
9411 <title>License</title>
9412 <sect1 id="license.indexdata">
9413 <title>Index Data Copyright</title>
9415 Copyright © ©right-year; Index Data.
9418 All rights reserved.
9421 Redistribution and use in source and binary forms, with or without
9422 modification, are permitted provided that the following conditions are met:
9427 Redistributions of source code must retain the above copyright
9428 notice, this list of conditions and the following disclaimer.
9433 Redistributions in binary form must reproduce the above copyright
9434 notice, this list of conditions and the following disclaimer in the
9435 documentation and/or other materials provided with the distribution.
9440 Neither the name of Index Data nor the names of its contributors
9441 may be used to endorse or promote products derived from this
9442 software without specific prior written permission.
9447 THIS SOFTWARE IS PROVIDED BY INDEX DATA ``AS IS'' AND ANY
9448 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
9449 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
9450 DISCLAIMED. IN NO EVENT SHALL INDEX DATA BE LIABLE FOR
9451 ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
9452 DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
9453 SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
9454 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
9455 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
9456 OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
9461 <appendix id="indexdata">
9462 <title>About Index Data</title>
9464 Index Data is a consulting and software-development enterprise that
9465 specializes in library and information management systems. Our
9466 interests and expertise span a broad range of related fields, and one
9467 of our primary, long-term objectives is the development of a powerful
9468 information management
9469 system with open network interfaces and hyper-media capabilities.
9471 We make this software available free of charge, on a fairly unrestrictive
9472 license; as a service to the networking community, and to further the
9473 development of quality software for open network communication.
9475 We'll be happy to answer questions about the software, and about ourselves
9481 <street>Amagerfælledvej 56</street>
9482 <postcode>2300 Copenhagen S</postcode>
9483 <country>Denmark</country>
9484 Email <email>info@indexdata.dk</email>
9488 The Hacker's Jargon File has the following to say about the
9490 prefix "YA" in the name of a software product.
9494 Yet Another. adj. 1. Of your own work: A
9495 humorous allusion often used in titles to acknowledge that the
9496 topic is not original, though the content is. As in "Yet Another
9497 AI Group" or "Yet Another Simulated Annealing Algorithm".
9499 others' work: Describes something of which there are already far
9504 <appendix id="credits">
9505 <title>Credits</title>
9507 This appendix lists individuals that have contributed in the development
9508 of &yaz;. Some have contributed with code, while others have provided bug
9509 fixes or suggestions. If we're missing somebody, of if you, for
9510 whatever reason, don't like to be listed here, let us know.
9520 Morten Bøgeskov
9541 Mads Bondo Dydensborg
9550 Morten Garkier Hendriksen
9607 Tom André Øverland
9613 <!-- Keep this comment at the end of the file
9616 nxml-child-indent: 1