1 <!-- $Id: tools.xml,v 1.22 2003-03-18 13:30:21 adam Exp $ -->
2 <chapter id="tools"><title>Supporting Tools</title>
5 In support of the service API - primarily the ASN module, which
6 provides the pro-grammatic interface to the Z39.50 APDUs, &yaz; contains
7 a collection of tools that support the development of applications.
10 <sect1 id="tools.query"><title>Query Syntax Parsers</title>
13 Since the type-1 (RPN) query structure has no direct, useful string
14 representation, every origin application needs to provide some form of
15 mapping from a local query notation or representation to a
16 <token>Z_RPNQuery</token> structure. Some programmers will prefer to
17 construct the query manually, perhaps using
18 <function>odr_malloc()</function> to simplify memory management.
19 The &yaz; distribution includes two separate, query-generating tools
20 that may be of use to you.
23 <sect2 id="PQF"><title>Prefix Query Format</title>
26 Since RPN or reverse polish notation is really just a fancy way of
27 describing a suffix notation format (operator follows operands), it
28 would seem that the confusion is total when we now introduce a prefix
29 notation for RPN. The reason is one of simple laziness - it's somewhat
30 simpler to interpret a prefix format, and this utility was designed
31 for maximum simplicity, to provide a baseline representation for use
32 in simple test applications and scripting environments (like Tcl). The
33 demonstration client included with YAZ uses the PQF.
38 The PQF have been adopted by other parties developing Z39.50
39 software. It is often referred to as Prefix Query Notation
44 The PQF is defined by the pquery module in the YAZ library.
45 There are two sets of function that have similar behavior. First
46 set operates on a PQF parser handle, second set doesn't. First set
47 set of functions are more flexible than the second set. Second set
48 is obsolete and is only provided to ensure backwards compatibility.
51 First set of functions all operate on a PQF parser handle:
54 #include <yaz/pquery.h>
56 YAZ_PQF_Parser yaz_pqf_create (void);
58 void yaz_pqf_destroy (YAZ_PQF_Parser p);
60 Z_RPNQuery *yaz_pqf_parse (YAZ_PQF_Parser p, ODR o, const char *qbuf);
62 Z_AttributesPlusTerm *yaz_pqf_scan (YAZ_PQF_Parser p, ODR o,
63 Odr_oid **attributeSetId, const char *qbuf);
66 int yaz_pqf_error (YAZ_PQF_Parser p, const char **msg, size_t *off);
69 A PQF parser is created and destructed by functions
70 <function>yaz_pqf_create</function> and
71 <function>yaz_pqf_destroy</function> respectively.
72 Function <function>yaz_pqf_parse</function> parses query given
73 by string <literal>qbuf</literal>. If parsing was successful,
74 a Z39.50 RPN Query is returned which is created using ODR stream
75 <literal>o</literal>. If parsing failed, a NULL pointer is
77 Function <function>yaz_pqf_scan</function> takes a scan query in
78 <literal>qbuf</literal>. If parsing was successful, the function
79 returns attributes plus term pointer and modifies
80 <literal>attributeSetId</literal> to hold attribute set for the
81 scan request - both allocated using ODR stream <literal>o</literal>.
82 If parsing failed, yaz_pqf_scan returns a NULL pointer.
83 Error information for bad queries can be obtained by a call to
84 <function>yaz_pqf_error</function> which returns an error code and
85 modifies <literal>*msg</literal> to point to an error description,
86 and modifies <literal>*off</literal> to the offset within last
87 query were parsing failed.
90 The second set of functions are declared as follows:
93 #include <yaz/pquery.h>
95 Z_RPNQuery *p_query_rpn (ODR o, oid_proto proto, const char *qbuf);
97 Z_AttributesPlusTerm *p_query_scan (ODR o, oid_proto proto,
98 Odr_oid **attributeSetP, const char *qbuf);
100 int p_query_attset (const char *arg);
103 The function <function>p_query_rpn()</function> takes as arguments an
104 &odr; stream (see section <link linkend="odr">The ODR Module</link>)
105 to provide a memory source (the structure created is released on
106 the next call to <function>odr_reset()</function> on the stream), a
107 protocol identifier (one of the constants <token>PROTO_Z3950</token> and
108 <token>PROTO_SR</token>), an attribute set reference, and
109 finally a null-terminated string holding the query string.
112 If the parse went well, <function>p_query_rpn()</function> returns a
113 pointer to a <literal>Z_RPNQuery</literal> structure which can be
114 placed directly into a <literal>Z_SearchRequest</literal>.
115 If parsing failed, due to syntax error, a NULL pointer is returned.
118 The <literal>p_query_attset</literal> specifies which attribute set
119 to use if the query doesn't specify one by the
120 <literal>@attrset</literal> operator.
121 The <literal>p_query_attset</literal> returns 0 if the argument is a
122 valid attribute set specifier; otherwise the function returns -1.
126 The grammar of the PQF is as follows:
130 query ::= top-set query-struct.
132 top-set ::= [ '@attrset' string ]
134 query-struct ::= attr-spec | simple | complex | '@term' term-type
136 attr-spec ::= '@attr' [ string ] string query-struct
138 complex ::= operator query-struct query-struct.
140 operator ::= '@and' | '@or' | '@not' | '@prox' proximity.
142 simple ::= result-set | term.
144 result-set ::= '@set' string.
148 proximity ::= exclusion distance ordered relation which-code unit-code.
150 exclusion ::= '1' | '0' | 'void'.
152 distance ::= integer.
154 ordered ::= '1' | '0'.
156 relation ::= integer.
158 which-code ::= 'known' | 'private' | integer.
160 unit-code ::= integer.
162 term-type ::= 'general' | 'numeric' | 'string' | 'oid' | 'datetime' | 'null'.
166 You will note that the syntax above is a fairly faithful
167 representation of RPN, except for the Attribute, which has been
168 moved a step away from the term, allowing you to associate one or more
169 attributes with an entire query structure. The parser will
170 automatically apply the given attributes to each term as required.
174 The @attr operator is followed by an attribute specification
175 (<literal>attr-spec</literal> above). The specification consists
176 of optional an attribute set, an attribute type-value pair and
177 a sub query. The attribute type-value pair is packed in one string:
178 an attribute type, a dash, followed by an attribute value.
179 The type is always an integer but the value may be either an
180 integer or a string (if it doesn't start with a digit character).
184 Version 3 of the Z39.50 specification defines various encoding of terms.
185 Use the <literal>@term </literal> <replaceable>type</replaceable>,
186 where type is one of: <literal>general</literal>,
187 <literal>numeric</literal>, <literal>string</literal>
188 (for InternationalString), ..
189 If no term type has been given, the <literal>general</literal> form
190 is used which is the only encoding allowed in both version 2 - and 3
191 of the Z39.50 standard.
194 <example><title>PQF queries</title>
196 <para>Queries using simple terms.
202 <para>Boolean operators.
204 @or "dylan" "zimmerman"
205 @and @or dylan zimmerman when
206 @and when @or dylan zimmerman
210 Reference to result sets.
217 Attributes for terms.
220 @attr 1=4 @attr 4=1 "self portrait"
221 @attr exp1 @attr 1=1 CategoryList
222 @attr gils 1=2008 Copenhagen
223 @attr 1=/book/title computer
229 @prox 0 3 1 2 k 2 dylan zimmerman
233 Specifying term type.
235 @term string "a UTF-8 string, maybe?"
240 @or @and bob dylan @set Result-1
242 @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
244 @and @attr 2=4 @attr gils 1=2038 -114 @attr 2=2 @attr gils 1=2039 -109
249 <sect2 id="CCL"><title>Common Command Language</title>
252 Not all users enjoy typing in prefix query structures and numerical
253 attribute values, even in a minimalistic test client. In the library
254 world, the more intuitive Common Command Language (or ISO 8777) has
255 enjoyed some popularity - especially before the widespread
256 availability of graphical interfaces. It is still useful in
257 applications where you for some reason or other need to provide a
258 symbolic language for expressing boolean query structures.
262 The <ulink url="http://europagate.dtv.dk/">EUROPAGATE</ulink>
263 research project working under the Libraries programme
264 of the European Commission's DG XIII has, amongst other useful tools,
265 implemented a general-purpose CCL parser which produces an output
266 structure that can be trivially converted to the internal RPN
267 representation of &yaz; (The <literal>Z_RPNQuery</literal> structure).
268 Since the CCL utility - along with the rest of the software
269 produced by EUROPAGATE - is made freely available on a liberal
270 license, it is included as a supplement to &yaz;.
273 <sect3><title>CCL Syntax</title>
276 The CCL parser obeys the following grammar for the FIND argument.
277 The syntax is annotated by in the lines prefixed by
278 <literal>‐‐</literal>.
282 CCL-Find ::= CCL-Find Op Elements
285 Op ::= "and" | "or" | "not"
286 -- The above means that Elements are separated by boolean operators.
288 Elements ::= '(' CCL-Find ')'
291 | Qualifiers Relation Terms
292 | Qualifiers Relation '(' CCL-Find ')'
293 | Qualifiers '=' string '-' string
294 -- Elements is either a recursive definition, a result set reference, a
295 -- list of terms, qualifiers followed by terms, qualifiers followed
296 -- by a recursive definition or qualifiers in a range (lower - upper).
298 Set ::= 'set' = string
299 -- Reference to a result set
301 Terms ::= Terms Prox Term
303 -- Proximity of terms.
307 -- This basically means that a term may include a blank
309 Qualifiers ::= Qualifiers ',' string
311 -- Qualifiers is a list of strings separated by comma
313 Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
314 -- Relational operators. This really doesn't follow the ISO8777
318 -- Proximity operator
322 <example><title>CCL queries</title>
324 The following queries are all valid:
336 (dylan and bob) or set=1
340 Assuming that the qualifiers <literal>ti</literal>,
341 <literal>au</literal>
342 and <literal>date</literal> are defined we may use:
348 au=(bob dylan and slow train coming)
350 date>1980 and (ti=((self portrait)))
356 <sect3><title>CCL Qualifiers</title>
359 Qualifiers are used to direct the search to a particular searchable
360 index, such as title (ti) and author indexes (au). The CCL standard
361 itself doesn't specify a particular set of qualifiers, but it does
362 suggest a few short-hand notations. You can customize the CCL parser
363 to support a particular set of qualifiers to reflect the current target
364 profile. Traditionally, a qualifier would map to a particular
365 use-attribute within the BIB-1 attribute set. However, you could also
366 define qualifiers that would set, for example, the
371 A CCL profile is a set of predefined CCL qualifiers that may be
373 The YAZ client reads its CCL qualifiers from a file named
374 <filename>default.bib</filename>. Each line in the file has the form:
378 <replaceable>qualifier-name</replaceable>
379 [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable>
380 [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable> ...
384 where <replaceable>qualifier-name</replaceable> is the name of the
385 qualifier to be used (eg. <literal>ti</literal>),
386 <replaceable>type</replaceable> is attribute type in the attribute
387 set (Bib-1 is used if no attribute set is given) and
388 <replaceable>val</replaceable> is attribute value.
389 The <replaceable>type</replaceable> can be specified as an
390 integer or as it be specified either as a single-letter:
391 <literal>u</literal> for use,
392 <literal>r</literal> for relation,<literal>p</literal> for position,
393 <literal>s</literal> for structure,<literal>t</literal> for truncation
394 or <literal>c</literal> for completeness.
395 The attributes for the special qualifier name <literal>term</literal>
396 are used when no CCL qualifier is given in a query.
399 <example><title>CCL profile</title>
401 Consider the following definition:
411 Three qualifiers are defined, <literal>ti</literal>,
412 <literal>au</literal> and <literal>ranked</literal>.
413 <literal>ti</literal> and <literal>au</literal> both set
414 structure attribute to phrase (s=1).
415 <literal>ti</literal>
416 sets the use-attribute to 4. <literal>au</literal> sets the
418 When no qualifiers are used in the query the structure-attribute is
419 set to free-form-text (105).
422 You can combine attributes. To Search for "ranked title" you
425 ti,ranked=knuth computer
427 which will use "relation is ranked", "use is title", "structure is
433 <sect3><title>CCL API</title>
435 All public definitions can be found in the header file
436 <filename>ccl.h</filename>. A profile identifier is of type
437 <literal>CCL_bibset</literal>. A profile must be created with the call
438 to the function <function>ccl_qual_mk</function> which returns a profile
439 handle of type <literal>CCL_bibset</literal>.
443 To read a file containing qualifier definitions the function
444 <function>ccl_qual_file</function> may be convenient. This function
445 takes an already opened <literal>FILE</literal> handle pointer as
446 argument along with a <literal>CCL_bibset</literal> handle.
450 To parse a simple string with a FIND query use the function
453 struct ccl_rpn_node *ccl_find_str (CCL_bibset bibset, const char *str,
454 int *error, int *pos);
457 which takes the CCL profile (<literal>bibset</literal>) and query
458 (<literal>str</literal>) as input. Upon successful completion the RPN
459 tree is returned. If an error occur, such as a syntax error, the integer
460 pointed to by <literal>error</literal> holds the error code and
461 <literal>pos</literal> holds the offset inside query string in which
466 An English representation of the error may be obtained by calling
467 the <literal>ccl_err_msg</literal> function. The error codes are
468 listed in <filename>ccl.h</filename>.
472 To convert the CCL RPN tree (type
473 <literal>struct ccl_rpn_node *</literal>)
474 to the Z_RPNQuery of YAZ the function <function>ccl_rpn_query</function>
475 must be used. This function which is part of YAZ is implemented in
476 <filename>yaz-ccl.c</filename>.
477 After calling this function the CCL RPN tree is probably no longer
478 needed. The <literal>ccl_rpn_delete</literal> destroys the CCL RPN tree.
482 A CCL profile may be destroyed by calling the
483 <function>ccl_qual_rm</function> function.
487 The token names for the CCL operators may be changed by setting the
488 globals (all type <literal>char *</literal>)
489 <literal>ccl_token_and</literal>, <literal>ccl_token_or</literal>,
490 <literal>ccl_token_not</literal> and <literal>ccl_token_set</literal>.
491 An operator may have aliases, i.e. there may be more than one name for
492 the operator. To do this, separate each alias with a space character.
496 <sect2 id="tools.cql"><title>CQL</title>
498 <ulink url="http://www.loc.gov/z3950/agency/zing/cql/">CQL</ulink>
499 - Common Query Language - was defined for the
500 <ulink url="http://www.loc.gov/z3950/agency/zing/srw/">SRW</ulink>
502 In many ways CQL has a similar syntax to CCL.
503 The objective of CQL is different. Where CCL aims to be
504 an end-user language, CQL is <emphasis>the</emphasis> protocol
505 query language for SRW.
509 If you are new to CQL, read the
510 <ulink url="http://zing.z3950.org/cql/intro.html">Gentle
511 Introduction</ulink>.
515 The CQL parser in &yaz; provides the following:
519 It parses and validates a CQL query.
524 It generates a C structure that allows you to convert
525 a CQL query to some other query language, such as SQL.
530 The parser converts a valid CQL query to PQF, thus providing a
531 way to use CQL for both SRW/SRU servers and Z39.50 targets at the
537 The parser converts CQL to
538 <ulink url="http://www.loc.gov/z3950/agency/zing/cql/xcql.html">
540 XCQL is an XML representation of CQL.
541 XCQL is part of the SRW specification. However, since SRU
542 supports CQL only, we don't expect XCQL to be widely used.
543 Furthermore, CQL has the advantage over XCQL that it is
549 <sect3 id="tools.cql.parsing"><title>CQL parsing</title>
551 A CQL parser is represented by the <literal>CQL_parser</literal>
552 handle. Its contents should be considered &yaz; internal (private).
554 #include <yaz/cql.h>
556 typedef struct cql_parser *CQL_parser;
558 CQL_parser cql_parser_create(void);
559 void cql_parser_destroy(CQL_parser cp);
561 A parser is created by <function>cql_parser_create</function> and
562 is destroyed by <function>cql_parser_destroy</function>.
565 To parse a CQL query string, the following function
568 int cql_parser_string(CQL_parser cp, const char *str);
570 A CQL query is parsed by the <function>cql_parser_string</function>
571 which takes a query <parameter>str</parameter>.
572 If the query was valid (no syntax errors), then zero is returned;
573 otherwise a non-zero error code is returned.
577 int cql_parser_stream(CQL_parser cp,
578 int (*getbyte)(void *client_data),
579 void (*ungetbyte)(int b, void *client_data),
582 int cql_parser_stdio(CQL_parser cp, FILE *f);
584 The functions <function>cql_parser_stream</function> and
585 <function>cql_parser_stdio</function> parses a CQL query
586 - just like <function>cql_parser_string</function>.
587 The only difference is that the CQL query can be
588 fed to the parser in different ways.
589 The <function>cql_parser_stream</function> uses a generic
590 byte stream as input. The <function>cql_parser_stdio</function>
591 uses a <literal>FILE</literal> handle which is opened for reading.
595 <sect3 id="tools.cql.tree"><title>CQL tree</title>
597 The the query string is validl, the CQL parser
598 generates a tree representing the structure of the
603 struct cql_node *cql_parser_result(CQL_parser cp);
605 <function>cql_parser_result</function> returns the
606 a pointer to the root node of the resulting tree.
609 Each node in a CQL tree is represented by a
610 <literal>struct cql_node</literal>.
611 It is defined as follows:
613 #define CQL_NODE_ST 1
614 #define CQL_NODE_BOOL 2
615 #define CQL_NODE_MOD 3
623 struct cql_node *modifiers;
624 struct cql_node *prefixes;
628 struct cql_node *left;
629 struct cql_node *right;
630 struct cql_node *modifiers;
631 struct cql_node *prefixes;
636 struct cql_node *next;
641 There are three kinds of nodes, search term (ST), boolean (BOOL),
645 The search term node has five members:
649 <literal>index</literal>: index for search term.
650 If an index is unspecified for a search term,
651 <literal>index</literal> will be NULL.
656 <literal>term</literal>: the search term itself.
661 <literal>relation</literal>: relation for search term.
666 <literal>modifiers</literal>: relation modifiers for search
667 term. The <literal>modifiers</literal> is a simple linked
668 list (NULL for last entry). Each relation modifier node
669 is of type <literal>MOD</literal>.
674 <literal>prefixes</literal>: index prefixes for search
675 term. The <literal>prefixes</literal> is a simple linked
676 list (NULL for last entry). Each prefix node
677 is of type <literal>MOD</literal>.
684 The boolean node represents both <literal>and</literal>,
685 <literal>or</literal>, not as well as
690 <literal>left</literal> and <literal>right</literal>: left
691 - and right operand respectively.
696 <literal>modifiers</literal>: proximity arguments.
701 <literal>prefixes</literal>: index prefixes.
702 The <literal>prefixes</literal> is a simple linked
703 list (NULL for last entry). Each prefix node
704 is of type <literal>MOD</literal>.
711 The modifier node is a "utility" node used for name-value pairs,
712 such as prefixes, proximity arguements, etc.
716 <literal>name</literal> name of mod node.
721 <literal>value</literal> value of mod node.
726 <literal>next</literal>: pointer to next node which is
727 always a mod node (NULL for last entry).
734 <sect3 id="tools.cql.pqf"><title>CQL to PQF conversion</title>
736 Conversion to PQF (and Z39.50 RPN) is tricky by the fact
737 that the resulting RPN depends on the Z39.50 target
738 capabilities (combinations of supported attributes).
739 In addition, the CQL and SRW operates on index prefixes
740 (URI or strings), whereas the RPN uses Object Identifiers
744 The CQL library of &yaz; defines a <literal>cql_transform_t</literal>
745 type. It represents a particular mapping between CQL and RPN.
746 This handle is created and destroyed by the functions:
748 cql_transform_t cql_transform_open_FILE (FILE *f);
749 cql_transform_t cql_transform_open_fname(const char *fname);
750 void cql_transform_close(cql_transform_t ct);
752 The first two functions create a tranformation handle from
753 either an already open FILE or from a filename respectively.
756 The handle is destroyed by <function>cql_transform_close</function>
757 in which case no further reference of the handle is allowed.
760 When a <literal>cql_transform_t</literal> handle has been created
761 you can convert to RPN.
763 int cql_transform_buf(cql_transform_t ct,
764 struct cql_node *cn, char *out, int max);
766 This function converts the CQL tree <literal>cn</literal>
767 using handle <literal>ct</literal>.
768 For the resulting PQF, you supply a buffer <literal>out</literal>
769 which must be able to hold at at least <literal>max</literal>
773 If conversion failed, <function>cql_transform_buf</function>
774 returns a non-zero error code; otherwise zero is returned
775 (conversion successful).
778 If you wish to be able to produce a PQF result in a different
779 way, there are two alternatives.
781 void cql_transform_pr(cql_transform_t ct,
783 void (*pr)(const char *buf, void *client_data),
786 int cql_transform_FILE(cql_transform_t ct,
787 struct cql_node *cn, FILE *f);
789 The former function produces output to a user-defined
790 output stream. The latter writes the result to an already
791 open <literal>FILE</literal>.
794 <sect3 id="tools.cql.map">
795 <title>Specification of CQL to RPN mapping</title>
797 The file supplied to functions
798 <function>cql_transform_open_FILE</function>,
799 <function>cql_transform_open_fname</function> follows
800 a structure found in many Unix utilities.
801 It consists of mapping specifications - one per line.
802 Lines starting with <literal>#</literal> are ignored (comments).
805 Each line is of the form
807 <replaceable>CQL pattern</replaceable><literal> = </literal> <replaceable> RPN equivalent</replaceable>
811 An RPN pattern is a simple attribute list. Each attribute pair
814 [<replaceable>set</replaceable>] <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>
816 The attribute <replaceable>set</replaceable> is optional.
817 The <replaceable>type</replaceable> is the attribute type,
818 <replaceable>value</replaceable> the attribute value.
821 The following CQL patterns are recognized:
824 <literal>qualifier.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
828 This pattern is invoked when a CQL qualifier, such as
829 dc.title is converted. <replaceable>set</replaceable>
830 and <replaceable>name</replaceable> is the index set and qualifier
832 Typically, the RPN specifies an equivalent use attribute.
835 For terms not bound by a qualifier the pattern
836 <literal>qualifier.srw.serverChoice</literal> is used.
837 Here, the prefix <literal>srw</literal> is defined as
838 <literal>http://www.loc.gov/zing/cql/srw-indexes/v1.0/</literal>.
839 If this pattern is not defined, the mapping will fail.
844 <literal>relation.</literal><replaceable>relation</replaceable>
848 This pattern specifies how a CQL relation is mapped to RPN.
849 <replaceable>pattern</replaceable> is name of relation
850 operator. Since <literal>=</literal> is used as
851 separator between CQL pattern and RPN, CQL relations
852 including <literal>=</literal> cannot be
853 used directly. To avoid a conflict, the names
854 <literal>ge</literal>,
855 <literal>eq</literal>,
856 <literal>le</literal>,
857 must be used for CQL operators, greater-than-or-equal,
858 equal, less-than-or-equal respectively.
859 The RPN pattern is supposed to include a relation attribute.
862 For terms not bound by a relation, the pattern
863 <literal>relation.scr</literal> is used. If the pattern
864 is not defined, the mapping will fail.
867 The special pattern, <literal>relation.*</literal> is used
868 when no other relation pattern is matched.
874 <literal>relationModifier.</literal><replaceable>mod</replaceable>
878 This pattern specifies how a CQL relation modifier is mapped to RPN.
879 The RPN pattern is usually a relation attribute.
885 <literal>structure.</literal><replaceable>type</replaceable>
889 This pattern specifies how a CQL structure is mapped to RPN.
890 Note that this CQL pattern is somewhat to similar to
891 CQL pattern <literal>relation</literal>.
892 The <replaceable>type</replaceable> is a CQL relation.
895 The pattern, <literal>structure.*</literal> is used
896 when no other structure pattern is matched.
897 Usually, the RPN equivalent specifies a structure attribute.
903 <literal>position.</literal><replaceable>type</replaceable>
907 This pattern specifies how the anchor (position) of
908 CQL is mapped to RPN.
909 The <replaceable>type</replaceable> is one
910 of <literal>first</literal>, <literal>any</literal>,
911 <literal>last</literal>, <literal>firstAndLast</literal>.
914 The pattern, <literal>position.*</literal> is used
915 when no other position pattern is matched.
921 <literal>set.</literal><replaceable>prefix</replaceable>
925 This specification defines a CQL index set for a given prefix.
926 The value on the right hand side is the URI for the set -
927 <emphasis>not</emphasis> RPN. All prefixes used in
928 qualifier patterns must be defined this way.
934 <example><title>CQL to RPN mapping file</title>
936 This simple file defines two index sets, three qualifiers and three
937 relations, a position pattern and a default structure.
939 <programlisting><![CDATA[
940 set.srw = http://www.loc.gov/zing/cql/srw-indexes/v1.0/
941 set.dc = http://www.loc.gov/zing/cql/dc-indexes/v1.0/
943 qualifier.srw.serverChoice = 1=1016
944 qualifier.dc.title = 1=4
945 qualifier.dc.subject = 1=21
951 position.any = 3=3 6=1
957 With the mappings above, the CQL query
961 is converted to the PQF:
963 @attr 1=1016 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "computer"
965 by rules <literal>qualifier.srw.serverChoice</literal>,
966 <literal>relation.scr</literal>, <literal>structure.*</literal>,
967 <literal>position.any</literal>.
974 is rejected, since <literal>position.right</literal> is
980 >my = "http://www.loc.gov/zing/cql/dc-indexes/v1.0/" my.title = x
984 @attr 1=4 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "x"
989 <sect3 id="tools.cql.xcql"><title>CQL to XCQL conversion</title>
991 Conversion from CQL to XCQL is trivial and does not
992 require a mapping to be defined.
993 There three functions to choose from depending on the
994 way you wish to store the resulting output (XML buffer
997 int cql_to_xml_buf(struct cql_node *cn, char *out, int max);
998 void cql_to_xml(struct cql_node *cn,
999 void (*pr)(const char *buf, void *client_data),
1001 void cql_to_xml_stdio(struct cql_node *cn, FILE *f);
1003 Function <function>cql_to_xml_buf</function> converts
1004 to XCQL and stores result in a user supplied buffer of a given
1008 <function>cql_to_xml</function> writes the result in
1009 a user defined output stream.
1010 <function>cql_to_xml_stdio</function> writes to a
1016 <sect1 id="tools.oid"><title>Object Identifiers</title>
1019 The basic YAZ representation of an OID is an array of integers,
1020 terminated with the value -1. The &odr; module provides two
1021 utility-functions to create and copy this type of data elements:
1025 Odr_oid *odr_getoidbystr(ODR o, char *str);
1029 Creates an OID based on a string-based representation using dots (.)
1030 to separate elements in the OID.
1034 Odr_oid *odr_oiddup(ODR odr, Odr_oid *o);
1038 Creates a copy of the OID referenced by the <emphasis>o</emphasis>
1040 Both functions take an &odr; stream as parameter. This stream is used to
1041 allocate memory for the data elements, which is released on a
1042 subsequent call to <function>odr_reset()</function> on that stream.
1046 The OID module provides a higher-level representation of the
1047 family of object identifiers which describe the Z39.50 protocol and its
1048 related objects. The definition of the module interface is given in
1049 the <filename>oid.h</filename> file.
1053 The interface is mainly based on the <literal>oident</literal> structure.
1054 The definition of this structure looks like this:
1058 typedef struct oident
1063 int oidsuffix[OID_SIZE];
1069 The proto field takes one of the values
1078 If you don't care about talking to SR-based implementations (few
1079 exist, and they may become fewer still if and when the ISO SR and ANSI
1080 Z39.50 documents are merged into a single standard), you can ignore
1081 this field on incoming packages, and always set it to PROTO_Z3950
1082 for outgoing packages.
1086 The oclass field takes one of the values
1108 corresponding to the OID classes defined by the Z39.50 standard.
1110 Finally, the value field takes one of the values
1168 again, corresponding to the specific OIDs defined by the standard.
1172 The desc field contains a brief, mnemonic name for the OID in question.
1180 struct oident *oid_getentbyoid(int *o);
1184 takes as argument an OID, and returns a pointer to a static area
1185 containing an <literal>oident</literal> structure. You typically use
1186 this function when you receive a PDU containing an OID, and you wish
1187 to branch out depending on the specific OID value.
1195 int *oid_ent_to_oid(struct oident *ent, int *dst);
1199 Takes as argument an <literal>oident</literal> structure - in which
1200 the <literal>proto</literal>, <literal>oclass</literal>/, and
1201 <literal>value</literal> fields are assumed to be set correctly -
1202 and returns a pointer to a the buffer as given by <literal>dst</literal>
1204 representation of the corresponding OID. The function returns
1205 NULL and the array dst is unchanged if a mapping couldn't place.
1206 The array <literal>dst</literal> should be at least of size
1207 <literal>OID_SIZE</literal>.
1211 The <function>oid_ent_to_oid()</function> function can be used whenever
1212 you need to prepare a PDU containing one or more OIDs. The separation of
1213 the <literal>protocol</literal> element from the remainder of the
1214 OID-description makes it simple to write applications that can
1215 communicate with either Z39.50 or OSI SR-based applications.
1223 oid_value oid_getvalbyname(const char *name);
1227 takes as argument a mnemonic OID name, and returns the
1228 <literal>/value</literal> field of the first entry in the database that
1229 contains the given name in its <literal>desc</literal> field.
1233 Finally, the module provides the following utility functions, whose
1234 meaning should be obvious:
1238 void oid_oidcpy(int *t, int *s);
1239 void oid_oidcat(int *t, int *s);
1240 int oid_oidcmp(int *o1, int *o2);
1241 int oid_oidlen(int *o);
1246 The OID module has been criticized - and perhaps rightly so
1247 - for needlessly abstracting the
1248 representation of OIDs. Other toolkits use a simple
1249 string-representation of OIDs with good results. In practice, we have
1250 found the interface comfortable and quick to work with, and it is a
1251 simple matter (for what it's worth) to create applications compatible
1252 with both ISO SR and Z39.50. Finally, the use of the
1253 <literal>/oident</literal> database is by no means mandatory.
1254 You can easily create your own system for representing OIDs, as long
1255 as it is compatible with the low-level integer-array representation
1262 <sect1 id="tools.nmem"><title>Nibble Memory</title>
1265 Sometimes when you need to allocate and construct a large,
1266 interconnected complex of structures, it can be a bit of a pain to
1267 release the associated memory again. For the structures describing the
1268 Z39.50 PDUs and related structures, it is convenient to use the
1269 memory-management system of the &odr; subsystem (see
1270 <link linkend="odr-use">Using ODR</link>). However, in some circumstances
1271 where you might otherwise benefit from using a simple nibble memory
1272 management system, it may be impractical to use
1273 <function>odr_malloc()</function> and <function>odr_reset()</function>.
1274 For this purpose, the memory manager which also supports the &odr;
1275 streams is made available in the NMEM module. The external interface
1276 to this module is given in the <filename>nmem.h</filename> file.
1280 The following prototypes are given:
1284 NMEM nmem_create(void);
1285 void nmem_destroy(NMEM n);
1286 void *nmem_malloc(NMEM n, int size);
1287 void nmem_reset(NMEM n);
1288 int nmem_total(NMEM n);
1289 void nmem_init(void);
1290 void nmem_exit(void);
1294 The <function>nmem_create()</function> function returns a pointer to a
1295 memory control handle, which can be released again by
1296 <function>nmem_destroy()</function> when no longer needed.
1297 The function <function>nmem_malloc()</function> allocates a block of
1298 memory of the requested size. A call to <function>nmem_reset()</function>
1299 or <function>nmem_destroy()</function> will release all memory allocated
1300 on the handle since it was created (or since the last call to
1301 <function>nmem_reset()</function>. The function
1302 <function>nmem_total()</function> returns the number of bytes currently
1303 allocated on the handle.
1307 The nibble memory pool is shared amongst threads. POSIX
1308 mutex'es and WIN32 Critical sections are introduced to keep the
1309 module thread safe. Function <function>nmem_init()</function>
1310 initializes the nibble memory library and it is called automatically
1311 the first time the <literal>YAZ.DLL</literal> is loaded. &yaz; uses
1312 function <function>DllMain</function> to achieve this. You should
1313 <emphasis>not</emphasis> call <function>nmem_init</function> or
1314 <function>nmem_exit</function> unless you're absolute sure what
1315 you're doing. Note that in previous &yaz; versions you'd have to call
1316 <function>nmem_init</function> yourself.
1322 <!-- Keep this comment at the end of the file
1327 sgml-minimize-attributes:nil
1328 sgml-always-quote-attributes:t
1331 sgml-parent-document: "yaz.xml"
1332 sgml-local-catalogs: nil
1333 sgml-namecase-general:t