X-Git-Url: http://lists.indexdata.com/cgi-bin?a=blobdiff_plain;f=doc%2Fbook.xml;h=10f54bdbb6b6a2bf71ddac204194b53ad0d3cbd5;hb=a7c6b853ed128f05ef6d30a8687c5029e75f69e9;hp=b64b93a623b1314db49e20e7e785bb3b0496290e;hpb=dd16b462c655c7783c4831a8c0b99605957c5f6e;p=metaproxy-moved-to-github.git diff --git a/doc/book.xml b/doc/book.xml index b64b93a..10f54bd 100644 --- a/doc/book.xml +++ b/doc/book.xml @@ -1,11 +1,34 @@ - + + + + + + %common; + + + + +]> + + Metaproxy - User's Guide and Reference - MikeTaylor + AdamDickmeiss - AdamDickmeiss + MarcCromme + + + MikeTaylor 2006 @@ -32,10 +55,10 @@ using the filter API. - The terms under which Metaproxy will be distributed have yet to be - established, but it will not necessarily be open source; so users - should not at this stage redistribute the code without explicit - written permission from the copyright holders, Index Data ApS. + Metaproxy is not open-source software, but + may be freely downloaded, unpacked, inspected, built and run for + evaluation purposes. Deployment requires a separate, commercial, + license. @@ -56,7 +79,7 @@ Metaproxy - is a standalone program that acts as a universal router, proxy and + is a stand alone program that acts as a universal router, proxy and encapsulated metasearcher for information retrieval protocols such as Z39.50, and in the future SRU and SRW. @@ -84,13 +107,13 @@ being more powerful, flexible, configurable and extensible. Among its many advantages over the older, more pedestrian work are support for multiplexing (encapsulated metasearching), routing by - database name, authentication and authorisation and serving local + database name, authentication and authorization and serving local files via HTTP. Equally significant, its modular architecture facilitites the creation of pluggable modules implementing further functionality. - This manual will briefly describe Metaproxy's licensing situation + This manual will describe how to install Metaproxy before giving an overview of its architecture, then discussing the key concept of a filter in some depth and giving an overview of the various filter types, then discussing the configuration file @@ -105,26 +128,60 @@ - - - - The Metaproxy Licence - - - No decision has yet been made on the terms under which - Metaproxy will be distributed. - - It is possible that, unlike - other Index Data products, metaproxy may not be released under a - free-software licence such as the GNU GPL. Until a decision is - made and a public statement made, then, and unless it has been - delivered to you other specific terms, please treat Metaproxy as - though it were proprietary software. - The code should not be redistributed without explicit - written permission from the copyright holders, Index Data ApS. - + + The Metaproxy License + + + + You are allowed to download this software for evaluation purposes. + You can unpack it, build it, run it, see how it works and how it fits + your needs, all at zero cost. + + + + + You may NOT deploy the software. For the purposes of this license, + deployment means running it for any purpose other than evaluation, + whether or not you or anyone else makes a profit from doing so. If + you wish to deploy the software, you must first contact Index Data and + arrange to purchase a DEPLOYMENT LICENCE. If you are unsure + whether or not your proposed use of the software constitutes + deployment, email us at info@indexdata.com + for clarification. + + + + + You may modify your copy of the software (fix bugs, add features) + if you need to. We encourage you to send your changes back to us for + integration into the master copy, but you are not obliged to do so. You + may NOT pass your changes on to any other party. + + + + + There is NO WARRANTY for this software, to the extent permitted by + applicable law. We provide the software ``as is'' without warranty of + any kind, either expressed or implied, including, but not limited to, the + implied warranties of MERCHANTABILITY and FITNESS FOR A + PARTICULAR PURPOSE. The entire risk as to the quality and + performance of the software is with you. Should the software prove + defective, you assume the cost of all necessary servicing, repair or + correction. In no event unless required by applicable law will we be + liable to you for damages, arising out of the use of the software, + including but not limited to loss of data or data being rendered + inaccurate. + + + + + All rights to the software are reserved by Index Data except where + this license explicitly says otherwise. + + + - + Installation @@ -149,7 +206,8 @@ Boost - The popular C++ library. + The popular C++ library. Initial versions of Metaproxy + was built with 1.33.0. Version 1.33.1 works too. @@ -163,7 +221,7 @@ for more information. - We have succesfully used Metaproxy with Boost using the compilers + We have successfully built Metaproxy using the compilers GCC version 4.0 and Microsoft Visual Studio 2003/2005. @@ -240,18 +298,198 @@
- Installation on Debian + Installation on Debian GNU/Linux - ### To be written + All dependencies for Metaproxy are available as + Debian + packages for the sarge (stable in 2005) and etch (testing in 2005) + distributions. -
+ + The procedures for Debian based systems, such as + Ubuntu is probably similar + + + There is currently no official Debian package for YAZ++. + And the Debian package for YAZ is probably too old. + Update the /etc/apt/sources.list + to include the Index Data repository. + See YAZ' Download Debian + for more information. + + + apt-get install libxslt1-dev + apt-get install libyazpp-dev + apt-get install libboost-dev + apt-get install libboost-thread-dev + apt-get install libboost-date-time-dev + apt-get install libboost-program-options-dev + apt-get install libboost-test-dev + + + With these packages installed, the usual configure + make + procedure can be used for Metaproxy as outlined in + . + +
Installation on Windows - ### To be written + Metaproxy can be compiled with Microsoft + Visual Studio. + Version 2003 (C 7.1) and 2005 (C 8.0) is known to work. +
+ Boost + + Get Boost from its home page. + You also need Boost Jam (an alternative to make). + That's also available from the Boost home page. + The files to be downloaded are called something like: + boost_1_33-1.exe + and + boost-jam-3.1.12-1-ntx86.zip. + Unpack Boost Jam first. Put bjam.exe + in your system path. Make a command prompt and ensure + it can be found automatically. If not check the PATH. + The Boost .exe is a self-extracting exe with + complete source for Boost. Compile that source with + Boost Jam (An alternative to Make). + The compilation takes a while. + For Visual Studio 2003, use + + bjam "-sTOOLS=vc-7_1" + + Here vc-7_1 refers to a "Toolset" (compiler system). + For Visual Studio 2005, use + + bjam "-sTOOLS=vc-8_0" + + To install the libraries in a common place, use + + bjam "-sTOOLS=vc-7_1" install + + (or vc-8_0 for VS 2005). + + + By default, the Boost build process installs the resulting + libraries + header files in + \boost\lib, \boost\include. + + + For more information about installing Boost refer to the + getting started + pages. +
+ +
+ Libxslt + + Libxslt can be downloaded + for Windows from + here. + + + Libxslt has other dependencies, but these can all be downloaded + from the same site. Get the following: + iconv, zlib, libxml2, libxslt. + +
+ +
+ YAZ + + YAZ can be downloaded + for Windows from + here. + +
+ +
+ YAZ++ + + Get YAZ++ as well. + Version 1.0 or later is required. For now get it from + Index Data's + Snapshot area. + + + YAZ++ includes NMAKE makefiles, similar to those found in the + YAZ package. + +
+ +
+ Metaproxy + + Metaproxy is shipped with NMAKE makefiles as well - similar + to those found in the YAZ++/YAZ packages. Adjust this Makefile + to point to the proper locations of Boost, Libxslt, Libxml2, + zlib, iconv, yaz and yazpp. + + + + DEBUG + + If set to 1, the software is + compiled with debugging libraries (code generation is + multi-threaded debug DLL). + If set to 0, the software is compiled with release libraries + (code generation is multi-threaded DLL). + + + + + BOOST + + + Boost install location + + + + + + BOOST_VERSION + + + Boost version (replace . with _). + + + + + + BOOST_TOOLSET + + + Boost toolset. + + + + + + LIBXSLT_DIR, + LIBXML2_DIR .. + + + Specify the locations of Libxslt, libiconv, libxml2 and + libxslt. + + + + + + + + After successful compilation you'll find + metaproxy.exe in the + bin directory. + +
+ + +
@@ -281,7 +519,7 @@ In general, packages are doctored as they pass through Metaproxy. For example, when the proxy performs authentication - and authorisation on a Z39.50 Init request, it removes the + and authorization on a Z39.50 Init request, it removes the authentication credentials from the package so that they are not passed onto the back-end server; and when search-response packages are obtained from multiple servers, they are merged @@ -320,7 +558,7 @@ The word ``filter'' is sometimes used rather loosely, in two different ways: it may be used to mean a particular type of filter, as when we speak of ``the - auth_simplefilter'' or ``the multi filter''; or it may be used + auth_simple filter'' or ``the multi filter''; or it may be used to be a specific instance of a filter within a Metaproxy configuration. For example, a single configuration will often contain multiple instances of the @@ -377,7 +615,7 @@ as part of Metaproxy, and others may be provided by third parties and dynamically loaded. They all conform to the same simple API of essentially two methods: configure() is - called at startup time, and is passed a DOM tree representing that + called at startup time, and is passed an XML DOM tree representing that part of the configuration file that pertains to this filter instance: it is expected to walk that tree extracting relevant information; and process() is called every @@ -391,6 +629,7 @@ others are sinks: they consume packages and return a result (z3950_client, backend_test, + bounce, http_file); the others are true filters, that read, process and pass on the packages they are fed @@ -398,7 +637,9 @@ log, multi, query_rewrite, + record_transform, session_shared, + sru_z3950, template, virt_db). @@ -410,7 +651,7 @@ We now briefly consider each of the types of filter supported by the core Metaproxy binary. This overview is intended to give a - flavour of the available functionality; more detailed information + flavor of the available functionality; more detailed information about each type of filter is included below in the reference guide to Metaproxy filters. @@ -428,11 +669,34 @@ The filters are here listed in alphabetical order: + +
<literal>auth_simple</literal> (mp::filter::AuthSimple) - Simple authentication and authorisation. The configuration + Simple authentication and authorization. The configuration specifies the name of a file that is the user register, which lists username:password pairs, one per line, colon separated. When a session begins, it @@ -451,7 +715,7 @@ <literal>backend_test</literal> (mp::filter::Backend_test) - A sink that provides dummy responses in the manner of the + A partial sink that provides dummy responses in the manner of the yaz-ztest Z39.50 server. This is useful only for testing. Seriously, you don't need this. Pretend you didn't even read this section. @@ -459,13 +723,31 @@
+ <literal>bounce</literal> + (mp::filter::Bounce) + + A sink that swallows all packages, + and returns them almost unprocessed. + It never sends any package of any type further down the row, but + sets Z39.50 packages to Z_Close, and HTTP_Request packages to + HTTP_Response err code 400 packages, and adds a suitable bounce + message. + The bounce filter is usually added at end of each filter chain + config.xml to prevent infinite hanging of for example HTTP + requests packages when only the Z39.50 client partial sink + filter is found in the + route. + +
+ +
<literal>frontend_net</literal> (mp::filter::FrontendNet) A source that accepts Z39.50 connections from a port specified in the configuration, reads protocol units, and feeds them into the next filter in the route. When the result is - revceived, it is returned to the original origin. + received, it is returned to the original origin.
@@ -473,8 +755,12 @@ <literal>http_file</literal> (mp::filter::HttpFile) - A sink that returns the contents of files from the local - filesystem in response to HTTP requests. (Yes, Virginia, this + A partial sink which swallows only HTTP_Request packages, and + returns the contents of files from the local + filesystem in response to HTTP requests. + It lets Z39.50 packages and all other forthcoming package types + pass untouched. + (Yes, Virginia, this does mean that Metaproxy is also a Web-server in its spare time. So far it does not contain either an email-reader or a Lisp interpreter, but that day is surely coming.) @@ -486,7 +772,8 @@ (mp::filter::Log) Writes logging information to standard output, and passes on - the package unchanged. + the package unchanged. A log file name can be specified, as well + as multiple different logging formats. @@ -494,7 +781,7 @@ <literal>multi</literal> (mp::filter::Multi) - Performs multicast searching. + Performs multi-database searching. See the extended discussion of virtual databases and multi-database searching below. @@ -514,6 +801,21 @@ + +
+ <literal>record_transform</literal> + (mp::filter::RecordTransform) + + This filter acts only on Z3950 present requests, and let all + other types of packages and requests pass untouched. It's use is + twofold: blocking Z3950 present requests, which the backend + server does not understand and can not honor, and transforming + the present syntax and elementset name according to the rules + specified, to fetch only existing record formats, and transform + them on the fly to requested record syntaxes. + +
+
<literal>session_shared</literal> (mp::filter::SessionShared) @@ -531,6 +833,16 @@
+ +
+ <literal>sru_z3950</literal> + (mp::filter::SRUtoZ3950) + + This filter transforms valid + SRU/GET or SRU/SOAP requests to Z3950 requests, and wraps the + received hit counts and XML records into suitable SRU response messages. + +
<literal>template</literal> @@ -547,7 +859,7 @@ <section> <title><literal>virt_db</literal> - (mp::filter::Virt_db) + (mp::filter::VirtualDB) Performs virtual database selection: based on the name of the database in the search request, a server is selected, and its @@ -564,13 +876,16 @@ <literal>z3950_client</literal> (mp::filter::Z3950Client) - Performs Z39.50 searching and retrieval by proxying the + A partial sink which swallows only Z39.50 packages. + It performs Z39.50 searching and retrieval by proxying the packages that are passed to it. Init requests are sent to the address specified in the VAL_PROXY otherInfo attached to the request: this may have been specified by client, or generated by a virt_db filter earlier in the route. Subsequent requests are sent to the same address, which is remembered at Init time in a Session object. + HTTP_Request packages and all other forthcoming package types + are passed untouched.
@@ -595,34 +910,10 @@ - frontend_sru (source) - - - Receive SRU (and perhaps SRW) requests. - - - - - sru2z3950 (filter) - - - Translate SRU requests into Z39.50 requests. - - - - sru_client (sink) - SRU searching and retrieval. - - - - - srw_client (sink) - - - SRW searching and retrieval. + SRU/GET and SRU/SOAP searching and retrieval. @@ -660,26 +951,10 @@ implementation detail - they could just as well have been written in YAML or Lisp-like S-expressions, or in a custom syntax.) - - Since XML has been chosen, an XML schema, - config.xsd, is provided for validating - configuration files. This file is supplied in the - etc directory of the Metaproxy distribution. It - can be used by (among other tools) the xmllint - program supplied as part of the libxml2 - distribution: - - - xmllint --noout --schema etc/config.xsd my-config-file.xml - - - (A recent version of libxml2 is required, as - support for XML Schemas is a relatively recent addition.) -
- Overview of XML structure + Overview of the config file XML structure All elements and attributes are in the namespace . @@ -739,14 +1014,13 @@ The following is a small, but complete, Metaproxy configuration file (included in the distribution as - metaproxy/etc/config0.xml). + metaproxy/etc/config1.xml). This file defines a very simple configuration that simply proxies - to whatever backend server the client requests, but logs each + to whatever back-end server the client requests, but logs each request and response. This can be useful for debugging complex client-server dialogues. - + @@ -761,13 +1035,14 @@ + ]]> It works by defining a single route, called - start, which consists of a sequence of three + start, which consists of a sequence of four filters. The first and last of these are included by reference: their <filter> elements have refid attributes that refer to filters defined @@ -775,18 +1050,51 @@ middle filter is included inline in the route. - The three filters in the route are as follows: first, a + The four filters in the route are as follows: first, a frontend_net filter accepts Z39.50 requests from any host on port 9000; then these requests are passed through a log filter that emits a message for each request; they are then fed into a z3950_client - filter, which forwards the requests to the client-specified - backend Z39.509 server. When the response arrives, it is handed + filter, which forwards all Z39.50 requests to the client-specified + back-end Z39.509 server. Those Z39.50 packages are returned by the + z3950_client filter, with the response data + filled by the external Z39.50 server targeted. + All non-Z39.50 packages are passed through to the + bounce filter, which definitely bounces + everything, including fish, bananas, cold pyjamas, + mutton, beef and trout packages. + When the response arrives, it is handed back to the log filter, which emits another - message; and then to the front-end filter, which returns the - response to the client. + message; and then to the frontend_net filter, + which returns the response to the client.
+
+ Config file syntax checking + + The distribution contains RelaxNG Compact and XML syntax checking + files, as well as XML Schema files. These are found in the + distribution paths + + xml/schema/metaproxy.rnc + xml/schema/metaproxy.rng + xml/schema/metaproxy.xsd + + and can be used to verify or debug the XML structure of + configuration files. For example, using the utility + xmllint, syntax checking is done like this: + + xmllint --noout --schema xml/schema/metaproxy.xsd etc/config-local.xml + xmllint --noout --relaxng xml/schema/metaproxy.rng etc/config-local.xml + + (A recent version of libxml2 is required, as + support for XML Schemas is a relatively recent addition.) + + + You can of course use any other RelaxNG or XML Schema compliant tool + you wish. + +
@@ -797,38 +1105,254 @@
Introductory notes - - Lark's vomit - - This chapter goes into a level of technical detail that is - probably not necessary in order to configure and use Metaproxy. - It is provided only for those who like to know how things work. - You should feel free to skip on to the next section if this one - doesn't seem like fun. - - Two of Metaproxy's filters are concerned with multiple-database operations. Of these, virt_db can work alone to control the routing of searches to one of a number of servers, - while multi can work with the output of - virt_db to perform multicast searching, merging - the results into a unified result-set. The interaction between - these two filters is necessarily complex: it reflecting the real, - irreducible complexity of multicast searching in a protocol such - as Z39.50 that separates initialisation from searching, and in - which the database to be searched is not known at initialisation + while multi can work together with + virt_db to perform multi-database searching, merging + the results into a unified result-set - ``metasearch in a box''. + + + The interaction between + these two filters is necessarily complex: it reflects the real, + irreducible complexity of multi-database searching in a protocol such + as Z39.50 that separates initialization from searching, and in + which the database to be searched is not known at initialization time. - Hold on tight - this may get a little hairy. + It's possible to use these filters without understanding the + details of their functioning and the interaction between them; the + next two sections of this chapter are ``HOW-TO'' guides for doing + just that. However, debugging complex configurations will require + a deeper understanding, which the last two sections of this + chapters attempt to provide.
-
+
Virtual databases with the <literal>virt_db</literal> filter + Working alone, the purpose of the + virt_db + filter is to route search requests to one of a selection of + back-end databases. In this way, a single Z39.50 endpoint + (running Metaproxy) can provide access to several different + underlying services, including those that would otherwise be + inaccessible due to firewalls. In many useful configurations, the + back-end databases are local to the Metaproxy installation, but + the software does not enforce this, and any valid Z39.50 servers + may be used as back-ends. + + + For example, a virt_db + filter could be set up so that searches in the virtual database + ``lc'' are forwarded to the Library of Congress bibliographic + catalogue server, and searches in the virtual database ``marc'' + are forwarded to the toy database of MARC records that Index Data + hosts for testing purposes. A virt_db + configuration to make this switch would look like this: + + + + lc + z3950.loc.gov:7090/voyager + + + marc + indexdata.dk/marc + +]]> + + As well as being useful in it own right, this filter also provides + the foundation for multi-database searching. + +
+ + +
+ Multi-database search with the <literal>multi</literal> filter + + To arrange for Metaproxy to broadcast searches to multiple back-end + servers, the configuration needs to include two components: a + virt_db + filter that specifies multiple + <target> + elements, and a subsequent + multi + filter. Here, for example, is a complete configuration that + broadcasts searches to both the Library of Congress catalogue and + Index Data's tiny testing database of MARC records: + + + + + + + + 10 + @:9000 + + + + lc + z3950.loc.gov:7090/voyager + + + marc + indexdata.dk/marc + + + all + z3950.loc.gov:7090/voyager + indexdata.dk/marc + + + + + 30 + + + + +]]> + + (Using a + virt_db + filter that specifies multiple + <target> + elements but without a subsequent + multi + filter yields surprising and undesirable results, as will be + described below. Don't do that.) + + + Metaproxy can be invoked with this configuration as follows: + + ../src/metaproxy --config config-simple-multi.xml + + And thereafter, Z39.50 clients can connect to the running server + (on port 9000, as specified in the configuration) and search in + any of the databases + lc (the Library of Congress catalogue), + marc (Index Data's test database of MARC records) + or + all (both of these). As an example, a session + using the YAZ command-line client yaz-client is + here included (edited for brevity and clarity): + + base lc +Z> find computer +Search was a success. +Number of hits: 10000, setno 1 +Elapsed: 5.521070 +Z> base marc +Z> find computer +Search was a success. +Number of hits: 10, setno 3 +Elapsed: 0.060187 +Z> base all +Z> find computer +Search was a success. +Number of hits: 10010, setno 4 +Elapsed: 2.237648 +Z> show 1 +[marc]Record type: USmarc +001 11224466 +003 DLC +005 00000000000000.0 +008 910710c19910701nju 00010 eng +010 $a 11224466 +040 $a DLC $c DLC +050 00 $a 123-xyz +100 10 $a Jack Collins +245 10 $a How to program a computer +260 1 $a Penguin +263 $a 8710 +300 $a p. cm. +Elapsed: 0.119612 +Z> show 2 +[VOYAGER]Record type: USmarc +001 13339105 +005 20041229102447.0 +008 030910s2004 caua 000 0 eng +035 $a (DLC) 2003112666 +906 $a 7 $b cbc $c orignew $d 4 $e epcn $f 20 $g y-gencatlg +925 0 $a acquire $b 1 shelf copy $x policy default +955 $a pc10 2003-09-10 $a pv12 2004-06-23 to SSCD; $h sj05 2004-11-30 $e sj05 2004-11-30 to Shelf. +010 $a 2003112666 +020 $a 0761542892 +040 $a DLC $c DLC $d DLC +050 00 $a MLCM 2004/03312 (G) +245 10 $a 007, everything or nothing : $b Prima's official strategy guide / $c created by Kaizen Media Group. +246 3 $a Double-O-seven, everything or nothing +246 30 $a Prima's official strategy guide +260 $a Roseville, CA : $b Prima Games, $c c2004. +300 $a 161 p. : $b col. ill. ; $c 28 cm. +500 $a "Platforms: Nintendo GameCube, Macintosh, PC, PlayStation 2 computer entertainment system, Xbox"--P. [4] of cover. +650 0 $a Video games. +710 2 $a Kaizen Media Group. +856 42 $3 Publisher description $u http://www.loc.gov/catdir/description/random052/2003112666.html +Elapsed: 0.150623 +Z> +]]> + + As can be seen, the first record in the result set is from the + Index Data test database, and the second from the Library of + Congress database. The result-set continues alternating records + round-robin style until the point where one of the databases' + records are exhausted. + + + This example uses only two back-end databases; more may be used. + There is no limitation imposed on the number of databases that may + be metasearched in this way: issues of resource usage and + administrative complexity dictate the practical limits. + + + What happens when one of the databases doesn't respond? By default, + the entire multi-database search fails, and the appropriate + diagnostic is returned to the client. This is usually appropriate + during development, when technicians need maximum information, but + can be inconvenient in deployment, when users typically don't want + to be bothered with problems of this kind and prefer just to get + the records from the databases that are available. To obtain this + latter behavior add an empty + <hideunavailable> + element inside the + multi filter: + + + + ]]> + + Under this regime, an error is reported to the client only if + all the databases in a multi-database search + are unavailable. + +
+ + +
+ What's going on? + + Lark's vomit + + This section goes into a level of technical detail that is + probably not necessary in order to configure and use Metaproxy. + It is provided only for those who like to know how things work. + You should feel free to skip on to the next section if this one + doesn't seem like fun. + + + + Hold on tight - this may get a little hairy. + + In the general course of things, a Z39.50 Init request may carry with it an otherInfo packet of type VAL_PROXY, whose value indicates the address of a Z39.50 server to which the @@ -847,32 +1371,18 @@ >the HTTP 1.1 specification. - The role of the virt_db filter is to rewrite - this otherInfo packet dependent on the virtual database that the - client wants to search. For example, a virt_db - filter could be set up so that searches in the virtual database - ``lc'' are forwarded to the Library of Congress server, and - searches in the virtual database ``id'' are forwarded to the toy - GILS database that Index Data hosts for testing purposes. A - virt_db configuration to make this switch would - look like this: + Within Metaproxy, Search requests that are part of the same + session as an Init request that carries a + VAL_PROXY otherInfo are also annotated with the + same information. The role of the virt_db + filter is to rewrite this otherInfo packet dependent on the + virtual database that the client wants to search. - - - lc - z3950.loc.gov:7090/Voyager - - - id - indexdata.dk/gils - - ]]> When Metaproxy receives a Z39.50 Init request from a client, it doesn't immediately forward that request to the back-end server. Why not? Because it doesn't know which - back-end server to forward it to until the client sends a search + back-end server to forward it to until the client sends a Search request that specifies the database that it wants to search in. Instead, it just treasures the Init request up in its heart; and, later, the first time the client does a search on one of the @@ -888,7 +1398,7 @@ frontend_net filter. The virt_db filter knows nothing about it; in fact, because the Init request that is received from the client - doesn't get forwarded until a Search reqeust is received, the + doesn't get forwarded until a Search request is received, the virt_db filter (and the z3950_client filter behind it) doesn't even get invoked at Init time. The only thing that a @@ -896,24 +1406,58 @@ VAL_PROXY otherInfo in the requests that pass through it. + + It is possible for a virt_db filter to contain + multiple + <target> + elements. What does this mean? Only that the filter will add + multiple VAL_PROXY otherInfo packets to the + Search requests that pass through it. That's because the virtual + DB filter is dumb, and does exactly what it's told - no more, no + less. + If a Search request with multiple VAL_PROXY + otherInfo packets reaches a z3950_client + filter, this is an error. That filter doesn't know how to deal + with multiple targets, so it will either just pick one and search + in it, or (better) fail with an error message. + + + The multi filter comes to the rescue! This is + the only filter that knows how to deal with multiple + VAL_PROXY otherInfo packets, and it does so by + making multiple copies of the entire Search request: one for each + VAL_PROXY. Each of these new copies is then + passed down through the remaining filters in the route. (The + copies are handled in parallel though the + spawning of new threads.) Since the copies each have only one + VAL_PROXY otherInfo, they can be handled by the + z3950_client filter, which happily deals with + each one individually. When the results of the individual + searches come back up to the multi filter, it + merges them into a single Search response, which is what + eventually makes it back to the client. +
-
+ +
A picture is worth a thousand words (but only five hundred on 64-bit architectures) - + - + - Diagram showing the progress of packages through the filters - during a simple virtual-database search and a multi-database - search. + [Here there should be a diagram showing the progress of + packages through the filters during a simple virtual-database + search and a multi-database search, but is seems that your + tool chain has not been able to include the diagram in this + document.]