1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1//EN"
3 "http://www.oasis-open.org/docbook/xml/4.1/docbookx.dtd"
5 <!ENTITY % local SYSTEM "local.ent">
7 <!ENTITY manref SYSTEM "manref.xml">
8 <!ENTITY progref SYSTEM "progref.xml">
9 <!ENTITY % common SYSTEM "common/common.ent">
11 <!-- Next line allows imagedata/@format="PDF" and is taken from
12 http://lists.oasis-open.org/archives/docbook/200303/msg00163.html
14 <!ENTITY % local.notation.class "| PDF">
15 <!-- Next line is necessary for some XML parsers, for reasons I
16 don't understand. I got this from
17 http://lists.oasis-open.org/archives/docbook/200303/msg00180.html
19 <!NOTATION PDF SYSTEM "PDF">
21 <!-- $Id: book.xml,v 1.50 2007-01-18 09:20:52 adam Exp $ -->
24 <title>Metaproxy - User's Guide and Reference</title>
27 <firstname>Adam</firstname><surname>Dickmeiss</surname>
30 <firstname>Marc</firstname><surname>Cromme</surname>
33 <firstname>Mike</firstname><surname>Taylor</surname>
36 <releaseinfo>&version;</releaseinfo>
38 <year>2005-2007</year>
39 <holder>Index Data ApS</holder>
43 This manual is part of Metaproxy version &version;.
46 Metaproxy is a universal router, proxy and encapsulated
47 metasearcher for information retrieval protocols. It accepts,
48 processes, interprets and redirects requests from IR clients using
49 standard protocols such as
50 <ulink url="&url.z39.50;">ANSI/NISO Z39.50</ulink>
51 (and in the future <ulink url="&url.sru;">SRU</ulink>
52 and <ulink url="&url.srw;">SRW</ulink>), as
53 well as functioning as a limited
54 <ulink url="&url.http;">HTTP</ulink> server.
55 Metaproxy is configured by an XML file which
56 specifies how the software should function in terms of routes that
57 the request packets can take through the proxy, each step on a
58 route being an instantiation of a filter. Filters come in many
59 types, one for each operation: accepting Z39.50 packets, logging,
60 query transformation, multiplexing, etc. Further filter-types can
61 be added as loadable modules to extend Metaproxy functionality,
65 Metaproxy is <emphasis>not</emphasis> open-source software, but
66 may be freely downloaded, unpacked, inspected, built and run for
67 evaluation purposes. Deployment requires a separate, commercial,
73 <imagedata fileref="common/id.png" format="PNG"/>
76 <imagedata fileref="common/id.eps" format="EPS"/>
83 <chapter id="introduction">
84 <title>Introduction</title>
88 <ulink url="&url.metaproxy;">Metaproxy</ulink>
89 is a stand alone program that acts as a universal router, proxy and
90 encapsulated metasearcher for information retrieval protocols such
91 as <ulink url="&url.z39.50;">Z39.50</ulink>, and in the future
92 <ulink url="&url.sru;">SRU</ulink> and <ulink url="&url.srw;">SRW</ulink>.
93 To clients, it acts as a server of these protocols: it can be searched,
94 records can be retrieved from it, etc.
95 To servers, it acts as a client: it searches in them,
96 retrieves records from them, etc. it satisfies its clients'
97 requests by transforming them, multiplexing them, forwarding them
98 on to zero or more servers, merging the results, transforming
99 them, and delivering them back to the client. In addition, it
100 acts as a simple <ulink url="&url.http;">HTTP</ulink> server; support
101 for further protocols can be added in a modular fashion, through the
102 creation of new filters.
107 Fish, bananas, cold pyjamas,
108 Mutton, beef and trout!
109 - attributed to Cole Porter.
112 Metaproxy is a more capable alternative to
113 <ulink url="&url.yazproxy;">YAZ Proxy</ulink>,
114 being more powerful, flexible, configurable and extensible. Among
115 its many advantages over the older, more pedestrian work are
116 support for multiplexing (encapsulated metasearching), routing by
117 database name, authentication and authorization and serving local
118 files via HTTP. Equally significant, its modular architecture
119 facilitites the creation of pluggable modules implementing further
123 This manual will describe how to install Metaproxy
124 before giving an overview of its architecture, then discussing the
125 key concept of a filter in some depth and giving an overview of
126 the various filter types, then discussing the configuration file
127 format. After this come several optional chapters which may be
128 freely skipped: a detailed discussion of virtual databases and
129 multi-database searching, some notes on writing extensions
130 (additional filter types) and a high-level description of the
131 source code. Finally comes the reference guide, which contains
132 instructions for invoking the <command>metaproxy</command>
133 program, and detailed information on each type of filter,
138 <chapter id="license">
139 <title>The Metaproxy License</title>
140 <orderedlist numeration="arabic">
143 You are allowed to download this software for evaluation purposes.
144 You can unpack it, build it, run it, see how it works and how it fits
145 your needs, all at zero cost.
150 You may NOT deploy the software. For the purposes of this license,
151 deployment means running it for any purpose other than evaluation,
152 whether or not you or anyone else makes a profit from doing so. If
153 you wish to deploy the software, you must first contact Index Data and
154 arrange to purchase a DEPLOYMENT LICENCE. If you are unsure
155 whether or not your proposed use of the software constitutes
156 deployment, email us at <literal>info@indexdata.com</literal>
162 You may modify your copy of the software (fix bugs, add features)
163 if you need to. We encourage you to send your changes back to us for
164 integration into the master copy, but you are not obliged to do so. You
165 may NOT pass your changes on to any other party.
170 There is NO WARRANTY for this software, to the extent permitted by
171 applicable law. We provide the software ``as is'' without warranty of
172 any kind, either expressed or implied, including, but not limited to, the
173 implied warranties of MERCHANTABILITY and FITNESS FOR A
174 PARTICULAR PURPOSE. The entire risk as to the quality and
175 performance of the software is with you. Should the software prove
176 defective, you assume the cost of all necessary servicing, repair or
177 correction. In no event unless required by applicable law will we be
178 liable to you for damages, arising out of the use of the software,
179 including but not limited to loss of data or data being rendered
185 All rights to the software are reserved by Index Data except where
186 this license explicitly says otherwise.
192 <chapter id="installation">
193 <title>Installation</title>
195 Metaproxy depends on the following tools/libraries:
197 <varlistentry><term><ulink url="&url.yazplusplus;">YAZ++</ulink></term>
200 This is a C++ library based on <ulink url="&url.yaz;">YAZ</ulink>.
204 <varlistentry><term><ulink url="&url.libxslt;">Libxslt</ulink></term>
206 <para>This is an XSLT processor - based on
207 <ulink url="&url.libxml2;">Libxml2</ulink>. Both Libxml2 and
208 Libxslt must be installed with the development components
209 (header files, etc.) as well as the run-time libraries.
213 <varlistentry><term><ulink url="&url.boost;">Boost</ulink></term>
216 The popular C++ library. Initial versions of Metaproxy
217 was built with 1.33.0. Version 1.33.1 works too.
224 In order to compile Metaproxy a modern C++ compiler is
225 required. Boost, in particular, requires the C++ compiler
226 to facilitate the newest features. Refer to Boost
227 <ulink url="&url.boost.compilers.status;">Compiler Status</ulink>
228 for more information.
231 We have successfully built Metaproxy using the compilers
232 <ulink url="&url.gcc;">GCC</ulink> version 4.0 and
233 <ulink url="&url.vstudio;">Microsoft Visual Studio</ulink> 2003/2005.
236 <section id="installation.unix">
237 <title>Installation on Unix (from Source)</title>
239 Here is a quick step-by-step guide on how to compile all the
240 tools that Metaproxy uses. Only few systems have none of the required
241 tools binary packages. If, for example, Libxml2/libxslt are already
242 installed as development packages use those (and omit compilation).
249 gunzip -c libxml2-version.tar.gz|tar xf -
257 gunzip -c libxslt-version.tar.gz|tar xf -
268 gunzip -c yaz-version.tar.gz|tar xf -
276 gunzip -c yazpp-version.tar.gz|tar xf -
287 gunzip -c boost-version.tar.gz|tar xf -
298 gunzip -c metaproxy-version.tar.gz|tar xf -
307 <section id="installation.debian">
308 <title>Installation on Debian GNU/Linux</title>
310 All dependencies for Metaproxy are available as
311 <ulink url="&url.debian;">Debian</ulink>
312 packages for the sarge (stable in 2005) and etch (testing in 2005)
316 The procedures for Debian based systems, such as
317 <ulink url="&url.ubuntu;">Ubuntu</ulink> is probably similar
320 There is currently no official Debian package for YAZ++.
321 And the Debian package for YAZ is probably too old.
322 Update the <filename>/etc/apt/sources.list</filename>
323 to include the Index Data repository.
324 See YAZ' <ulink url="&url.yaz.download.debian;">Download Debian</ulink>
325 for more information.
328 apt-get install libxslt1-dev
329 apt-get install libyazpp-dev
330 apt-get install libboost-dev
331 apt-get install libboost-thread-dev
332 apt-get install libboost-date-time-dev
333 apt-get install libboost-program-options-dev
334 apt-get install libboost-test-dev
337 With these packages installed, the usual configure + make
338 procedure can be used for Metaproxy as outlined in
339 <xref linkend="installation.unix"/>.
343 <section id="installation.windows">
344 <title>Installation on Windows</title>
346 Metaproxy can be compiled with Microsoft
347 <ulink url="&url.vstudio;">Visual Studio</ulink>.
348 Version 2003 (C 7.1) and 2005 (C 8.0) is known to work.
350 <section id="installation.windows.boost">
353 Get Boost from its <ulink url="&url.boost;">home page</ulink>.
354 You also need Boost Jam (an alternative to make).
355 That's also available from the Boost home page.
356 The files to be downloaded are called something like:
357 <filename>boost_1_33-1.exe</filename>
359 <filename>boost-jam-3.1.12-1-ntx86.zip</filename>.
360 Unpack Boost Jam first. Put <filename>bjam.exe</filename>
361 in your system path. Make a command prompt and ensure
362 it can be found automatically. If not check the PATH.
363 The Boost .exe is a self-extracting exe with
364 complete source for Boost. Compile that source with
365 Boost Jam (An alternative to Make).
366 The compilation takes a while.
367 For Visual Studio 2003, use
369 bjam "-sTOOLS=vc-7_1"
371 Here <literal>vc-7_1</literal> refers to a "Toolset" (compiler system).
372 For Visual Studio 2005, use
374 bjam "-sTOOLS=vc-8_0"
376 To install the libraries in a common place, use
378 bjam "-sTOOLS=vc-7_1" install
380 (or vc-8_0 for VS 2005).
383 By default, the Boost build process installs the resulting
384 libraries + header files in
385 <literal>\boost\lib</literal>, <literal>\boost\include</literal>.
388 For more information about installing Boost refer to the
389 <ulink url="&url.boost.getting.started;">getting started</ulink>
394 <section id="installation.windows.libxslt">
395 <title>Libxslt</title>
397 <ulink url="&url.libxslt;">Libxslt</ulink> can be downloaded
399 <ulink url="&url.libxml2.download.win32;">here</ulink>.
402 Libxslt has other dependencies, but these can all be downloaded
403 from the same site. Get the following:
404 iconv, zlib, libxml2, libxslt.
408 <section id="installation.windows.yaz">
411 <ulink url="&url.yaz;">YAZ</ulink> can be downloaded
413 <ulink url="&url.yaz.download.win32;">here</ulink>.
417 <section id="installation.windows.yazplusplus">
420 Get <ulink url="&url.yazplusplus;">YAZ++</ulink> as well.
421 Version 1.0 or later is required. For now get it from
423 <ulink url="&url.snapshot.download;">Snapshot area</ulink>.
426 YAZ++ includes NMAKE makefiles, similar to those found in the
431 <section id="installation.windows.metaproxy">
432 <title>Metaproxy</title>
434 Metaproxy is shipped with NMAKE makefiles as well - similar
435 to those found in the YAZ++/YAZ packages. Adjust this Makefile
436 to point to the proper locations of Boost, Libxslt, Libxml2,
437 zlib, iconv, yaz and yazpp.
441 <varlistentry><term><literal>DEBUG</literal></term>
443 If set to 1, the software is
444 compiled with debugging libraries (code generation is
445 multi-threaded debug DLL).
446 If set to 0, the software is compiled with release libraries
447 (code generation is multi-threaded DLL).
452 <term><literal>BOOST</literal></term>
455 Boost install location
461 <term><literal>BOOST_VERSION</literal></term>
464 Boost version (replace . with _).
470 <term><literal>BOOST_TOOLSET</literal></term>
479 <term><literal>LIBXSLT_DIR</literal>,
480 <literal>LIBXML2_DIR</literal> ..</term>
483 Specify the locations of Libxslt, libiconv, libxml2 and
492 After successful compilation you'll find
493 <literal>metaproxy.exe</literal> in the
494 <literal>bin</literal> directory.
502 <chapter id="architecture">
503 <title>The Metaproxy Architecture</title>
505 The Metaproxy architecture is based on three concepts:
506 the <emphasis>package</emphasis>,
507 the <emphasis>route</emphasis>
508 and the <emphasis>filter</emphasis>.
512 <term>Packages</term>
515 A package is request or response, encoded in some protocol,
516 issued by a client, making its way through Metaproxy, send to or
517 received from a server, or sent back to the client.
520 The core of a package is the protocol unit - for example, a
521 Z39.50 Init Request or Search Response, or an SRU searchRetrieve
522 URL or Explain Response. In addition to this core, a package
523 also carries some extra information added and used by Metaproxy
527 In general, packages are doctored as they pass through
528 Metaproxy. For example, when the proxy performs authentication
529 and authorization on a Z39.50 Init request, it removes the
530 authentication credentials from the package so that they are not
531 passed onto the back-end server; and when search-response
532 packages are obtained from multiple servers, they are merged
533 into a single unified package that makes its way back to the
542 Packages make their way through routes, which can be thought of
543 as programs that operate on the package data-type. Each
544 incoming package initially makes its way through a default
545 route, but may be switched to a different route based on various
546 considerations. Routes are made up of sequences of filters (see
555 Filters provide the individual instructions within a route, and
556 effect the necessary transformations on packages. A particular
557 configuration of Metaproxy is essentially a set of filters,
558 described by configuration details and arranged in order in one
559 or more routes. There are many kinds of filter - about a dozen
560 at the time of writing with more appearing all the time - each
561 performing a specific function and configured by different
565 The word ``filter'' is sometimes used rather loosely, in two
566 different ways: it may be used to mean a particular
567 <emphasis>type</emphasis> of filter, as when we speak of ``the
568 auth_simple filter'' or ``the multi filter''; or it may be used
569 to be a specific <emphasis>instance</emphasis> of a filter
570 within a Metaproxy configuration. For example, a single
571 configuration will often contain multiple instances of the
572 <literal>z3950_client</literal> filter. In
573 operational terms, of these is a separate filter. In practice,
574 context always make it clear which sense of the word ``filter''
578 Extensibility of Metaproxy is primarily through the creation of
579 plugins that provide new filters. The filter API is small and
580 conceptually simple, but there are many details to master. See
582 <link linkend="extensions">extensions</link>.
588 Since packages are created and handled by the system itself, and
589 routes are conceptually simple, most of the remainder of this
590 document concentrates on filters. After a brief overview of the
591 filter types follows, along with some thoughts on possible future
598 <chapter id="filters">
599 <title>Filters</title>
603 <title>Introductory notes</title>
605 It's useful to think of Metaproxy as an interpreter providing a small
606 number of primitives and operations, but operating on a very
607 complex data type, namely the ``package''.
610 A package represents a Z39.50 or SRU/W request (whether for Init,
611 Search, Scan, etc.) together with information about where it came
612 from. Packages are created by front-end filters such as
613 <literal>frontend_net</literal> (see below), which reads them from
614 the network; other front-end filters are possible. They then pass
615 along a route consisting of a sequence of filters, each of which
616 transforms the package and may also have side-effects such as
617 generating logging. Eventually, the route will yield a response,
618 which is sent back to the origin.
621 There are many kinds of filter: some that are defined statically
622 as part of Metaproxy, and others may be provided by third parties
623 and dynamically loaded. They all conform to the same simple API
624 of essentially two methods: <function>configure()</function> is
625 called at startup time, and is passed an XML DOM tree representing that
626 part of the configuration file that pertains to this filter
627 instance: it is expected to walk that tree extracting relevant
628 information; and <function>process()</function> is called every
629 time the filter has to processes a package.
632 While all filters provide the same API, there are different modes
633 of functionality. Some filters are sources: they create
635 (<literal>frontend_net</literal>);
636 others are sinks: they consume packages and return a result
637 (<literal>backend_test</literal>,
638 <literal>bounce</literal>,
639 <literal>http_file</literal>,
640 <literal>z3950_client</literal>);
641 the others are true filters, that read, process and pass on the
642 packages they are fed
643 (<literal>auth_simple</literal>,
644 <literal>log</literal>,
645 <literal>multi</literal>,
646 <literal>query_rewrite</literal>,
647 <literal>record_transform</literal>,
648 <literal>session_shared</literal>,
649 <literal>sru_z3950</literal>,
650 <literal>template</literal>,
651 <literal>virt_db</literal>).
656 <section id="overview.filter.types">
657 <title>Overview of filter types</title>
659 We now briefly consider each of the types of filter supported by
660 the core Metaproxy binary. This overview is intended to give a
661 flavor of the available functionality; more detailed information
662 about each type of filter is included below in
663 <xref linkend="reference"/>.
666 The filters are here named by the string that is used as the
667 <literal>type</literal> attribute of a
668 <literal><filter></literal> element in the configuration
669 file to request them, with the name of the class that implements
670 them in parentheses. (The classname is not needed for normal
671 configuration and use of Metaproxy; it is useful only to
675 The filters are here listed in alphabetical order:
682 New virt_db-alike that does inteligent peer choice, explain merging,
683 adds FD&N to explain. Keeps init responses (like "virt_db Classic"),
684 makes routing choices based on local explain knowledge. Ref IDDI
687 Filter to convert Explain Classic to ZeeRex.
689 CQL2PQF (which needs augmented ZeeRex) - MARC for Talis.
693 Figure out what additional information we need in:
694 ZeeRex (check against D3.1)
695 Init request (e.g. loop detection)
696 Query package (e.g. number of hops)
697 Query response (e.g. record source)
702 <title><literal>auth_simple</literal>
703 (mp::filter::AuthSimple)</title>
705 Simple authentication and authorization. The configuration
706 specifies the name of a file that is the user register, which
707 lists <varname>username</varname>:<varname>password</varname>
708 pairs, one per line, colon separated. When a session begins, it
709 is rejected unless username and passsword are supplied, and match
710 a pair in the register. The configuration file may also specific
711 the name of another file that is the target register: this lists
712 lists <varname>username</varname>:<varname>dbname</varname>,<varname>dbname</varname>...
713 sets, one per line, with multiple database names separated by
714 commas. When a search is processed, it is rejected unless the
715 database to be searched is one of those listed as available to
721 <title><literal>backend_test</literal>
722 (mp::filter::Backend_test)</title>
724 A partial sink that provides dummy responses in the manner of the
725 <literal>yaz-ztest</literal> Z39.50 server. This is useful only
726 for testing. Seriously, you don't need this. Pretend you didn't
727 even read this section.
732 <title><literal>bounce</literal>
733 (mp::filter::Bounce)</title>
735 A sink that swallows <emphasis>all packages</emphasis>,
736 and returns them almost unprocessed.
737 It never sends any package of any type further down the row, but
738 sets Z39.50 packages to Z_Close, and HTTP_Request packages to
739 HTTP_Response err code 400 packages, and adds a suitable bounce
741 The bounce filter is usually added at end of each filter chain
742 config.xml to prevent infinite hanging of for example HTTP
743 requests packages when only the Z39.50 client partial sink
744 filter is found in the
750 <title><literal>frontend_net</literal>
751 (mp::filter::FrontendNet)</title>
753 A source that accepts Z39.50 connections from a port
754 specified in the configuration, reads protocol units, and
755 feeds them into the next filter in the route. When the result is
756 received, it is returned to the original origin.
761 <title><literal>http_file</literal>
762 (mp::filter::HttpFile)</title>
764 A partial sink which swallows only HTTP_Request packages, and
765 returns the contents of files from the local
766 filesystem in response to HTTP requests.
767 It lets Z39.50 packages and all other forthcoming package types
770 does mean that Metaproxy is also a Web-server in its spare time. So
771 far it does not contain either an email-reader or a Lisp
772 interpreter, but that day is surely coming.)
777 <title><literal>load_balance</literal>
778 (mp::filter::LoadBalance)</title>
780 Performs load balancing for incoming Z39.50 init requests.
781 It is used together with the <literal>virt_db</literal> filter,
782 but unlike the <literal>multi</literal> filter it does send an
783 entire session to only one of the virtual backends. The
784 <literal>load_balance</literal> filter is assuming that
785 all backend targets have equal content, and chooses the backend
786 with least load cost for a new session.
789 This filter is experimental and yet not mature for heavy load
797 <title><literal>log</literal>
798 (mp::filter::Log)</title>
800 Writes logging information to standard output, and passes on
801 the package unchanged. A log file name can be specified, as well
802 as multiple different logging formats.
807 <title><literal>multi</literal>
808 (mp::filter::Multi)</title>
810 Performs multi-database searching.
812 <link linkend="multidb">the extended discussion</link>
813 of virtual databases and multi-database searching below.
818 <title><literal>query_rewrite</literal>
819 (mp::filter::QueryRewrite)</title>
821 Rewrites Z39.50 Type-1 and Type-101 (``RPN'') queries by a
822 three-step process: the query is transliterated from Z39.50
823 packet structures into an XML representation; that XML
824 representation is transformed by an XSLT stylesheet; and the
825 resulting XML is transliterated back into the Z39.50 packet
832 <title><literal>record_transform</literal>
833 (mp::filter::RecordTransform)</title>
835 This filter acts only on Z3950 present requests, and let all
836 other types of packages and requests pass untouched. It's use is
837 twofold: blocking Z3950 present requests, which the backend
838 server does not understand and can not honor, and transforming
839 the present syntax and elementset name according to the rules
840 specified, to fetch only existing record formats, and transform
841 them on the fly to requested record syntaxes.
846 <title><literal>session_shared</literal>
847 (mp::filter::SessionShared)</title>
849 When this is finished, it will implement global sharing of
850 result sets (i.e. between threads and therefore between
851 clients), yielding performance improvements especially when
852 incoming requests are from a stateless environment such as a
853 web-server, in which the client process representing a session
854 might be any one of many. However:
858 This filter is not yet completed.
864 <title><literal>sru_z3950</literal>
865 (mp::filter::SRUtoZ3950)</title>
867 This filter transforms valid
868 SRU GET/POST/SOAP searchRetrieve requests to Z3950 init, search,
869 and present requests, and wraps the
870 received hit counts and XML records into suitable SRU response
872 The <literal>sru_z3950</literal> filter processes also SRU
873 GET/POST/SOAP explain requests, returning
874 either the absolute minimum required by the standard, or a full
875 pre-defined ZeeReX explain record.
877 <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
878 standard pages and the
879 <ulink url="&url.sru.explain;">SRU Explain</ulink> pages
880 for more information on the correct explain syntax.
881 SRU scan requests are not supported yet.
886 <title><literal>template</literal>
887 (mp::filter::Template)</title>
889 Does nothing at all, merely passing the packet on. (Maybe it
890 should be called <literal>nop</literal> or
891 <literal>passthrough</literal>?) This exists not to be used, but
892 to be copied - to become the skeleton of new filters as they are
893 written. As with <literal>backend_test</literal>, this is not
894 intended for civilians.
899 <title><literal>virt_db</literal>
900 (mp::filter::VirtualDB)</title>
902 Performs virtual database selection: based on the name of the
903 database in the search request, a server is selected, and its
904 address added to the request in a <literal>VAL_PROXY</literal>
905 otherInfo packet. It will subsequently be used by a
906 <literal>z3950_client</literal> filter.
908 <link linkend="multidb">the extended discussion</link>
909 of virtual databases and multi-database searching below.
914 <title><literal>z3950_client</literal>
915 (mp::filter::Z3950Client)</title>
917 A partial sink which swallows only Z39.50 packages.
918 It performs Z39.50 searching and retrieval by proxying the
919 packages that are passed to it. Init requests are sent to the
920 address specified in the <literal>VAL_PROXY</literal> otherInfo
921 attached to the request: this may have been specified by client,
922 or generated by a <literal>virt_db</literal> filter earlier in
923 the route. Subsequent requests are sent to the same address,
924 which is remembered at Init time in a Session object.
925 HTTP_Request packages and all other forthcoming package types
926 are passed untouched.
932 <title><literal>zeerex_explain</literal>
933 (mp::filter::ZeerexExplain)</title>
935 This filter acts as a sink for
936 Z39.50 explain requests, returning a static ZeeReX
937 Explain XML record from the config section. All other packages
940 <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
942 for more information on the correct explain syntax.
946 This filter is not yet completed.
955 <section id="future.directions">
956 <title>Future directions</title>
958 Some other filters that do not yet exist, but which would be
959 useful, are briefly described. These may be added in future
960 releases (or may be created by third parties, as loadable
966 <term><literal>frontend_cli</literal> (source)</term>
969 Command-line interface for generating requests.
974 <term><literal>sru_client</literal> (sink)</term>
977 SRU/GET and SRU/SOAP searching and retrieval.
982 <term><literal>opensearch_client</literal> (sink)</term>
985 A9 OpenSearch searching and retrieval.
995 <chapter id="configuration">
996 <title>Configuration: the Metaproxy configuration file format</title>
1000 <title>Introductory notes</title>
1002 If Metaproxy is an interpreter providing operations on packages, then
1003 its configuration file can be thought of as a program for that
1004 interpreter. Configuration is by means of a single XML file, the name
1005 of which is supplied as the sole command-line argument to the
1006 <command>metaproxy</command> program. (See
1007 <xref linkend="reference"/> below for more information on invoking
1012 <section id="overview.xml.structure">
1013 <title>Overview of the config file XML structure</title>
1015 All elements and attributes are in the namespace
1016 <ulink url="http://indexdata.com/metaproxy"/>.
1017 This is most easily achieved by setting the default namespace on
1018 the top-level element, as here:
1021 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1024 The top-level element is <metaproxy>. This contains a
1025 <start> element, a <filters> element and a
1026 <routes> element, in that order. <filters> is
1027 optional; the other two are mandatory. All three are
1031 The <start> element is empty, but carries a
1032 <literal>route</literal> attribute, whose value is the name of
1033 route at which to start running - analogous to the name of the
1034 start production in a formal grammar.
1037 If present, <filters> contains zero or more <filter>
1038 elements. Each filter carries a <literal>type</literal> attribute
1039 which specifies what kind of filter is being defined
1040 (<literal>frontend_net</literal>, <literal>log</literal>, etc.)
1041 and contain various elements that provide suitable configuration
1042 for a filter of its type. The filter-specific elements are
1044 <xref linkend="reference"/>.
1045 Filters defined in this part of the file must carry an
1046 <literal>id</literal> attribute so that they can be referenced
1050 <routes> contains one or more <route> elements, each
1051 of which must carry an <literal>id</literal> element. One of the
1052 routes must have the ID value that was specified as the start
1053 route in the <start> element's <literal>route</literal>
1054 attribute. Each route contains zero or more <filter>
1055 elements. These are of two types. They may be empty, but carry a
1056 <literal>refid</literal> attribute whose value is the same as the
1057 <literal>id</literal> of a filter previously defined in the
1058 <filters> section. Alternatively, a route within a filter
1059 may omit the <literal>refid</literal> attribute, but contain
1060 configuration elements similar to those used for filters defined
1061 in the <filters> section. (In other words, each filter in a
1062 route may be included either by reference or by physical
1068 <section id="example.configuration">
1069 <title>An example configuration</title>
1071 The following is a small, but complete, Metaproxy configuration
1072 file (included in the distribution as
1073 <literal>metaproxy/etc/config1.xml</literal>).
1074 This file defines a very simple configuration that simply proxies
1075 to whatever back-end server the client requests, but logs each
1076 request and response. This can be useful for debugging complex
1077 client-server dialogues.
1079 <screen><![CDATA[<?xml version="1.0"?>
1080 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1081 <start route="start"/>
1083 <filter id="frontend" type="frontend_net">
1086 <filter id="backend" type="z3950_client">
1091 <filter refid="frontend"/>
1092 <filter type="log"/>
1093 <filter refid="backend"/>
1094 <filter type="bounce"/>
1100 It works by defining a single route, called
1101 <literal>start</literal>, which consists of a sequence of four
1102 filters. The first and last of these are included by reference:
1103 their <literal><filter></literal> elements have
1104 <literal>refid</literal> attributes that refer to filters defined
1105 within the prior <literal><filters></literal> section. The
1106 middle filter is included inline in the route.
1109 The four filters in the route are as follows: first, a
1110 <literal>frontend_net</literal> filter accepts Z39.50 requests
1111 from any host on port 9000; then these requests are passed through
1112 a <literal>log</literal> filter that emits a message for each
1113 request; they are then fed into a <literal>z3950_client</literal>
1114 filter, which forwards all Z39.50 requests to the client-specified
1115 back-end Z39.509 server. Those Z39.50 packages are returned by the
1116 <literal>z3950_client</literal> filter, with the response data
1117 filled by the external Z39.50 server targeted.
1118 All non-Z39.50 packages are passed through to the
1119 <literal>bounce</literal> filter, which definitely bounces
1120 everything, including fish, bananas, cold pyjamas,
1121 mutton, beef and trout packages.
1122 When the response arrives, it is handed
1123 back to the <literal>log</literal> filter, which emits another
1124 message; and then to the <literal>frontend_net</literal> filter,
1125 which returns the response to the client.
1128 <section id="checking.xml.syntax">
1129 <title>Config file syntax checking</title>
1131 The distribution contains RelaxNG Compact and XML syntax checking
1132 files, as well as XML Schema files. These are found in the
1135 xml/schema/metaproxy.rnc
1136 xml/schema/metaproxy.rng
1137 xml/schema/metaproxy.xsd
1139 and can be used to verify or debug the XML structure of
1140 configuration files. For example, using the utility
1141 <filename>xmllint</filename>, syntax checking is done like this:
1143 xmllint --noout --schema xml/schema/metaproxy.xsd etc/config-local.xml
1144 xmllint --noout --relaxng xml/schema/metaproxy.rng etc/config-local.xml
1146 (A recent version of <literal>libxml2</literal> is required, as
1147 support for XML Schemas is a relatively recent addition.)
1150 You can of course use any other RelaxNG or XML Schema compliant tool
1158 <chapter id="multidb">
1159 <title>Virtual databases and multi-database searching</title>
1163 <title>Introductory notes</title>
1165 Two of Metaproxy's filters are concerned with multiple-database
1166 operations. Of these, <literal>virt_db</literal> can work alone
1167 to control the routing of searches to one of a number of servers,
1168 while <literal>multi</literal> can work together with
1169 <literal>virt_db</literal> to perform multi-database searching, merging
1170 the results into a unified result-set - ``metasearch in a box''.
1173 The interaction between
1174 these two filters is necessarily complex: it reflects the real,
1175 irreducible complexity of multi-database searching in a protocol such
1176 as Z39.50 that separates initialization from searching, and in
1177 which the database to be searched is not known at initialization
1181 It's possible to use these filters without understanding the
1182 details of their functioning and the interaction between them; the
1183 next two sections of this chapter are ``HOW-TO'' guides for doing
1184 just that. However, debugging complex configurations will require
1185 a deeper understanding, which the last two sections of this
1186 chapters attempt to provide.
1191 <section id="multidb.virt_db">
1192 <title>Virtual databases with the <literal>virt_db</literal> filter</title>
1194 Working alone, the purpose of the
1195 <literal>virt_db</literal>
1196 filter is to route search requests to one of a selection of
1197 back-end databases. In this way, a single Z39.50 endpoint
1198 (running Metaproxy) can provide access to several different
1199 underlying services, including those that would otherwise be
1200 inaccessible due to firewalls. In many useful configurations, the
1201 back-end databases are local to the Metaproxy installation, but
1202 the software does not enforce this, and any valid Z39.50 servers
1203 may be used as back-ends.
1206 For example, a <literal>virt_db</literal>
1207 filter could be set up so that searches in the virtual database
1208 ``lc'' are forwarded to the Library of Congress bibliographic
1209 catalogue server, and searches in the virtual database ``marc''
1210 are forwarded to the toy database of MARC records that Index Data
1211 hosts for testing purposes. A <literal>virt_db</literal>
1212 configuration to make this switch would look like this:
1214 <screen><![CDATA[<filter type="virt_db">
1216 <database>lc</database>
1217 <target>z3950.loc.gov:7090/voyager</target>
1220 <database>marc</database>
1221 <target>indexdata.com/marc</target>
1223 </filter>]]></screen>
1225 As well as being useful in it own right, this filter also provides
1226 the foundation for multi-database searching.
1231 <section id="multidb.multi">
1232 <title>Multi-database search with the <literal>multi</literal> filter</title>
1234 To arrange for Metaproxy to broadcast searches to multiple back-end
1235 servers, the configuration needs to include two components: a
1236 <literal>virt_db</literal>
1237 filter that specifies multiple
1238 <literal><target></literal>
1239 elements, and a subsequent
1240 <literal>multi</literal>
1241 filter. Here, for example, is a complete configuration that
1242 broadcasts searches to both the Library of Congress catalogue and
1243 Index Data's tiny testing database of MARC records:
1245 <screen><![CDATA[<?xml version="1.0"?>
1246 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1247 <start route="start"/>
1250 <filter type="frontend_net">
1251 <threads>10</threads>
1254 <filter type="virt_db">
1256 <database>lc</database>
1257 <target>z3950.loc.gov:7090/voyager</target>
1260 <database>marc</database>
1261 <target>indexdata.com/marc</target>
1264 <database>all</database>
1265 <target>z3950.loc.gov:7090/voyager</target>
1266 <target>indexdata.com/marc</target>
1269 <filter type="multi"/>
1270 <filter type="z3950_client">
1271 <timeout>30</timeout>
1273 <filter type="bounce"/>
1276 </metaproxy>]]></screen>
1279 <literal>virt_db</literal>
1280 filter that specifies multiple
1281 <literal><target></literal>
1282 elements but without a subsequent
1283 <literal>multi</literal>
1284 filter yields surprising and undesirable results, as will be
1285 described below. Don't do that.)
1288 Metaproxy can be invoked with this configuration as follows:
1290 <screen>../src/metaproxy --config config-simple-multi.xml</screen>
1292 And thereafter, Z39.50 clients can connect to the running server
1293 (on port 9000, as specified in the configuration) and search in
1294 any of the databases
1295 <literal>lc</literal> (the Library of Congress catalogue),
1296 <literal>marc</literal> (Index Data's test database of MARC records)
1298 <literal>all</literal> (both of these). As an example, a session
1299 using the YAZ command-line client <literal>yaz-client</literal> is
1300 here included (edited for brevity and clarity):
1302 <screen><![CDATA[$ yaz-client @:9000
1306 Search was a success.
1307 Number of hits: 10000, setno 1
1311 Search was a success.
1312 Number of hits: 10, setno 3
1316 Search was a success.
1317 Number of hits: 10010, setno 4
1320 [marc]Record type: USmarc
1323 005 00000000000000.0
1324 008 910710c19910701nju 00010 eng
1328 100 10 $a Jack Collins
1329 245 10 $a How to program a computer
1335 [VOYAGER]Record type: USmarc
1337 005 20041229102447.0
1338 008 030910s2004 caua 000 0 eng
1339 035 $a (DLC) 2003112666
1340 906 $a 7 $b cbc $c orignew $d 4 $e epcn $f 20 $g y-gencatlg
1341 925 0 $a acquire $b 1 shelf copy $x policy default
1342 955 $a pc10 2003-09-10 $a pv12 2004-06-23 to SSCD; $h sj05 2004-11-30 $e sj05 2004-11-30 to Shelf.
1345 040 $a DLC $c DLC $d DLC
1346 050 00 $a MLCM 2004/03312 (G)
1347 245 10 $a 007, everything or nothing : $b Prima's official strategy guide / $c created by Kaizen Media Group.
1348 246 3 $a Double-O-seven, everything or nothing
1349 246 30 $a Prima's official strategy guide
1350 260 $a Roseville, CA : $b Prima Games, $c c2004.
1351 300 $a 161 p. : $b col. ill. ; $c 28 cm.
1352 500 $a "Platforms: Nintendo GameCube, Macintosh, PC, PlayStation 2 computer entertainment system, Xbox"--P. [4] of cover.
1353 650 0 $a Video games.
1354 710 2 $a Kaizen Media Group.
1355 856 42 $3 Publisher description $u http://www.loc.gov/catdir/description/random052/2003112666.html
1360 As can be seen, the first record in the result set is from the
1361 Index Data test database, and the second from the Library of
1362 Congress database. The result-set continues alternating records
1363 round-robin style until the point where one of the databases'
1364 records are exhausted.
1367 This example uses only two back-end databases; more may be used.
1368 There is no limitation imposed on the number of databases that may
1369 be metasearched in this way: issues of resource usage and
1370 administrative complexity dictate the practical limits.
1373 What happens when one of the databases doesn't respond? By default,
1374 the entire multi-database search fails, and the appropriate
1375 diagnostic is returned to the client. This is usually appropriate
1376 during development, when technicians need maximum information, but
1377 can be inconvenient in deployment, when users typically don't want
1378 to be bothered with problems of this kind and prefer just to get
1379 the records from the databases that are available. To obtain this
1380 latter behavior add an empty
1381 <literal><hideunavailable></literal>
1383 <literal>multi</literal> filter:
1385 <screen><![CDATA[ <filter type="multi">
1387 </filter>]]></screen>
1389 Under this regime, an error is reported to the client only if
1390 <emphasis>all</emphasis> the databases in a multi-database search
1396 <section id="multidb.what">
1397 <title>What's going on?</title>
1399 <title>Lark's vomit</title>
1401 This section goes into a level of technical detail that is
1402 probably not necessary in order to configure and use Metaproxy.
1403 It is provided only for those who like to know how things work.
1404 You should feel free to skip on to the next section if this one
1405 doesn't seem like fun.
1409 Hold on tight - this may get a little hairy.
1412 In the general course of things, a Z39.50 Init request may carry
1413 with it an otherInfo packet of type <literal>VAL_PROXY</literal>,
1414 whose value indicates the address of a Z39.50 server to which the
1415 ultimate connection is to be made. (This otherInfo packet is
1416 supported by YAZ-based Z39.50 clients and servers, but has not yet
1417 been ratified by the Maintenance Agency and so is not widely used
1418 in non-Index Data software. We're working on it.)
1419 The <literal>VAL_PROXY</literal> packet functions
1420 analogously to the absoluteURI-style Request-URI used with the GET
1421 method when a web browser asks a proxy to forward its request: see
1423 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2"
1424 >Request-URI</ulink>
1426 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616.html"
1427 >the HTTP 1.1 specification</ulink>.
1430 Within Metaproxy, Search requests that are part of the same
1431 session as an Init request that carries a
1432 <literal>VAL_PROXY</literal> otherInfo are also annotated with the
1433 same information. The role of the <literal>virt_db</literal>
1434 filter is to rewrite this otherInfo packet dependent on the
1435 virtual database that the client wants to search.
1438 When Metaproxy receives a Z39.50 Init request from a client, it
1439 doesn't immediately forward that request to the back-end server.
1440 Why not? Because it doesn't know <emphasis>which</emphasis>
1441 back-end server to forward it to until the client sends a Search
1442 request that specifies the database that it wants to search in.
1443 Instead, it just treasures the Init request up in its heart; and,
1444 later, the first time the client does a search on one of the
1445 specified virtual databases, a connection is forged to the
1446 appropriate server and the Init request is forwarded to it. If,
1447 later in the session, the same client searches in a different
1448 virtual database, then a connection is forged to the server that
1449 hosts it, and the same cached Init request is forwarded there,
1453 All of this clever Init-delaying is done by the
1454 <literal>frontend_net</literal> filter. The
1455 <literal>virt_db</literal> filter knows nothing about it; in
1456 fact, because the Init request that is received from the client
1457 doesn't get forwarded until a Search request is received, the
1458 <literal>virt_db</literal> filter (and the
1459 <literal>z3950_client</literal> filter behind it) doesn't even get
1460 invoked at Init time. The <emphasis>only</emphasis> thing that a
1461 <literal>virt_db</literal> filter ever does is rewrite the
1462 <literal>VAL_PROXY</literal> otherInfo in the requests that pass
1466 It is possible for a <literal>virt_db</literal> filter to contain
1468 <literal><target></literal>
1469 elements. What does this mean? Only that the filter will add
1470 multiple <literal>VAL_PROXY</literal> otherInfo packets to the
1471 Search requests that pass through it. That's because the virtual
1472 DB filter is dumb, and does exactly what it's told - no more, no
1474 If a Search request with multiple <literal>VAL_PROXY</literal>
1475 otherInfo packets reaches a <literal>z3950_client</literal>
1476 filter, this is an error. That filter doesn't know how to deal
1477 with multiple targets, so it will either just pick one and search
1478 in it, or (better) fail with an error message.
1481 The <literal>multi</literal> filter comes to the rescue! This is
1482 the only filter that knows how to deal with multiple
1483 <literal>VAL_PROXY</literal> otherInfo packets, and it does so by
1484 making multiple copies of the entire Search request: one for each
1485 <literal>VAL_PROXY</literal>. Each of these new copies is then
1486 passed down through the remaining filters in the route. (The
1487 copies are handled in parallel though the
1488 spawning of new threads.) Since the copies each have only one
1489 <literal>VAL_PROXY</literal> otherInfo, they can be handled by the
1490 <literal>z3950_client</literal> filter, which happily deals with
1491 each one individually. When the results of the individual
1492 searches come back up to the <literal>multi</literal> filter, it
1493 merges them into a single Search response, which is what
1494 eventually makes it back to the client.
1499 <section id="multidb.picture">
1500 <title>A picture is worth a thousand words (but only five hundred on 64-bit architectures)</title>
1504 <imagedata fileref="multi.pdf" format="PDF" scale="50"/>
1507 <imagedata fileref="multi.png" format="PNG"/>
1510 <!-- Fall back if none of the images can be used -->
1512 [Here there should be a diagram showing the progress of
1513 packages through the filters during a simple virtual-database
1514 search and a multi-database search, but is seems that your
1515 tool chain has not been able to include the diagram in this
1519 <!-- ### This used to work with an older version of DocBook
1521 <para>Caption: progress of packages through filters.</para>
1524 </inlinemediaobject>
1531 <chapter id="extensions">
1532 <title>Writing extensions for Metaproxy</title>
1533 <para>### To be written</para>
1539 <chapter id="classes">
1540 <title>Classes in the Metaproxy source code</title>
1544 <title>Introductory notes</title>
1546 <emphasis>Stop! Do not read this!</emphasis>
1547 You won't enjoy it at all. You should just skip ahead to
1548 <xref linkend="reference"/>,
1550 <!-- The remainder of this paragraph is lifted verbatim from
1551 Douglas Adams' _Hitch Hiker's Guide to the Galaxy_, chapter 8 -->
1552 you things you really need to know, like the fact that the
1553 fabulously beautiful planet Bethselamin is now so worried about
1554 the cumulative erosion by ten billion visiting tourists a year
1555 that any net imbalance between the amount you eat and the amount
1556 you excrete whilst on the planet is surgically removed from your
1557 bodyweight when you leave: so every time you go to the lavatory it
1558 is vitally important to get a receipt.
1561 This chapter contains documentation of the Metaproxy source code, and is
1562 of interest only to maintainers and developers. If you need to
1563 change Metaproxy's behavior or write a new filter, then you will most
1564 likely find this chapter helpful. Otherwise it's a waste of your
1565 good time. Seriously: go and watch a film or something.
1566 <citetitle>This is Spinal Tap</citetitle> is particularly good.
1569 Still here? OK, let's continue.
1572 In general, classes seem to be named big-endianly, so that
1573 <literal>FactoryFilter</literal> is not a filter that filters
1574 factories, but a factory that produces filters; and
1575 <literal>FactoryStatic</literal> is a factory for the statically
1576 registered filters (as opposed to those that are dynamically
1581 <section id="individual.classes">
1582 <title>Individual classes</title>
1584 The classes making up the Metaproxy application are here listed by
1585 class-name, with the names of the source files that define them in
1590 <title><literal>mp::FactoryFilter</literal>
1591 (<filename>factory_filter.cpp</filename>)</title>
1593 A factory class that exists primarily to provide the
1594 <literal>create()</literal> method, which takes the name of a
1595 filter class as its argument and returns a new filter of that
1596 type. To enable this, the factory must first be populated by
1597 calling <literal>add_creator()</literal> for static filters (this
1598 is done by the <literal>FactoryStatic</literal> class, see below)
1599 and <literal>add_creator_dyn()</literal> for filters loaded
1605 <title><literal>mp::FactoryStatic</literal>
1606 (<filename>factory_static.cpp</filename>)</title>
1608 A subclass of <literal>FactoryFilter</literal> which is
1609 responsible for registering all the statically defined filter
1610 types. It does this by knowing about all those filters'
1611 structures, which are listed in its constructor. Merely
1612 instantiating this class registers all the static classes. It is
1613 for the benefit of this class that <literal>struct
1614 metaproxy_1_filter_struct</literal> exists, and that all the filter
1615 classes provide a static object of that type.
1620 <title><literal>mp::filter::Base</literal>
1621 (<filename>filter.cpp</filename>)</title>
1623 The virtual base class of all filters. The filter API is, on the
1624 surface at least, extremely simple: two methods.
1625 <literal>configure()</literal> is passed an XML DOM tree representing
1626 that part of the configuration file that pertains to this filter
1627 instance, and is expected to walk that tree extracting relevant
1628 information. And <literal>process()</literal> processes a
1629 package (see below). That surface simplicity is a bit
1630 misleading, as <literal>process()</literal> needs to know a lot
1631 about the <literal>Package</literal> class in order to do
1637 <title><literal>mp::filter::AuthSimple</literal>,
1638 <literal>Backend_test</literal>, etc.
1639 (<filename>filter_auth_simple.cpp</filename>,
1640 <filename>filter_backend_test.cpp</filename>, etc.)</title>
1642 Individual filters. Each of these is implemented by a header and
1643 a source file, named <filename>filter_*.hpp</filename> and
1644 <filename>filter_*.cpp</filename> respectively. All the header
1645 files should be pretty much identical, in that they declare the
1646 class, including a private <literal>Rep</literal> class and a
1647 member pointer to it, and the two public methods.
1650 The source file for each filter needs to supply:
1655 A definition of the private <literal>Rep</literal> class.
1660 Some boilerplate constructors and destructors.
1665 A <literal>configure()</literal> method that uses the
1666 appropriate XML fragment.
1671 Most important, the <literal>process()</literal> method that
1672 does all the actual work.
1679 <title><literal>mp::Package</literal>
1680 (<filename>package.cpp</filename>)</title>
1682 Represents a package on its way through the series of filters
1683 that make up a route. This is essentially a Z39.50 or SRU APDU
1684 together with information about where it came from, which is
1685 modified as it passes through the various filters.
1690 <title><literal>mp::Pipe</literal>
1691 (<filename>pipe.cpp</filename>)</title>
1693 This class provides a compatibility layer so that we have an IPC
1694 mechanism that works the same under Unix and Windows. It's not
1695 particularly exciting.
1700 <title><literal>mp::RouterChain</literal>
1701 (<filename>router_chain.cpp</filename>)</title>
1708 <title><literal>mp::RouterFleXML</literal>
1709 (<filename>router_flexml.cpp</filename>)</title>
1716 <title><literal>mp::Session</literal>
1717 (<filename>session.cpp</filename>)</title>
1724 <title><literal>mp::ThreadPoolSocketObserver</literal>
1725 (<filename>thread_pool_observer.cpp</filename>)</title>
1732 <title><literal>mp::util</literal>
1733 (<filename>util.cpp</filename>)</title>
1735 A namespace of various small utility functions and classes,
1736 collected together for convenience. Most importantly, includes
1737 the <literal>mp::util::odr</literal> class, a wrapper for YAZ's
1743 <title><literal>mp::xml</literal>
1744 (<filename>xmlutil.cpp</filename>)</title>
1746 A namespace of various XML utility functions and classes,
1747 collected together for convenience.
1753 <section id="other.source.files">
1754 <title>Other Source Files</title>
1756 In addition to the Metaproxy source files that define the classes
1757 described above, there are a few additional files which are
1758 briefly described here:
1762 <term><literal>metaproxy_prog.cpp</literal></term>
1765 The main function of the <command>metaproxy</command> program.
1770 <term><literal>ex_router_flexml.cpp</literal></term>
1773 Identical to <literal>metaproxy_prog.cpp</literal>: it's not clear why.
1778 <term><literal>test_*.cpp</literal></term>
1781 Unit-tests for various modules.
1787 ### Still to be described:
1788 <literal>ex_filter_frontend_net.cpp</literal>,
1789 <literal>filter_dl.cpp</literal>,
1790 <literal>plainfile.cpp</literal>,
1791 <literal>tstdl.cpp</literal>.
1797 <reference id="reference">
1798 <title>Reference</title>
1801 The material in this chapter is drawn directly from the individual
1802 manual entries. In particular, the Metaproxy invocation section is
1803 available using <command>man metaproxy</command>, and the section
1804 on each individual filter is available using the name of the filter
1805 as the argument to the <command>man</command> command.
1812 <!-- Keep this comment at the end of the file
1817 sgml-minimize-attributes:nil
1818 sgml-always-quote-attributes:t
1821 sgml-parent-document: nil
1822 sgml-local-catalogs: nil
1823 sgml-namecase-general:t