1 <!-- $Id: book.xml,v 1.13 2006-04-22 13:28:05 adam Exp $ -->
3 <title>Metaproxy - User's Guide and Reference</title>
5 <firstname>Mike</firstname><surname>Taylor</surname>
8 <firstname>Adam</firstname><surname>Dickmeiss</surname>
12 <holder>Index Data ApS</holder>
16 Metaproxy is a universal router, proxy and encapsulated
17 metasearcher for information retrieval protocols. It accepts,
18 processes, interprets and redirects requests from IR clients using
19 standard protocols such as ANSI/NISO Z39.50 (and in the future SRU
20 and SRW), as well as functioning as a limited
21 HTTP server. Metaproxy is configured by an XML file which
22 specifies how the software should function in terms of routes that
23 the request packets can take through the proxy, each step on a
24 route being an instantiation of a filter. Filters come in many
25 types, one for each operation: accepting Z39.50 packets, logging,
26 query transformation, multiplexing, etc. Further filter-types can
27 be added as loadable modules to extend Metaproxy functionality,
31 The terms under which Metaproxy will be distributed have yet to be
32 established, but it will not necessarily be open source; so users
33 should not at this stage redistribute the code without explicit
34 written permission from the copyright holders, Index Data ApS.
39 <chapter id="introduction">
40 <title>Introduction</title>
44 <ulink url="http://www.indexdata.com/metaproxy/">Metaproxy</ulink>
45 is a standalone program that acts as a universal router, proxy and
46 encapsulated metasearcher for information retrieval protocols such
47 as Z39.50, and in the future SRU and SRW. To clients, it acts as a
49 protocols: it can be searched, records can be retrieved from it,
50 etc. To servers, it acts as a client: it searches in them,
51 retrieves records from them, etc. it satisfies its clients'
52 requests by transforming them, multiplexing them, forwarding them
53 on to zero or more servers, merging the results, transforming
54 them, and delivering them back to the client. In addition, it
55 acts as a simple HTTP server; support for further protocols can be
56 added in a modular fashion, through the creation of new filters.
61 Cold bananas, fish, pyjamas,
62 Mutton, beef and trout!
63 - attributed to Cole Porter.
66 Metaproxy is a more capable alternative to
67 <ulink url="http://www.indexdata.com/yazproxy/">YAZ Proxy</ulink>,
68 being more powerful, flexible, configurable and extensible. Among
69 its many advantages over the older, more pedestrian work are
70 support for multiplexing (encapsulated metasearching), routing by
71 database name, authentication and authorisation and serving local
72 files via HTTP. Equally significant, its modular architecture
73 facilitites the creation of pluggable modules implementing further
77 This manual will briefly describe Metaproxy's licensing situation
78 before giving an overview of its architecture, then discussing the
79 key concept of a filter in some depth and giving an overview of
80 the various filter types, then discussing the configuration file
81 format. After this come several optional chapters which may be
82 freely skipped: a detailed discussion of virtual databases and
83 multi-database searching, some notes on writing extensions
84 (additional filter types) and a high-level description of the
85 source code. Finally comes the reference guide, which contains
86 instructions for invoking the <command>metaproxy</command>
87 program, and detailed information on each type of filter,
94 <chapter id="licence">
95 <title>The Metaproxy Licence</title>
97 <emphasis role="strong">
98 No decision has yet been made on the terms under which
99 Metaproxy will be distributed.
101 It is possible that, unlike
102 other Index Data products, metaproxy may not be released under a
103 free-software licence such as the GNU GPL. Until a decision is
104 made and a public statement made, then, and unless it has been
105 delivered to you other specific terms, please treat Metaproxy as
106 though it were proprietary software.
107 The code should not be redistributed without explicit
108 written permission from the copyright holders, Index Data ApS.
114 <chapter id="architecture">
115 <title>The Metaproxy Architecture</title>
117 The Metaproxy architecture is based on three concepts:
118 the <emphasis>package</emphasis>,
119 the <emphasis>route</emphasis>
120 and the <emphasis>filter</emphasis>.
124 <term>Packages</term>
127 A package is request or response, encoded in some protocol,
128 issued by a client, making its way through Metaproxy, send to or
129 received from a server, or sent back to the client.
132 The core of a package is the protocol unit - for example, a
133 Z39.50 Init Request or Search Response, or an SRU searchRetrieve
134 URL or Explain Response. In addition to this core, a package
135 also carries some extra information added and used by Metaproxy
139 In general, packages are doctored as they pass through
140 Metaproxy. For example, when the proxy performs authentication
141 and authorisation on a Z39.50 Init request, it removes the
142 authentication credentials from the package so that they are not
143 passed onto the back-end server; and when search-response
144 packages are obtained from multiple servers, they are merged
145 into a single unified package that makes its way back to the
154 Packages make their way through routes, which can be thought of
155 as programs that operate on the package data-type. Each
156 incoming package initially makes its way through a default
157 route, but may be switched to a different route based on various
158 considerations. Routes are made up of sequences of filters (see
167 Filters provide the individual instructions within a route, and
168 effect the necessary transformations on packages. A particular
169 configuration of Metaproxy is essentially a set of filters,
170 described by configuration details and arranged in order in one
171 or more routes. There are many kinds of filter - about a dozen
172 at the time of writing with more appearing all the time - each
173 performing a specific function and configured by different
177 The word ``filter'' is sometimes used rather loosely, in two
178 different ways: it may be used to mean a particular
179 <emphasis>type</emphasis> of filter, as when we speak of ``the
180 auth_simplefilter'' or ``the multi filter''; or it may be used
181 to be a specific <emphasis>instance</emphasis> of a filter
182 within a Metaproxy configuration. For example, a single
183 configuration will often contain multiple instances of the
184 <literal>z3950_client</literal> filter. In
185 operational terms, of these is a separate filter. In practice,
186 context always make it clear which sense of the word ``filter''
190 Extensibility of Metaproxy is primarily through the creation of
191 plugins that provide new filters. The filter API is small and
192 conceptually simple, but there are many details to master. See
194 <link linkend="extensions">extensions</link>.
200 Since packages are created and handled by the system itself, and
201 routes are conceptually simple, most of the remainder of this
202 document concentrates on filters. After a brief overview of the
203 filter types follows, along with some thoughts on possible future
210 <chapter id="filters">
211 <title>Filters</title>
215 <title>Introductory notes</title>
217 It's useful to think of Metaproxy as an interpreter providing a small
218 number of primitives and operations, but operating on a very
219 complex data type, namely the ``package''.
222 A package represents a Z39.50 or SRU/W request (whether for Init,
223 Search, Scan, etc.) together with information about where it came
224 from. Packages are created by front-end filters such as
225 <literal>frontend_net</literal> (see below), which reads them from
226 the network; other front-end filters are possible. They then pass
227 along a route consisting of a sequence of filters, each of which
228 transforms the package and may also have side-effects such as
229 generating logging. Eventually, the route will yield a response,
230 which is sent back to the origin.
233 There are many kinds of filter: some that are defined statically
234 as part of Metaproxy, and others may be provided by third parties
235 and dynamically loaded. They all conform to the same simple API
236 of essentially two methods: <function>configure()</function> is
237 called at startup time, and is passed a DOM tree representing that
238 part of the configuration file that pertains to this filter
239 instance: it is expected to walk that tree extracting relevant
240 information; and <function>process()</function> is called every
241 time the filter has to processes a package.
244 While all filters provide the same API, there are different modes
245 of functionality. Some filters are sources: they create
247 (<literal>frontend_net</literal>);
248 others are sinks: they consume packages and return a result
249 (<literal>z3950_client</literal>,
250 <literal>backend_test</literal>,
251 <literal>http_file</literal>);
252 the others are true filters, that read, process and pass on the
253 packages they are fed
254 (<literal>auth_simple</literal>,
255 <literal>log</literal>,
256 <literal>multi</literal>,
257 <literal>query_rewrite</literal>,
258 <literal>session_shared</literal>,
259 <literal>template</literal>,
260 <literal>virt_db</literal>).
265 <section id="overview.filter.types">
266 <title>Overview of filter types</title>
268 We now briefly consider each of the types of filter supported by
269 the core Metaproxy binary. This overview is intended to give a
270 flavour of the available functionality; more detailed information
271 about each type of filter is included below in
272 <link linkend="filterref"
273 >the reference guide to Metaproxy filters</link>.
276 The filters are here named by the string that is used as the
277 <literal>type</literal> attribute of a
278 <literal><filter></literal> element in the configuration
279 file to request them, with the name of the class that implements
280 them in parentheses. (The classname is not needed for normal
281 configuration and use of Metaproxy; it is useful only to
285 The filters are here listed in alphabetical order:
289 <title><literal>auth_simple</literal>
290 (mp::filter::AuthSimple)</title>
292 Simple authentication and authorisation. The configuration
293 specifies the name of a file that is the user register, which
294 lists <varname>username</varname>:<varname>password</varname>
295 pairs, one per line, colon separated. When a session begins, it
296 is rejected unless username and passsword are supplied, and match
297 a pair in the register. The configuration file may also specific
298 the name of another file that is the target register: this lists
299 lists <varname>username</varname>:<varname>dbname</varname>,<varname>dbname</varname>...
300 sets, one per line, with multiple database names separated by
301 commas. When a search is processed, it is rejected unless the
302 database to be searched is one of those listed as available to
308 <title><literal>backend_test</literal>
309 (mp::filter::Backend_test)</title>
311 A sink that provides dummy responses in the manner of the
312 <literal>yaz-ztest</literal> Z39.50 server. This is useful only
313 for testing. Seriously, you don't need this. Pretend you didn't
314 even read this section.
319 <title><literal>frontend_net</literal>
320 (mp::filter::FrontendNet)</title>
322 A source that accepts Z39.50 connections from a port
323 specified in the configuration, reads protocol units, and
324 feeds them into the next filter in the route. When the result is
325 revceived, it is returned to the original origin.
330 <title><literal>http_file</literal>
331 (mp::filter::HttpFile)</title>
333 A sink that returns the contents of files from the local
334 filesystem in response to HTTP requests. (Yes, Virginia, this
335 does mean that Metaproxy is also a Web-server in its spare time. So
336 far it does not contain either an email-reader or a Lisp
337 interpreter, but that day is surely coming.)
342 <title><literal>log</literal>
343 (mp::filter::Log)</title>
345 Writes logging information to standard output, and passes on
346 the package unchanged.
351 <title><literal>multi</literal>
352 (mp::filter::Multi)</title>
354 Performs multicast searching.
356 <link linkend="multidb">the extended discussion</link>
357 of virtual databases and multi-database searching below.
362 <title><literal>query_rewrite</literal>
363 (mp::filter::QueryRewrite)</title>
365 Rewrites Z39.50 Type-1 and Type-101 (``RPN'') queries by a
366 three-step process: the query is transliterated from Z39.50
367 packet structures into an XML representation; that XML
368 representation is transformed by an XSLT stylesheet; and the
369 resulting XML is transliterated back into the Z39.50 packet
375 <title><literal>session_shared</literal>
376 (mp::filter::SessionShared)</title>
378 When this is finished, it will implement global sharing of
379 result sets (i.e. between threads and therefore between
380 clients), yielding performance improvements especially when
381 incoming requests are from a stateless environment such as a
382 web-server, in which the client process representing a session
383 might be any one of many. However:
387 This filter is not yet completed.
393 <title><literal>template</literal>
394 (mp::filter::Template)</title>
396 Does nothing at all, merely passing the packet on. (Maybe it
397 should be called <literal>nop</literal> or
398 <literal>passthrough</literal>?) This exists not to be used, but
399 to be copied - to become the skeleton of new filters as they are
400 written. As with <literal>backend_test</literal>, this is not
401 intended for civilians.
406 <title><literal>virt_db</literal>
407 (mp::filter::Virt_db)</title>
409 Performs virtual database selection: based on the name of the
410 database in the search request, a server is selected, and its
411 address added to the request in a <literal>VAL_PROXY</literal>
412 otherInfo packet. It will subsequently be used by a
413 <literal>z3950_client</literal> filter.
415 <link linkend="multidb">the extended discussion</link>
416 of virtual databases and multi-database searching below.
421 <title><literal>z3950_client</literal>
422 (mp::filter::Z3950Client)</title>
424 Performs Z39.50 searching and retrieval by proxying the
425 packages that are passed to it. Init requests are sent to the
426 address specified in the <literal>VAL_PROXY</literal> otherInfo
427 attached to the request: this may have been specified by client,
428 or generated by a <literal>virt_db</literal> filter earlier in
429 the route. Subsequent requests are sent to the same address,
430 which is remembered at Init time in a Session object.
436 <section id="future.directions">
437 <title>Future directions</title>
439 Some other filters that do not yet exist, but which would be
440 useful, are briefly described. These may be added in future
441 releases (or may be created by third parties, as loadable
447 <term><literal>frontend_cli</literal> (source)</term>
450 Command-line interface for generating requests.
455 <term><literal>frontend_sru</literal> (source)</term>
458 Receive SRU (and perhaps SRW) requests.
463 <term><literal>sru2z3950</literal> (filter)</term>
466 Translate SRU requests into Z39.50 requests.
471 <term><literal>sru_client</literal> (sink)</term>
474 SRU searching and retrieval.
479 <term><literal>srw_client</literal> (sink)</term>
482 SRW searching and retrieval.
487 <term><literal>opensearch_client</literal> (sink)</term>
490 A9 OpenSearch searching and retrieval.
500 <chapter id="configuration">
501 <title>Configuration: the Metaproxy configuration file format</title>
505 <title>Introductory notes</title>
507 If Metaproxy is an interpreter providing operations on packages, then
508 its configuration file can be thought of as a program for that
509 interpreter. Configuration is by means of a single file, the name
510 of which is supplied as the sole command-line argument to the
511 <command>metaproxy</command> program. (See
512 <link linkend="progref">the reference guide</link>
513 below for more information on invoking Metaproxy.)
516 The configuration files are written in XML. (But that's just an
517 implementation detail - they could just as well have been written
518 in YAML or Lisp-like S-expressions, or in a custom syntax.)
521 Since XML has been chosen, an XML schema,
522 <filename>config.xsd</filename>, is provided for validating
523 configuration files. This file is supplied in the
524 <filename>etc</filename> directory of the Metaproxy distribution. It
525 can be used by (among other tools) the <command>xmllint</command>
526 program supplied as part of the <literal>libxml2</literal>
530 xmllint --noout --schema etc/config.xsd my-config-file.xml
533 (A recent version of <literal>libxml2</literal> is required, as
534 support for XML Schemas is a relatively recent addition.)
538 <section id="overview.xml.structure">
539 <title>Overview of XML structure</title>
541 All elements and attributes are in the namespace
542 <ulink url="http://indexdata.dk/yp2/config/1"/>.
543 This is most easily achieved by setting the default namespace on
544 the top-level element, as here:
547 <yp2 xmlns="http://indexdata.dk/yp2/config/1">
550 The top-level element is <yp2>. This contains a
551 <start> element, a <filters> element and a
552 <routes> element, in that order. <filters> is
553 optional; the other two are mandatory. All three are
557 The <start> element is empty, but carries a
558 <literal>route</literal> attribute, whose value is the name of
559 route at which to start running - analogous to the name of the
560 start production in a formal grammar.
563 If present, <filters> contains zero or more <filter>
564 elements. Each filter carries a <literal>type</literal> attribute
565 which specifies what kind of filter is being defined
566 (<literal>frontend_net</literal>, <literal>log</literal>, etc.)
567 and contain various elements that provide suitable configuration
568 for a filter of its type. The filter-specific elements are
570 <link linkend="filterref">the reference guide below</link>.
571 Filters defined in this part of the file must carry an
572 <literal>id</literal> attribute so that they can be referenced
576 <routes> contains one or more <route> elements, each
577 of which must carry an <literal>id</literal> element. One of the
578 routes must have the ID value that was specified as the start
579 route in the <start> element's <literal>route</literal>
580 attribute. Each route contains zero or more <filter>
581 elements. These are of two types. They may be empty, but carry a
582 <literal>refid</literal> attribute whose value is the same as the
583 <literal>id</literal> of a filter previously defined in the
584 <filters> section. Alternatively, a route within a filter
585 may omit the <literal>refid</literal> attribute, but contain
586 configuration elements similar to those used for filters defined
587 in the <filters> section. (In other words, each filter in a
588 route may be included either by reference or by physical
594 <section id="example.configuration">
595 <title>An example configuration</title>
597 The following is a small, but complete, Metaproxy configuration
598 file (included in the distribution as
599 <literal>metaproxy/etc/config0.xml</literal>).
600 This file defines a very simple configuration that simply proxies
601 to whatever backend server the client requests, but logs each
602 request and response. This can be useful for debugging complex
603 client-server dialogues.
606 <?xml version="1.0"?>
607 <yp2 xmlns="http://indexdata.dk/yp2/config/1">
608 <start route="start"/>
610 <filter id="frontend" type="frontend_net">
613 <filter id="backend" type="z3950_client">
618 <filter refid="frontend"/>
620 <filter refid="backend"/>
626 It works by defining a single route, called
627 <literal>start</literal>, which consists of a sequence of three
628 filters. The first and last of these are included by reference:
629 their <literal><filter></literal> elements have
630 <literal>refid</literal> attributes that refer to filters defined
631 within the prior <literal><filters></literal> section. The
632 middle filter is included inline in the route.
635 The three filters in the route are as follows: first, a
636 <literal>frontend_net</literal> filter accepts Z39.50 requests
637 from any host on port 9000; then these requests are passed through
638 a <literal>log</literal> filter that emits a message for each
639 request; they are then fed into a <literal>z3950_client</literal>
640 filter, which forwards the requests to the client-specified
641 backend Z39.509 server. When the response arrives, it is handed
642 back to the <literal>log</literal> filter, which emits another
643 message; and then to the front-end filter, which returns the
644 response to the client.
651 <chapter id="multidb">
652 <title>Virtual databases and multi-database searching</title>
656 <title>Introductory notes</title>
658 <title>Lark's vomit</title>
660 This chapter goes into a level of technical detail that is
661 probably not necessary in order to configure and use Metaproxy.
662 It is provided only for those who like to know how things work.
663 You should feel free to skip on to the next section if this one
664 doesn't seem like fun.
668 Two of Metaproxy's filters are concerned with multiple-database
669 operations. Of these, <literal>virt_db</literal> can work alone
670 to control the routing of searches to one of a number of servers,
671 while <literal>multi</literal> can work with the output of
672 <literal>virt_db</literal> to perform multicast searching, merging
673 the results into a unified result-set. The interaction between
674 these two filters is necessarily complex: it reflecting the real,
675 irreducible complexity of multicast searching in a protocol such
676 as Z39.50 that separates initialisation from searching, and in
677 which the database to be searched is not known at initialisation
681 Hold on tight - this may get a little hairy.
684 In the general course of things, a Z39.50 Init request may carry
685 with it an otherInfo packet of type <literal>VAL_PROXY</literal>,
686 whose value indicates the address of a Z39.50 server to which the
687 ultimate connection is to be made. (This otherInfo packet is
688 supported by YAZ-based Z39.50 clients and servers, but has not yet
689 been ratified by the Maintenance Agency and so is not widely used
690 in non-Index Data software. We're working on it.)
691 The <literal>VAL_PROXY</literal> packet functions
692 analogously to the absoluteURI-style Request-URI used with the GET
693 method when a web browser asks a proxy to forward its request: see
695 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2"
698 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616.html"
699 >the HTTP 1.1 specification</ulink>.
702 The role of the <literal>virt_db</literal> filter is to rewrite
703 this otherInfo packet dependent on the virtual database that the
704 client wants to search. For example, a <literal>virt_db</literal>
705 filter could be set up so that searches in the virtual database
706 ``lc'' are forwarded to the Library of Congress server, and
707 searches in the virtual database ``id'' are forwarded to the toy
708 GILS database that Index Data hosts for testing purposes. A
709 <literal>virt_db</literal> configuration to make this switch would
713 <filter type="virt_db">
715 <database>lc</database>
716 <target>z3950.loc.gov:7090/Voyager</target>
719 <database>id</database>
720 <target>indexdata.dk/gils</target>
722 </filter>]]></screen>
724 When Metaproxy receives a Z39.50 Init request from a client, it
725 doesn't immediately forward that request to the back-end server.
726 Why not? Because it doesn't know <emphasis>which</emphasis>
727 back-end server to forward it to until the client sends a search
728 request that specifies the database that it wants to search in.
729 Instead, it just treasures the Init request up in its heart; and,
730 later, the first time the client does a search on one of the
731 specified virtual databases, a connection is forged to the
732 appropriate server and the Init request is forwarded to it. If,
733 later in the session, the same client searches in a different
734 virtual database, then a connection is forged to the server that
735 hosts it, and the same cached Init request is forwarded there,
739 All of this clever Init-delaying is done by the
740 <literal>frontend_net</literal> filter. The
741 <literal>virt_db</literal> filter knows nothing about it; in
742 fact, because the Init request that is received from the client
743 doesn't get forwarded until a Search reqeust is received, the
744 <literal>virt_db</literal> filter (and the
745 <literal>z3950_client</literal> filter behind it) doesn't even get
746 invoked at Init time. The <emphasis>only</emphasis> thing that a
747 <literal>virt_db</literal> filter ever does is rewrite the
748 <literal>VAL_PROXY</literal> otherInfo in the requests that pass
756 <chapter id="extensions">
757 <title>Writing extensions for Metaproxy</title>
758 <para>### To be written</para>
764 <chapter id="classes">
765 <title>Classes in the Metaproxy source code</title>
769 <title>Introductory notes</title>
771 <emphasis>Stop! Do not read this!</emphasis>
772 You won't enjoy it at all. You should just skip ahead to
773 <link linkend="refguide">the reference guide</link>,
775 <!-- The remainder of this paragraph is lifted verbatim from
776 Douglas Adams' _Hitch Hiker's Guide to the Galaxy_, chapter 8 -->
777 you things you really need to know, like the fact that the
778 fabulously beautiful planet Bethselamin is now so worried about
779 the cumulative erosion by ten billion visiting tourists a year
780 that any net imbalance between the amount you eat and the amount
781 you excrete whilst on the planet is surgically removed from your
782 bodyweight when you leave: so every time you go to the lavatory it
783 is vitally important to get a receipt.
786 This chapter contains documentation of the Metaproxy source code, and is
787 of interest only to maintainers and developers. If you need to
788 change Metaproxy's behaviour or write a new filter, then you will most
789 likely find this chapter helpful. Otherwise it's a waste of your
790 good time. Seriously: go and watch a film or something.
791 <citetitle>This is Spinal Tap</citetitle> is particularly good.
794 Still here? OK, let's continue.
797 In general, classes seem to be named big-endianly, so that
798 <literal>FactoryFilter</literal> is not a filter that filters
799 factories, but a factory that produces filters; and
800 <literal>FactoryStatic</literal> is a factory for the statically
801 registered filters (as opposed to those that are dynamically
806 <section id="individual.classes">
807 <title>Individual classes</title>
809 The classes making up the Metaproxy application are here listed by
810 class-name, with the names of the source files that define them in
815 <title><literal>mp::FactoryFilter</literal>
816 (<filename>factory_filter.cpp</filename>)</title>
818 A factory class that exists primarily to provide the
819 <literal>create()</literal> method, which takes the name of a
820 filter class as its argument and returns a new filter of that
821 type. To enable this, the factory must first be populated by
822 calling <literal>add_creator()</literal> for static filters (this
823 is done by the <literal>FactoryStatic</literal> class, see below)
824 and <literal>add_creator_dyn()</literal> for filters loaded
830 <title><literal>mp::FactoryStatic</literal>
831 (<filename>factory_static.cpp</filename>)</title>
833 A subclass of <literal>FactoryFilter</literal> which is
834 responsible for registering all the statically defined filter
835 types. It does this by knowing about all those filters'
836 structures, which are listed in its constructor. Merely
837 instantiating this class registers all the static classes. It is
838 for the benefit of this class that <literal>struct
839 metaproxy_1_filter_struct</literal> exists, and that all the filter
840 classes provide a static object of that type.
845 <title><literal>mp::filter::Base</literal>
846 (<filename>filter.cpp</filename>)</title>
848 The virtual base class of all filters. The filter API is, on the
849 surface at least, extremely simple: two methods.
850 <literal>configure()</literal> is passed a DOM tree representing
851 that part of the configuration file that pertains to this filter
852 instance, and is expected to walk that tree extracting relevant
853 information. And <literal>process()</literal> processes a
854 package (see below). That surface simplicitly is a bit
855 misleading, as <literal>process()</literal> needs to know a lot
856 about the <literal>Package</literal> class in order to do
862 <title><literal>mp::filter::AuthSimple</literal>,
863 <literal>Backend_test</literal>, etc.
864 (<filename>filter_auth_simple.cpp</filename>,
865 <filename>filter_backend_test.cpp</filename>, etc.)</title>
867 Individual filters. Each of these is implemented by a header and
868 a source file, named <filename>filter_*.hpp</filename> and
869 <filename>filter_*.cpp</filename> respectively. All the header
870 files should be pretty much identical, in that they declare the
871 class, including a private <literal>Rep</literal> class and a
872 member pointer to it, and the two public methods. The only extra
873 information in any filter header is additional private types and
874 members (which should really all be in the <literal>Rep</literal>
875 anyway) and private methods (which should also remain known only
876 to the source file, but C++'s brain-damaged design requires this
877 dirty laundry to be exhibited in public. Thanks, Bjarne!)
880 The source file for each filter needs to supply:
885 A definition of the private <literal>Rep</literal> class.
890 Some boilerplate constructors and destructors.
895 A <literal>configure()</literal> method that uses the
896 appropriate XML fragment.
901 Most important, the <literal>process()</literal> method that
902 does all the actual work.
909 <title><literal>mp::Package</literal>
910 (<filename>package.cpp</filename>)</title>
912 Represents a package on its way through the series of filters
913 that make up a route. This is essentially a Z39.50 or SRU APDU
914 together with information about where it came from, which is
915 modified as it passes through the various filters.
920 <title><literal>mp::Pipe</literal>
921 (<filename>pipe.cpp</filename>)</title>
923 This class provides a compatibility layer so that we have an IPC
924 mechanism that works the same under Unix and Windows. It's not
925 particularly exciting.
930 <title><literal>mp::RouterChain</literal>
931 (<filename>router_chain.cpp</filename>)</title>
938 <title><literal>mp::RouterFleXML</literal>
939 (<filename>router_flexml.cpp</filename>)</title>
946 <title><literal>mp::Session</literal>
947 (<filename>session.cpp</filename>)</title>
954 <title><literal>mp::ThreadPoolSocketObserver</literal>
955 (<filename>thread_pool_observer.cpp</filename>)</title>
962 <title><literal>mp::util</literal>
963 (<filename>util.cpp</filename>)</title>
965 A namespace of various small utility functions and classes,
966 collected together for convenience. Most importantly, includes
967 the <literal>mp::util::odr</literal> class, a wrapper for YAZ's
973 <title><literal>mp::xml</literal>
974 (<filename>xmlutil.cpp</filename>)</title>
976 A namespace of various XML utility functions and classes,
977 collected together for convenience.
983 <section id="other.source.files">
984 <title>Other Source Files</title>
986 In addition to the Metaproxy source files that define the classes
987 described above, there are a few additional files which are
988 briefly described here:
992 <term><literal>metaproxy_prog.cpp</literal></term>
995 The main function of the <command>metaproxy</command> program.
1000 <term><literal>ex_router_flexml.cpp</literal></term>
1003 Identical to <literal>metaproxy_prog.cpp</literal>: it's not clear why.
1008 <term><literal>test_*.cpp</literal></term>
1011 Unit-tests for various modules.
1017 ### Still to be described:
1018 <literal>ex_filter_frontend_net.cpp</literal>,
1019 <literal>filter_dl.cpp</literal>,
1020 <literal>plainfile.cpp</literal>,
1021 <literal>tstdl.cpp</literal>.
1028 <chapter id="refguide">
1029 <title>Reference guide</title>
1031 The material in this chapter is drawn directly from the individual
1032 manual entries. In particular, the Metaproxy invocation section is
1033 available using <command>man metaproxy</command>, and the section
1034 on each individual filter is available using the name of the filter
1035 as the argument to the <command>man</command> command.
1039 <section id="progref">
1040 <title>Metaproxy invocation</title>
1045 <section id="filterref">
1046 <title>Reference guide to Metaproxy filters</title>
1053 <!-- Keep this comment at the end of the file
1058 sgml-minimize-attributes:nil
1059 sgml-always-quote-attributes:t
1062 sgml-parent-document: "main.xml"
1063 sgml-local-catalogs: nil
1064 sgml-namecase-general:t
1065 nxml-child-indent: 1