1 <!-- $Id: comstack.xml,v 1.3 2001-08-13 09:42:54 adam Exp $ -->
2 <chapter><title id="comstack">The COMSTACK Module</title>
4 <sect1><title>Synopsis (blocking mode)</title>
10 int size = 0, length_incoming;
11 char *protocol_package;
12 int protocol_package_length;
13 char server_address[] = "myserver.com:2100";
16 stack = cs_create(tcpip_type, 1, PROTO_Z3950);
18 perror("cs_create"); /* note use of perror() here since we have no stack yet */
22 status = cs_connect(stack, server_address);
24 cs_perror(stack, "cs_connect");
28 status = cs_put(stack, protocol_package, protocol_package_length);
30 cs_perror(stack, "cs_put");
34 /* Now get a response */
36 length_incoming = cs_get(stack, &buf, &size);
37 if (!length_incoming) {
38 fprintf(stderr, "Connection closed\n");
40 } else if (length_incoming < 0) {
41 cs_perror(stack, "cs_get");
45 /* Do stuff with buf here */
55 <sect1><title>Introduction</title>
59 subsystem provides a transparent interface to different types of transport
60 stacks for the exchange of BER-encoded data. At present, the
61 RFC1729 method (BER over TCP/IP), and Peter Furniss' XTImOSI
62 stack are supported, but others may be added in time. The philosophy of the
63 module is to provide a simple interface by hiding unused options and
64 facilities of the underlying libraries. This is always done at the risk
65 of losing generality, and it may prove that the interface will need
70 The interface is implemented in such a fashion that only the
71 sub-layers constructed to the transport methods that you wish to
72 use in your application are linked in.
76 You will note that even though simplicity was a goal in the design,
77 the interface is still orders of magnitudes more complex than the
78 transport systems found in many other packages. One reason is that
79 the interface needs to support the somewhat different requirements of
80 the different lower-layer communications stacks; another important reason is
81 that the interface seeks to provide a more or less industrial-strength
82 approach to asynchronous event-handling. When no function is allowed
83 to block, things get more complex - particularly on the server
84 side. We urge you to have a look at the demonstration client and server
85 provided with the package. They are meant to be easily readable and
86 instructive, while still being at least moderately useful.
90 <sect1><title>Common Functions</title>
92 <sect2><title>Managing Endpoints</title>
95 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
99 Creates an instance of the protocol stack - a communications endpoint.
100 The <literal>type</literal> parameter determines the mode of communication.
101 At present, the values
102 <literal>tcpip_type</literal>
104 <literal>mosi_type</literal>
105 are recognized. The function returns a null-pointer if a system error
106 occurs. The <literal>blocking</literal> parameter should be one if you wish
107 the association to operate in blocking mode, zero otherwise. The
108 <literal>protocol</literal> field should be one of
109 <literal>PROTO_SR</literal> or <literal>PROTO_Z3950</literal>.
114 int cs_close(COMSTACK handle);
118 Closes the connection (as elegantly as the lower layers will permit),
119 and releases the resources pointed to by the
120 <literal>handle</literal>
122 <literal>handle</literal>
123 should not be referenced again after this call.
128 We really need a soft disconnect, don't we?
133 <sect2><title>Data Exchange</title>
136 int cs_put(COMSTACK handle, char *buf, int len);
141 <literal>buf</literal>
142 down the wire. In blocking mode, this function will return only when a
143 full buffer has been written, or an error has occurred. In nonblocking
144 mode, it's possible that the function will be unable to send the full
145 buffer at once, which will be indicated by a return value of 1. The
146 function will keep track of the number of octets already written; you
147 should call it repeatedly with the same values of <literal>buf</literal>
148 and <literal>len</literal>, until the buffer has been transmitted.
149 When a full buffer has been sent, the function will return 0 for
150 success. -1 indicates an error condition (see below).
154 int cs_get(COMSTACK handle, char **buf, int *size);
158 Receives a PDU from the peer. Returns the number of bytes
159 read. In nonblocking mode, it is possible that not all of the packet can be
160 read at once. In this case, the function returns 1. To simplify the
161 interface, the function is
162 responsible for managing the size of the buffer. It will be reallocated
163 if necessary to contain large packages, and will sometimes be moved
164 around internally by the subsystem when partial packages are read. Before
166 <function>cs_get</function>
167 for the fist time, the buffer can be initialized to the null pointer,
168 and the length should also be set to 0 - cs_get will perform a
169 <function>malloc(2)</function>
170 on the buffer for you. When a full buffer has been read, the size of
171 the package is returned (which will always be greater than 1). -1
172 indicates an error condition.
176 See also the <function>cs_more()</function> function below.
180 int cs_more(COMSTACK handle);
184 The <function>cs_more()</function> function should be used in conjunction
185 with <function>cs_get</function> and
186 <function>select(2)</function>.
187 The <function>cs_get()</function> function will sometimes
188 (notably in the TCP/IP mode) read more than a single protocol package
189 off the network. When this happens, the extra package is stored
190 by the subsystem. After calling <function>cs_get()</function>, and before
191 waiting for more input, You should always call
192 <function>cs_more()</function>
193 to check if there's a full protocol package already read. If
194 <function>cs_more()</function>
196 <function>cs_get()</function>
197 can be used to immediately fetch the new package. For the
199 subsystem, the function should always return 0, but if you want your
200 stuff to be protocol independent, you should use it.
205 The <function>cs_more()</function>
206 function is required because the RFC1729-method
207 does not provide a way of separating individual PDUs, short of
208 partially decoding the BER. Some other implementations will carefully
209 nibble at the packet by calling
210 <function>read(2)</function>
211 several times. This was felt to be too inefficient (or at least
212 clumsy) - hence the call for this extra function.
217 int cs_look(COMSTACK handle);
221 This function is useful when you're operating in nonblocking
223 <function>select(2)</function>
224 tells you there's something happening on the line. It returns one of
225 the following values:
229 <varlistentry><term>CS_NONE</term><listitem><para>
230 No event is pending. The data found on the line was not a complete package.
231 </para></listitem></varlistentry>
233 <varlistentry><term>CS_CONNECT</term><listitem><para>
234 A response to your connect request has been received. Call
235 <function>cs_rcvconnect</function>
236 to process the event and to finalize the connection establishment.
237 </para></listitem></varlistentry>
239 <varlistentry><term>CS_DISCON</term><listitem><para>
240 The other side has closed the connection (or maybe sent a disconnect
241 request - but do we care? Maybe later). Call
242 <function>cs_close</function> to close your end of the association as well.
243 </para></listitem></varlistentry>
245 <varlistentry><term>CS_LISTEN</term><listitem><para>
246 A connect request has been received. Call <function>cs_listen</function>
247 to process the event.
248 </para></listitem></varlistentry>
250 <varlistentry><term>CS_DATA</term><listitem><para>
251 There's data to be found on the line. Call <function>cs_get</function>
253 </para></listitem></varlistentry>
258 You should be aware that even if
259 <function>cs_look()</function>
260 tells you that there's an event event pending, the corresponding
261 function may still return and tell you there was nothing to be found.
262 This means that only part of a package was available for reading. The
263 same event will show up again, when more data has arrived.
268 int cs_fileno(COMSTACK h);
272 Returns the file descriptor of the association. Use this when
273 file-level operations on the endpoint are required
274 (<function>select(2)</function> operations, specifically).
280 <sect1><title>Client Side</title>
283 int cs_connect(COMSTACK handle, void *address);
287 Initiate a connection with the target at <literal>address</literal>
288 (more onaddresses below). The function will return 0 on success, and 1 if
289 the operation does not complete immediately (this will only
290 happen on a nonblocking endpoint). In this case, use
291 <function>cs_rcvconnect</function> to complete the operation,
292 when <function>select(2)</function> reports input pending on the
297 int cs_rcvconnect(COMSTACK handle);
301 Complete a connect operation initiated by <function>cs_connect()</function>.
302 It will return 0 on success; 1 if the operation has not yet completed (in
303 this case, call the function again later); -1 if an error has occurred.
308 <sect1><title>Server Side</title>
311 To establish a server under the <application>inetd</application> server, you
316 COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
321 The <literal>socket</literal> parameter is an established socket (when
322 your application is invoked from <application>inetd</application>, the
323 socket will typically be 0.
324 The following parameters are identical to the ones for
325 <function>cs_create</function>.
329 int cs_bind(COMSTACK handle, void *address, int mode)
333 Binds a local address to the endpoint. Read about addresses below. The
334 <literal>mode</literal> parameter should be either
335 <literal>CS_CLIENT</literal> or <literal>CS_SERVER</literal>.
339 int cs_listen(COMSTACK handle, char *addr, int *addrlen);
343 Call this to process incoming events on an endpoint that has been
344 bound in listening mode. It will return 0 to indicate that the connect
345 request has been received, 1 to signal a partial reception, and -1 to
346 indicate an error condition.
350 COMSTACK cs_accept(COMSTACK handle);
354 This finalizes the server-side association establishment, after
355 cs_listen has completed successfully. It returns a new connection
356 endpoint, which represents the new association. The application will
357 typically wish to fork off a process to handle the association at this
358 point, and continue listen for new connections on the old
359 <literal>handle</literal>.
367 char *cs_addrstr(COMSTACK);
371 on an established connection to retrieve the host-name of the remote host.
375 <para>You may need to use this function with some care if your
376 name server service is slow or unreliable
381 <sect1><title>Addresses</title>
384 The low-level format of the addresses are different depending on the
385 mode of communication you have chosen. A function is provided by each
386 of the lower layers to map a user-friendly string-form address to the
387 binary form required by the lower layers.
391 struct sockaddr_in *tcpip_strtoaddr(char *str);
393 struct netbuf *mosi_strtoaddr(char *str);
397 The format for TCP/IP addresses is straightforward:
401 <host> [ ':' <portnum> ]
405 The <literal>hostname</literal> can be either a domain name or an IP address.
406 The port number, if omitted, defaults to 210.
410 For OSI, the format is
414 [ <t-selector> '/' ] <host> [ ':' <port> ]
418 The transport selector is given as an even number of hex digits.
422 You'll note that the address format for the OSI mode are just a subset
423 of full presentation addresses. We use presentation addresses because
424 xtimosi doesn't, in itself, allow access to the X.500 Directory
425 service. We use a limited form, because we haven't yet come across an
426 implementation that used more of the elements of a full p-address. It
427 is a fairly simple matter to add the rest of the elements to the
428 address format as needed, however: Xtimosi <emphasis>does</emphasis>
429 support the full P-address structure.
433 In both transport modes, the special hostname "@" is mapped
434 to any local address (the manifest constant <literal>INADDR_ANY</literal>).
435 It is used to establish local listening endpoints in the server role.
439 When a connection has been established, you can use
443 char cs_addrstr(COMSTACK h);
447 to retrieve the host name of the peer system. The function returns a pointer
448 to a static area, which is overwritten on the next call to the function.
453 We have left the issue of X.500 name-to-address mapping open, for the
454 moment. It would be a simple matter to provide a table-based mapping,
455 if desired. Alternately, we could use the X.500 client-function that
456 is provided with the ISODE (although this would defeat some of the
457 purpose of using ThinOSI in the first place. We have been told that it
458 should be within the realm of the possible to implement a lightweight
459 implementation of the necessary X.500 client capabilities on top of
460 ThinOSI. This would be the ideal solution, we feel. On the other hand, it
461 still remains to be seen just what role the Directory will play in a world
462 populated by ThinOSI and other pragmatic solutions.
468 <sect1><title>Diagnostics</title>
471 All functions return -1 if an error occurs. Typically, the functions
472 will return 0 on success, but the data exchange functions
473 (<function>cs_get</function>, <function>cs_put</function>,
474 <function>cs_more</function>) follow special rules. Consult their
479 When a function (including the data exchange functions) reports an
480 error condition, use the function
481 <function>cs_errno()</function> to determine the cause of the
482 problem. The function
486 void cs_perror(COMSTACK handle char *message);
490 works like <function>perror(2)</function> and prints the
491 <literal>message</literal> argument, along with a system message, to
492 <literal>stderr</literal>. Use the character array
496 extern const char *cs_errlist[];
500 to get hold of the message, if you want to process it differently.
505 const char *cs_stackerr(COMSTACK handle);
509 Returns an error message from the lower layer, if one has been
514 <sect1><title>Enabling OSI Communication</title>
516 <sect2><title>Installing Xtimosi</title>
518 Although you will have to down-load Peter Furniss' XTI/mOSI
519 implementation for yourself, we've tried to make the integration as
524 The latest version of xtimosi will generally be under
528 ftp://pluto.ulcc.ac.uk/ulcc/thinosi/xtimosi/
532 When you have down-loaded and unpacked the archive, it will (we assume)
533 have created a directory called <literal>xtimosi</literal>.
534 We suggest that you place this directory <emphasis>in the same
535 directory</emphasis> where you unpacked the &yaz;
536 distribution. This way, you shouldn't have to fiddle with the
537 makefiles of &yaz; beyond uncommenting a few lines.
541 Go to <literal>xtimosi/src</literal>, and type "make libmosi.a/".
542 This should generally create the library, ready to use.
547 The currently available release of xtimosi has some inherent
548 problems that make it disfunction on certain platforms - eg. the
549 Digital OSF/1 workstations. It is supposedly primarily a
550 compiler problem, and we hope to see a release that is generally
551 portable. While we can't guarantee that it can be brought to work
552 on your platform, we'll be happy to talk to you about problems
553 that you might see, and relay information to the author of the
554 software. There are some signs that the <application>gcc</application>
555 compiler is more likely to produce a fully functional library, but this
556 hasn't been verified (we think that the problem is limited to the use
557 of hexadecimal escape-codes used in strings, which are silently
558 ignored by some compilers).
561 A problem has been encountered in the communication with
562 ISODE-based applications. If the ISODE presentation-user calls
563 <function>PReadRequest()</function> with a timeout value different
564 from <literal>OK</literal> or <literal>NOTOK</literal>,
565 he will get an immediate TIMEOUT abort when receiving large (>2041
566 bytes, which is the SPDU-size that the ISODE likes to work with) packages
567 from an xtimosi-based implementation (probably most
568 other implementations as well, in fact). It seems to be a flaw in the
569 ISODE API, and the workaround (for ISODE users) is to either not
570 use an explicit timeout (switching to either blocking or
571 nonblocking mode), or to check that the timer really has expired
572 before closing the connection.
577 The next step in the installation is to modify the makefile in the toplevel
579 directory. The place to change is in the top of the file, and is
580 clearly marked with a comment.
584 Now run <literal>make</literal> in the &yaz; toplevel directory (do a
585 <literal>make clean</literal> first, if the system has been previously
586 made without OSI support). Use the &yaz;
587 <application>yaz-ztest</application> and <application>yaz-client</application>
588 demo programs to verify that OSI communication works OK. Then, you can go
589 ahead and try to talk to other implementations.
594 Our interoperability experience is limited to version
595 7 of the Nordic SR-Nett package, which has had several
596 protocol errors fixed from the earlier releases. If you have
597 problems or successes in interoperating with other
598 implementations, we'd be glad to hear about it, or to help
599 you make things work, as our resources allow.
604 If you write your own applications based on &yaz;, and you wish to
605 include OSI support, the procedure is equally simple. You should
606 include the <filename>xmosi.h</filename> header file in addition to
607 <filename>comstack.h</filename>. <filename>xmosi.h</filename>
608 will define the manifest constant <literal>mosi_type</literal>, which you
609 should pass to the <function>cs_create()</function> function. In
610 addition, you should use the function <function>mosi_strtoaddr()</function>
611 rather than <function>tcpip_strtoaddr()</function> when you need to
616 When you link your application, you should include (after the
617 <filename>libyaz.a</filename> library) the <literal>libmosi.a</literal>
618 library, and the <filename>librfc.a</filename> library provided with
619 &yaz; (for OSI transport).
622 As always, it can be very useful, if not essential, to have a look at the
623 example applications to see how things are done.
627 <sect2><title>OSI Transport</title>
630 Xtimosi requires an implementation of the OSI transport service under
631 the X/OPEN XTI API. We provide an implementation of the RFC1006
632 encapsulation of OSI/TP0 in TCP/IP (through the Berkeley Sockets API),
633 as an independent part of &yaz; (it's found under the
634 <filename>rfc1006</filename> directory).
635 If you have access to an OSI transport provider under XTI,
636 you should be able to make that work too, although it may require
637 tinkering with the <function>mosi_strtoaddr()</function> function.
641 <sect2><title>Presentation Context Management</title>
644 To simplify the implementation, we use Peter Furniss' alternative (PRF)
646 for the Control of the presentation negotiation phase. This format
647 is enabled by default when you
652 The current version of &yaz; does <emphasis>not</emphasis> support
653 presentation-layer negotiation of response record formats. The primary
654 reason is that we have had access to no other SR or Z39.50
655 implementations over OSI that used this
656 method. Secondarily, we believe that the EXPLAIN facility is a superior
657 mechanism for relaying target capabilities in this respect. This is not to
658 say that we have no intentions of supporting presentation context
659 negotiation - we have just hitherto given it a lower priority than other
660 aspects of the protocol.
663 One thing is certain: The addition of this capability to &yaz; should
664 have only a minimal impact on existing applications, and on the
665 interface to the software in general. Most likely, we will add an extra
666 layer of interface to the processing of EXPLAIN records, which will
667 convert back and forth between <literal>oident</literal> records (see
668 section <link linkend="oid">Object Identifiers</link>) and direct or
669 indirect references, given the current association setup. Implementations
670 based on any of the higher-level interfaces will most likely not have to
675 <sect1><title>Summary and Synopsis</title>
678 #include <comstack.h>
680 #include <tcpip.h> /* this is for TCP/IP support */
681 #include <xmosi.h> /* and this is for mOSI support */
683 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
685 COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
688 int cs_bind(COMSTACK handle, int mode);
690 int cs_connect(COMSTACK handle, void *address);
692 int cs_rcvconnect(COMSTACK handle);
694 int cs_listen(COMSTACK handle);
696 COMSTACK cs_accept(COMSTACK handle);
698 int cs_put(COMSTACK handle, char *buf, int len);
700 int cs_get(COMSTACK handle, char **buf, int *size);
702 int cs_more(COMSTACK handle);
704 int cs_close(COMSTACK handle);
706 int cs_look(COMSTACK handle);
708 struct sockaddr_in *tcpip_strtoaddr(char *str);
710 struct netbuf *mosi_strtoaddr(char *str);
714 void cs_perror(COMSTACK handle char *message);
716 const char *cs_stackerr(COMSTACK handle);
718 extern const char *cs_errlist[];
724 <!-- Keep this comment at the end of the file
729 sgml-minimize-attributes:nil
730 sgml-always-quote-attributes:t
733 sgml-parent-document: "yaz.xml"
734 sgml-local-catalogs: "../../docbook/docbook.cat"
735 sgml-namecase-general:t