@setfilename ilu-commonlisp.info
@settitle Using ILU with Common Lisp
@ifclear largerdoc
@titlepage
@title Using ILU with Common Lisp
@author Bill Janssen @code{<janssen@@parc.xerox.com>}
@author Doug Cutting @code{<cutting@@parc.xerox.com>}
@sp
@today{}
@sp
Copyright @copyright{} 1993 Xerox Corporation@*
All Rights Reserved.
@end titlepage
@ifinfo
@node Top, ,(dir),(dir)
@top Using ILU with Common Lisp
@end ifinfo
@syncodeindex vr cp
@syncodeindex pg cp
@syncodeindex tp cp
@syncodeindex fn cp
@end ifclear

This document is for the @language{Common Lisp} programmer who wishes to use 
@system{ILU}.  The first section explains the mappings from the @system{ILU} Interface 
Specification Language into the @language{Common Lisp} language, the second discusses 
concepts necessary for people exporting @system{ILU} modules from @language{Common Lisp}.  In 
general, people who need only to use pre-existing @system{ILU} modules should only need 
to read the first section.  You should understand the types and concepts 
supported by @system{ILU} before reading this document.  (@xref{ILU Concepts}.)

@section @system{ILU} Mappings to @language{Common Lisp}

@subsection Generating Common Lisp Surrogate and True Stubs
@cindex Stub generation
@cindex Common Lisp stub generation
The program @system{ILU} @program{lisp-stubber}
@pindex lisp-stubber
takes a interface specification (an @file{.isl} file) and generates lisp 
code to provide both client-side and server-side support for the interface.
The files are generated in the current working directory.
In particular, the following files are generated:

@itemize @bullet
@item
@file{@metavar{interface-name}-sysdcl.lisp} -- tells @module{PDEFSYS}
@footnote{@xref{The ILU Common Lisp Portable DEFSYSTEM Module},
 for a description of the @module{PDEFSYS} package.} how to compile and load the other files.  It defines a Common Lisp module @cl{:@metavar{<interface>}}, which describes the code needed to support both surrogate and true use of the interface.  This file is often called a @dfn{sysdcl} for the module.
@item
@file{@metavar{interface-name}-basics.lisp} -- contains lisp code needed by clients of the module; and
@item
@file{@metavar{interface-name}-server-procs.lisp} -- contains lisp code needed by module implementations.
@end itemize
@subsection Packages & Symbols

Runtime code is in the @language{Common Lisp} package @module{ilu}.

Names from interface specifications are transformed into Lisp names
(case-insensitive) by inserting hyphens at lower-to-upper case transitions.
Hyphens that are already present are maintained as is.@footnote{This causes
problems; the ISL names "FooBar" and "foo-bar" map to the same @language{Common Lisp} name.  Something will have to change.}

A separate package is defined for each interface with
@fn{defpackage}.  The name of this package
is taken from the name of the interface.  This package uses the packages 
@module{common-lisp} and @module{ilu}.  The @language{Common Lisp}
names of all entities defined in the ISL
are exported from the package, including types, classes, constants,
accessors, type predicates,
generic functions, exceptions, etc. Such symbols are also shadowed, to avoid 
conflicts with used packages. For example, given the following interface:

@codeexample
INTERFACE MyInterface END;
EXCEPTION TotalLoser : Person;
TYPE Person = OBJECT
  METHODS
     FriendsP (someone : Person) : Boolean
       RAISES TotalLoser END
  END;
@end codeexample

the stubber generates the following @code{defpackage}:

@codeexample
(defpackage :my-interface
  (:use :common-lisp :ilu)
  (:shadow #:total-loser #:person #:friends-p)
  (:export #:total-loser #:person #:friends-p))
@end codeexample

This allows symbols defined in the @module{commonlisp} package
to be used by the automatically generated code in the generated package,
but it also means that the user needs to be careful about using any
generated package.  In general, we recommend that you explicitly specify
the full name of symbols from @system{ILU} interfaces.

@subsection Types

@system{ILU} types appear in @language{Common Lisp} as follows:

@enumerate
@item
Exceptions are represented with @language{CL} conditions, defined by @fn{define-condition}.
All ILU conditions 
are subtypes of @type{ilu:rpc-exception}, which is a 
@type{serious-condition}. If a value is 
specified for an exception it may be accessed with
@method{ilu:exception-value} on the 
condition signalled.
@item
Constants are implemented in @language{CL} by a value of the 
appropriate type, defined with @fn{defconstant}.
@item
Type aliases (one type name specified as another) are implemented 
with @kwd{deftype}.
@item
Sequences are implemented as @type{list}s, except for sequences of characters, which are implemented as @cl{simple-string}s.
@item
BOOLEAN values are represented as normal @language{Common Lisp} logical
values, typically either @cl{commonlisp:t} or @cl{commonlisp:nil}.
@item
Arrays and sequences of CHARACTER (regular or SHORT) are implemented as
@type{simple-string}s.
@item
Arrays of bytes are implemented with objects of type
@type{(simple-array (unsigned-byte 8) (*))}.
@item
Other arrays are implemented as @type{simple-array}s.
@item
Unions are implemented as a cons'ed value, with the cdr containing
the union type discriminant, and the cdr containing the actual value.
@item
Enumerations are implemented with symbols, as in@*
@code{(deftype answer () `(member 'yes 'no 'maybe))}
@item
Record types are implemented with @language{CL} @kwd{defstruct}.
@item
Classes are implemented with @language{CLOS} @kwd{defclass}.

Private slots are created for methods which are specified as
@kwd{functional}, and the runtime caches the value of this method
in such slots after the first call to the method.

Methods always take as their first argument the object which they are a method 
on.  Subsequent arguments are those specified in the @file{.isl} file.
Methods that have @isl{OUT} or @isl{INOUT} arguments may return multiple
values.  In general, the parameters to a method are the @isl{IN} and
@isl{INOUT} parameters specified in the @language{ISL} interface, but not
the @isl{OUT} parameters.  The return values from a method are the
specified return value for the @language{ISL} method, if any, followed
by the @isl{INOUT} and @isl{OUT} parameters for the method, if any, in
the order in which they appear in the @language{ISL} specification
of the method.
@end enumerate

@section Using a Module from Common Lisp
@cindex Using a Module from Common Lisp

To use a module from @language{Common Lisp}, you must first have loaded
the @system{PDEFSYS} ``system'' that describes the module.  Typically,
for an @system{ILU} interface called @metavar{Foo}, the system
can be loaded by invoking @cl{(pdefsys:load-system :foo)}.
Next, you must bind an instance of an object from that interface.
The most common way of doing this is to receive an instance
of an object from a method called on another object.  But to get the
first object exported
by that module, one can use either @cl{ilu:sbh->instance} or
@cl{ilu:lookup}.

@deffn {Lisp Function} ilu:sbh->instance (@metavar{PUTATIVE-TYPE-NAME} @type{symbol}) (@metavar{SBH} @type{string}) &optional (@metavar{MOST-SPECIFIC-TYPE-ID} @type{simple-string} @constant{mstid of specified PUTATIVE-TYPE}) => @type{ilu:rpc-object}

Accepts an @system{ILU} string binding handle and @language{Common Lisp} type name, and attempts to locally bind an instance of that type with the OID specified in the string binding handle.  If no such instance exists locally, a
surrogate instance is created and returned.  If a true instance exists
locally, that instance will be returned.

@end deffn

@deffn {Lisp Function} ilu:lookup (@metavar{PUTATIVE-TYPE-NAME} @type{symbol}) (@metavar{OID} @type{simple-string}) => @type{(or nil ilu:rpc-object)}

This routine will find and return an object with the OID @metavar{OID}, if
such an object has been registered in the local domain via the @system{ILU}
simple binding protocol.  @footnote{The simple binding protocol is experimental
in release 1.8 of @system{ILU}, and may change without warning in later
releases.}  @xref{Publishing a Common Lisp True Object}, for an example.

@end deffn

Various @system{ILU} attributes of an object type may be discovered at run time with the generic function @cl{ilu:ilu-class-info}.

@deffn {Lisp Function} ilu:ilu-class-info (@metavar{DISC} @type{(or ilu:ilu-object type-name)}) (@metavar{WHAT} @type{keyword}) => @type{(or string boolean list)}

This routine will return the specified piece of information about the @system{ILU} class
specified with @metavar{DISC}, which may be either a @language{CLOS} class name,
or an instance of the class, and with @metavar{WHAT}, which identifies which piece of information to
return.  @metavar{WHAT} may have the following values:
@itemize @bullet
@item 
@cl{:authentication} -- what kind of authentication, if any, is expected by the methods of this class
@item
@cl{:brand} -- the brand of the object type, if any
@item
@cl{:collectible-p} -- whether or not the object type participates in the @system{ILU} distributed GC
@item
@cl{:doc-string} -- the doc string specified for the object type
@item
@cl{:id} -- the @system{ILU} unique ID for the object type
@item
@cl{:ilu-version} -- which version of @system{ILU} the stubber that generated the code for this object type came from
@item
@cl{:methods} -- a list of the methods of the object type
@item
@cl{:optional-p} -- whether values of this class are allowed to be @cl{cl:nil} (a @system{CORBA} excrescence)
@item
@cl{:name} -- the @system{ILU} name of the object type
@end itemize

@end deffn

@section Implementing a Module in Common Lisp
@cindex Common Lisp True Modules
@cindex Common Lisp Servers

For each @system{ILU} class @class{@metavar{interface}.@metavar{otype}},
@system{ILU} will define, in the file
@file{@metavar{interface}-server-procs.lisp},
a @system{CLOS} class called @cl{@metavar{interface}:@metavar{otype}.IMPL}.
To implement a true object for @class{@metavar{interface}.@metavar{otype}},
one should further subclass this @system{CLOS} class,
and override all of its methods.  In particular, do not let
any of the default methods for the class be called from your methods for it.

@system{ILU} supports,
in each address space, multiple instances of something called
a @dfn{kernel server}, each of which in turn
supports some set of object instances.
A kernel server @dfn{export}s its objects by making them available
to other modules.  It may do so via one or more @dfn{port}s, which are
abstractly a tuple of (@metavar{rpc protocol}, @metavar{transport type},
@metavar{transport address}).  For example, a typical port might
provide access to a kernel server's objects
via @code{(Sun RPC, TCP/IP, UNIX port 2076)}.  Another port on the
same kernel server might provide access to the objects via
@code{(Xerox Courier, XNS SPP, XNS port 1394)}.

When creating an instance of a true object, a kernel server for it,
and an instance id (the name by which the kernel server knows it) for
it must be determined.  These may be specified explicitly by use of the
keyword arguments to @cl{commonlisp:make-instance}
@cl{:ilu-kernel-server} and @cl{:ilu-instance-handle}, respectively.  If they are
not specified explicitly,
the variable @cl{ilu:*default-server*} will be bound,
and its value will be used; a default instance
handle, unique relative to the kernel server, will be generated.

A kernel server may be created by instantiating the class
@class{ilu:kernel-server}.  The keyword argument @cl{:id} may
be specified to select a name for the server.  Note that @system{ILU}
object IDs, which consist of the kernel server ID, plus the instance
handle of the object on that server, must be unique ``across space
and time'', as the saying goes.  If no kernel server id is specified,
@system{ILU} will generate one automatically, using an algorithm
that provides a high probability of uniqueness.  If you explicitly
specify a kernel server ID, a good technique is to use a prefix or
suffix which uniquely identifies some domain in which you can assure
the uniqueness of the remaining part of the ID.  For example, when
using @system{ILU} at some project called NIFTY at some internet
site in the IP domain @code{department.company.com}, one might use
kernel server IDs with names like
@code{something.NIFTY.department.company.com}.

@codeexample
=> @userinput{(make-instance 'ilu:kernel-server :id "FOO-SERVER-1")}
#<ILU:KERNEL-SERVER "FOO-SERVER-1">
=> @userinput{(make-instance 'ilu:kernel-server)}
#<ILU:KERNEL-SERVER "121.2.100.231.1404.2c7577eb.3e5a28f">
=>
@end codeexample

@deffn {Lisp @cl{cl:make-instance}} {ilu:kernel-server} &key {(id @type{string} @cl{nil})} {(unix-port @type{fixnum} @cl{0})} {(object-table @type{list of 2 elements} @cl{nil})} => @type{ilu:kernel-server}

Creates and returns an instance of @type{ilu:kernel-server}.  If @metavar{id} is specified, the server has that value for its server ID.  If @metavar{unix-port} is specified, the server attempts to `listen' on that UNIX port, if the notion of a UNIX port is applicable.  If @metavar{object-table} is specified, it must consist of a list of two functions.  The first function must take a string, which is the instance handle of a desired object on this kernel server, and return a value of type @type{ilu:ilu-true-object}.  The second funtion must free up any resources used by this object table.

@end deffn

To export a module for use by other modules,
simply instantiate one or more instances
of your subtypes of @cl{@metavar{interface}:@metavar{otype}.IMPL} (which
will inherit from @type{ilu:ilu-true-object}.

@codeexample
=> @userinput{(make-instance 'foo:my-bar.impl :ilu-kernel-server s)}
#<FOO:MY-BAR.IMPL 0x3b32e8 "1">
=>
@end codeexample

@deffn {Lisp @cl{cl:make-instance}} {ilu:ilu-true-object} &key {(ilu-kernel-server @type{ilu:kernel-server} @cl{nil})} {(ilu-instance-handle @type{string} @cl{nil})} => @type{ilu:ilu-true-object}

Creates and returns an instance of @type{ilu:ilu-true-object}.  If @metavar{ilu-true-server} is specified, the instance is created on the specified server.  If @metavar{ilu-instance-handle} is specified, that instance handle is used.

@end deffn

The simplest @language{Common Lisp} ``server'' code would
look something like:

@codeexample
(defun start-server ()
  (make-instance 'foo:my-bar.impl))
@end codeexample

which will create an instance of @cl{FOO:MY-BAR.IMPL} and export it via
a default server.

It's also possible to find out who is making the call:

@defvr {Lisp Variable} ilu:*caller-identity*

The identity of the caller is bound to the special variable
@cl{ilu:*caller-identity*}.  It is a string which begins with the name
of an identity scheme, followed by an identity in that scheme.  For example,
an identity in the SunRPC UNIX identity scheme would be something like
@code{"sunrpc-unix:2345,67@@13.12.11.10"}
(i.e., @code{"sunrpc-unix:@metavar{<uid>},@metavar{<gid>}@@@metavar{<hostname>}"}).  If no
identity is furnished, a zero-length string is bound.

@end defvr

@node Publishing a Common Lisp True Object
@subsection Publishing

To enable users of your module find the exported objects, you may
register the string binding handle of the object or objects, along
with their type IDs, in any name service or registry that is convenient
for you.  In release 1.6 of @system{ILU}, we are supporting an experimental
simple binding method that allows you to ``publish'' an object,
which registers it in a domain-wide registry, and
then to withdraw the object, if necessary.  Potential clients can
find the string binding handle and type ID of the object by calling
a lookup function.  @strong{Note that this interface and service
is experimental, and may be supported differently in future releases
of the @system{ILU} system.}

@deffn {Lisp Function} ilu:publish (@metavar{OBJ} @type{ilu:rpc-object}) => @type{boolean}

Accepts an @cl{ilu:rpc-object} instance and registers it with some
domain-wide registration service.  The object is known by its
@dfn{object ID} (OID), which is composed of the ID of its kernel server, plus
a server-relative instance ID, typically composed as
@code{@metavar{instance-ID}@@@metavar{server-ID}}.  Clients may find
the object by looking up the OID via the @cl{ilu:lookup} function.
The function returns non-@constant{cl:nil} if the publication succeeded.

@end deffn

@deffn {Lisp Function} ilu:withdraw (@metavar{OBJ} @type{ilu:rpc-object}) => @type{boolean}

If @metavar{OBJ} is registered, and if it was published by the same
address space that is calling @cl{withdraw}, its registration is
withdrawn.  The function returns non-@constant{cl:nil} if the object
is no longer published.

@end deffn

If you wanted to create an instance, and publish it, the code for
starting a service might look something like this:

@codeexample
(defun start-server ()
  (let* ((ks (make-instance 'ilu:kernel-server
                ;; specify the service id
                :id "service.localdomain.company.com"))
         (si (make-instance 'foo:my-bar.impl
                ;; specify the server
                :ilu-kernel-server ks
                ;; specify the instance handle
                :ilu-instance-handle "theServer")))
    ;; the OID for "si" is now "theServer@@service.localdomain.company.com"
    (ilu:publish si)
    si))
@end codeexample

Someone who wanted to use this service could then find it with the following:

@codeexample
(defun find-server ()
  (ilu:lookup 'foo:bar "theServer@@service.localdomain.company.com"))
@end codeexample

@subsection Debugging

To help with finding errors in your methods, the variable @cl{*debug-uncaught-conditions*}
is provided.

@defvar ilu:*debug-uncaught-conditions*

If @cl{cl:t}, causes a server to invoke the debugger when an unhandled error in user code
is encountered, rather than the default action of signalling an exception back to the
caller.  The default value is @cl{cl:nil}.

@end defvar

@section Dumping an image with @system{ILU}

@system{ILU} has dynamic runtime state.  In particular, after it is
initialized, it uses several @language{Common Lisp} threads to maintain
part of its state, and may also keep open connections
on operating system communication interfaces.
If you wish to dump an image containing @system{ILU},
you must dump the image before initializing the @module{ILU} module.

Initialization occurs automatically whenever a instance of @cl{ilu:ilu-object}
or @cl{ilu:rpc-server} is created.  Thus you should not create
any instances of either true or surrogate @system{ILU} objects before
dumping the image.  However, you may load all the interface code for any
interfaces that you are using, before dumping the image.

Initialization may also be accomplished by an explicit
call to @cl{ilu:initialize-ilu}:

@deffn {Lisp Function} ilu:initialize-ilu

Initializes the @module{ILU} module.

@end deffn

You may check to see whether the system has been initialized by
examining the variable @cl{ilu::*ilu-initialized*}, which is @cl{t} iff
@cl{ilu:initialize-ilu} has been invoked.

@section The Portable DEFSYSTEM Module

@system{ILU} support uses a portable implementation
of @cl{DEFSYSTEM} to specify modules to @language{Common Lisp}.
@xref{The ILU Common Lisp Portable DEFSYSTEM Module},
for details of this system.

@section ILU Common Lisp Lightweight Processes

@system{ILU} currently assumes the existence of lightweight process,
or thread, support in your @language{Common Lisp} implementation.
It uses these internally via a generic veneer, described
fully in @ref{The ILU Common Lisp Lightweight Process System}.

@section Porting ILU to a New Common Lisp Implementation

The @language{Lisp} support provided with @system{ILU} includes
support for the @system{Franz Allegro Common Lisp 4.x} implementation.
To use @language{ILU} with other @language{Common Lisp} implementations,
please see @ref{Porting ILU to Common Lisp Implementations}.
