2 High-level interface

This high-level interface between a VM and a process is analogous to the C library interface of a Unix-like operating system. The Low-level interface corresponds to the system call interface of a Unix-like operating system.

2.1 Using #lang marketplace and friends

Programs written for Marketplace differ from normal Racket modules only in their selection of language. A Racket module written with #lang marketplace, such as the echo server in TCP echo server, specifies a sequence of definitions and startup actions for an application. Typically, initial actions spawn application processes and nested VMs, which in turn subscribe to sources of events from the outside world.

At present, there’s just #lang marketplace. In future, there will be languages providing greater support for flow control, responsibility transfer, and other networking concepts.

2.2 Using Marketplace as a library

Instead of using Racket’s #lang feature, ordinary Racket programs can use Marketplace features by requiring Marketplace modules directly.

Such programs need to use ground-vm to start the ground-level VM explicitly. They also need to explicitly start any drivers they need; for example, the file "examples/echo-plain.rkt" uses ground-vm along with tcp and an initial endpoint action:

(ground-vm tcp

(subscriber (tcp-channel ? (tcp-listener 5999) ?)

(match-conversation (tcp-channel from to _)

(on-presence (spawn (echoer from to))))))

2.3 Constructing transitions

Each action-tree must be an action tree. It’s fine to include no action trees, in which case the transition merely updates the state of the process without taking any actions.

In the case of transition/no-state, the value (void) is used for the process state. transition/no-state is useful for processes that are stateless other than the implicit state of their endpoints.

• '();

• #f;

• (void);

• a pair of action trees; or

• an action.

When performing actions, a VM will traverse an action-tree in left-to-right order.

'(), (void), and #f may be present in action-trees: when the VM reaches such a value, it ignores it and continues with the next leaf in the tree.

For example, all of the following are valid action trees which will send messages 1, 2 and 3 in that order:

(list (send-message 1)

(send-message 2)

(send-message 3))

(list (list (send-message 1))

(cons (send-message 2) (cons '() (send-message 3))))

(cons (cons (send-message 1)

(send-message 2))

(list #f #f (send-message 3)))

Because #f and (void) are valid, ignored, members of an action-tree, and and when can be used to selectively include actions in an action-tree:

(list (first-action)

(when (condition?)

(optional-action))

(final-action))

(list (first-action)

(and (condition?)

(optional-action))

(final-action))

Finally, these inert placeholders can be used to represent "no action at all" in a transition:

(transition new-state) ; No action-trees at all

(transition new-state '())

(transition new-state (void))

(transition new-state #f)

For example,

(sequence-actions (transition 'x

(send-message (list 'message 0)))

(send-message (list 'message 1))

(send-message (list 'message 2))

(lambda (old-state)

(transition (cons 'y old-state)

(send-message (list 'message 3))))

(send-message (list 'message 4)))

produces the equivalent of

(transition (cons 'y 'x)

(send-message (list 'message 0))

(send-message (list 'message 1))

(send-message (list 'message 2))

(send-message (list 'message 3))

(send-message (list 'message 4)))

2.4 Creating endpoints

The primitive action that creates new endpoints is add-endpoint, but because endpoints are the most flexible and complex point of interaction between a process and its VM, a collection of macros helps streamline endpoint setup.

Almost everything is optional in an endpoint definition. The only mandatory part is the topic.

For example, a minimal endpoint subscribing to all messages would be:

(subscriber ?)

A minimal publishing endpoint would be:

(publisher ?)

While topic patterns are ordinary Racket data with embedded ? wildcards (see Messages and Topics), all the other patterns in an endpoint definition are match-patterns. In particular note that ? is a wildcard in a topic pattern, while _ is a wildcard in a match-pattern.

2.4.1 Receiving messages

The following endpoint subscribes to all messages, but only handles some of them:

(subscriber ?

(on-message

['ping (send-message 'pong)]

['hello (list (send-message 'goodbye)

(quit))]))

2.4.2 Action-only vs. State updates

If not, however, the handler expressions are expected to return plain action trees.

This way, simple handlers that do not need to examine the process state, and simply act in response to whichever event triggered them, can be written without the clutter of threading the process state value through the code.

For example, a simple endpoint could be written either as

(subscriber 'ping

(on-message ['ping (send-message 'pong)]))

or, explicitly accessing the endpoint’s process’s state,

(subscriber 'ping

(match-state old-state

(on-message ['ping (transition old-state

(send-message 'pong))])))

2.4.3 Handling presence and absence events

By default, no actions are taken on such events, but on-presence and on-absence handlers override this behaviour.

For example, say process A establishes the following endpoint:

(subscriber 'ping

(on-presence (send-message 'pinger-arrived))

(on-absence  (send-message 'pinger-departed))

(on-message ['ping (send-message 'pong)]))

Some time later, process B takes the following endpoint-establishing action:

(let-fresh (ping-endpoint-name pong-waiter-name)

(name-endpoint ping-endpoint-name

(publisher 'ping

(on-presence

(list (name-endpoint pong-waiter-name

(subscriber 'pong

(on-message

['pong (list (delete-endpoint ping-endpoint-name)

(delete-endpoint pong-waiter-name))])))

(send-message 'ping))))))

The sequence of events will be:

1. Process A’s on-presence handler will run, and the 'pinger-arrived message will be sent. At the same time,In the current implementation, one happens before the other, but it is nondeterministic which is run first. process B’s on-presence handler runs, installing a second endpoint and sending the 'ping message.

2. Process A’s endpoint receives the 'ping message, and sends the 'pong message.

3. Process B’s second endpoint receives the 'pong message, and deletes both of process B’s endpoints.

4. The on-absence handler in process A runs, sending the 'pinger-departed message.

One possible trace of messages in the VM containing processes A and B is

'pinger-arrived

'ping

'pong

'pinger-departed

By sending the 'ping message only once the on-presence handler has fired, process B ensures that someone is listening for pings.

This way, if process B starts before process A, then B will automatically wait until A is ready to receive ping requests before issuing any.

2.4.4 Exit reasons

2.4.5 Updating endpoints

If, when an endpoint is created, an existing endpoint with an equal? name is already present, then if the existing and to-be-added endpoints have exactly equal roles (meaning equal orientations, interest-types, and topic patterns), the handlers for the endpoint are updated without emitting presence or absence notifications.

This dubious feature can be used to avoid "glitching" of presence signals. A future release of this library will include better automatic support for avoiding such transients.

2.4.6 Who am I talking to?

The carried role describes the intersection of interests between the current endpoint and the peer endpoint, and so can proxy for the identity of the other party. It is in a sense a description of the scope of the current conversation.

It is most common to simply use match-conversation to extract the role-topic alone, since it is seldom necessary to examine role-orientation (since it’s guaranteed to be complementary to the orientation of the current endpoint) or role-interest-type.

See Examples for examples of the use of match-conversation and friends.

2.4.7 Participating in a conversation vs. observing conversations

The core build-endpoint form takes an expression evaluating to a role, rather than a simple topic. This gives full control over the new endpoint’s Orientation and InterestType.

The other forms exist for convenience, since usually the orientation and interest-type is known statically, and only the topic varies dynamically:

• publisher and subscriber are for ordinary participation in conversations;

• observe-subscribers and observe-publishers are for observing conversations without participating in them; and

• observe-subscribers/everything and observe-publishers/everything are like the ordinary observe-... variants, but use interest-type 'everything instead of 'observer.

The publisher, observe-subscribers and observe-subscribers/everything forms create publisher-oriented endpoints, and subscriber, observe-publishers and observe-publishers/everything create subscriber-oriented endpoints. The rationale for this is that as a participant, the code should declare the role being played; but as an observer, the code should declare the roles being observed.

2.4.8 Naming endpoints

Endpoint names can be used to update or delete endpoints.

(let-fresh (my-name)

(name-endpoint my-name

(subscriber ?

(on-message [_ (list (delete-endpoint my-name)

...)]))))

2.5 Deleting endpoints

If reason is supplied, it is included in the corresponding action, and made available in any resulting absence-events.

2.6 Sending messages and feedback

2.7 Creating processes

If #:pid is supplied, the associated identifier is bound to the child process’s PID in both boot-expr and the parent’s k-expr.

The spawn/continue variation includes a k-expr, which will run in the parent process after the child process has been created. Note that k-expr must return a transition, since parent-state-pattern is always supplied for these variations.

2.8 Exiting and killing processes

If reason is supplied, it is included in the corresponding action, and made available in any resulting absence-events.

Terminating the current process is as simple as:

(quit)

When a process raises an exception that it does not catch, its containing VM catches the exception and turns it into an implicit quit action. In that case, the reason will be the raised exception itself.

2.9 Cooperative scheduling

The state of the yielding process will be matched against state-pattern when the process is resumed, and k-expr must evaluate to a transition.

2.10 Creating nested VMs

If #:vm-pid is present, the corresponding identifier is bound in the boot-action expressions to the container-relative PID of the new VM itself. If #:boot-pid is present, however, the corresponding identifier is bound to the new-VM-relative PID of the primordial process in the new VM.

2.11 Relaying across layers

Marketplace’s actions can apply to either of those two networks. By default, actions apply to the VM of the acting process directly, but using at-meta-level to wrap an action level-shifts the action to make it apply at the level of the acting process’s VM’s container instead.

For example, wrapping an endpoint in at-meta-level adds a subscription to the VM’s container’s network. Instead of listening to sibling processes of the acting process, the new endpoint will listen to sibling processes of the acting process’s VM. In this example, the primordial process in the nested VM creates an endpoint in the VM’s own network, the ground VM:

(spawn-vm

(at-meta-level

(subscriber (tcp-channel ? (tcp-listener 5999) ?) ...)))

In this example, a new process is spawned as a sibling of the nested VM rather than as a sibling of its primordial process:

(spawn-vm

(at-meta-level

(spawn (transition/no-state (send-message 'hello-world)))))

Compare to this example, which spawns a sibling of the nested VM’s primordial process:

(spawn-vm

(spawn (transition/no-state (send-message 'hello-world))))