Originally published in Rebol Forces.

Reb-IT!
Author: Jeff Kreis
Date: Feb, 2001
Resources: Official Internet Protocol Standards

Contents

Writing your own Protocol Handler

Hi.  I'm not even sure what the core pdf says on this topic, but here is a brief document I put together to help anyone who is interested in writing their own protocol.  It covers the concepts and basics.  I'll probably be unprepared for any real deep questions, but if you play with some of the stuff below you should be well on your way to hacking custom protocols like a pro!

-jeff

Below is a simple port handler.  I've labeled a few places "Voodoo" which means don't worry about what that part really does, for know, just think of those as magic incantations you have to do to make this example work properly:

make root-protocol [
 
;-- ( voodoo )
port-flags: system/standard/port-flags/pass-thru
 
open: func [port][
print "OPEN!"
;- When you get to open
; the URL has been parsed:
print [
"user is" port/user newline
"pass is" port/pass newline
"host is" port/host newline
"port-id is" port/port-id newline
"path is" port/path newline
"target is" port/target newline
]
 
;-- Say how big we are
port/state/tail: 2000
port/state/index: 0
 
;-- Take our flags and merge
; them into the port/state/flags ( Voodoo )
port/state/flags: port/state/flags or port-flags
]
 
copy: func [port][print "COPY" random 1000]
insert: func [port data][
print ["INSERT data:" mold data]
"OK"
]
read: func [port][print "READ"]
close: func [port][print "CLOSE"]
pick: func [port][
print ["PICK:" port/state/index]
port/state/index ** 2
]
;-- This installs the handler ( Voodoo )
; args: scheme name, this object, protocol port-id (
; 80 for http, 25 smtp, etc..)
net-utils/net-install simple self 0
]

Put the above in a script and DO it, then try these operations at the console:

x: open simple://usr:pass@host.dom/path/target
 
pick x 3
 
pick x 4
 
x: skip x 4
 
pick x 2
 
insert x ["foo" "bar"]
 
close x
 
x: read simple://usr:pass@host.dom/path/target

What you should observe is that OPENING this port handler and doing PICKs and INSERTs on the port does a rather different thing than doing a READ on this port handler.  PICK will print out the port/state/index and return that index to the power of 2.  READS call OPEN, COPY and CLOSE.  COPY returns a random number between 1 and 1000.

What this all goes to show is that the port mechanism doesn't necessarily have to be used in network protocols, but can be used for various things you would like to behave like a port.  A port is a generalization of series, an interface to an external series. So if you think about the Rebol Protocol handlers, they are an attempt to bend the Internet protocols into looking as much like external series as possible.

OPEN PICK INSERT COPY READ CLOSE in PORT CONTEXT

Keep in mind that we have defined OPEN, PICK, INSERT, COPY, READ, and CLOSE inside the port's context.  If we need to use the global versions of these Rebol functions, you must use explicit paths to SYSTEM/WORDS, ie:

system/words/pick "12345" 3

Weird looking errors can result if you forget to use the global definitions and instead refer to one of these locally defined functions within your protocol handler.

If you need to use these functions repeatedly in your protocol handler, it helps to make some equivalences at the top of your protocol handler, ie:

s-pick: get in system/words 'pick
 
...
 
pick: func [port][ s-pick "12345" port/state/index ]

SOME VOODOO

PORT/STATE/FLAGS: this is a setting for the port which determines how your port is called.  You OR in your settings to what you find there at the end of the OPEN function.  The two major types are PASS-THRU ports and DIRECT ports.  For now, stick with using PASS-THRU.  Most of the network protocol handlers are written as PASS-THRU ports with the exception of the SMTP:// protocol handler.

NET-UTILS/NET-INSTALL is the function responsible for turning your port handler into one of the globally accessible protocol schemes. You pass NET-INSTALL the scheme name, which is a word, SELF (the object that defines your protocol handler) and the port-id.  We used 0 because we're not actually opening a network port.

If we were opening a network port, you would set the appropriate port-id and usually you would allow the root-protocol OPEN function to handle opening the low-level tcp port for us.  This is usually accomplished by first calling OPEN-PROTO from within our open function:

open: func [port][
open-proto port
...

This is a preferred way to get your tcp port started because OPEN-PROTO will also take care of handling proxy connections for you. Afterwards, the actual live tcp port is found inside port/sub-port.  This is the port you can pass to READ-IO and WRITE-IO.  SUB-PORTS are opened, by default, with the /lines refinement.

ROOT-PROTOCOL BIG MAMA

At the base of all the protocols is a protocol called the Root-Protocol.  This thing has the phases of a network transaction defined: INIT, OPEN, READ, WRITE, PICK, and others (To see, print mold root-protocol — keep in mind that you'll see the function OPEN and OPEN-PROTO. They are actually the same function.)

Other protocols make themselves out of the root protocol changing what they need but letting the root-protocol take care a lot of the repetitive work.  For instance, the INIT function in the root-protocol takes care of parsing your url into pieces so that when you get to OPEN you have everything you need.  Your protocol usually does not need to define an INIT function most of the time, just taking the one derived from the root-protocol.

The root-protocol is largely a functioning protocol handler in its own right.  You can see the basic network functionality of the root-protocol handler by looking at WHOIS and FINGER.  Here is how both WHOIS and FINGER are created in Rebol:

make Root-Protocol [
open-check: [[any [port/user ""]] none]
 
net-utils/net-install Finger self 79
net-utils/net-install Whois make self [] 43
]

The open-check thing means "send this, expect that".  The root-protocol will take care of doing that for you if you have an open-check, close-check or a write-check (they are NONE by default in the root-protocol).  So above FINGER and WHOIS are almost the same thing as the root-protocol itself: just open this port, send the port/user and return the result.

DAYTIME protocol is entirely the root-protocol.  Here is how it is defined:

make Root-Protocol [
net-utils/net-install Daytime self 13
]

There is no open check, just connect to tcp port 13, and read what is there.

You can create a clone of any protocol handler very easily.  Here we create a clone of http:

make system/schemes/http/handler [
net-utils/net-install web self 80
]

Now you can do:

read web://www.rebol.com

You could also clone a handler and make a change to one of the protocol phase functions, redefine CLOSE, or READ, etc..

It might all seem a little bizarre, but it really is a pretty simple system.  A port-handler has certain functions that are expected to take care of the different parts of a port transaction.  Define the ones you need.. let the Root-Protocol take care of the common cases.

NET-UTILS

Net-utils is an object containing utility functions for the protocols. There's a number of them so they're all bundled together to prevent crowding the global name space.

For example, in the case of FINGER and WHOIS, the root protocol OPEN-PROTO function will pass open-check to NET-UTILS/CONFIRM along with the sub-port (which is the actual tcp level port). NET-UTILS/CONFIRM takes care of inserting the send data (if any) into the port and confirming the response back from the port — firing an error if they don't match.  A NONE for the expect part means it doesn't look for a response, and a NONE for the send part means it doesn't send anything initially.  These checks can also look for multiple responses:

open-check: [none ["HELLO" "HOWDY"]]

The above open-check would cause open-proto to send nothing but read from the newly opened tcp port looking for either "HELLO" or "HOWDY". If it doesn't find either of those it will generate an error.

NET-UTILS also contains NET-LOG which is called from CONFIRM to help trace network activity, the URL-parser which is invoked in root-protocol/INIT, and a bunch of proxy stuff.  NET-UTILS is a whole pocket knife of protocol handler utilities.

SYNCHRONICITY

Right now the protocols are synchronous. Every thing is written like "Send this, then wait for the result and see if it's this other thing".  Asynchronous protocols (which are coming soon to a Rebol near you) are going to be like "Send this, and let me know some time in the near or distant future when you've got something for me to look at". It'll be very different approach and the protocols will all have to be revamped.