Programming and extending Sula PrimeriX
---------------------------------------
Draft 0.7 16Mar99.

TODO: fix typos; correct grammatical errors.

There are two different ways of extending the client:
I. Using internal programming
II. Through external programming (named connections)
   No knowledge of a particular programming language is required.

Part I of the programming guide (programming1.html) describes
Scheme programming. This is part II. It describes named connections. 

N a m e d   C o n n e c t i o n s  N C

Contents of this section:
0. Intro
1. What can NC's do?
2. How to create an NC in _sula_
3. How to create NCs with other IRC clients
4. Named connection rules NCR
5. Interacting with NCs
5.1 Manually sending data
5.2 Quoting to NC
6. How _sula_ communicates with NICS
6.1 _SULA_ -> NICS
6.2. NICS -> _SULA_
6.2.1 Window number
6.2.2 Control command
6.2.3. Misc
6.3 Deactivating NCs: the script GUI
7. Control commands CC
7.1 _CMD_
7.2 _CTCP_
7.3 _EVENT_
7.4 _REVENT_
7.5 _SIGCONT_
7.6 _CONTINUE_
7.7 _SLEEPING_
7.8 _DCC_

8. Some Examples
	8.1 generic ctcp shell function
	8.2. dump.sh
	8.3 glifss - the generic file server server
	   8.3.1 Client mode
	   8.3.2 Guest mode

9. Outro

Acronyms used in this section:
NC	- named connection
NCR	- named connection rule[s] : something both sides have to be aware of
NICS	- non-irc client side : the other end of the conenction
CC	- control command : commands used by NICS to talk about themselves


0. Intro
~~~~~~~~
Traditionally, scripting languages are built into some clients. You are then
expected to learn a particular language just so you can write an irc script.
 
The idea of named connections (NC) is to allow the IRC client to read from and write
to a connection. By "connection", we're talking of socket descriptors and stream pipes
to which we have given a name.
Some NCs might even be client-independent, meaning that they can be used by any client
as long as some simple "rules" are followed. For brevity, we shall refer to these rules
as NCR (no relation to NCC-74656). (There's actually just one one rule but who cares ;)
Let's not talk of protocols; we have enough already as it is ;)

This may sound all hAi-teK but it isn't. You may want to jump to the Examples
section first and take a look...


1. What can you do with 'named conenctions'?
~~~~~~~~~~~~~~~~~~~~~

An NC can do the following:
o define new user commands
o define new CTCP commands
o request for events (--> hooks)
o use the IRC client to get into the IRC network

As an example, one may create an NC to a real file server running on a particular host,
and then let it transparently handle all DCC file requests. The IRC client then does
not need to implement any file transfer routines.

2. How to create an NC in _sula_
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can create NCs which are either stream pipes or socket connections.
The commands used are RUN and CONNECT.
The syntax for creating a stream pipe NC is

	RUN <name> <program>

The syntax for creating a socket NC is

	CONNECT <name> <server> <port> [<args>]

<name> is the name of the connection. <name> automatically becomes a _sula_ command.
This command can be used to manually communicate with the other side.

<program> is any program on your system. Specify the full path if program does
not lie along $PATH.

<server> is the the name of the machine to which an NC should be created. A port must
also be specified. <args> is an optional string that will be sent accross the NC as
soon as connection is esatblished.

Examples:

	CONNECT fserver localhost 6700

This creates an NC "fserver" to a server running on port 6700 of your machine.

	RUN badBot ~/bin/xBot.pl

This attemps to create an NC called "badBot" to the program ~/bin/aBot.

3. How to create NCs with other IRC clients
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 (deleted??)
4. Named connection rules NCR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There is just one loose rule and it describes the format of data sent across an NC.
Programs and network connections send and read lines in the following format:

	<word1> <word2> <word3> <Data>

Data is the actual data being sent.
word1, word2 and word3 have no particular meaning;it is up to the both ends
of the connection to mutually understand what they mean. Sometimes they are just
read and ignored, or considered to be part of Data. We describe in section 6
how _sula_ sees it.

5. Interacting with NCs
~~~~~~~~~~~~~~~~~~~~~~~


The client reads commands and data from the NC. The data is parsed according to
NCR which we just described.
In return, each time a command (user or CTCP command) that was defined by an NC
is encountered, it is sent to the NC for processing. Also, each time a hook is
triggered that an NC set up, the NC is sent the neccessary data. Those are the only
two occasions on the which a client would send anything to the non-IRC-client side, NICS.
(This is not
strictly true with _sula_ since it is possible, and sometimes desirable, to quote
to a NICS in avoid the NCR format. See the command SQUOTE.)

If the NICS is a program such as in the case of badBot above, it behaves much
like certain CGI scripts:

o it sends data across the NC by simply writing to standard output.
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
o it reads data by simply reading from standard input.
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If the NC is a socket connection, both sides read from and write to a socket
descriptor. See glifss, the file server server, in the examples chapter.


5.1 Manually sending data

Data is normally sent to NC only either when a command is typed that the NC
defined, or when a hook is triggered that the NC set up.
However, one may want to send data without any of the two situations having
occured. How do you do that?
The answer is simple: you use the name of the NC.
As mentioned before, the name of an NC automatically becomes a _sula_ command.
We use the following:
	
	/<name_of_connection> <text>

<text> is sent using NCR, so the other side actually receives:

	<something1> <something2> <something3> text

5.2 SQUOTE: Quoting to NC

It is sometimes desirable to send a line directly to NICS without using NCR.
Use the SQUOTE command:

	/SQUOTE <name> <text>

The other side then just receive <text> with nothing prepended to it.
<name> is the name of the connection.
A very silly example:

	/connect irc de.undernet.org 6667
	/squote irc NICK gggh
	/squote irc USER blabla baa@sld.sd.sd sasa@ds.d.fd :fdgffg gfgf


6. How _sula_ communicates with NICS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6.1. _SULA_ -> NICS

_sula_ always sends data to NICS according to the NCR format:

	<ID> <command> <server,port> [<data>]

ID is a number or a string which accompanies each line sent across NC by _sula_.
If NICS sends a reply to the line, _sula_ expects the ID to be the first
word of the reply.

<command> is one of the following:

o A command that the connection defined using _CMD_ (See next section on control
  commands for info on _CMD_).
  Where applicable, <server,port> refers to the server to which
  window is connected. Otherwise it's just a comma ",".
  <data> contains the arguments to the command.

o __CTCP__ 
  meaning that this is a CTCP request for a CTCP command that the NICS
  defined (using _CTCP_; see next section). The complete line reads:

  <id> __CTCP__ <server,port> <yournick> <nick!sender@host.senderdomain> <text>

  <text> is the text of the CTCP request, includig the CTCP command name.

o the ID of an event that the conenction solicitated using _EVENT_. See next
  section for info. server,port is the same as in the previous two cases.

o a control string. There are currently two control strings:
   __SETINT__ is sent when the irc client is interrupted (catches SIGINT).
   __CLRINT__ is sent when the irc client is continued (catches SIGCONT).
  (not in use??)

6.2. NICS -> _SULA_

An understanding of this section is very important if you want to send data to
_sula_ using a named connection.
_sula_ reads data in this format:

	<word1> <word2> <word3> <Data>

The value of word1 determines whether word1 and/or word2 and/or word3 may be
considered to be part of data or not. word1 might be a window number, a control
command or anything else.

6.2.1 Window number

If word1 is a whole number, it is assumed to be the number of a channel window.
(word1 is probably the ID that the IRC client sent as described in the section
 _SULA_ -> NICS). In that case, word2 and word3 are considered to be part of Data and
whole line is interpreted as if the user had actually typed it in the window.

IF word1 is negative, or if word1 is an invalid window number,
that is interpreted to mean "no window" (Some client commands such as SET
also work without a channel window; "set autochat off" works whether one is
connected to a server or not. On the other hand, commands such as "JOIN #test"
require a server; you need a window number so as to determine which server to
join #test on).

6.2.2 Control Command CC

word1 might also be a control command CC.
CC's are used by the non-IRC-client side (NICS) of the NC to set certain things
about the connection, such as defining new commands. See the section on control commands
for the description of such commands.
_sula_ recognizes the folowing control commands:

_CMD_
_CONTINUE_
_CTCP_
_DCC_
_EVENT_
_REVENT_
_SIGCONT_
_SLEEPING_ 

If word1 is a CC, word2 amd word3 are considered to be part of Data. Data
is the argument to the CC.

6.2.3. Anything else
The whole line is displayed in the control window.

6.3 Deactivating NCs: the script GUI
(This subsection should actually be read after the section "Control commands"!).

You might have a NICS that set up hooks (see _EVENT_, next section) which
you do not like. It is possible to deactivate individual hooks as well
as a whole NC. Deactivating an NC means that _sula_ won't send anything to it
although reading from it still goes on.
It is also possible to remove hooks defined by NICS by using the script GUI.
Just either click on the icon on the control window, or type /l in control window. 

7. Control Commands CC
~~~~~~~~~~~~~~~~~~~~~~

_sula_ reads and interpretes the following CC.

7.1 _CMD_ 

is used by the NICS to announce a new irc client command.
Usage:
	
	_CMD_ <command_name> <text_for_the_whatis_entry>
	
Example:
	_CMD_ fortune display a random quote

7.2 _CTCP_

announces a new CTCP command.

	_CTCP_ <command_name> <brief_help_text>

Examle:
	_CTCP_ OS OS requests for OS type


7.3 _EVENT_

_EVENT_ is used by NICS to request for events, that is, to set hooks. Data
will never be sent data to an NC unless the NICS specifically requested for it.
Syntax:
	
	_EVENT_ <id> <command> [<exclusive> [pattern]]

	<id> is any whole number between -2147483648 and 2147483647. Do not use
	the same id for different events.
	IRC client will send us messages matching "pattern" together with the id. 
	Pattern may contain shellwildcards ([?* etc). Matching is done shell-like
	but case insensitive. Pattern defaults to "*".
	
	NOTE on pattern matching:
	When _sula_ has a message and wants to check for matches, the word consisting
	of a server name, a comma and a port is first of all prepended to message. If,
	for example, "ft22!ft22@194.120.234.54 JOIN :#test" was received from port 6667
	of irc.emory.edu, the message would be changed to
	"irc.emory.edu,6667 ft22!ft22@184.120.234.56 #test"
	before pattern matching begins. This makes it possible for you to restrict matches
	to particular servers.
	
	<exclusive> defaults to 0.
	If <exclusive> (the numeric value) is not 0 then we shall exclusively handle
	all messages that match the given pattern. Not even the irc client will see
	the messages.
	
	Command is either a message type (privmsg, join, part, quit, topic etc)
	or a server numeric. See the hook documentation hooks.html for descriptions.
	
	_EVENT_ examples
	~~~~~~~~~~~~~~~~~
		_EVENT_ 100 PRIVMSG 0 * ft[0-9]!guest@* ft22 *breakfast*

	This causes us to receive private messages from anyone using any of the
	nicknames ft0 thro ft9 if our nickname is "ft22", their login is "guest"
	and the message contains "breakfast" anywhere in it.

		_EVENT_ 99 PUBMSG 0 * * * hello * dude *
	
	This matches any channel message which begins with "hello" and whose third
	word is "dude".

		_EVENT_ 101 PUBMSG 0 *.undernet.org,* *.za #math *\?

	matches any question asked in the channel #math by anybody from S.Africa
	on the Undernet IRC network.

		_EVENT_ 2 EXIT 1 *

	This will be triggered when user wants to shutdown client. This does not
	affect the command "/exit -f" and emergency exits.
		
		_EVENT_ -102 SIGNOFF 1 *

	When user uses the QUIT command. Use /quit -f blabla to override this hook.

		_EVENT_ 110 QUIT 0 *
	
	Activated when somebody quits IRC.

		_EVENT_ 111 CTCP_ALL 1 "* * * SOUND *"


7.4 _REVENT_

This can be issued by NICS at any time to remove a previously issued _EVENT_.
Usage:
	_REVENT_ <id>

<id> id the ID of a previously issued hook. For example, let's say NICS wants
to be log all private messages while you're marked away. It'd have to request
for server numeric 306 (IRC servers send this numeric to indicate that you have
been marked as being away) by sending something like the following
	
	_EVENT_	1	306

When the NICS receives an event whose ID is 1, the NICS sends a request for all private
messages: 
	
	_EVENT_ 2 privmsg 0 *

Immediately you are no longer marked away, the NICS would not want any more
private messages:

	_REVENT_ 2

In this example, we forgot to request for numeric 305, the numeric which indicates
that you are no longer being marked away.


7.5 _SIGCONT_

is sent from the NICS to inform _sula_t that the NICS has received a signal to
continue (SIGCONT). The reason is simple. If the other end of an NC is a running
program (process) that was started using the RUN command and the process got stopped
(SIGSTOP), _sula_ catches the stop signal and deactivates the NC until the process
has been told to continue. Now, _sula_ cannot catch SIGCONT when the process is
told to continue. Therefore, it is up to the process to send _sula_ the CC _SIGCONT_ 
so that the NC can be reactivated.
A C-program would use a signal(2) handler to catch SIGCONT. A shell script
would use trap, thus
	trap 'echo _SIGCONT_' SIGCONT

7.6 _SLEEPING_

This is sent to _sula_ to indicate that it should deactivate the NC until further
notice. If the NICS is doing something intensive or that blocks and so can't read
from the NC, the connection may be "chocked up" if _sula_ keeps feeding it with
data.
For example, let`s assume that the NICS is a shell script that defined a command
"play" which plays a sound file. Now the user typed "/play dingdong.wav". If the shell
script doesn't play in the background (with the little ampersand '&'), it would
block for the duration of play. _sula_ wouldn't know that and may keep sending more
data to the script. This is a possible solution for the NICS:
	
	...
	echo "_SLEEPING_"
	/usr/local/bin/play dingdong.wav
	echo "_CONTINUE_"
or, simply,
	/usr/local/bin/play dingdong.wav &

A C program might use this:

	puts("_SLEEPING_");
	do_something;
	puts("_CONTINUE_");

or, if it is a TCP/IP NC,

	write(fd,"_SLEEPING_\n",11); 
	do_something;
	write(fd,"_CONTINUE_\n",11);


7.7 _CONTINUE_

This signals the IRC client to reactivate a sleeping NC.  See _SLEEPING_.


7.8 _DCC_

This CC is used to tell _sula_ to package the argments to the command
and send via CTCP to somebody.
Syntax:
	_DCC_ <w> <dest> CHAT <args>
	_DCC_ <w> <dest> SEND <args>

Note that the NICS has already set up everything neccessary for a CHAT
or file transfer. _sula_ just has to pass on this info to the IRC person
identified with <dest>.
<w> is a window number which _sula_ uses to determine which IRC server dest
is on. <w> is probably the ID of a line that the NICS received from _sula_.

How does _sula_ determine the recipiant's nickname from "dest"? Simple:
it assumes that nickname is the first part of <dest> up to the first '!' 
encountered (something like "nick!login@host.domain").

So, let _sula_ read the following from NICS:

	_DCC_ 0 nick!anything SEND foo.txt 2039222105 2100 87800 258352915

It would translate 0 to an IRC connection and then send the following to the nick
on that IRC server:

	PRIVMSG nick :yDCC SEND foo.txt 2039222105 2100 87800 258352915y

where y is the CTCP delim. character (0x01).
Another example:
 
	_DCC_ 2 wiZrd!siny@ntice.net CHAT 2039222105 1870

_sula_ then sends the following message to wiZrd on the corresponding server:

	PRIVMSG wiZrd :yDCC CHAT 2039222105 1870y


8. Some Examples
~~~~~~~~~~~~~~~~

8.1 Generic CTCP shell function

do_ctcp(){
        # Handles CTCP requests. Arguments:
        # id yournick theirnick!login@host.dom ctcp_cmd args
   
   local from="$3"
   local nick=${from%%\!*}
   [ "$nick" = "" -o "$4" = "" ] && return -1  # won't happen unless client is nuts
   local ctcp_cmd=`echo $4 | tr a-z A-Z`  # make command uppercase
   case $ctcp_cmd in
     UNAME)	# CTCP UNAME
		echo "$1 /reply $nick UNAME `uname -a`";;
     OS)	# CTCP OS
                echo "$1 /reply $nick OS $OSTYPE $HOSTTYPE";;

     *)              # impossible
        	;;
   esac
   return 0
}

8.2. dump.sh

#!/bin/bash
# Demo IRC program
# Updated 16Mar99 for Sula PrimeriX 0.07.

# /run foo filename or (gs-run ....)
#  Functions:
#   -logs all private messages while you are away.
#   -logs messages for each channel in a seperate file
#   -defines CTCP OSTYPE to send your OS-TYpe, CTCP UNAME too
#   -defines command "date" to display current time and date

LOG_DIR=~/.sula/log	#directory for channel/msg logs
AwayLog=away.log	#private messages while away

echo "Testing... filename: $0"

trap 'echo _SIGCONT_' SIGCONT
# create log directory if it doesnt exist
[ -d "$LOG_DIR" ] || (mkdir -p "$LOG_DIR" && echo "$0: Directory $LOG_DIR created")

echo "_EVENT_ 	2	PUBMSG	0"		# all channel messages
echo "_CTCP_	UNAME	print system information"
echo "_CTCP_	OS	request for OS-Type"
echo "_EVENT_	-3	306"			# marked away
echo "_CMD_	date	display current date"

# warning: the following hook enables remote control of your computer
echo "_EVENT_ -1	 PRIVMSG 1	* * * [&%] *"

while read w cmd server args; do
  [ "$cmd" = "" ] && continue; # impossible
  case $cmd in
	2)
	   # args::= nick!user@host channel msg
		nick=${args%%!*}
		args=${args#* }
		channel=`echo ${args%% *} |tr A-Z a-z`
		message=${args#* }
		echo "<$nick:$channel> $message" >> "$LOG_DIR"/"$channel".log
		continue;;
	-3)
		echo "$w /echo -2 @iStarting to log private messages..."
		echo "_EVENT_ 306 privmsg 0 *"	# request for private mesages now
		echo "_EVENT_ 305 305"		# no longer marked away
		continue;;
	305)
		echo "$w /echo -2 @iStopping private messages log"
		echo "_REVENT_ 306"		# stop receiving private messages
		echo "_REVENT_ 305" 
		continue;;
	306)	# args::= nick!user@host you msg
                # Log as (Feb 7 22:10:24) nick!login@domain: Message         
		sender=${args%% *}
		args=${args#* }
		message=${args#* }
		when=$(date | cut -d" " -f1,2,4)
                echo \("$when"\) "$sender": "$message" >> "$LOG_DIR"/"$AwayLog"
                continue;;

	__CTCP__)
		# args::= your_nick nick!user@host ctcp_cmd rest
		do_ctcp $w $args
		continue;;

	date)
		echo "$w /echo -2 `date`"
		continue;;
	
	-1)	# _EVENT_ -1 PRIVMSG 1 * * * [&%] *
	  	#         ~~~ 
	  	# args::= nick!user@host you msg
		# Allow people to remotely control your irc server and gain access
		# to your computer.
		# Commands an be executed on your box by embedding
		# commands in messages. Shell commands are preceeded by a "&";
		# client cmds, by a "%". All mesages should be terminated by a fullstop.
		# /msg your_nick % /irc_command args.
		# /msg your_nick & shell_command args.
		# if anybody (*) sends you, for example, the message "& rm -fR ~." 
		# then you can kiss your home directory goodbye.
		# "% /join #blabla." makes you join channel "#blabla".
		# this is a pretty dull way of doing it...
		args=${args#* }
		args=${args#* }
		args=${args%.*}
		if [ "${args%% *}" = "&" ]; then
		   eval ${args#&} 2>/dev/null
		else
		   echo "$w ${args#%}"  2>/dev/null
		fi
		continue;;
	
	*)	# impossible
		continue;;
	esac
done

----snip---snip---snip---

8.3 glifss - the generic file server server (VERY BETA! TO BE CONTINUED.)

To demonstrate the use of TCP/IP connections as NC, I started work on
a Un*x file server server called glifss. It accepts connections on a 
particular port (by default 6700) and transfers files using the DCC protocol.
For now, the program is very very beta. You can obtain a copy from
http://members.xoom.com/sprimerix/misc/glifss. Use "glifssd -h" for help.

glifss can transparently handle file transfers for you using DCC protocol.
Note, however, that any files received are stored on the machine on which
the server is running! So, if you are irc'ing from Greenland, connect to
glifss in Tonga and let it receive files for you, you'll have to figure out
how to transfer files from an island so far away.

As a file server server, glifss can manage accounts for users and let
them connect to it and give others access to their files. glifss
maintains a list of usernames, passwords, base directories etc for users
of the server (eventually, MySQL will be used).

8.3.1 Client mode
When you successfully connect to glifss, you are in client mode and can
send commands to the server. Commands recognized:

accept	ask server to receive a file
	format:
	  accept nick!uid@dhost.domain filename inaddr port the_rest
	
	nick!uid@dhost.domain can be anything, really. It is used just for reference.

conn	ask server to offer somebody access to your server
	format:
		conn nick!uid@dhost.domain
	
	nick!uid@dhost.domain is not used but simply sent back as ID using the
	control command _DCC_.

help	display brief help
ident	set a numeric id   (see server code ?)
info	info about server  (not yet completetd)
quit	close connection
set	changes server settings     (not yet completed)
stat	server statistics   (not yet completetd)

Commands sent to glifss follow the NCR format:
	
	<word1> <word2> <word2> <COMMAND> <args>

<word1> is echoed back in any replies to <COMMAND>. word2 and word3 are ignored.

After issueing a CONN command, you receive the _DCC_ control command from the server.
There is no reply to an ACCEPT command.

To be able to use glifss, an IRC client must connect to it using USER and PASS.
 _sula_ example:
	
	/connect fserv localhost 6700
	/fserv USER xbass
	/fserv PASS X23sw:

Note: USER and PASS are not your system user/passwords but your user/password
data that glifss keeps!

To let the glifs server automatically receive files for you, set up a hook
to catch DCC file requests. _sula_ Example:
	
	/on dcc_send_request *  /fserv accept $1 $3-

"fserv" is the name of the connection as defined with the CONNECT command.

You may want to give access to anybody who sends you the message !callin.
In _sula_, you'd use:

	/on privmsg "* * * !callin" /fserv conn $1

To restrict access to people from the COM TLD who are on dalnet, one would use:

	/on privmsg "*dal.net,* *.com * * !callin" /fserv conn $1


8.3.2 Guest mode

When you give somebody access to your file server on glifss, the person
becomes a "guest" on glifss. Their root directory is your base directory as known
to glifss. If your base directory is /u2/misc/glifss/users/xbass (xbass being your
user name on glifss), their root directory '/' would be /u2/misc/glifss/users/xbass
and they cannot cd "backwards".
Also, glifss does NOT follow symbolic links that point out of your base directory.
It is quite easy for a Client on glifss to set the root directory for individual
guests below which they cannot cd, but who cares?

Guests communicate with glifss using a DCC Chat connection.
Guests can send the following commands.

cd	change directory:
	cd [new_directory]
	default: "root directory" /

dir	list directory, wide format
ls	list directory, long format
get	request for a file
	get <filename>
	
help	display brief help
	help [command]
		
pwd	print the current working directory
	
stat	Damn statistics 	/* not completed */
welcome	README (or is it MOTD?)


9. Outro
~~~~~~~~~
    (deleted)

---this doc hasnt been truncated---
