Interpreted GUI Programming.

by Peter van Eerten.

§1 Introduction

Shellscripts are often used in a Unix enviroment. By using shellscripts, it is possible to automate daily tasks in a fast and efficient way. In the traditional Unix environments Bourne Shell and Korn Shell are most common, while in Linux the Bourne Again Shell (BASH) is the default.
	Since the shell is capable of interpreting the three cornerstones of a programming language, namely sequence, choice and repetition, in a way a shell script can be regarded as a program which is interpreted by the shell. A shell script is able to solve similar problems as the problems solved by other interpreted programming languages, like Awk, Python, Perl and so on.
	In Linux, almost all interpreted programming languages and shell scripts all have one feature in common: the lack of a decent Graphical User Interface (abbreviated as GUI in this article). The problems solved by these languages always output to a console or a text file, while a Unix desktop often has a graphical nature.
	In this article we will consider several existing solutions to this problem and propose the GTK-server as a generic possibility to have GUI access to interpreted programming. A basic knowledge of shell scripting or interpreted programming may help to understand the issues discussed below.

§2 DTKSH

The Common Desktop Environment (CDE) available in Solaris and Tru64Unix provides a convenient way of shell script programming with graphical capabilities. The CDE comes with a shell which is Korn based, and which contains extra built-in command to access CDE widgets: the DeskTop KornSHell or DTKSH.
	To get an impression of the functionality of this shell, let's take a look at an example DTKSH script, which generates a simple window with a button to exit:
_______________________________

#!/usr/dt/bin/dtksh

activateCB()
{
echo "Pushbutton activated; normal termination."
exit 0
}

XtInitialize TOPLEVEL dttest1 Dtksh $0

XtSetValues $TOPLEVEL title:"Test1"

XtCreateManagedWidget BBOARD bboard XmBulletinBoard $TOPLEVEL \
    resizePolicy:RESIZE_NONE height:150 width:250 \
    background:SkyBlue

XtCreateManagedWidget BUTTON pushbutton XmPushButton $BBOARD \
    background:goldenrod \
    foreground:MidnightBlue \
    labelString:"Push Here" \
    height:30 width:100 x:75 y:60 shadowThickness:3

XtAddCallback $BUTTON activateCallback activateCB

XtRealizeWidget $TOPLEVEL

XtMainLoop
_______________________________

This is the resulting dialog:

Without going too deep into the details of this code, two things can be observed. First, DTKSH contains additional GUI commands next to the standard shell commands. So the graphical programming is implemented within the shell itself. Second, DTKSH is able to register callbacks. A callback is a function or subroutine which is executed when the user presses a button or performs some other action on the GUI. As can be seen in the DTKSH example above, the function describing the buttonaction (activeCB) is passed to the interpreter, which will execute this function when the button is actually pressed.
	Since DTKSH has these two properties, it is impossible to port DTKSH separately to another type of desktop (like KDE or Gnome). The intrinsic GUI commands relate directly to the CDE GUI, and the processing of callbacks depends on the signals generated by the CDE widgets. Therefore the only way of using DTKSH in Linux is by porting and installing the complete CDE desktop.

§3 Xdialog, Zenity, Getgui

An important attempt to realize interpreted GUI programming for Linux was made by the XDialog project. A similar project is Zenity, which actually does the same thing, but is part of the Gnome desktop. Also Getgui, which has similar functionality, but only uses the X API, aims to realize interpreted GUI programming. With these tools it is possible to create different types of dialog boxes, containing predefined types of widgets. It is possible to create dialogs with checklists, with radiolists, calenders, treeviews, and so on. Also plain dialogs like a messagebox or inputbox can be created.
	Below a short example of some Xdialog code:

_______________________________

#!/bin/sh

Xdialog --title "MESSAGE BOX" \
        --msgbox "Hi, this is a simple message box. You can use this to
display any message you like. The box will remain until
you press the ENTER key." 10 41

case $? in
  0)
    echo "OK";;
  255)
    echo "Box closed.";;
esac
_______________________________

This will deliver the following dialog:

If we compare this code with DTKSH, it is clear that the architecture is completely different. Xdialog is a separate binary, which must be executed within a shell script. The example above uses plain Bourne Shell. With the use of arguments to the Xdialog binary, the actual dialog is created. The callback mechanism is implemented by quering the exit code of the invoked binary, which in Bourne Shell is kept in the '$?'-construct.
	The big advantage of this approach is, that Xdialog and friends have no dependency to a particular desktop. Xdialog will work with KDE, Gnome and even CDE, as long as the GTK libraries are installed. The Getgui project does not even need GTK.
	The disadvantage however is, that there always will be a limitation to the layout of your windows. Xdialog and friends cannot position widgets on a window determined by the programmer. Neither do these projects give access to the full range of GUI capabilities, like menubars, statusbars, subwindows and so on. Finally the callback mechanism is limited, since the program can only know which widget was activated by the user after the GUI is exited.

§4 Requirements to interpreted GUI programming

With the previous remarks in mind it appears that the existing solutions are not sufficient to the needs of interpreted GUI programming. It should be possible to have full GUI access to shell scripts similar to the possibilities in DTKSH without depending on a particular desktop. Let's define the requirements for interpreted GUI programming as follows:

1) no dependency to a particular desktop;
2) no dependency to a particular interpreted language or shell script;
3) full access to all widgets;
4) flexible callback mechanism e.g. quering of signals.

Now, it is clear from these requirements that an individual implementation on a per-language basis is impossible. To investigate the sourcecode of existing shell- and programming language implementations, adapting these in a way that extra GUI commands are added to realize GUI programming, is far too much work and beyond a realistic project target.
	Therefore we must think of an external binary, like Xdialog, which can be executed separately. However, this external binary must be able to deliver access to the full range of widgets available, and must not limit the programmer to predefined constructs. In fact, it would be nice if the original GUI API functions can be used directly. Also, this external binary must continue to exist, in order to capture callback signals on the widgets; signals which can be queried and processed by the shell script or interpreted program.

§5 The proposition

I thought of a way to realize the requirements mentioned previously, while being able to use nice looking widgets. Having used GTK before, and being familiar with it's API, it was a logical choice to investigate it's possibilities for interpreted GUI programming. (Instead, the X API or QT API could have been used also.)
	The biggest problem was processing the callback sinals. Reading incoming API functions and passing them to the GTK library is one thing, but how to handle the GTK callback mechanism? The main function in a regular C program using the GTK API always ends with a call to the function gtk_main, which handles the signals and updates the widgets. This is similar to DTKSH, which always ends with XtMainLoop. However, the address or name of a function connected to a button in an interpreted program may be passed to the external GTK binary, but the external GTK binary never is able to execute this function when the button is pressed, since this execution in it's turn needs an interpreter!
	Therefore, the external binary has to keep track of the signals emitted by the widgets. If a click on a button occurs, the callback function in the external GTK binary itself must pass this back to the interpreted program, which in it's turn can execute the appropriate function or subroutine. In other words: the binary needs an intrinsic function to keep track of signals and pass these back. This leads to the problem of the 2-way communication with the interpreted program. Furthermore, as stated, an interpreted program never can use the gtk_main function, and therefore must update the GUI in a different way. This leads to the (second) problem of updating the GUI.
	First the problem of communication. In what way should the interpreted program exchange information with the external binary? The Awk, CShell and KSH interpreter are able to communicate with an external binary using the bidirectional pipe. This pipe is created using special commands within the shellscript. For example:

_______________________________

#!/bin/ksh

bc |&

print -p "3*4"
read -p RESULT

echo $RESULT
_______________________________

This piece of KSH code executes the binary calculator 'bc', which will remain in memory as a background process. Here, commands passed to the pipe (by using '-p' as argument to 'print') deliver a result into the variable $RESULT. Actually, the bidirectional pipe is nothing but the regular STDIN/STDOUT, to which commands are passed, and from which results are read.
	Not every intepreted language is able to communicate this way. As of the moment of writing this article, next to AWK, KSH and CShell, at least the languages newLisp, CLISP, Prolog, PHP and VB Script can do this, using a different syntax. Therefore, to make sure other languages can use our external binary as well, other types of communication interfaces need to be implemented. A very common and well-known interface is the named pipe.
	The advantage of a named pipe is, that it behaves like a file. As long as the interpreted language is able to read from and write to a file, it is also able to communicate with an external binary using a named pipe. For example:

_______________________________

#!/bin/bash

if [ ! -p /tmp/pipe ]
then
    mkfifo /tmp/pipe
fi

echo "3*4" > /tmp/pipe &
RESULT=`cat /tmp/pipe`

echo $RESULT
_______________________________

After executing this script, it will keep waiting for 'the other side' to read the pipe. Now run cat /tmp/pipe | bc > /tmp/pipe from another prompt. The script in it's turn will continue after this line has been executed, and prompt the result: 12. The echo command is redirected to the pipe file, which acts as communication interface between the script and the binary. The example above could have been implemented in a far easier way, using the regular pipe symbols. However, we need the external binary to exist forever, and not to exist only once when reading the contents of the pipe. If the external binary is started, and tries to keep reading the named pipe forever, we can pass and receive messages to and from our interpreted program. If it runs forever, that is, as long as the interpreted program runs, it can also capture callback signals in a consistent way.
	Also a communication interface can be implemented using TCP/IP or UDP/IP. In this way, the interpreted program has to connect to and communicate with a socket which is created by the external binary.
	The second problem was how to update the state of the widgets if a call to gtk_main is not possible? Fortunately, there is an easy solution for this. GTK provides functions to update widgets at any moment. For example, if the interpreted program sends a gtk_main_iteration in a mainloop, the GTK library will update the state of the widgets and also will process callback signals and events. And as already discussed, the callback function itself must be implemented in the external binary, in a way, that it will return the widgetID which has caused the signal to the interpreted program.

§6 Implementation notes

Let's summarize our investigation so far. We need an external binary which gives access to all GUI widgets and is able to capture callback signals. Also it needs a communication interface where the interpreted program can send the GTK API calls to, and can receive information about occured signals from.
	Sending and receiving information using the described bidirectional- and named pipe will unavoidably take place in a synchronous way. This leads to the concept of a server, similar to a webserver or mailserver. The interpreted program then behaves like a client, sending information to the server, which always will reply. This is why I have called the external binary 'GTK-server'.
	Also, for the commands coming in, they need to be looked up in the GTK libraries. It is therefore necessary to know in advance the properties of each GTK function; properties being the type of return value, the signal attached to the widget, and the amount of arguments. It is possible to program these properties hardcoded, but since the GTK libraries evolve and might change, I decided to put the API properties in an external configuration file. This file is a plain text file which can be changed freely by any user, depending on which GTK API is used and how it is used.

§7 Example usage

Now how would the usage of the GTK-server look like? First, it needs to be started with an argument pointing out which communication interface is being used. Secondly, the interpreted program sends GTK functions with an s-expression syntax to the GTK-server, which will return a result code. This is how the GUI is created. Finally, a mainloop must exist where the client program continuously updates the GUI and interrogates the GTK-server for callback signals. Below a simple demonstration program with Korn shell:

_______________________________

#!/bin/ksh

function GTK
{
print -p $1
read -p TMP
}

gtk-server stdin |&

GTK "gtk_init NULL NULL"
GTK "gtk_window_new 0"; WINDOW=$TMP
GTK "gtk_window_set_title $WINDOW Title"
GTK "gtk_table_new 30 30 1"; TABLE=$TMP
GTK "gtk_container_add $WINDOW $TABLE"
GTK "gtk_label_new \"GTK-server demo\""; LABEL=$TMP
GTK "gtk_table_attach_defaults $TABLE $LABEL 1 29 3 7"
GTK "gtk_button_new_with_label Exit"; BUTTON=$TMP
GTK "gtk_table_attach_defaults $TABLE $BUTTON 10 28 20 29"
GTK "gtk_widget_show_all $WINDOW"

EVENT=0

while [[ $EVENT != $BUTTON ]]
do
    GTK "gtk_server_callback WAIT"
    EVENT=$TMP
done

GTK "gtk_exit 0"
_______________________________

This is the result:

The three stages are immediately visible: starting the gtk-server, building the GUI and running through a mainloop. The only thing not visible here is the place where the GUI is actually updated by GTK. This is performed by the function gtk_server_callback, which is not part of GTK, but belongs to the GTK-server itself. This function does not only interrogate the GTK library for callback signals, but the argument wait also takes care of processing the states of the widgets, updating inputfields, display pressed buttons, process events and so on.
	Now how does the GTK-server know on which event to return a value? How does it know that clicking the button with a mouse must return the button ID? This is determined in the configuration file. When defining the configuration for a GTK button, it must be described that the GTK-server connects the 'click' signal to the button:

GTK_LIB_FUNCTION = gtk_button_new, clicked, WIDGET, 0

	So, when the GTK-server is started, it first will read the configuration file containing the description of the used GTK functions. The button configuration shown describes the actual name of the function (gtk_button_new), the signal which will be connected to it (clicked), the value it will return (a widget ID), and finally, how many arguments the function takes (0).
	But how do we know which signals are available for a widget? As may be clear from the last KSH example, the used GTK commands are real existing functions, which normally would be used in a C program. They just have been embedded in a plain shell script. The big advantage of this is, that the standard GTK documentation also applies to our interpreted GUI programming. (Only the internal commands of the GTK-server, like gtk_server_callback, need extra documentation - see below.) So, to find out which signals can be used for a button, just read the GTK documentation for the widget. The actual GTK signal names can be used in the GTK-server configuration file.

§8 A brief overview part 1: internal commands

Let us look briefly to some features of the GTK-server. Right now, the GTK-server can handle three internal commands. The first command is gtk_server_callback. This command already has been discussed. It was clear that it tries to find out which callback signal has occured. Currently, gtk_server_callback command can handle three different arguments:

1.	The argument '0' only checks if a widget has emitted a callback signal. If not, the GTK-server will return immediately to the client program with the result '0'. However, if there was a signal, the widgetID which emitted the signal will be returned. No updating of GTK widgets and events is performed.
2.	The argument '1' checks if a widget has emitted a callback signal. If not, the GTK-server does not return, but keeps waiting for a signal. If there was a signal, the widgetID which emitted the signal is returned. Also all updating of GTK widgets and events is performed. Instead of '1' the words 'wait' or 'WAIT' can be used. This setting suits most situations.
3.	The argument '2' checks if a widget has emitted a callback signal. If not, the GTK-server will return to the client program with result '0', after it has updated all pending GTK widgets and events. If there was a signal, the widgetID is returned. Instead of '2' the words 'update' or 'UPDATE' may be used.

Actually, the last argument to gtk_server_callback is more like a convenience argument; the same result can be achieved by using argument '0' and performing the updating of GTK widgets in your client script.
	Now let's look at the second GTK-server command. With GTK it is possible to connect more signals to the same widget. To enable this for interpreted GUI programming, an extra command was added to the GTK-server: gtk_server_connect. This command takes 3 arguments, namely, the widgetID, the name of the signal and a user-defined description. So if the signal occurs on the specified widget, the user-defined description is returned to the client program. This can be catched in the mainloop, as is visible from the following extended example program:

_______________________________

#!/bin/ksh

function GTK
{
print -p $1
read -p TMP
}

gtk-server stdin |&

GTK "gtk_init NULL NULL"
GTK "gtk_window_new 0"; WINDOW=$TMP
GTK "gtk_window_set_title $WINDOW \"GTK-server demo\""
GTK "gtk_table_new 30 30 1"; TABLE=$TMP
GTK "gtk_container_add $WINDOW $TABLE"
GTK "gtk_label_new \"Hello world!\""; LABEL=$TMP
GTK "gtk_table_attach_defaults $TABLE $LABEL 1 29 3 7"
GTK "gtk_button_new_with_label Exit"; BUTTON=$TMP
GTK "gtk_table_attach_defaults $TABLE $BUTTON 10 28 20 29"
GTK "gtk_widget_show_all $WINDOW"

GTK "gtk_server_connect $BUTTON enter blabla"

EVENT=0

while [[ $EVENT != $BUTTON ]]
do
    GTK "gtk_server_callback WAIT"
    EVENT=$TMP
    if [[ $EVENT = "Blabla" ]]
    then
        echo "Other signal occured."
    fi
done

GTK "gtk_exit 0"
_______________________________

Now the signal enter is connected to the button also. When the mousepointer enters the button area, the GTK-server will return to the program with the user-defined description Blabla. The interpreted program in it's turn can take action once it recognized this description.
	The third and last internal command gtk_server_version returns a string containing the current version of the GTK-server.

§9 A brief overview part 2: the config file

Some more words about the configuration file. The GTK-server reads the configuration in a configfile before it actually processes the command received from the client program. But how does the GTK-server knows where the configuration file resides?
	The name of the configuration file is always 'gtk-server.cfg'. Then there is a predefined search order which is followed. First, the GTK-server looks in the same directory as the client program for a configuration file. If a configuration file exists, this file will be used and the GTK-server will search no further. In this way, for each interpreted program an individual configfile can be created.
	If there is no file with the name 'gtk-server.cfg' in the directory of your program, the GTK-server tries to read the environment variable GTK_SERVER_CONFIG. This variable should contain the full path including filename to the configurationfile.
	If the environment setting GTK_SERVER_CONFIG cannot be read, the GTK-server will look in the Unix default directory's: first in the /etc directory, and if not found there, it will look in /usr/local/etc. If no configuration file was found at all, the GTK-server will print an error message and exit.
	It was already mentioned that every GTK function used in the interpreted program must be defined in the configuration file. The syntax of this definition can be described as follows:

GTK_LIB_FUNCTION = <func>, <callback>, <type>, <amount>, <arg1>...<argx>

To repeat what previously has been said, the function name must be entered, the signal to which the widget listens, the type of information the function returns, the amount of arguments it contains and also the type of each individual argument. To give another example:

GTK_LIB_FUNCTION = gtk_label_new, NONE, WIDGET, 1, STRING

Here, a label is defined. The function gtk_label_new will create a label on your GUI. The definition in the configfile determines no callback signals to be attached to the label, and a widgetID to be returned. There is only 1 argument to gtk_label_new which is of the type 'string'.
	Next to a description of each individual GTK function, also the name of the actual GTK library you want to use must be defined:

GTK_LIB_NAME = libgtk-x11-2.0.so

It is clear that the GTK library must be available in your library path (LD_LIBRARY_PATH) in order to realize GTK functions.
	The last option to discuss is the logfile. It is possible to start the GTK-server with an argument forcing the creation of a logfile. For example:

gtk-server stdin log

In the logfile the GTK-server writes the commands it receives from the client script, and it also writes it's responses. In this way it is very easy to trace where the GUI programming goes wrong. Now, the 'gtk-server.cfg' configfile also must define where the logfile is created:

GTK_SERVER_LOG = /tmp

The resulting logfile will appear in the /tmp directory. A logfile always will have the name 'gtk-server.log'.
	This concludes the brief overview of the functionality of the GTK-server.

§10 Conclusion

The original problem of GUI access to interpreted languages can be solved to a great extent with the design described above. Our investigation concentrated on GTK, but other graphical toolkits (QT, X-Forms) probably will work also. All requirements have been met: no dependency to a particular desktop, no dependency to a particular intepreted language, full access to all widgets and a flexible callback signal handling.
	The advantages of a binary like the GTK-server are obvious. There's no need for a complicated compile process, no need to learn C or C++, no need to explore and hack sourcecode of existing interpreted languages to add GUI commands, there is no limitation to predefined widgets and dialogs, also all widgets are within reach, both GTK1.x and GTK2.x are within reach, and finally, fast tracing of GTK functions is possible due to the fact that the client program is interpreted and due to the availability of a logfile.
	However, the downside must be mentioned also. A few complicated GTK functions return their result in a pointer. For example, the function gtk_get_start_iter will return the position of the text insertion point in it's second argument, which actually is a pointer to a memory address. Some interpreted languages however can allocate memory and read the contents of such pointers (like newLisp), but plain shell scripts cannot. Secondly, the implemented GTK-server communication interfaces cannot be handled properly by some interpreted languages (like Haskell and REXX, I never got these working). Consequently it must be concluded that a 100% coverage of GUI widgets for all interpreted languages using the GTK-server architecture cannot be reached.
	In spite of this, the GTK-server provides an elegant way for most interpreted languages to create a GUI which is completely self-defined and which does not rely on a limited set of fixed widgets.

_______________________________

URL's

DTKSH		http://docs.sun.com/?q=dtksh
The GTK-server	http://www.gtk-server.org
Xdialog			http://xdialog.dyns.net
getGUI			http://getgui.sourceforge.net

Document created with AbiWord 2.0.6.