Plugin architecture with REALbasic and RBScript

This article gives some ideas on how to use REALbasic's RBScript feature to add plugins to a REALbasic application.

A plugin is hereby defined as a program that can be added to the already-built application and executed at the app's discretion.

My example

In the next paragraphs I will suggest ideas. I will base them on the following example:

Design for a search app, similar to how Apple's Spotlight works.

The app shall be a tool to search for text in many files. While the app has hard-coded routines to search for text in a single file, it wants to allow plugins to convert binary files into text files where appropriate. Imagine documents from applications such as Word (.doc), Acrobat (.pdf) and even REALbasic (.rbp), which, when opened in a plain text editor, show a lot of gibberish that you don't want to have searched. Instead, you want it all readable.

Hence, the plugins for this search app will basically function like this:

The app will pass each file, before looking at it, to the plugins, so that the plugins can convert the file to text if necessary. Each plugin gets to look at the file, decide whether it can understand it. If it can, it returns the readable text, which the app then searches for the contents the user wants to find.

Defining the Plugin API

The plugin will have one or more more functions that the app can call. The functions need to be well defined. This should best be written down first, into a separate document, for later reference. And it needs to be kept up-to-date all the time, of course.

Here are a few functions that should never be missed in such an API:

• The plugin should be able to learn the app's version and maybe also the version of the plugin API separately. Note that it can happen that you later decide to add functionality or even change some, and plugins should be able to react accordingly. Thus the plugin should be able to retrieve or be passed a numeric version number for easy comparison.
• Similarly, the app should learn the plugin's version, mostly to be able to show it to the user, but it might also sometimes help to let the app add work-arounds for problems of certain versions of known plugins.
• Additionally, it may be useful that the plugin lets the app know on which API version it is based. That way, if the app changes the API radically, it could reject loading the plugin and let the user know that he needs to get an updated version of the plugin.
• As RBScripts cannot store data by themselves between several invocations, the app should provide a way for them to store data across calls to plugin functions. I suggest adding at least two functions that store and retrieve Strings, indexed by a name (as String). The app stores those separately for each plugin.

For our example, here are the functions it shall implement:

• CanHandleExtension (extension as String) as Boolean - gets passed a file's extension (without the dot) and returns true if it may be able to read and convert it. The app only calls the next function (ConvertToText) if it returns true.
• ConvertToText (fileRef as File) as String - gets passed the file reference and then can read it and try to convert it to text. If it succeeds, it shall return a non-empty string containing the text that can then be searched by the app. If it returns an empty string, this means that this plugin could not handle this file. The "File" class is defined below.

That's all the functions the plugin needs to provide.

Next, we define the functions the plugin can invoke to help in its task. These functions will be implemented by the app:

• StoreValue (id as String, value as String) -- saves 'value' in a Dictionary with index 'id' persistently.
• RetrieveValue (id as String, ByRef value as String) as Boolean -- fetches value for 'id' from the Dictionary, returns true if found. Does not alter 'value' if no value has been stored for 'id' yet.
• OSPlatform () as String -- returns "osx" when running on Mac OS X, "win" for Windows and "linux" for Linux.
• SystemCall (command as String) as String -- invokes a shell command with parameters via RB's Shell class.
• class File -- a class with the following methods:
  • ReadAll () as String -- returns the entire contents of the file as a string (with nil encoding).
  • ShellPath () as String -- returns the file's ShellPath as a string.
  • Constructor (path as String) -- creates a new "File" object from a shell path.

With these methods, a plugin can refer to a file, read its contents or pass it to a command line tool for conversion (which may be useful to convert .doc and .pdf files), or even read neighboring files in case it's a set of files that need to be read for the conversion.

Note that RBScript already offers a set of basic functions by default, such as many string functions. But apart from those, any other functions that are available to a RB application need to be explicitly made available to the plugin. This means that the plugin runs in a sandbox where it can't mess with files or other system resources unless the app allows it by providing operations for that explicitly via a custom class assigned to the Context property of a RBScript instance.

Plugin file format

It's up to you to define how you want plugins for your app be formatted so that you can identify and load them.

Basically, always use a format that allows you to have more than just one RBScript loaded. You'll probably want to have separate scripts for different tasks.

My usual solution is to use one XML file per plugin that includes not only the script code but also other attributes a plugin might want to provide, such as version, name, descriptive text, embedded pictures (base64 encoded) and so on. Optionally, you might require the plugin to be a folder, in which the main xml file resides plus additional files the plugin might need (e.g. larger picture files).

I suggest an XML with the following attributes:

<plugin
   internalName="...a never changing and unique name to identify this plugin..."
   visibleName="...what the user sees as the name for this plugin..."
   internalVersion="...an ever-increasing number, preferrably integer..."
   visibleVersion="...a text, preferrably in the common version scheme..."
   apiVersion="...the number of the API version used here..." >
  <method name="...name of a function..."
    args="parameter names and types, in RB syntax" returns="...a type...">
     ... here goes the RBScript code for this function ...
     (note that you need to convert a few characters, such as "<" and ">" need to
     appear here as "&lt;" and "&gt;" or it would confuse the XML structure)
  </method>
  ... and possibly more functions as <method> tags
</plugin>

Note: Declaring the arguments and return types is useful to let the app know which parameters the plugin expects. This would help, for instance if the API the plugin uses is newer than the app it is used with (shouldn't happen if the user keeps all his apps up-to-date, but it's still better to always clarify the expectations as verbose as possible in an API): The app may then supply default values to the unknown arguments.

For our example, I've decided to go a simpler way, though. All methods will be just declared in one large RbScript file, and then code is generated by the plugin loader to access either of those functions.

For simplicity, I simply assume that there is one text file ending in .rbs (for "RBScript", which Yuma uses as well), containing all RBScript code and nothing else.

A simple plugin to handle plain text files would look like this:

function CanHandleExtension (ext as String) as Boolean
  return ext="txt" or ext="xml"
end function

function ConvertToText (f as File) as String
  return f.ReadAll
end function

Another plugin to convert Word .doc files could look like this (note: this uses a tool for Mac OS X - I don't know how to do it on Linux and Windows):

function CanHandleExtension (ext as String) as Boolean
  return ext = "doc" and OSPlatform() = "osx"
end function

function ConvertToText (f as File) as String
  dim path as String = f.ShellPath
  return SystemCall ("textutil -convert txt -stdout "+path)
end function

Implementation of the API in the application

At this point, this article doesn't go much deeper for now.

I have implemented a demo application which implements what was outlined above. The app loads plugins, then takes a file and tries to operate each plugin on the file, until one succeeds.

Download the RBScript Plugin demo project here

Suspending a script waiting for input or events

A script may have to wait for an RB event to occur before it can continue.

For example, if a script wants to download a file from the internet, a called function in the Context class would start the download and then somehow have to wait until the "DownloadComplete" event or something like that is called. The script, however, wants to wait until this event has occured. The solution is to run the script in a thread, and then the downoad is started, the script is put to sleep using a Semaphore. Once the "completed" event gets called, the script is resumed with the result passed to it.

Download a demo project I made for REALWorld 2006.

Catching runtime errors (exceptions) in executed scripts

RBScript has a few long standing bugs that prevent us from catching exceptions in scripts as it was intended (and even documented).

Fact is that when an executed RBScript raises an exception, the RBScript.RuntimeError event that's meant for this is not called. Also, scripts do not even know the class RuntimeException. Lastly, if a Context method gets called by a script, and that method raises an exception, even different rules apply as to how they can be caught.

The overall solution is to both wrap the RBScript.Run method and the script code in a try...catch handler to catch exceptions in the script.

The RBScript Plugin demo project (see above) implements this.

You can also read my presentation I made for REALWorld 2006 on this.