Friday, 2 October 2009


I have a new favorite web framework, among the many web frameworks that I've never had an opportunity to use in anger. web2py is a full-stack framework written in Python. It seems to be pretty comprehensive and easy to use.

I've started to look at it because I've written some Spotfire extensions that use R for computation and some people have expressed interest in a server version.

General criteria are:
  • Linux and Windows
  • Extensible by modifying text files
  • Accessible and understandable by a typical R user
  • Self-contained and easy to install
I first took a look at using either Rapache or biocep, which are my current favorite R server solutions. Admittedly I haven't used either of them either.

Since Rapache doesn't have a Windows version, it needs to be run within a VM. I really like the idea of using standard Apache server configuration procedures along with either standard R scripts or "brew" templates. But I lost patience with working in a Linux terminal within a VMPlayer window.

Biocep seems to be pretty rich and I've heard good things about it. But it's lacking in documentation. It also is focused on managing R engines rather than on a whole web stack. There's nothing wrong with that, it just isn't what I was looking for.

So after revisiting the R server options, I decided to look at other languages in which server development is more front-and-center. Since Linux is of interest, I set aside ASP.NET. As usual, Java offers a plethora of options but there's bunches of XML configuration involved in most of them and the learning curve is steep.

Turning back to the Python offerings, the web2py framework stands out as being self-contained, full-featured, and very approachable. As it's written and configured in Python, it also seems to be approachable to R programmers.

In my particular use case I need to invoke R batch jobs and do some file management operations from Spotfire which is a .NET application. I like keeping simple things simple, so I favor using XML + HTTP for this. Perhaps it's too naive, but this keeps the client and server very much technology neutral. Authentication, encryption, etc. is then handled at the HTTP transport level via tried-and-true techniques.

web2py fits really well with this use case. You can define an XML-RPC service by simply adding an annotation to a Python method. The whole controller class for my "RunR()" service is below:

from import Service
service = Service(globals())

def call():
return service()

def index(): return dict()

def RunR(scriptText, dirKey):
return "Script executed in directory " + dirKey

Of course, the RunR() method needs to do more than just return a string. My thinking is that it'll invoke "R CMD BATCH" using the Python "subprocess" package. This is essentially what I'm doing from Spotfire locally using the "Process" class in C#.

It turns out that it's easy to implement an XML-RPC client in C#. The XML-RPC.NET assembly from Cook Computing makes it a simple matter of declaring an interface:

using CookComputing.XmlRpc;

namespace CalculationExample
public interface IRManager : IXmlRpcProxy
string RunR(string scriptText,
string sessionDirKey);

You then use a factory method to construct an object of this type and work with it as a standard C# object.

This is basically the point I've reached. On the client side, there's a bit of C# work that's pretty easy to have the code use the local or remote methods based on an option setting. On the server side, there's a bit of Python work with "subprocess" and file management methods. As I've done lots of C# and little to no Python the first task is easy and the second more challenging.


I've recently noticed that the old Insightful web site has been taken down. Browsing to now redirects to the Spotfire web site.

The good news is the Spotfire web site now lists all of the currently available S+ products. The bad news is all the other information that was up there is hard or impossible to find.

Happily, at least some of the content is available via the "Internet Archive Wayback Machine" at*/