by Mike Bennett
What to know before debating type systems is a really good article discussing static and dynamic type systems and type inference. I found it via Dry Sellers CodeBetter.com blog post Sunday Thought: Embracing the Type System.
Related to that, if you’d like to learn more about type inference algorithms, have a read of Code Commit’s blog post What is Hindley-Milner? (and why is it cool?). The post gives an easy to understand overview of the Hindley-Milner type inference algorithm.
From a practical software development perspective, I see that according to Slashdot the speed of Javascript in Firefox has been improved by 20% to 30%. Firefox uses the SpiderMonkey Javascript Engine. Apparently the speed improvement is due to enabling type inference in the Javascript engine.
I wonder will SpiderMonkey’s speed improvements have a significant impact on Thwonk’s speed, esp. when I finish off the Thwonk code for caching and reusing JIT compiled Javascript?
by Mike Bennett
On and off over the years I’ve developed various software projects for fun. One of those projects is called Thwonk.
Today, I’m releasing the C, SQL and Javascript code for the backend of Thwonk and placing it under an MIT License (means you can take the code and reuse it in both open and closed software projects).
Thwonk is a bit like node.js, except it’s for messaging and online communication systems. The purpose of Thwonk is to enable developers to easily and quickly create and prototype new kinds of Internet messaging and communication systems.
A key aspect of Thwonk is that you program the messaging system rules and constraints in Javascript. I’ve taken the Spidermonkey Javascript engine, extended it, and built it into some custom written C daemons that handle and hide the complexity of messaging systems.
For example, with Thwonk you can easily develop a mailing list that trims all mailing list posts to 140 characters (a Twitter-like mailing list!). In Thwonk, this Twitter-like mailing list works as follows:
1. A message (in this case an email) arrives at a Thwonk email address, such as play@thwonk.com
2. Thwonk daemons take the message, look at the destination address, and based on the address find the Javascript associated with that destination address
4. Thwonk daemons then run the Javascript associated with the address (in a sandboxed environment)
3. The Javascript does whatever you’ve programmed it to do with the incoming message, e.g. trim the message to 140 characters
4. Your Javascript code then sends outgoing messages, e.g. send trimmed message to all play@thwonk.com subscribers
You can easily imagine other kinds of messaging rules, e.g. a mailing list where everyone on the list can only post once a day, or where all mailing list subscribers HAVE to post once a day (otherwise automatically unsubscribe them). As Thwonk matures I’ll extend it so you’ll be able to play with the rules of other online communication systems, such as web forums.
With this first version of Thwonk you can create new kinds of mailing lists in Javascript. You don’t have to worry about handling multiple messages arriving at once, nor worry about message queues, managing message delivery protocols, handling high and low loads, resource management (Javascript in an infinite loop is automatically stopped), etc. As Thwonk develops I’ll add support for other messaging protocols and communication frameworks, such as Web Services, XMPP, Twitter and Facebook.
The source code for Thwonk is up on Github:
www.github.com/mikemb/thwonk (see backend dir)
This alpha version 2 code compiles to a set of daemons that run on Unix servers, talk to SQL databases, and run Javascript (in a sandboxed environment on a backend server).
Thwonk provides multiple message queues, both incoming and outgoing – so you can easily have many types of parallel messaging systems running on a single Thwonk server. There’s also a virtual file system (VFS) for storing and publishing files, esp. handy for building messaging archives and publishing them on web. One great benefit of the VFS is you can serialize and deserialize Javascript objects to it, which makes storing state information easy, e.g. store a list of the mailing list subscribers in a Javascript Object or as a JSON file. There’s also security models, user rights and synchronization management (you don’t need to worry about handling multiple messages arriving at once) – but I won’t get into that today.
There is a fronted for Thwonk, which I’ve written in Django and Python. I want to tidy the frontend up a bit before publishing the code. With the frontend, anyone can create their own Thwonks and write the Javascript for the Thwonk rules in a simple online web IDE. In the next few weeks I’ll setup a public Thwonk server you can play with it.
This version of Thwonk compiles against the 1.6 branch of Spidermonkey on Linux. I’ll be shortly updating the code to support the recent 1.8 branch of Spidermonkey and recent versions of the mailutils dev libraries. There are automated build scripts for the backend, i.e. configure; make. Thwonk currently depends on spider monkey 1.6, libmailutils 1.2 and mysql dev libraries.
A few immediate todos are:
1. Setup Thwonk on a server for other’s to play with, i.e. enable people to create a Thwonk and program the Javascript for it.
2. Add in documentation explaining how to get Thwonk up and running on your own server.
3. Simplify the setup and configuration (there’s a bunch of hardcoded configuration options that absolutely should be stored in the database and external configuration file – naughty me!).
4. Provide examples of writing messaging systems in Thwonk Javascript.
5. Publish architecture documents, i.e. how do the various components and dameons in Thwonk fit together and communicate.
6. Add unit testing.
7. Add support for web services, i.e. both accept and publish message via a web API
8. Refactor Virtual File System
Over the next few weeks I’ll go into more details of Thwonk and provide examples of how to program with it. If you’re wondering how fast Thwonk is, as in how many messages it can handle per second – the non-specific answer is lots and its got a distributed architecture so it should scale across multiple cores and computers. I’ll re-run my evaluation scripts and graph the results of the number of messages it can handle per second. IIRC from the last time I tested Thwonk it was able to process messages much faster than the mail server could accept and deliver them.
Thwonk was inspired by a previous project called BumpList, which I co-created with Jonah Brucker-Cohen. Thwonk is our new project – you’re welcome to get involved!
