A feature I’ve been meaning to add to Open Server Platform for a while is a web management system, where an administrator can login and manage the cluster and the servlets running on it. I’d like there to be a user friendly interface for administrators to start, stop and migrate servlets across the different nodes of the system and a way to upload a servlet file and have it compiled and distributed across the cluster.

The simplest way to start serving web content with Erlang is to use the inets server and the httpd service. This is a HTTP/1.1 server built into the Erlang distribution that supports some more advanced features, most interesting of all, the ability to use Erlang to dynamically generate content. It is however very poorly documented, and there are a few very annoying things I came across that I’m posting to hopefully help anyone else trying to get it working.

1. The order of modules in the {modules, []} directive matters, if you want to have mod_dir work, it needs to be specified *AFTER* the mod_alias.
2. The logging is rather horrid, the transfer.log will not log anything except for HTTP 200 for every request, even if it failed.
3. You must specify {bind_address, any} in the configuration to use the httpd:reload_config function, otherwise it will return {error, not_started}
4. If you just want to server static content, you will need at a minimum the following modules: mod_get, mod_head, mod_log, mod_actions and mod_range. However, adding mod_alias is recommended along with the {directory_index, ["index.html"]} directive to stop it from failing (HTTP 500) on a directory request.
5. To use dynamic content, create a module that exports callbacks of the form: function(SessionID, _Env, _Input). To write Str back to the client, use the mod_esi:deliver(SessionID, Str) function.

I hope that this helps out!

Peace and chow,

Ranok

Today I finally got around to going through and testing the RC for version 0.3 of Open Server Platform. Everything seemed to work as planned, except for a few known issues that will be fixed in the next release, which should be coming down the pipe soon.

Improvements:

• Added replication nodes to increase fail-safe reliability
• Added commands to the administration console to stop and live migrate applications
• Can use configuration files to assist in the start up of OSP
• Many bug fixes and documentation updates
• The HTTPd servlet example now supports large files and the HEAD command

So, please check it out and let me know what you think!

Peace and chow,

Ranok

After taking off the afternoon due to a migraine, and napping for a few hours until it passed, I was looking over some old projects, and decided to get motivated to work on my various Erlang hackery projects. I checked out a fresh copy of Open Server Platform on my new computer, and decided to hack on it some more. There are a number of loose ends I’d like to tie up before the 0.3 release, but, before I could get down to coding, SVN get my so frustrated that I could no longer deal with it and switched to git, creating a new repository on Github. I will keep the Google Code repository the ‘defacto’ repository, where the safer code gets committed, but use Github for the more bleeding edge development due to it’s simplified branching and merging (among other things).

In the new git setup, I have two branches (aside from master, which follows the SVN repository): otp and no-otp. The no-otp version is currently the stable code that runs just fine, but doesn’t take advantage of Erlang’s OTP framework. The otp branch is the more cutting edge OTP aware version, which I hope to fully migrate to soon. With full support for OTP, I should be able to use an already existing distribution platform, and more battle tested redundancy.

This evening, I fixed a long standing bug in the example HTTPd where the server would crash when opening large files due to a shortcut I took with first implementing the server. Originally, the server would read the entire file into a string and then send that to the client, as you can probably see, there is a problem when the server tries to read in a 3.9 GB file (my test file). Now, my servlet takes a much more sane approach, read in the file 1 kilobyte at time, sending that to the client before reading more. This new approach works perfectly, though I had to add some messiness for handling CGI/PHP files and different MIME types. I also added support for the HTTP HEAD command.

Peace and chow,

Ranok

Seeing as I’m going to have no homework for the next semester (!!) I’m hoping to get some more projects underway in my down time. I’m going to use this post as a dump of my current interests and as a road-map for where I want to go in the next few months. I’m sure this will change as time goes on, but I need to start somewhere!

Projects in progress:

• FANG – I’d like to polish the multi-processing system, add namespaces, and tie it into the Erlang stdlib. Also add macros (real macros!) to FANG.
• OSP – I’m going to be using OSP as a basis for LadieBug (mentioned below). This will hopefully give me perspective into what I should work on in OSP to make it more friendly for development.

New projects:

• I’d like to take advantage of LaunchPad’s ‘Personal Package Archive’ system to host a few of my own Ubuntu packages, namely a more up-to-date version of Erlang’s OTP system.
• LadieBug – To take advantage of OSP’s distributed data store, I’d like to implement my own caching, recursive DNS server that would be both DNSSEC and DNSCurve compliant and would share the cache over the entire cluster, making it more scalable and reduce the number of needless queries.

That’s about all I can think of at the moment, if anyone would like to help me on any of these, feel free to comment below and we can get in touch.

Peace and chow,

Ranok

In spirit of last night’s COSI meeting, I figured I should get ahead of the game and post a little bit about what I’ve been up to with OSP. There were a number of shortcomings with the 0.2 release in terms of permanent storage and replication. Since r65 (version 0.2) there have been a number of changes, mostly some old ones centered on ERLFS and the most recent ones, which tweak the mnesia settings. Below are some important revisions and their impact:

• r88 – Now will autosense is a node has a disk, and will determine whether or not to keep a copy of the applications table on the disk or just in RAM
• r89 – Made gen-join.pl which asks the user a bunch of questions to make a join script for easier setup
• r93 – When you first install OSP, you should run ./setup-osp.sh to initialize the database on the first/master node
• r96 – Added the add-backup-server, which will make a disk copy of all the tables to that backup server incase of node failure, there’s a backup copy

While thse changes are small, they are still very needed in creating an actual application. Currently, OSP is at r97, only 3 away from the big 100. I thought that we should have a mini OSP party when we reach r100, but since OSP is distributed, so should the party. So when you get word of the 100th commit, please raise a glass or eat a slice of cake.