Justin Rudd’s Drivel

I’ve been reading a few Erlang weblogs lately.  One that I’ve been reading the most is Yariv Sadan.  He has a post titled Ericsson’s Biggest PR Blunder: Forgetting to Tell Us that Erlang programming is fun.  He gives some information about Yaws (a dynamic content webserver written in Erlang), ejabberd (a Jabber server written in Erlang), and the required Ericsson projects (phone switches using Erlang, etc.).  It is a well written post.  Most of Yariv’s post are informational and easy to read (except his Smerl posts.  But that is because I don’t grasp the Erlang syntax a 100% yet).

One thing I’ve noticed is that Erlang users tend to push you towards the Mnesia for your storage needs.  But there seem to be some issues with its handling of data.  Especially large amounts of data.  But I can’t seem to find any information about what qualifies as a large amount of data.

But Mnesia has some very cool features.  I especially like the location transparency of data.  I can start off with the data on a local node, and then at a later time move that data onto a remote node.  Or possibly replicate the data to another node for scale out scenarios.   But I can do all of these without the client code caring.

I was reading in The Erlang Fix by Joel Reymont about his experience Writing low-pain massively scalable multiplayer servers.  Joel’s article is the best I’ve seen when it comes to writing backend code (Yariv tends to focus on web development with Erlang).  He describes how Mnesia was used for sharing configuration data between nodes, how it has multiple masters and slaves.  He describes the architecture for getting users connected to game servers, the login protocol, etc.  I can see how a lot of what he did could be applied to projects that I work on.

For example, we have upstream systems that send us data.  Right now that happens with a queuing system.  But I can see how each upstream system could first go through a Gateway to be directed to a real instance.  Then the data that comes in from upstream is owned by that node, but is replicated to other nodes to allow for fault tolerance.  As with my Scala experiments, I can’t seem to grasp how I’m going to process the incoming data.  This is what I’m struggling with.  Too many years in an OO and procedural world I guess

A simple example, when an order gets assigned to a warehouse, that order comes into my team’s software.  If the order doesn’t exist, then we create it and decide when it is going to ship out.  If the order does exist, then we figure out what the differences are (added an item, removed an item, etc.) and apply them.  So I can see doing a Mnesia lookup for the order.  Taking what comes back and doing pattern matching on it to do the next step instead of an if statement like in procedural world.  It makes sense to me.  But applying it is the hard part.  Not to mention getting all the data out of Oracle into Mnesia :)  I don’t even want to think what that would take.

It is an interesting experiment.  I’ve heard rumors that groups within Amazon are looking at Erlang as well.  The idea of services that just don’t go down is appealing to all groups within Amazon.  That would just allow us to worry about things that we don’t have control over like the network and true high severity situations instead of a lot of the fire drills that we do.

But in a lot of the comments I’ve read, Erlang is definitely one of those “bus” languages.  What they mean by that is, you need to ensure that whoever is writing it has a backup for the case where he/she is hit by a bus.

You know what?  I hate this logic.  We’ve got C++ code that is 6+ years old.  I know C++ pretty good.  And I struggle to understand what the hell that code is doing.  Would the code in Erlang be harder to pick up?  Probably.  But like Smalltalk, Erlang seems to only have a handful of constructs.  Once you understand them, the rest is figuring out the logic, which is hard no matter what the language is.