Archive for the ‘Software Development’ Category.

Facebook scales!

I’m really impressed with Facebook. It’s fun (totally addictive in fact), and I think it’s actually a better way to keep in touch with friends than email.

But what I’m really impressed with is how well they’re handling a massive growth in traffic. As of Feb 23 they’ve gone from zero to almost 18 million users in just over three years, an increase of 240% since last July alone.

From the Facebook blog five weeks ago:

…almost ten million different users sign into the site every day, or more than half the user base. During our biggest peaks - Sunday & Monday night around 10EST - more than one million people will be simultaneously logged into the site.

Friendster is famous for missing a huge opportunity in part because of poor performance (40 second page load times), and MySpace’s well-known and frequent reliability problems have continually frustrated their users.

According to Alexa (as of today, April 11) Facebook’s traffic is only slightly higher than Friendster’s (who have since redesigned their architecture), and still far behind Myspace’s. (Alexa’s stats may not be that reliable however.)

Regardless of the actual numbers, they’re obviously doing the right things to keep on top of the massive growth so far. Unfortunately though, I haven’t been able to find out many technical details except that they have two terabytes of RAM used by memcached.

UPDATE April 16: Apparently their traffic increased by another 50% during the month of March! (Link via Matthew Burpee on Facebook).

Announcing GigPark

I’m excited to announce that I’ve joined a new venture, I’ll be leading the development for GigPark!

GigPark is a new service that will help you recommend your favorite service providers to friends and family, instantly get service recommendations from people you trust, and build your service business through word-of-mouth. Excited yet?! Me too!!

GigPark Logo

Please check out the GigPark blog for more information.

Process XML in Java more easily by… not using Java?

Java and XML are like peanut butter and chocolate, right? They just go together, don’t they? Jon Bosak of Sun summed it up most famously as “XML gives Java something to do“.

XML gives Java developers something to do
I think I see Sun’s master plan: the more work Java developers must do to solve a given problem, the more of them we’ll need to have. The more Java developers we have, the more popular Java appears. (The only problem left to solve is how to profit from that, but I digress). The key to this plan’s success is lack of syntactic support for XML, and the added bonus is that even more developers will occupy their time building endless numbers of libraries! “Why make trillions when we can make… BILLIONS?!” :-)

There are some decent libraries for handling XML in Java, but most are quite heavyweight, requiring you to create Java classes (in the case of the XML binding variety) or at least a config file (XML-based, naturally!).

There is another way
I’ve written before about using other languages with Java, and Groovy has some really nice features for handling XML. Groovy compiles to Java bytecode so you can write an XML handling class in Groovy and use it within your Java app. Here’s an example that uses Groovy’s GPath to read in an XML file from Sun’s XML DOM tutorial, and prints some of the elements. Just for fun, we’re loading the XML file directly by http:

def slideShow = new XmlSlurper().parse("http://java.sun.com/webservices/jaxp/dist/1.1/docs/tutorial/dom/samples/slideSample01.xml")
println("slideshow author attribute: ${slideShow['@author']}")

def slides = slideShow.slide
for (slide in slides) {
    println("slide title element: " + slide.title)
    for (item in slide.item) {
        println("  item text: ${item}")
    }
}

Here’s the Java version as a comparison. Honestly, nobody really uses the DOM API because it’s so painful (hence the existence of so many libraries to do the job), but this is the out-of-the-box Java way:

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
URL url = new URL("http://java.sun.com/webservices/jaxp/dist/1.1/docs/tutorial/dom/samples/slideSample01.xml");
InputStream stream = url.openStream();
DocumentBuilder builder = factory.newDocumentBuilder();
Document document = builder.parse(stream);

Element rootElement = document.getDocumentElement();
String author = rootElement.getAttribute("author");
System.out.println("slideshow author attribute: " + author);

NodeList slides = rootElement.getChildNodes();

for (int i = 0; i < slides.getLength(); i++) {
    if (slides.item(i).getNodeType() == Node.ELEMENT_NODE) {
        Node slide = slides.item(i);
        NodeList slideChildren = slide.getChildNodes();
        for (int j = 0; j < slideChildren.getLength(); j++) {
            Node child = slideChildren.item(j);
            if (child.getNodeType() == Node.ELEMENT_NODE) {
                if (child.getNodeName().equals("title")) {
                    System.out.println("slide title element: "
                            + child.getTextContent());
                } else if (child.getNodeName().equals("item")) {
                    System.out.println("  item text: "
                            + child.getTextContent());
                }
            }
        }
    }
}

I have omitted error handling.

Using XPath in Java can make our job easier when it comes to searching a tree for a node (you wouldn’t want to see this done with pure DOM manipulation!), but we’ll still still have to deal with an org.w3c.dom.Node object that is the result:

XPath xpath = XPathFactory.newInstance().newXPath();
String expression = "/slideshow/slide[title = 'Overview']";
Node overview = (Node) xpath.evaluate(expression, document, XPathConstants.NODE);

Here’s the same thing done in Groovy, a simple one-liner:

def overview = slideShow.slide.find { it.title == "Overview" }

Two interesting things here are the use of a closure, and the use of == which might look wrong to a sharp-eyed Java programmer. Groovy uses == for equality and the method is() to test for identity. This is the opposite of Java but leads to more readable code because typically we compare for equality much more often than identity. And == works with null in Groovy, so your typical Java comparison (x != null && x.equals(y)) becomes x == y in Groovy, which looks more like what we mean.

Creating XML is also easy in Groovy using MarkupBuilder, here is the code to create our sample XML file:

def doc = new StreamingMarkupBuilder().bind {
    slideshow(title:"Sample Slide Show", date:"Date of publication", author:"Yours Truly") {
        slide(type:"all") {
            title("Wake up to WonderWidgets!")
        }
        slide(type:"all") {
            title("Overview")
            item("item") {
                mkp.yield("Why ")
                em("WonderWidgets")
                mkp.yield(" are great")
            }
            item()
            item("item") {
                mkp.yield("Who ")
                em("buys")
                mkp.yield(" WonderWidgets")
            }
        }

        mkp.yield("normal text here")
    }
}

System.out << doc

It’s easy to read because it has a visually similar structure to the XML that it’s creating, and the syntax is quite terse. Groovy has similar builders for other things besides XML, for example to create SWT and Swing UI’s. You can even create your own. More on how this is implemented coming soon…

There are other choices besides Groovy, of course, I haven’t tried it yet but Scala looks like it might be a good choice. (For that matter there are other choices besides XML!)

JavaScript - finally, the name makes sense

JavaScript began life in 1996 as an embedded scripting language in the Netscape browser. It was originally named LiveScript but was renamed at the last minute, presumably to ride the coat-tails of Java which was growing fast as a client-side programming platform.

Since then many dialects have been implemented on many different platforms, including the ECMA-262 standard, but historically it’s success has mainly been on the client side, as a scripting language for UI elements. There were some early attempts at using it on the server, including Netscape’s LiveWire scripting and Microsoft’s (non-default) option for server-side JScript in ASP pages, but I don’t believe either of those were widely used.

This seems to be changing lately though, and the release of Java 6 makes it official: it includes a scripting API, and though the API aims to stay relatively language neutral the default scripting engine is a slightly modified version of Rhino, the JavaScript engine.

So Sun is finally beginning to position Java the platform as being about the JVM rather than the language, the approach that Microsoft took with .NET from the start. Sun is simply trying to stay relevant amidst the growing shift to dynamic languages. Phobos is a JavaScript-based server that is part of Sun’s GlassFish project, and one of its stated goals is to “appeal to developers who, for a variety of reasons, are hard for Java EE to reach”. But it’s a welcome change for Java developers who now have more language options while still developing for the Java platform. “Don’t worry boss, it’s JEE”. ;-)

Java developers now have options for building DSL’s using metaprogramming, and for more efficient declaration of complex literals. Just imagine how much cleaner and shorter your jUnit tests would be if you set up your test fixtures like this:

var expected = {firstName:"John", lastname:"Smith", salary: 80000}

That’s JavaScript object literal syntax. Try it.

Of course there are a variety of “scripting” languages available that can run on the JVM, most have been around for a few years already. Jython (née JPython) has been around since 1997(!). Groovy (though suffering slightly from infighting and controversy) has been around since at least 2003. And Sun just hired the two core JRuby developers. (There are also pre-compiled languages like Scala).

But JavaScript seems to be the one growing most quickly in popularity, partly because it’s shipped by default with Java 6, and partly because there is appeal in using only one language; since we’re forced to use JavaScript on the client side we might as well use it on the server too. I think the latter is a pretty weak reason though, it’s good to have a working knowledge of several languages and to use the right tool for the job. (When all you have is a hammer…) Also, though it still lacks modularity, new features like array comprehensions and destructuring assignment are bringing JavaScript more in line with other scripting languages.

Or perhaps Java developers just like the name.

Building Scalable Web Sites Book

When Building Scalable Web Sites came out earlier this year (written by Cal Henderson of Flickr) I was immediately interested because I’m totally fascinated with the architecture of very large scale systems. But a quick browse through the table of contents changed my mind, it looked really basic and there were only two chapters dedicated to the “scalable” part of the subject. I was interested because the title was “Building Scalable Web Sites”, not just “Building Web Sites”.

But I happened to pick it up in a book store recently and actually flip through it a bit, and it looked interesting so I bought it. I’m glad I did, it was a surprisingly good read. It reminds me a bit of The Pragmatic Programmer: most of the material is fairly basic, but it’s worth reading even for experts because it’s well written, reasonably short, yet quite thorough.

It focuses mainly on PHP and MySQL (on Linux), but really this book is about general principles not implementation details, and those principles are equally applicable to other languages and platforms.

For a list of other technical books that I recommend, take a look at my reading page.