unethical blogger - JavaScript

V8 and FastCGI, Exploring an Idea

R. Tyler Croy — Mon, 09 Mar 2009 09:49:58 +0000

Over the past couple years I've talked a lot of trash about JavaScript (really, a lot) but I've slowly started to come around to a more neutral stance, I actually hate browsers, I like JavaScript just fine by itself! While the prototype-based object system is a little weird at first coming from a more classical object-oriented background, the concept grows on you the more you use it.

Since I hate browsers so much (I really do), I was pleased as punch to hear that Google's V8 JavaScript Engine was embeddable. While WebKit's JavaScriptCore is quite a nice JavaScript engine, it doesn't lend itself to being embedded the same way that V8 does. The only immediate downside to V8 is that it's written entirely in C++, which does provide some hurdles to embedding (for example, I'm likely never going to be able to embed it into a Mono application), but for the majority of cases embedding the engine into a project shouldn't be all that difficult.

A few weekends ago I started exploring the possibility of running server-side JavaScript courtesy of V8, after reading about mod_v8 I felt more confident to try my project: FastJS.

In a nutshell, FastJS is a FastCGI server to process server-side JavaScript, this means FastJS can hook up to Lighttpd, Nginx, or even Apache via mod_fcgi. Currently FastJS is in a state of "extremely unstable and downright difficult", there's not a lot there as I'm exploring what should be provided by the FastJS server-side software, and what should be provided by JavaScript libraries. As it stands now, FastJS preloads the environment with jQuery 1.3.2 and a "fastjs" object which contains some important callbacks like:

fastjs.write() // write to the output stream
fastjs.log() // write to the FastCGI error.lgo
fastjs.source() // Include and execute other JavaScript files

On the server side, a typical request looks something like this (for now):

2009-03-09 05:04:06: (response.c.114) Response-Header:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-type: text/html
X-FastJS-Request: 1
X-FastJS-Process: 11515
X-FastJS-Engine: V8
Date: Mon, 09 Mar 2009 09:04:06 GMT
Server: lighttpd/1.4.18

Below is an example of the current test page "index.fjs":

var index = new Object();
 
index.header = function() {
	fastjs.write("<html><head><title>FastJS</title></head>");
	fastjs.write("<body>");
	fastjs.write("<center><h2>FastJS Test Page</h2></center><br/>");
};
 
index.footer = function() { 
	fastjs.write("</body></html>");
};
 
index.dump_attributes = function(title, obj) {
	fastjs.write("<br/><strong>");
	fastjs.write(title);
	fastjs.write("</strong><br/><hr/>");
 
	for (var k in obj) {
		fastjs.write(k + " = ");
 
		if (typeof(obj[k]) != "string")
			fastjs.write(typeof(obj[k]));
		else
			fastjs.write(obj[k]);
 
		fastjs.write("<br/>\n");
	}
};
 
(function() {
	index.header();
 
	fastjs.source("pages/test.fjs");
 
	index.dump_attributes("window", window);
	index.dump_attributes('location', location);
	index.dump_attributes("fastjs.env", fastjs.env);
	index.dump_attributes("fastjs.fcgi_env", fastjs.fcgi_env);
 
 
	index.footer();
 
	fastjs.log("This should go into the error.log");
})();

The code above generates a page that looks pretty basic, but informative nonetheless (click to enlarge):

Pretty fun in general to play with, I think I'm near on the point where I can stop writing more of my terrible C/C++ code and get back into the wonderful land of JavaScript. As it stands now, here's what still needs to be done:

Proper handling of erroring scripts via an informative 500 page that reports on the error
Templating? Lots of fastjs.write() calls are likely to drive you mad
Performance concerns? As of now, the whole stack (jQuery + .fjs) are evaluated every page request.
Tests! I should really get around to writing some level of integration tests to make sure that FastJS is returning expected results for particular chunks of .fjs scripts

The project is hosted on GitHub right now, here and is under a 2-clause BSD license.

Lazily loading attributes.

R. Tyler Croy — Mon, 15 Sep 2008 07:27:41 +0000

I found myself talking to Jason today about the virtues of getattr(), setattr(), and hasattr() in Python and "abusing" the dynamic nature of the language which reminded me of some lazy-loading code I wrote a while back. In February I found the need to have portions of the logic behind one of our web applications fetch data once per-request. The nature of the web applications we're building on top of the MySpace, Hi5 and Facebook platforms require some level of network data-access (traditionally via REST-like APIs). This breaks our data access model into the following tiers:

Working with network-centric data resources is difficult in any scenario (desktop, mobile, web) but the particularly difficult thing about network data access in the mod_python-driven request model is that it will be synchronous (mod_python doesn't support "asynchronous pages" like ASP.NET does). This means every REST call to Facebook, for example, is going to block execution of the request handler until the REST request to Facebook's API tier completes.

 def request_handler(self, *args, **kwargs):
    fb_uid = kwargs.get('fb_sig_user')
    print "Fetching the name for %s" % fb_uid
    print time.time()
    name = facebook.users.getInfo(uid=fb_uid) 
    ### WAIT-WAIT-WAIT-WAIT-WAIT
    print time.time()      
    ### Continue generating the page...

There is also a network hit (albeit minor) for accessing cached data or data stored in databases. The general idea is that we'll need to have some level of data resident in memory through-out a request that can differ widely from request-to-request.

Lazy loading in Python

To help avoid unnecessary database access or network access I wrote a bit of class-sugar to make this a bit easier and more fail-proof:

class LazyProgrammer(object):
    ''' 
        LazyProgrammer allows for lazily-loaded attributes on the subclasses 
        of this object. In order to enable lazily-loaded attributes define
        "_X_attr_init()" for the attribute "obj.X"
    '''
    def __getattr__(self, name):
        rc = object.__getattribute__(self, '_%s_attr_init')()
        setattr(self, name, rc) 
        return rc

This makes developing with network-centric web applications a bit easier, for example, if I have a "friends" lazily-loading attribute off the base "FacebookRequest" class, all developers writing code subclassing FacebookRequest can simply refer to self.friends and feel confident they aren't incurring unnecessary bandwidth hits, and the friends-list fetching code is located in once spot. If one-per-request starts to become too resource intensive as well, it'd be trivial to override the _friends_attr_init() method to hit a caching server instead of the REST servers first, without needing to change any code "downstream."

Lazy loading in C#

Since C# is not a dynamically-typed language like Python or JavaScript, you can't implement lazily-loaded attributes in the same fashion (calling something like setattr()) but you can "abuse" properties in a manner similar to the C# singleton pattern, to get the desired effect:

using System;
using System.Collections.Generic;
 
public class LazySharp
{
	#region "Lazy Members"
	private Dictionary<int, string> _names = null;
	#endregion
 
	#region "Lazy Properties"
	public Dictionary<int, string> Names
	{
		get {
			if (this._names == null)
				this._names = this.SomeExpensiveCall();
			return this._names;
		}
	}
	#endregion
}

Admittedly I don't find myself writing Facebook/MySpace/Hi5 applications these days on top of ASP.NET so I cannot say I actually use the class above in production, but conceptually it makes sense.

Lazy loading attributes I find useful in the more hodge-podge situations, where code and feature-sets have both grown organically over time, they're not for everybody but I figured I'd share anyways.

Resurgange of the shell.

R. Tyler Croy — Thu, 11 Sep 2008 07:57:36 +0000

Two things happened in such short proximity time-wise that I can't help but thing they're somehow related to the larger shift to interpreters. Earlier this week Miguel introduced csharp shell which forced me to dust off my shoddy Mono 1.9 build and rebuild Mono from Subversion just because this is too interesting to pass up on.

One of my favorite aspects of using IronPyhton, or Python for that matter is the interpreter which allows for prototyping that doesn't involve creating little test apps that I have to build to prove a point. For example, I can work through fetching a web page in the csharp shell really easily, instead of creating a silly little application, compiling, fixing errors, and recompiling:

tyler@pineapple:~/source/mono-project/mono> csharp
Mono C# Shell, type "help;" for help
 
Enter statements below.
csharp> using System;
csharp> Console.WriteLine("This changes everything.");
This changes everything.
csharp> String url = "http://tycho.usno.navy.mil/cgi-bin/timer.pl";
csharp> using System.Web;
csharp> using System.Net;
csharp> using System.IO;
csharp> using System.Text;
csharp> HttpWebRequest req = HttpWebRequest.Create(url);
<interactive>(1,17): error CS0266: Cannot implicitly convert type `System.Net.WebRequest' to `System.Net.HttpWebRequest'. An explicit conversion exists (are you missing a cast?)
csharp> HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;
csharp> HttpWebResponse response = req.GetResponse() as HttpWebResponse;
csharp> StreamReader reader = new StreamReader(req.GetResponseStream() as Stream, Encoding.UTF8);
<interactive>(1,45): error CS1061: Type `System.Net.HttpWebRequest' does not contain a definition for `GetResponseStream' and no extension method `GetResponseStream' of type `System.Net.HttpWebRequest' could be found (are you missing a using directive or an assembly reference?)
csharp> StreamReader reader = new StreamReader(response.GetResponseStream() as Stream, Encoding.UTF8);
csharp> String result = reader.ReadToEnd();
csharp> Console.WriteLine(result);
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final"//EN>
<html>
<body>
<TITLE>What time is it?</TITLE>
<H2> US Naval Observatory Master Clock Time</H2> <H3>
<BR>Sep. 11, 07:29:02 UTC
<BR>Sep. 11, 03:29:02 AM EDT
<BR>Sep. 11, 02:29:02 AM CDT
<BR>Sep. 11, 01:29:02 AM MDT
<BR>Sep. 11, 12:29:02 AM PDT
<BR>Sep. 10, 11:29:02 PM AKDT
<BR>Sep. 10, 09:29:02 PM HAST
</H3><P><A HREF="http://tycho.usno.navy.mil">Time Service Department, US Naval Observatory</A>
 
</body></html>
csharp> reader.Close();
csharp> response.Close();
csharp>

I really think Miguel and Co. have adding something infinitely more useful in this Hackweek project than anything I've seen come out of recent hackweeks at Novell. The only feature request that I'd add along to the csharp shell would be "recording", i.e.:

tyler@pineapple:~/source/mono-project/mono> csharp
Mono C# Shell, type "help;" for help
 
Enter statements below.
csharp> Shell.record("public void Main(string[] args)");
recording...
csharp> using System;
csharp> Console.WriteLien("I prototyped this in csharp shell!");
<interactive>(1,10): error CS0117: `System.Console' does not contain a definition for `WriteLien'
/home/tyler/basket/lib/mono/2.0/mscorlib.dll (Location of the symbol related to previous error)
csharp> Console.WriteLine("I prototyped this in csharp shell!");
csharp> Shell.save_record("Hello.cs");
recording saved to "Hello.cs"

Which could conceptually generate the following file:

using System;
 
public class Hello
{
   public void Main(string[] args)
   {
     Console.WriteLine("I prototyped this in csharp shell!");
   }
}

JavaScript Shell

In addition to the C# shell, I've been playing with v8, the JavaScript engine that powers Google Chrome. The V8 engine is capable of being embedded easily, or running standalone, one of the examples they ship with is a JavaScript shell. I've created a little wrapper script to give me the ability to load jQuery into the V8 shell to prototype jQuery code without requiring a browser to be up and running:

tyler@pineapple:~/source/v8> ./shell
V8 version 0.3.0
> load("window-compat.js");
> load("jquery.js");
> $ = window.$
function (selector,context){return new jQuery.fn.init(selector,context);}
> x = [1, 5, 6, 12, 42];
1,5,6,12,42
> $.each(x, function(index) { print("x[" + index + "] = " + this); });
x[0] = 1
x[1] = 5
x[2] = 6
x[3] = 12
x[4] = 42
1,5,6,12,42
>

The contents of "window-compat.js" being:

/*
 * Providing stub "window" objects for jQuery
 */
 
if (typeof(window) == 'undefined') {
        window = new Object();
        document = window;
        self = window;
 
        navigator = new Object();
        navigator.userAgent = navigator.userAgent || 'Chrome v8 Shell';
 
        location = new Object();
        location.href = 'file:///dev/null';
        location.protocol = 'file:';
        location.host = ''; 
};

In general I don't really have anything insightful or especially interesting to add, but I wanted to put out my "+1" in support of both of these projects. Making any language or API more easily accessible through these shells/interpreters can really help developers double-check syntax, expected API behavior etc. Thanks Novell/Google, interpreters rock!

Data Binding with jQuery

R. Tyler Croy — Wed, 02 Apr 2008 14:52:38 +0000

Since I've come from the land of desktop application development, there are a few concepts that I don't think quite "made the voyage" from desktop/thick-client development to web/thin-client development. The concept of "data binding" is completely lost in my opinion in the land of Javascript and HTML (not to mention the concept of "controls" to begin with).A few weeks ago while exploring a couple other concepts for how to improve our overall frontend development at Slide I prototyped a means of "databinding" controls, or at the very least DOM elements to data-providing Javascript functions.

I've posted an example here of some of the data binding code I've written for experimentation purposes. In the example page linked, there is a <ul> tag that is "bound" to a Javascript function, the Javascript function creates an array of associative arrays inline (it could very well be powered by some AJAX-oriented Javascript with minor adjustments). Using the results of the "databind" function specified on the bindable element, it creates a set of child nodes to attach to the parent list. In effect, the following code:

<script type="text/javascript">
    var bindSimpleList = function() {
        var simple_list = new Array();
        simple_list.push({'contents' : 'List Item #1 OMG', 'onclick' : function() { alert("You clicked the first list item! lulz"); }});
        simple_list.push({'contents' : 'List Item #2', 'style' : 'border-bottom: 1px dashed red; width: 100px;'});
        simple_list.push({'contents' : '<strong>List Item #3</strong>'});
        return simple_list;
    }
</script>
<ul class="bindable" databind="bindSimpleList" databindto="li"</ul>
</ul>

Will generate the following DOM tree, after the "bind()" function has been run on page load:

<ul class="bindable" databind="bindSimpleList" databindto="li">
    <li onclick="alert('You clicked the first list item! lulz');">List Item #1 OMG</li>
    <li style="border-bottom: 1px dashed red; width: 100px;">List Item #2</li>
    <li><strong>List Item #3</strong></li>
</ul>

Since the code is relatively simple (in my opinion) I figured I would throw it out there in all it's minimalistic glory and get some general feedback on the concept before I go "all out" and create a full-on jQuery extension based off of the linked prototype above. I'm trying to think of ways to make it more powerful as well, built-in support for binding to the results of an asynchrounous call to a URL that returns JSON that would then create the elements is at the top of my TODO list at this point. Feedback, flames and actual useful critiques are all welcome; I'll be sure to post again once I have the time to create the jQuery extension for binding, this however is more experimental quality (i.e. don't use it, i'm not).

What do you think?

Did you know! Slide is hiring! Looking for talented engineers to write some good Python and/or JavaScript, feel free to contact me at tyler[at]slide

Srsly ur doin it wrong.

R. Tyler Croy — Mon, 24 Mar 2008 11:18:53 +0000

Via twitter I have been griping a bit about Javascript recently. It's quite possible that I've been complaining about it far more than I complain about other things via twitter, which is a tall order to match.

When addressing something as big and scary as say, a platform built on Javascript, it forces you into looking at Javascript in a way different than how I think most developers (myself included) have looked at Javascript. Most Javascript that I've seen has been hideous. Gobs and gobs of functions and procedural garbage thrown into a series of files that kinda makes sense, but really doesn't. It would seem that most developers charged with writing Javascript don't understand how to write object-oriented Javascript. In fact about two or three months ago when considering topics to discuss in a front-end developers meeting here at Slide, I bit the bullet, raised my hand and said "Can you explain how to do object-oriented Javascript? Because I honestly don't have a fucking clue."

In the past Javascript that I've written has been to compliment existing backend web-application code and front-end code, i.e. I wasn't looking at Javascript as one of the building blocks of my application, I was looking at it as a bit of mortar spread between the cracks to smooth out the surface of the application. The difference in how you start to use Javascript in a web application makes an enormous difference 6 months to a year down the road. How terrible your code (this isn't actually segregated to Javascript) is becomes far more apparent when other developers start to work with your code as well, it's tremendously embarrassing to have to answer questions like "where's the code that generates that one DOM element?" As a general rule, coding all by your lonesome, especially with a tight schedule, will produce less than clean results (unfortunately Javascript is one of the languages I've found where this is more of the norm than the exception).

A lot of what's driven the change from my Javascript being the mortar to being the bricks in my work has been the adoption of jQuery which I highly recommend along with the jQuery.ui library. jQuery makes developing Javascript feel like actual programming, instead of hackish-scripting, which means you'll start to view your Javascript code differently too. Dealing with scoping issues, and prototype-based programming in Javascript isn't all rainbows and butterflies but "doing it right" will help you sleep at night and help reduce the amount of embarrassing questions you'll have to answer to the next poor unfortunate soul that inherits your code.

Some of the resources I've found useful in getting over the barrier to object-oriented Javascript have been:

This Introduction to Object-Oriented Javascript on the Mozilla Developer Center
Almost anything John Resig writes, especially gems like his recent post on Simple Javascript Inheritance
The code included in some mainstream javascript libraries

I'm still not a huge fan of Javascript, but I'm hating it less these days :)

Javascript Wonks: "missing : after property id"

R. Tyler Croy — Fri, 21 Mar 2008 09:33:06 +0000

I've been doing work with OpenSocial recently and have used the opportunity to bring my ~~tolerance~~ talent in Javascript up a notch or two. In doing so, I've been slowly but surely running into a myriad of browser-specific quirks along with a few cross-browser gems that have left me thinking about putting some browser developers on my "To Anonymously Beat Up In Alleyway" list (so far, James Gosling, and this man top the list).

After working on a few "classes" tonight (the notion that Javascript is object-oriented still makes me chuckle) I ran into an interesting problem with some of my global-level "constants" defined in the same file that I was working in, that my "class" just so happened to make use of. As I tend to do when I fall into situations like this to where I can't tell if I'm hallucinating or if something with Javascript has gone awry, I called over Sergio (in-house CSS master and Javascript Lvl. 60 Mage).

Some background to how Javascript works
Javascript engines essentially have two "modes" that it runs over your code that you can spot errors in. The first mode, "parsing", is where you'll find syntax errors spewing into the Javascript console. If you've used any interpreted language before (Python, Java, C#, Ruby), this is really just "compilation". Using Python as an example, when you import a module (i.e. import some_module) the Python interpreter actually compiles your code into Python byte-code to be executed at a later date. The second mode, "execution", is where you'll run into your run-time errors, using an accessing an undefined object property, overrunning an array index, etc. In Python/Java terms, this is where your compiled byte-code is actually being run in the Python/Java virtual machine.

The gripe
The crux of the problem comes down to two different ways to declare an associative array in Javascript, the following two notations are both correct and both "work":
Notation #1

var mapped_values = {};
mapped_values['key'] = 'value';

Notation #2

var mapped_values = {'key' : 'value'};

Everything looks correct yes? (hint: say yes)

Incorrect, because of the point at which the two are evaluated. The first example will be notated at run-time, whereas the second example will be evaluated at parse/compile-time. Who cares right? The distinction becomes much more apparent when you start to use references to other code for your keys. Keep in mind, with both notations it is actually valid to have a key "undefined" when you declare your associative array. For example:

Notation #1

/* the variable "foo" is not defined */
var mapped_values = {};
mapped_values[foo] = 'value';

Notation #2

/* the variable "foo" is not defined */
var mapped_values = { foo : 'value' };

This still works perfectly fine, both at parse/compile-time and at run-time. Since I mentioned I'm working on OpenSocial, chances are I'm going to need to reference some of the OpenSocial code. So for the next example let's say I need to create an associative array with one of the keys defined by the OpenSocial container, using the two different notations I would write something like:
Notation #1

var mapped_values = {};
mapped_values[opensocial.DataRequest.PeopleRequestFields.FILTER] = opensocial.DataRequest.FilterType.ALL;

Notation #2

var mapped_values = {opensocial.DataRequest.PeopleRequestFields.FILTER : opensocial.DataRequest.FilterType.ALL};

Because of the different points in time at which the two notations above will be evaluated, #1 will properly "compile" and then execute correctly when called (regardless of the scope-level at which it is defined). The second one however, will fail to "compile" when the browser's rendering engine is loading the Javascript (also regardless of the scope-level at which it is defined), and will result in the following error at load-time:

"missing : after property id"
(verified in both IE6 and Firefox)

The issue here is that the variable "opensocial" is not defined at "compile-time" as far as the Javascript engine is concerned (which is fine) but the code attempts to access a property of that object, which causes the error at "compile-time". This will error at at *any* level (as far as I've tried) so it's not a scoping issue, just an unfortunate fact of life with how Javascript is loaded and eventually executed in the browser.

Sergio and I tried a few more examples (the scope of which was in side a function, not at the global level):

// Works in IE/FF
var test = { magic : 'thing' };
var test2 = {}; 
test2[rick.roll] = 'thing';
test2[alert] = 'somethingelse';
var test3 = {}; 
test3[function() { alert('sux'); }] = 'test';

// Fail in IE/FF
var test4  ={ opensocial['DataRequest']['PeopleRequestFields']['FILTER'] : opensocial['DataRequest']['FilterType']['ALL']} ;
var test5 = { rick['roll'] : 'thing'};
var test6 = { rick.roll : 'thing'};
var test7 = { function() { alert('sux'); } : 'test'};

Object accessor calls work in "test2" for example because that's going to be evaluated at run-time instead of at compile-time, as is happening in "test5" and "test6". I would love to be proven wrong on our analysis of the issue here (our tests were less than scientific, and there may have been Corona involved) but switching from inline-declarations for associative arrays (var t = {'k' : 'v'};) to the more sequential alternative (var t = {}; t['k'] = 'v';) solved the issue of the Javascript engine's parser spewing errors on the loading of the Javascript.

Food for thought I suppose.