elarson’s posterous

It has been really interesting to think about how jquery ui fits within the design of our applications at work and the concept of a "widget". On the one hand, a widget is very much a javascript/jquery plugin in that it effectively must do all its communication over with ajax and is in fact loaded via ajax. The converse is that the HTML generated often comes directly from the server. There is always a struggle in my mind where this division really should be and while "it depends" definitely holds true, the scenario that helps define best practices seems to allude finding a set of general rule of thumb ideals.

When I worked at Novell in college it was when they were venturing into the webmail space with Hula. The idea for Hula was a webmail app that rivaled desktop applications while still providing an open platform for both email and calendaring. I got to see demos of the UI and was always impressed with what they were doing. When I looked at the source there was a huge gap when trying to find templates that defined the look and feel. In talking to the developers, the entire UI basically was constructed via javascript and only passed data back and forth to the server. At the time it seemed like a perfect idea.

Well, time seems to be the best tester of software as Hula was eventually dropped. I remember seeing cool demos here and there, but it was obvious it was a huge burden building the UI in this way. The process was slow and difficult to debug. Like any major javascript application, testing across supported browsers is most definitely an issue. In the end, it seemed that while the design seemed solid, the actual practicalities of building a UI completely in javascript was too difficult to overcome. I'm not saying that this is what sunk the project, but it was clear it was an issue.

What is interesting then is that jQuery UI uses a similar pattern of forcing the widget to write the HTML. This really isn't a problem in this respect because the widgets are limited in scope and must be decoupled from any specific application. In my use case at work, a widget also encompasses a server component, which makes using jQuery UI widgets more difficult. There are things the server markup must provide that the widget knows about and our requirement for decoupling is much less.

The goal then is to find a way to create a widget that has a specific server component while still writing generic widgets that can be decoupled in terms of each other, while still keeping the ease of integration required. The design I'm coming up with then is effectively a hybrid approach that aims to allow the server side templates define the majority of the widget and rely on specific cues to communicate the coupling between the javascript and the widget markup.

A good example would be a list of inputs. If you select one input, the values of the others must be cleared. So, the markup for the inputs should all be given semantic class names to help make it clear what the set of inputs are. This is usually unique according to the question's unique ID. Then any sort of grouping element would also be used to delineate the border or limits within the DOM the widget should not effect. Finally, the markup should utilize an internal data structure for keeping track of state that is verified through DOM elements. In other words, a two pronged attack at making sure the widget does the right thing when different events occur.

My goal is to keep the template changes to a minimum past changing the semantic classes and rely on the javascript be flexible to handle small changes in markup as needed. The way this all will need to come together is through configuration. The widget initializer will need to include the border or root element from which the widget works as well as the selectors for any nodes that need to be used as reference points. In terms of actual code then, the basic design will be a widget that does the generic functionality and then a set of initializers that will handle adding the context specific aspects.

Hopefully this small change will help make the difference for implementing widgets in a way that provides the flexibility we're beginning to need. Also, my hope is that in the future we can write the actual widget javascript to be more modular from the beginning instead of having to special case things so much afterwards. While the code often seems pretty simple, it becomes clear there is too much coupling between the server templates and the client side javascript. The result of this coupling is that small DOM changes wreak havoc on the functionality, when arguably, it should be relatively benign changing the look and feel or style. Likewise, no one wants to be tied down dealing with massive abstractions and libraries, so simplicity is a must.

The design is still very much in the air, but the ideas seem to be settling. I don't imagine that when it is all said and done things will be radically different. I also think that limiting the changes is probably a good thing and will help provide a healthy constraint for simplicity.

In Selenium tests tend to get brittle very quickly. This is a function of testing something like the DOM that is effectively like machine code for the browser. What happens is changing the DOM to support some new feature written in Javascript almost always radically changes the tests. These changes typically have two aspects.

1. Changing the locators needed to find different elements
2. Changing the verify/assert statements to test a new attribute, style, value or element

Changing locators is almost a given and while you can minimize the changes by selecting as close to the necessary element as possible and through using unique identifiers (either IDs or classes have worked for me), it is always something of a struggle. Likewise, changing the tests to effectively reflect what changed can be difficult as well, since the change might be as simple as adding or removing a new class.

All that in mind, I always wished there was a simple way to test whether an element as a class. For example, if you had a div that needed an image as the background, testing for something like a 'button-bg' class or something similar seems really helpful. It seemed simple enough to implement, so I went about creating a 'assertElementHasClass' function.

First off, Selenium supports adding new functionality via user-extensions.js. This is just a file that will be included and in turn made available in your tests. My first cut at getting the required functionality was to have a helper function and eval the result. This was easy enough, but somewhat undesireable since it effectively masked what was being tested. That said it was a really easy way to get started.

The "correct" way to add your own functionality is to add a prototype to the Selenium object. There are a few guidelines listed in an example user-extensions.js, so I'd suggest taking a close look at that. It can also be somewhat helpful to check out the selenium-api.js which is where all the typical selenese commands are.

In the end here is what I came up with:

Selenium.prototype.assertElementHasClass = function (locator, cls) {
    var element = this.page().findElement(locator);
    var cls_name = element.className;
    var clses = cls_name.split(' ');
    var actual = false;
    for (var i=0; i < clses.length; i++) {
        if (clses[i] == cls) {
            actual = true;
        }
    }
    var expected = true;
    return actual == expected;
};

One thing to note is that the assert functions want you to return either true or false. Other examples I looked at used an Assert.match function for comparisons. I'm not really sure what the difference is at this point, but as I couldnt' get the Assert.match to work, it seems safe to avoid it.

Hope this helps someone!

I finally realized why having a database for client side Javascript is such a great idea. It took a while for me see a use case that really exposed why it is such a helpful concept. I always thought to myself that if you have an application pushing enough data to the client where you need to do SQL queries, then something just isn't designed correctly. I'm all for pushing work to the client when it helps speed up the response, but pushing large batches of data to the client can only hurt. It has never made sense then why toolkits like Gears provide a database feature.

Like so much in software, there is always a consideration of state. What does the world look like at a particular point in time. One of the more difficult aspects of Javascript is that the state can be difficult to determine because there are times where taking a look at some value might be misleading. A great example is if you check the status of an input. That input might change by some other event. The easy answer is to avoid randomly attaching events. The problem is things can get complicated when events start bubbling and they can be fired multiple times. I'm sure smarter folks than me have developed excellent skills working with the Javascript event models (and I hope they all contribute to jQuery), but for me, it is still very much a challenge to keep straight why I might see an event a few times when it seems like once would be enough (with bubbling considered).

Having a client side database takes much of this confusion out of the picture because you have a place to keep your current state that can be locked. Sessions are a great example where multiple requests can be considered in their own scope even though different threads may handle them. One common means for storing sessions is via a database. The general idea is that you have a single canonical resource for making a decision. Even though the browser might be in the middle of making changes, you have a means of saving and retriveing critical details about the state of the applicaiton without trusting the asynchronous nature of Javascript events.

I might still be misunderstanding the thinking behind a Javascript database, but assuming it gives me a place outside the constant changing of the browser DOM for keeping important state information, then I think it makes a lot more sense.

Since SXSW, I've been in something of a catch up mode trying to finish up a major upgrade to the latest jQuery at work and it has been pretty time consuming. I work on a pretty Javascript heavy application at work that has gone somewhat stale. They say if it's not broke, don't fix it and we did just that. The impetus then to go ahead and take the plunge to upgrade jQuery was a bug in IE 8 that had to do with the scrolling plugin we were using. A few tests were put together and we found that the latest jQuery would do the trick, so that started us down the path. We found out later the bug was actually an IE 8 release candidate bug that had been fixed in the released version. It was somewhat disheartening to do so much work on a bug that wasn't actually a bug. Since we have a shiny new jQuery, I'm not all that upset.

In our case the 80/20 rule was very much true. The first step was to change the script tag to the latest version, run the tests and see what breaks. After changing some of the API calls, the vast majority of things worked. I'd say around 80% in fact! The things that didn't work were a bit more troubling. One of the major changes was that we moved from using the old interface library to using the new jQuery-UI. This seemed to be a good idea as there were some helpful updates to the widget APIs as well as an entirely new widget and theme pattern that could be helpful. Unfortunately, what caused the most trouble was upgrading our code that used the old widgets.

One interesting aspect the jQuery team added was the inclusion of themes for UI widgets. This required making a difficult decision regarding how a UI widget is implemented in the DOM. The basic options are to 1) ask the user to declare the different components on page or 2) as the user to declare a container where the widget will fill in the required DOM elements. The first option is a tedious one since you will need to code for many different cases and define not only the required elements, but rigidity of the markup. The second option is also difficult because you have the obligation to expose the HTML the widget will write in order to allow setting the state of the widget (ie when a user goes back). The jQuery team chose the second option and it seems to be the correct decision. While it definitely caused me quite a bit of grief understanding how the widgets changed, in the end, the pattern seems like the best trade off.

Another aspect of the upgrade that I thought was interesting was the disparate documentation out there for making a jQuery plugin. I have been writing most of my javascript code as jQuery plugins in order to keep everything within their framework. My reasoning is that I can push the cross browser issues to the library and hopefully save myself some time. The problem is that there are quite a few plugin tutorials that all offer slightly slightly different views of how to properly write a plugin. After reading some mailing list posts and seeing the new widget API in jQuery-UI, it is clear that this problem is being addressed, albeit with caution. The unanswered questions regarding how to write a plugin stems from the flexibility found in Javascript. You can write code in such a wide variety of styles that choosing one or even recommending one is difficult. While I was not able to utilise it, the jQuery-UI widget API is a huge step towards providing a plugin pattern that I hope will eventually be extended to general jQuery plugins.

This upgrade has been pretty tough and I'm sure part of that was my own fault. My tendency was to try not to change the way the widget worked. My logic was that the algorithm had worked previously, so changing things radically would only present the opportunity for new, unknown bugs. While I still think this is true, one side effect of reimplementing things is that you have the opportunity to see what the underlying library is doing with less distractions as well as understanding potential pitfalls the previous author might have already solved. It is also relatively painless to make a go at reimplementing things and drop it if it becomes obvious you are on the wrong path. As I move forward I think my goal will be to make creating one off revisions of things as quickly and easily as possible. I'm not sure how best to do that but I have feeling if I streamline my experimentation workflow, the result will not only be more ideas getting fleshed out, but a better understanding of bugs that I'm fixing.

First off, I'm a fan of jQuery. It is a great library. My problem is I want to use more of it! I want a Rails/Django/Pylons/Your Framework Here for writing Javascript apps. The problem is that while jQuery provides a very powerful plugin architecture, it doesn't offer a set of conventions and helpers to make writing a Javascript app with jQuery consistent. I find that every time I step up to the Javascript plate with jQuery, my path is usually sporadic and rarely do I cover the same ground in term of application design, while many of the same requirements exist. With that in mind, what I'd like to have is a jQuery framework that does the following.

1. Presents a clear and obvious place for configuration. Many of the tutorials suggest a pattern of providing default options for a plugin, but the pattern is rarely the same.
   
2. Provide a simple way of defining different kinds of functions. I'll go into more later.
   
3. Theme and/or template support. This is already starting to be solved via jQuery-UI, but it is somewhat limited to widgets. It also infers specific details about how an item will be laid out. What I'm looking for is a generic pattern for taking a selector and adding some visible feature. This includes things like DOM nodes and styles/classes, but is not meant to provide a look and feel as much as a way of consistently mapping between a set of configured classes/selectors/ids and program logic.
   
4. Dispatching to an obvious set of exposed functions for a plugin interface. The pattern that seems prevalent is the $(selector).plugin($action, options) pattern. I'm cool with that. The question though is where do you go from there? I've created basic dispatchers, but the pattern is limited based on how you define that dispatcher and the functions accepting the result. Personally, I'd like to see a WSGI interface here, but that honestly seems doubtful as one of the killer features of jQuery is the chaining, so no big deal if it limited to cater to chainability.
   

Regarding the different function types, it is unclear the best way to organize a plugin. Looking at the jQuery UI code, there are some obvious patterns, but they are difficult to follow within the scope of creating your own plugin. There seems to be a lot of ramp up code involved that hides the basic patterns. As I mentioned above, it would be nice to have a set of function type helpers to reduce the boilerplate as well as simplify things. One good example is that most jQuery plugins work with a selector and without a selector. It would be nice to have an easy way of defining a functions that have either functionality or both. Likewise, having an obvious place for a simple library function seems helpful as well.

One thing I'm not looking for is another MVC pattern. I don't think it makes much sense as the DOM and jQuery's data features provide more than enough model enough, in addition to effectively defining the view in one way or another. What I'm thinking about be more like a stack of functions and hooks that you plug into. These would effectively just be namespaces that have assumptions. Something like:

$.plugin = {

    dispatch : 'action',

    actions : { 'show' : function () { ...}, 'hide' : function () { ... } },

    libs : { ... }

    config : {...}

    ui : { ... }


}

Now, I have no idea how that "ui" thing would work or what the "action" dispatcher is, but hopefully it is somewhat clear what the idea is. While jQuery rules, it is not obvious how to use all the amazing bits. There is still a lot of voodoo that happens in the Javascript world and having a Javascript specific set of constraints to keep things in order might be very helpful in pushing the limits that much farther. I'm not talking about things like Cappuccino that provide an entire GUI framework either. This is just a set of design patterns and helpers to make the basics a little more obvious so those folks who don't program in Javascript every day (ie most web developers) have a better chance of doing something really interesting. If constraints set you free, then that means Javascript, with its crazy features and style, is a prison.

I'll probably take a stab at the above idea, but if you're intersted in sending along code, I'd love to see it. Likewise, if you do the work, I'm happy to sit by and cheer you on. Is this a decent idea?

Kevnin Dangoor mentioned Jack, a Rack implementation in Javascript. While I'm a fan of Rack, I don't get Jack. Why you ask? Well, I'll tell you!

The reason Rack works as it does is because Ruby doesn't support functions as first class citizens. Yes, I realize that many Rubyist fancy their language as an entry to functional programming (I'm assuming b/c of blocks), but Ruby is far from functional. Python, on the other hand, while also far from functional, does allow passing functions as arguments. The WSGI specification, where Rack draws its inspiration, in fact works via passing callables (a duck type for functions) along with an environment. This is very similar to XSLT in how the "." represents the current context, the environment represents the current state of the request. Secondly, WSGI allows two places to adjust the request response. Again, this is very functional. You either adjust the input (the environment) or the output. Basically, through passing functions, you nest them together using the WSGI spec and when the last one calls the start_response function, the output bubbles back through the function hierarchy.

Based on how WSGI works then, it makes no sense whatsoever to implement a WSGI copy in a similar fashion to Rack. Javascript already supports the necessary tools to make a rather pure WSGI implementation (lets call it JSGI) using the same patterns. One distinction the Jack docs make is that Jack is meant for server side Javascript. Again, I don't think this is a problem whatsoever.

I don't want to suggest that Jack is a bad idea, but it seems like basing it on Rack lacks foresight into the actual use of the design. WSGI has proven itself in a very powerful way. There are a multitude of applications, libraries and middleware that all make it clear WSGI is successful technology. Rack on the other hand does not have the same track record. Including Rack in Rails 3 doesn't really count either as Rails 3 is not released and there are no examples of sites or services using it. It also seems like a shame that the functional features of Javascript are not being utilized. The sucess of jQuery, I believe, is largely a function of its functional approach and respect for the language. It seems that a similar functional approach should be taken implementing something like WSGI in Javascript.