Advertisement

Flapjax: A Programming Language for Ajax Applications

M Nithin

This article presents Flapjax, a language designed for contemporaryWeb applications. These applications communicate with servers and have rich, interactive interfaces. Flapjax provides two key features that simplify writing these applications. First, it provides event streams, a uniform abstraction for communication within a program as well as with external Web services. Second, the language itself is reactive: it automatically tracks data dependencies and propagates updates along those dataflows. This allows developers to write reactive interfaces in a declarative and compositional style. Flapjax is built on top of JavaScript. It runs on unmodified browsers and readily interoperates with existing JavaScript code. It is usable as either a programming language (that is compiled to JavaScript) or as a JavaScript library, and is designed for both uses. This paper presents the language, its design decisions, and illustrative examples drawn from several working Flapjax applications.

Introduction

The advent of broadband has changed the structure of application software. Increasingly, desktop applications are migrating to the Web. Programs that once made brief forays to the network now “live” there. The network servers they communicate with not only provide data but also store data, enabling networked persistence. Some applications, known as mashups, combine data from multiple sources. Often, applications process not only static data but also continuous streams of information, such as RSS news feeds.

This article presents Flapjax, a programming language built with such applications in mind. We make three key arguments, which this article will substantiate:

Event-driven reactivity is a natural programming model for Web applications.
Consistency should be a linguistic primitive.
Uniformity is possible when treating both external events

(those from remote machines) and internal ones (those from local devices such as the mouse). Uniformity enables better abstractions and also reduces the number of concepts needed for reasoning and validation. Rather than invent a language from fresh cloth, we chose to engineer Flapjax atop HTML and JavaScript (despite their warts). JavaScript offers three important benefits. First, it is found in all modern browsers, and has hence become a lingua franca. Second, it reifies the entire content of the current Web page into a single data structure called the
Document Object Model (DOM), so that developers can naturally address and modify all aspects of the current page (including its visual style). Third, it provides a primitive, XMLHttpRequest, that permits asynchronous communication (in the style called Ajax) without reloading the current page. This enables background communication with servers so Web applications can provide much of the reactivity of desktop applications. As a result of building atop JavaScript and HTML, Flapjax applications do not require plugins or other browser modifications; they can reuse existing JavaScript libraries; and the language can build on the existing knowledge of Web developers. A formal presentation would hide the many pragmatic benefits and decisions in the design of Flapjax. We therefore present it through a series of increasingly sophisticated examples, including uses and mashups of popular Web services.

These demonstrate how event streams, and automatic reaction to their changes, encourage several important software design principles including model-view clarification and policy-mechanism separation. We outline the implementation technique and its pragmatic choices. Finally, we discuss the use of the language in actual applications.

Language or Library? We have repeatedly referred to Flapjax as a language, but with a little extra effort, Flapjax can be used as a JavaScript library. That means the developer who does not want to add the Flapjax-to-JavaScript compiler to their toolchain can include the Flapjax library and program purely in JavaScript itself; in fact, most Flapjax applications are actually written this way.

Flapjax by Example
We present Flapjax as a programming language through examples. They are necessarily short, but Flapjax is a living, breathing language! I invite the reader to view and run the demos on the language site, read the documentation of the many primitives, and try out their own examples.

The Structure of JavaScript Programs

Before we study Flapjax, let us consider a very simple JavaScript program. It displays the time elapsed since starting or clicking on a button (figure-1, which elides some of the HTML scaffolding). The point of this program is to fill in a value for the curTime element on the HTML page. Consider the reasoning a developer must employ:

1. The value is ostensibly displayed by the second line of the function doEverySecond.
2. The value displayed is that of elapsedTime.

var timerID = null;
var elapsedTime = 0;
function doEverySecond() {
elapsedTime += 1;
document.getElementById("curTime")
.innerHTML = elapsedTime; }
function startTimer() {
timerId = setInterval("doEverySecond()", 1000); }
function resetElapsed() {
elapsedTime = 0; }
<body onload="startTimer()">
<input id="reset" type="button" value="Reset"
onclick="resetElapsed()"/>
<div id="curTime"> </div>
</body>

Figure-1. Elapsed Time in JavaScript

3. elapsedTime is set in the previous line.
4. But this depends on the invocation of doEverySecond.
5. doEverySecond is passed inside a string parameter to setInterval inside startTimer.
6. startTimer is called by the onload handler. . . so it appears that’s where the value comes from.
7. Is that it? No, there’s also the initialization of the variable elapsedTime at the top.
8. Oh wait: elapsedTime is also set within resetElapsed.
9. Does resetElapsed ever execute? Yes, it is invoked in the onclick.

Just to understand this tiny program, the developer needs to reason about timers, initialization, overlap, interference, and the structure of callbacks. (We trust the reader spotted the semantic bug?.)

The culprit here is not JavaScript, but the use of callbacks and their effect on program structure. Callbacks are invoked by a generic event loop (e.g., in the JavaScript runtime) which has no knowledge of the application’s logic, so it would be meaningless for a callback to compute and
return a non-trivial value. The return type of a callback is therefore the equivalent of void. That immediately means developers can no longer use types to guide their reasoning.

Furthermore, a void-typed function or method must have side-effects to be useful, so even local reasoning about a program’s data depends on global reasoning about the program’s control flow, destroying encapsulation and abstraction. Indeed, Myers has forcefully made similar critiques of
callbacks.

The problem of comprehension affects not only humans but also tools. For example, static analysis and verification engines must decipher a program’s intent from a highly fragmented description, and must reconstruct its dataflow from a rat’s nest of fragments of control. As previous work on model checking Web applications has shown, a more direct program structure is a great help in this regard.

The asynchronous nature of Ajax applications further compounds these problems. Because the primary composition operator is a side-effect, interleavings and interactions become the developer’s responsibility. The network does not guarantee message ordering; deployment can further affect  ordering (e.g., higher server loads, or users at a greater geographic
distance from servers than developers). All the usual problems of concurrency manifest, without even the small comfort of locking.

Despite this, callbacks appear necessary to receive notification of events, and are charged with propagating them through the system to keep the data model up-to-date. In an Ajax application, there are numerous sources of updates a program must process, such as:

1. initial data from each host
2. user actions (e.g., button clicks, mouse movements)
3. updates from a data stream
4. changes to data made in another concurrent session
5. acknowledgments from servers (e.g., in response to a
store request)
6. changes to the access-control policy

As a result, Ajax applications are agglomerations of callbacks with largely implicit control flows. Some operations exacerbate this: for instance, XMLHttpRequest requires a callback (onreadystatechange) that it invokes up to four times, providing the status in a field (not as a parameter to the callback).

The Flapjax Alternative

Flapjax endows JavaScript with a reactive semantics. In effect, if a developer defines y = f(x) and the value of x changes, the value of y is recomputed automatically. Concretely, Flapjax is JavaScript augmented with two new kinds of data. A behavior is like a variable—it always has a value—except that changes to its value propagate automatically; an event stream is a potentially infinite stream of discrete events whose new events trigger additional computation. The propagation of updated behavior values and new events is the responsibility of the language. Figure-2 uses the same timer example to illustrate these concepts. The elapsed time (always has a value, but the value keeps changing) is best represented as a behavior, while clicks on the reset button (may be clicked an arbitrary number of times, but the developer cannot anticipate when it will be clicked next) is best represented as an event stream. timerB(1000) creates a behavior that updates every second (i.e., 1000 milliseconds). The valueNow method extracts a snapshot of the behavior’s value at the time it is invoked (i.e., it does not update automatically). $E defines an event stream,

var nowB = timerB(1000);
var startTm = nowB.valueNow();
var clickTmsB = $E("reset", "click").snapshotE(nowB)
.startsWith(startTm);
var elapsedB = nowB - clickTmsB;
insertValueB(elapsedB, "curTime", "innerHTML");
<body onload="loader()">
<input id="reset" type="button" value="Reset"/>
<div id="curTime"> </div>
</body>

Figure-2. Elapsed Time in Flapjax in this case one per click of the button named reset. The result is a stream of DOM event objects. The snapshot method transforms this into a stream of the value—at the time of clicking—of the timer. startsWith converts the event stream into a behavior, initialized with startTm. Finally, insertValueB inserts the value of its behavior into the DOM. (Appendix A recapitulates all Flapjax operations used in this paper.) Obviously, the Flapjax code may not appear any “easier” to a first-time reader. What is salient is both what is and isn’t present. What is present is the composition of expressions, even when they involve I/O. A button is no longer just an imperative object; rather, $E("reset", "click") lets us treat it as a value that can be composed with and transformed by surrounding expressions. What is absent is the callbacks: the developer simply expresses the dependencies between expressions, and leaves it to the language to schedule updates.

Thus, nowB updates every second, and therefore so does elapsedB (which depends on nowB) and so does the value on the page (because elapsedB is inserted into it).

It is instructive to return to the pure JavaScript version.

The bug is that the time displayed on-screen—which we might think of as having to always represent the value of elapsedTime (i.e., a model-view relationship)—is undefined between when the user clicks on the Reset button and when the timer next fires. This is because the program resets elapsedTime but not does propagate it to the screen. The error is subtle to detect during testing because it is a function of when in this interval the user clicks on the button. Put differently, the developer has incurred the burden of synchronizing the on-screen value with the model of time, and even in such a simple program, it is possible to slip up. In contrast, the Flapjax developer has left maintenance of consistency to the language. In a sense, behaviors should not surprise JavaScript developers. When a program makes a change to the JavaScript DOM, the update is automatically propagated to the screen without the need to notify the browser’s renderer. In other words, the DOM already acts rather like a behavior! Flapjax thus asks why only the bottommost layer should provide this feature, and instead exposes to developers the same functionality previously reserved for a special case.

From Buttons to the Network (and Back)

The example above shows that button clicks are event streams and clocks are behaviors. Flapjax makes it possible for developers to treat all the other components of Ajax programs in these terms. The mouse’s location is a behavior. A text box can be treated either as a behavior (its current content) or as an event stream (a stream of changes), depending on which is more convenient. A server is simply a function that consumes an event stream of requests and produces an event stream of responses. This view is slowly also being adopted by industry.

As an illustration, consider a component that appears in many Web applications: a text buffer with auto-saving. (Many readers will recognize this from the mail composition buffer of Google Mail.) This is a component we reuse in several of our own applications. It is therefore instructive to consider its design and implementation. We will build the auto-save buffer incrementally to show how a developer might approach such a problem. At its simplest, the developer wants to create a text box that saves every t seconds:

 

function mkSaveBox(t) {
var draftBox = // make a <textarea>
setInterval("...XMLHttpRequest...", t*1000);
return draftBox; }
In fact, however, the buffer should also save whenever the
user clicks the Save button. That means the function needs
two parameters:
function mkSaveBox(t, btn) {
var draftBox = // make a <textarea>
// save every t seconds or when btn clicks
return draftBox; }

Now it isn’t quite as clear what to do: using setInterval, the save callback will run every t seconds, but is that what we want? Should it auto-save every t seconds regardless of Save clicks, or only t seconds after the last Save? Irrespective, should it auto-save even if there are no changes? And then, if some user needs it to auto-save after every keystroke, this abstraction is useless.

The problem is that the abstraction confuses mechanism—setting up timers, dispatching messages, and so on with policy. Obtaining this separation is not straightforward in JavaScript and most libraries fail to demonstrate it.

In Flapjax, the event stream is an excellent representation of policy. Thus, we would write,

function mkSaveBox(whenE) {
var draftBox = // make a <textarea>
// save every time there is an event on whenE
return draftBox; }
where whenE represents the policy. Armed with this abstraction,
the concrete policy is entirely up to the developer’s
imagination; here are three simple examples:
mkSaveBox(timerE(60000))
mkSaveBox($E(btn, "click"))
mkSaveBox(mergeE(timerE(60000),
$E(btn, "click")))


Respectively, these save every minute, every time the user clicks a button, or when either of these occurs. Now we complete the definition of mkSaveBox. First, we create the <textarea>:

var draftBox = TEXTAREA(); TEXTAREA creates a new <textarea> and exposes it as
a behavior. Flapjax defines similar constructors for all the HTML elements.

We now define a simple function that identifies the Web service and marshals the text buffer’s contents:

function makeRequest(v) {
return {url: "/saveValue", fields: {value: v},
request: "post"};}

We use Flapjax’s $B function to obtain a behavior carrying the current value of draftBox. ($B may be applied to any input element.) When a save event fires on whenE, we snapshot the current value of draftBox and use makeRequest to wrap the draft into a request:

var requestsE = whenE.snapshotE($B(draftBox))
.mapE(makeRequest);

Given this event stream of requests we invoke the function getWebServiceObjectE, which consumes a stream of server requests and returns a stream of server responses:

var savedE = getWebServiceObjectE(requestsE);

getWebServiceObjectE encapsulates both the call to XMLHttpRequest and its callback. When the callback is invoked to indicate that the response is ready, Flapjax fires an event carrying the response.

Presentation The above provides a complete implementation of the auto-save functionality, using event streams to represent the policy. We can go further: in our applications, we have found it valuable for the abstraction to indicate when the buffer is out-of-sync with the server (i.e., between edits and responses). We use a Cascading Style Sheet (CSS) annotation to alter the presentation by changing the editor’s border:

var changedE = $B(draftBox).changes();
styleE = mergeE(changedE.constantE("unsaved"),
savedE.constantE("saved"));
styleB = styleE.startsWith("saved");
insertValueB(styleB, draftBox, "className");}

changes creates an event stream that fires each time the value of $B(draftBox) changes (i.e., on each keystroke). constantE transforms the keyboard events (which indicate the buffer has changed) and the network responses (which indicate that the server has saved) to the strings "unsaved" and "saved" respectively. The merged event stream, styleE, propagates events from both its arguments. We use startsWith (seen earlier in figure 2) to transform the discrete event stream into a continuous behavior, styleB. The value carried by styleB, is the value of the last event, which is the current state of the auto-save buffer.

We need to specify an initial value for styleB to hold before the first event fires. Initially, the (empty) buffer is effectively “saved”. This behavior is inserted into the DOM as the CSS className; CSS class entries for saved and unsaved will correspondingly alter the box’s appearance.

Higher-Order Event Streams: Drag-and-Drop

So far, we have seen event streams of keystrokes and network requests. Flapjax events may, however, represent arbitrary actions. In particular, they can represent events much more complex than those exposed by the DOM. To illustrate this, we build a drag-and-drop event abstraction.

For simplicity, consider dragging and dropping a box:

<div id="target"
style="position: absolute;
border: 1px solid black">
Drag this box
</div>

The DOM provides mouseup, mousedown and mousemove events for each element. A drag-and-drop operation begins with a mousedown, followed by a sequence of mousemoves,
and ends with mouseup.

We wish to define a function that, given an element, produces an event stream of drags and a drop:4

dragE :: element -> EventStream (drag or drop)

Both drag and drop events are a record of three fields. The fields left and top are the mouse coordinates. The third field, drag or drop, carries the element being manipulated.
We begin with mousemove, which naturally leads to the creation of drag events:

function dragE(elt) {

return $E(elt,"mousemove").mapE(
function(mm) {
return { drag: elt,
left: mm.clientX,
top: mm.clientY };});}

The function above is permanently stuck dragging. We should start responding to mousemove events only after we register a mousedown event:

return $E(elt,"mousedown").mapE(
function(md) {
return $E(elt,"mousemove").mapE(
function(mm) {
return { drag: elt,
left: mm.clientX,
top: mm.clientY }})});

Above, the mousemove event stream is created only after an enclosing mousedown event fires. In fact, each mousedown produces a new stream of mousemove events. This code appears to have a runtime type error: it produces an event stream of event streams. Such higher-order event streams are in fact perfectly legal and semantically sound. The problem is that dragE ultimately needs the coordinates of the latest inner event stream (the latest drag sequence).

To flatten higher-order streams, Flapjax offers the primitive:

switchE :: EventStream (EventStream a)
-> EventStream a
switchE fires events from the latest inner event stream.
With switchE, we can easily fix our type error:
var moveEE = $E(elt,"mousedown")
.mapE(function(md) { ... as before ...
return moveEE.switchE();

We have not accounted for drop events, which should “turn off” the stream of drag events. We thus map over the stream of mouseup events and return a singleton drop event:

var moveEE = ... as before ...
var dropEE = $E(elt,"mouseup")
.mapE(function(mu) {
return oneE({ drop: elt,
left: mu.clientX,
top: mu.clientY })});

We can combine these two event streams with mergeE, which fires events from either of its arguments:

return mergeE(moveEE,dropEE).switchE();

Because switchE fires events from the latest inner event stream, when the mouse button is pressed, moveEE produces an event stream of drags. This becomes the latest inner event stream, and switchE thus produces a stream of drags. When the mouse button is released, dropEE produces a new event stream (oneE({ drop ... }). When this new event stream arrives, switchE stops forwarding drag events from the previous event stream. It fires the single drop event and waits for more (in this case, we know there is just one drop event).

When the mouse button is pressed again, moveEE produces a new stream of drags that supersede the earlier stream from dropEE. This abstraction therefore lets us drag
the box repeatedly.

Using Drag-and-Drop The dragE function merely reports the position where the target is dragged and dropped. It does not, as one might expect, actually move the target. This omission is intentional. We can easily move the target when it is dragged:

var posE = dragE("target");
insertValueE(posE.mapE(function(p)
{return p.left}),
"target","style","left");
insertValueE(posE.mapE(function(p)
{return p.top}),
"target","style","top");

However, by separating the drag-and-drop event stream from the action of moving the element, we’ve enabled a variety of alternate actions. For example, the following action ignores the drag events and immediately moves the target when it is dropped:

insertValueE(
posE.filterE(function(p) {return p.drop;})
.mapE(function(p) { return p.left;}),
"target","style","left");

In the next example, the target moves continuously but lags behind the mouse by 1 second:.

insertValueE(
posE.delayE(1000)
.mapE(function(p) {return p.left;}),
"target","style","left");

Further possibilities include confining the drag area, aborting drag operations, etc. Our example has omitted a startdrag event, which opens up a range of new uses. The reader might wish to also compare our approach to that of Arrowlets.

Compositional Interfaces: Building Filters

We built drag-and-drop by combining event streams that were extracted from a single DOM element. Flapjax also allows behaviors to be built from multiple, independently updating sources. We will illustrate this with an example taken from Resume.. Resume presents reviewers with a list of a job candidate. A large list is unusable without support for filtering and sorting.

Figure-3 defines two possible filters for selecting candidates: by sex and by score.

function pickSex() {
var ui = SELECT(
OPTION({ value: "Female" }, "Female"),
OPTION({ value: "Male" }, "Male"));
return {
dom: ui,
pred: function(person) {
return person.sex == $B(ui);
}};}
function pickScore() {
var ui = INPUT({ type: "text", size: 5 });
return {
dom: ui,
pred: function(person) {
return person.score == $B(ui);
}};}

Figure 3. Filters for Single Criteria

function pickFilter(filters) {
var options = mapKeys(function(k, _) {
return OPTION({ value: k }, k);},
filters);
var sel = SELECT(options);
var subFilter = filters[$B(sel)]();
return {
dom: SPAN(sel, " is ", subFilter.dom),
pred: subFilter.pred };};

Figure- 4.
Selecting Filters

Each function returns both the interface for the filter and the predicate that defines the filter. The interface elements are built using Flapjax’s constructors, such as SELECT and INPUT, which construct behaviors (just like TEXTAREA). We can display these DOM behaviors with insertDomB:

var filterObj = pickScore();
insertDomB(filterObj.dom,"filterDiv");
We can use the predicate to filter an array of candidates:
var filteredCandidates =
filter(filterObj.pred,candidates);

Observe in pickScore that $B(ui) is a behavior dependent on the current score. The value of filteredCandidates thus updates automatically as the user changes their desired score. We can then map over the list of candidates, transforming each candidate object to a string. The resulting strings can be inserted into the DOM. Flapjax tracks these data dependencies and automatically keeps them consistent.

function modalFilter(subFilter) {
var button = A({ href: "" }, "Update");
var subPred = subFilter.pred;
return {
dom: DIV(subFilter.dom, button),
pred: $E(button, "click")
.snapshotE(subPred)
.startsWith(subPred.valueNow()) };};

Figure-5. Filters with an Update Button

Real systems may have many available filters. If the user wants only one at any given time, displaying them all would clutter the screen. We might instead indicate the available filters in a drop-down box, and show only the controls for the selected filter. Figure-4 implements a filter selector. It chooses between filters of the form in figure-3, where each filter is an object with dom and pred fields (of the appropriate type). The result of pickFilter is also an object of the same type. The function is parameterized over a dictionary of available

filters; for example:basicFilters = {
"Sex": pickSex,
"Score": pickScore };
var filterObj = pickFilter(basicFilters);

We build the drop-down box (sel) by mapping over the names of the filters (options); we elide mapKeys—it maps a function over the key-value pairs of an object. The pickFilter interface displays sel and the interface for the selected filter (subFilter.dom). The predicate for pickFilter is that of the selected filter.

When the user chooses a different filter, $B(sel) updates, and so does subFilter.dom. Since the DOM depends on subFilter.dom, Flapjax automatically removes the interface of the old filter and replaces it with that of the new one. The developer does not need to engineer the DOM update. The filtering predicate (subFilter.pred) updates similarly, changing any displayed results that depend on it.

These filters update the list immediately when the user makes a selection in the filtering GUI. An alternate, modal interface would not affect the list until an Update button is clicked. We can reuse our existing filtering abstractions for modal filters. To do so, we build the interface by composing filters, exactly as we did in figure 4. When we’re done, we apply modalFilter (figure 5) to add an Update button:

var filterObj =
modalFilter(pickFilter(basicFilters));
As the user makes selections in the filtering interface,
subPred continuously updates in modalFilter. However,
modalFilter’s predicate is not the current value of
// flickrSearchRequest :: String -> Request
// Packages the search text into a Flickr API call.
function flickrSearchRequest(req) { ... }
// flickrSearchResponse :: Response -> Listof(Url)
// Extracts URLs from a Flickr API response.
function flickrSearchResponse(resp) { ... }
// makeImg :: Url -> Element
function makeImg(url) {
return IMG({ src: url }); }
var queryE = $B("search").changes().calmE(1000);
var requestE = queryE.mapE(flickrSearchRequest);
var responseE = getForeignWebServiceObjectE(requestE)
.mapE(flickrSearchResponse);
var imgs = DIV(map(makeImg, responseE.startsWith([])));
insertDomB(imgs, "thumbs");

Figure-6. Flickr Thumbnail Viewer

subPred, but a snapshot of its value when Update was last clicked. As a result, we get a modal filtering interface. By preserving the interface to a filter at each level, we obtain a variety of substitutable components. For example, filterObj may be any one of:

filterObj = pickScore();
filterObj = pickSex();
filterObj = pickFilter(
{ "Sex": pickSex, "Score": pickScore });
filterObj =
modalFilter(pickFilter(
{ "Sex": pickSex, "Score": pickScore }));
Regardless of the definition of filterObj, the code to apply
and display filters does not need to change:
insertDomB(filterObj.dom, "filterDiv");
var filteredCandidates =
filter(filterObj.pred, candidates);

In Resume, we have even more general combinators such as the conjunction and disjunction of multiple filtering options. Though it is unrelated to the notion of filtering itself, this idea of bundling interface with behavior is strongly reminiscent of Formlets.

Pipes (and Tubes) forWeb Services

As Unix showed many decades ago, pipelines are good component connectors. The etForeignWebServiceObjectE primitive extends this to the Web. Suppose, for instance, the developer wants to erect a pipeline from a text box to a Web service to the screen. Assuming the HTML document contains an input box with id search and a presentation element with id thumbs, the program in figure-6 extracts each

function(p) { resultE.sendEvent(
{ data: d, point: p })}};
requestE.mapE(function(req) {
geocoder.getLatLng(req.loc,
callback(req.data));});
return resultE;};

Figure-7. Geocoder Service as Event Stream Transformer query typed into the search box, sends the query to the photo sharing site flickr.com, obtains a list of thumbnails, and displays them in the DOM. In the definition of queryE, calmE builds a “muted” event stream for a given time period. By calming an event stream associated with a buffer’s keystrokes for a second, the developer can keep the system from responding to every keystroke, waiting for a pause when the user is not typing. We use this method frequently to provide smoother user interfaces.

Mashups: ComposingWeb Services

If Web services expose themselves as consumers and producers of event streams, they naturally fit the Flapjax mold. However, many external Web services are not designed this way. Some such as Twitter (www.twitter.com, which lets users broadcast short messages called tweets) return responses containing JavaScript code that must be eval’d to obtain a local callback. Others, such as Google Maps (maps.google.com), supply extensive (callback-based) libraries.

However, with just a little effort, these callbackbased APIs can be turned into event stream transformers that fit naturally into Flapjax applications. We first show this adaptation, then use it to build a mashup of Twitter and Google Maps.

Google Geocoder The Google Geocoder is a part of the Google Maps API. It accepts the name of a location (e.g., “Providence, RI”) and, if it successfully interprets the name, returns its latitude and longitude. The lookup occurs asynchronously on Google’s servers, so it is unsurprising that the Geocoder function uses a callback:

getLatLng :: String * (Point -> void) -> void

whose use tends to follow this template:

var data = ...
var callback = function(p) { ... };
getLatLng(data.location,callback);

An application that uses getLatLng continuously needs to associate points returned in the callback with data about the corresponding request; we can encapsulate in a closure:

var callback = function(d) {
return function(p) { ... }};
getLatLng(data.location,callback(data));

We will package this pattern into an event stream transformer from strings to points, along with an arbitrary datum that is passed from each request to its associated result:

EventStream { data: a, loc: String }
-> EventStream { data: a, point: Point }

We can easily map over a stream of requests:

function makeGoogleGeocoderE(requestE) {
var callback = ...;
requestE.mapE(function(req) {
getLatLng(req.loc,callback(req.data)); });}

However, the result is not available within the body of function(req) { ... }, so the event stream above does not produce meaningful events. Since results arrive asynchronously, they are conceptually a new event stream. The operation receiverE creates a primitive event stream with
no sources, so it does not fire any events:

var resultE = receiverE();
var callback = ...;
requestsE.mapE(...);
return resultE;

However, we can imperatively push an event to it using sendEvent. Since points are sent to the callback, the body of our callback becomes:

resultE.sendEvent({ data: d, point: p });

The complete function is shown in figure-7. It exposes itself as a pure Flapjax event transformer, completely hiding the internal callback. Using this pattern, we can erect a similar reactive interface—which can then be treated compositionally—to any Web service with a callback-based.

Twitter/Google Maps Mashup Now that we’ve seen how callback-basedWeb services can be turned into event stream transformers, we can combine Web services into a mashup.

Consider a simple mashup that takes Twitter’s most recent public tweets and plots them on a GoogleMap. This mashup operates in three steps: (1) Fetch the live feed of public tweets from Twitter. Tweets are accompanied by the location(e.g., “Providence, RI”) of their sender. (2) Using the Google Maps Geocoder API, transform these locations into latitudes and longitudes. (3) If the Geocoder recognizes the location, plot the corresponding tweet on an embedded Google Map. Since we have a live feed, repeat forever.

Figure-7 shows the code for the Geocoder event stream.

We can similarly build a public tweet event stream:

var googleMap = new google.maps
.Map2(document.getElementById("map"));
googleMap.setCenter(
new google.maps.LatLng(0, 0), 2);
// Fetch the live feed of public tweets
var tweetE = getTwitterPublicTweetsE();
// Transform locations into coordinates
var pointsE = makeGoogleGeocoderE(
tweetE.mapE(function(tweet) {
return { data: tweet.text,
location: tweet.user.location };}));
// Elide points the Geocoder did not recognize
makeMapOverlayE(googleMap,
pointsE.filterE(function(x) {
return x.point != false; }));

Figure-8. Mashup of Twitter and Google Maps

getTwitterPublicTweetsE ::
-> EventStream Tweet
We are using Google Maps to plot points returned by the
Geocoder. It thus suffices to build an event steam consumer:
makeMapOverlayE ::
GoogleMap * EventStream { data: String,
point: Point }
-> void

In the interest of space, we elide the definitions of these functions. However, they are similar in length and in spirit to the Geocoder function.

Figure-8 shows the code for our mashup (figure 9 and figure 10 show two instances of its execution). Aside from the initialization of the embedded map, the mashup is almost
directly a transcription of strategy outlined in prose above. Furthermore, it employs reusable abstractions that other mashups can also share.

FromWeb Services to Persistent Objects

Web services are good abstractions for procedural and message-passing interfaces, such as for finding a list of movies playing on a particular evening. They do not, however, directly model shared, mutable, persistent objects, such as a meeting in a group calendar. The Flapjax system provide a custom persistent object store to save such objects,and a client-side library to interface with it. The server maintains notions of identity for users and applications (with standard authentication details). It presents each application with a filesystem-like tree, which developers
can also browse through a trusted Web interface. The operation writePersistentObject associates an event stream with a location—which is a path through the tree— while readPersistentObject reflects the values stored at that location into a stream. Thus, referring to the draft-saver from section 2.3:

writePersistentObject(draftBox.changes(),
["draft"]);

saves drafts at the top-level “file” "draft". In practice, an application will wrap writePersistentObject in an abstraction. This can be used to erect another policy-mechanism separation: writePersistentObject is a mechanism, but various filters can be applied to the event stream it consumes to perform, e.g., rate limiting. readPersistentObject reverses the direction of binding to reflect persistent objects in the application, with extra parameters for the initial value and polling rate.

Access Control In keeping with the filesystem analogy, every persistent object is subject to an access-control policy. Naturally, this set of permissions can also change at any time. User interface elements that depend on the permissions should also update their appearance or behavior. The Flapjax primitive readPermissionsB produces a behavior representing the current permissions of a location in the persistent store. The application can use its value to
drive the user interface. For instance, if permsB is bound to the permission of a store location,

INPUT({type: "text",
disabled: !(permsB.has("WRITE", true))});

function EventStream(sources,update) {
this.sources = sources;
this.sinks = [ ]; // filled in by sources
for (var i = 0; i < sources.length; i++) {
this.sources[i].sinks.push(this); }
this.update = update; }

Figure-11. The Event Stream Constructor declaratively ties the disabling of the input box with lack of write permission. Thus, the user interface will automatically update in step with changing permissions.

 Implementation
The theory of Flapjax is rooted in signal processing: events and behaviors are essentially signal-processing abstractions. The theoretical underpinnings of Flapjax can be found in Cooper’s dissertation. Here, we focus on the implementation strategy that enables the above programs to run.

The EvaluationModel
The central idea behind Flapjax is push-driven dataflow evaluation. Flapjax converts JavaScript programs into dataflow graphs. Dataflow graphs are mostly-acyclic directed graphs from sources (clocks, user inputs, the network, etc.) to sinks (screen, network, etc.). When an event occurs at a source, Flapjax “pushes” its value through the graph. A graph node represents a computation; when it receives an event, a, it applies a function f (representing the computation) to a and may further propagate f(a) to its children. This push-based, demand-driven evaluation strategy is a good match for systems with many kinds of external stimuli that do not obey a single central clocking strategy.

Dataflow Graph Construction

Though we have presented event streams and behaviors as distinct entities, the astute reader will have guessed that they are almost duals of each other. Given a behavior, issuing an event whenever its value changes yields a corresponding event stream. Given an event stream and an initial value, continuously yielding the most recent value on the stream (and the initial value before the first event appears) gives a corresponding behavior. In Flapjax, nodes in the dataflow graphs are event streams while behaviors are derived objects. We describe the construction of a dataflow graph of event streams below.

To developers, an event stream is an abstract data type that may only bemanipulated with event stream combinators (e.g., mapE, calmE, etc). Internally, an event stream node is implemented as an object with three fields:

sources :: listof(EventStream)
sinks :: listof(EventStream)
update :: a -> (b or StopValue)

// EventStream a * EventStream a -> EventStream a
function mergeE(src1,src2) {
var update = function(a) {
return a; }
return new EventStream([src1,src2],update); }
// (a -> b) * EventStream a -> EventStream b
function mapE(f,src) {
var update = f;
return new EventStream([src],update); }
// (a -> Bool) * EventStream a -> EventStream a
function filterE(pred,src) {
var update = function(a) {
if (pred(a)) {
return a; }
else {
return StopValue; }};
return new EventStream([src],update); }
// EventStream (EventStream a) -> EventStream a
function switchE(srcE) {
var outE = new EventStream([], function(a) {
return a; });
var prevE = null;
var inE = new EventStream([srcE], function(aE) {
if (prevE) {
outE.sources.remove(prevE);
prevE.sinks.remove(outE); }
prevE = aE;
outE.sources.push(aE);
aE.sinks.push(outE);
return StopValue; });
return outE; }

Figure-12. Implementation of Event Stream Combinators sources and sinks specify a node’s position in the graph. When a value (of type a) is pushed to a node, Flapjax applies the node’s update function to the value. If update returns the StopValue sentinel, the value does not propagate further from the node. Otherwise the result, b, is propagated further by Flapjax’s evaluator. Consider mergeE, which builds a node that propagates all values from both its sources without transforming them:

merged = mergeE(src1,src2)

mergeE must build a new node by specifying the sources, sinks, and updater. The sources are the event streams src1 and src2. The update function propagates all values: function update(a) { return a; } Since Flapjax is push-driven, the node bound to merged must know the sinks to which it pushes values. However, function computes a new value n; if n is different from the current value it sets n as the current value and propagates to all sinks, otherwise it returns StopValue to prevent further propagation.

The Compiler

Flapjax can be viewed as a language or, with a little extra work, a library. Now we can explain precisely what this extra work is. The compiler consumes files containing HTML, JavaScript and Flapjax. Flapjax code is identified by the <script type="text/flapjax"> directive. The compiler transforms Flapjax code into JavaScript and produces standardWeb pages containing just HTML and JavaScript. It includes the Flapjax library and elaborates Flapjax source code to call library functions as necessary. Flapjax source code has JavaScript’s syntax, but the compiler’s elaboration gives it a reactive semantics. Furthermore, the compiler allows Flapjax and JavaScript code to seamlessly interoperate. This strategy of transparent reactivity has great advantage for beginning users and in teaching contexts. We outline its main components below. Implicit Lifting The principal task of the compiler is to automatically lift functions and operators to work over behaviors. The compiler does so by transforming function applications to invocations of liftB. This allows us to write expressions such as timerB(1000) + 1 where JavaScript’s + operator is applied to the timerB(1000) behavior. The compiler transforms the expression above to code equivalent to:

liftB(function(t) { return t + 1; },
timerB(1000))

The function liftB creates a node in the dataflow graph with timerB(1000) as the source and
function(t) { return t+1 } as the update function. Without the compiler, such transformations must be performed manually. In practice, this appears to be less onerous than it sounds, as our experience suggests. Without the compiler, the development cycle involves just editing code and refreshing the page in the browser. The compiler introduces another step in the development cycle which may sometimes be inconvenient.

JavaScript Interoperability
The compiler enables Flapjax code to interoperate with raw JavaScript. Flapjax already shares JavaScript’s namespace, so either language can readily use identifiers defined in the other. Consider Flapjax calling JavaScript functions. In the simplest case, if a JavaScript function is applied to behaviors, the compiler can lift the application. The JavaScript function is thus applied to the values carried by the behaviors, rather than the behaviors themselves (which it presumably would not comprehend). Whenever a behavior changes, the function is reapplied. For example, suppose filter is defined in JavaScript:

filter :: (a -> Bool) * listof(a) -> listof(a)
and consider the following Flapjax code:
var tock = filter(function(x) {
return (x % 2) == timerB(1000) % 2;
}, [0,1]);

filter expects a predicate and a list. The result of tock, however, is not a boolean, but a behavior carrying a boolean. The compiler thus wraps the predicate so that the current value of its result is extracted on application. Furthermore, when the result is invalidated (as it is every second), filter is reapplied. As a result, tock is a behavior that alternates between [0] and [1]. In the other direction—JavaScript calling Flapjax code— the compiler performs no transformations. There is no need to do so, since the natural way to invoke Flapjax from JavaScript is to treat it as a library with explicit calls to the event stream and behavior combinators.

Inline Flapjax The compiler provides one more convenience that we have not yet discussed in this article, called inline Flapjax. It recognizes the special matching tokens {! and !} 6 in any HTML context and treats the text contained within as Flapjax code. This code is expected to evaluate to a behavior. The compiler inserts the behavior into the document at that point without the need for additional code. For instance, given some function validCC that validates a credit card number and these JavaScript declarations—

function validColor(valid) {
return valid ? "aqua" : "cyan"; }
var ccNumField = $B("ccNum");
var ccNumValid = validCC(ccNumField);

—this inline Flapjax term this as a localized functional dependency rather than diffuse it into callbacks, which inverts the dependency structure.

Propagation
The recursive propagation model (which is implemented with trampolining, to prevent stack overflow) requires additional explanation, particularly to prevent some undesirable behavior. Consider the following expressions, where y is some numeric behavior:

var a = y + 0;
var b = y + a;
var c = b + 1;
var d = c % 2;

We would expect b to always be twice y, c to be odd, d to be 1, and so on. Unfortunately, nothing in our description above guarantees this. The update from y may recompute b before it recomputes a, which might trigger the subsequent recomputations, resulting in all the invariants above being invalidated. Of course, once the value of a updates, the invariants are restored. This temporary disruption is called a glitch in signal-processing lingo. (There is another, subtle problem above: some nodes may be computed more than once. Not only is this wasteful, this computation may be noticed in case the updater functions have side-effects.)

Fortunately, there is a simple solution: to use topological order. This prevents nodes from being evaluated before they should, and avoids repeated computation. To perform this, the node data structure tracks its partial-order rank in the dataflow graph. The Flapjax evaluator calls nodes’ update functions (figure-12) in topological order. Instead of propagating values immediately, the evaluator inserts them into a priority queue in topological order. The only obstacle to topological ordering is the presence of cycles in the graph. Cyclic dependencies without delays would, however, be ill-defined. Flapjax therefore expects that every cycle is broken by at least one delay (either the primitive delayE or a higher-level procedure, such as integration, that employs delays). This restores the delay-free sub-graph to a partial order.

 Primitive Event Streams and Callbacks

Flapjax programs do not directly use callbacks, so the Flapjax library encapsulates callback management code. The simplest function that encapsulates a callback is $E, which encapsulates a DOM callback and exposes it as an event stream of DOM events. For example:

var moveE = $E(document.body, "mousemove");
As we showed for the Geocoder (figure 7), we can use
sendEvent to encapsulate the callback-based DOM API:
function $E(elt, evt) {
var stream = receiverE();
var callback = function(e) {
stream.sendEvent(e); };
elt.addEventListener(evt, callback);
return stream; }

However, $E above never removes the callback. (It does not call the DOM function removeEventListener.) If the event stream moveE becomes unreachable from the rest of the program, we may expect it to be garbage collected. However, addEventListener creates an internal reference from elt to callback. Therefore, as long as elt is reachable, the event stream fires in perpetuity.

This scenario occurs when we have higher-order event streams, such as the drag-and-drop example, where a new stream of mousemove events is created for each mousedown event. switchE makes the previous mousemove unreachable by the program, though the element being dragged keeps a reference to the mousemove callback. Since
the element is never removed from the DOM, the previous mousemove callback is continually invoked. After a number of drag operations, the browser becomes noticeably slower. We solve this issue by adding an isDetached field to all event streams. If isDetached is set, the callback in $E removes itself, instead of propagating the event:

function $E(elt, evt) {
var stream = receiverE();
var callback = function(e) {
if (stream.isDetached) {
elt.removeEventListener(evt, callback); }
else {
stream.sendEvent(e); }};
elt.addEventListener(evt, callback);
return stream; }

isDetached is initialized to false. In the definition of switchE (figure 12), we set prevE.isDetached = true  when removing prevE from the dataflow graph. However, prevE may not itself be a $E expression, so the isDetached flag of its sources must be updated as well.
For any event stream isDetached is set if all its sinks are detached.

Library Design

The Flapjax library is unusual in that it serves both the compiler (as its runtime system) and developers using it directly. In both capacities it needs to be efficient; the latter is unusual as most languages’ runtime systems are not directly used by developers. Here we discuss some design decisions that have proven important over several years of use.

Functions versus Objects Flapjax encourages making Web applications more functional in style. This can, however, lead to deeply nested function applications, which are syntactically alien to many JavaScript developers. We have therefore found it convenient to make all the standard functions available as methods in the Behavior and EventStream prototypes. This means that instead of
var name = calmE(changes($B("name")), 300);

developers can write

var name = $B("name").changes().calmE(300);

which is arguably more readable than standard functional notation, since the left-to-right order of operations corresponds to the direction of dataflow. We do offer all these operations as standard functions also, so developers can use whichever style they favor.

Lifting Constants The compiler inserts behavior combinators automatically. To aid developers who do not use the compiler, Flapjax’s behavior combinators lift constant arguments to constant behaviors; this does not require compiler support. For example, the type of timerB is timerB :: Behavior Int -> Behavior Int

so that the interval may itself vary over time. Library users may, however, simply write timerB(1000). The function will treat 1000 as a constant behavior.

Element Addressing The library includes many DOM manipulation functions that consume HTML elements. There are many ways for JavaScript functions to acquire DOM elements;
one of the more common techniques is to give them a name (an id). All functions that consume elements also accept strings that name elements.

Reactive DOM Elements Developers may be concerned about the cost of Flapjax’s reactive element constructors. Perhaps hand-coding imperative DOM updates would be significantly
faster?

Indeed, a naive implementation of a constructor would rebuild the entire element on any update. For example, DIV(timerB(1000)) might construct a new <div> every second. Our implementation updates changes in-place, so only one <div> is constructed, but its text updates every second. This strategy significantly ameliorates such efficiency concerns.

 Evaluation

All the Flapjax code above is real. Developers can run the Flapjax compiler on Flapjax code to generate pure JavaScript applications which can then be deployed. As a result, all these programs execute on stock browsers. Flapjax has been public since October 2006, and has been used by several third-parties (i.e., non-authors) to build working applications:

Data Grid The Data Grid application was developed by a commercial consulting firm based in London. They were able to successfully construct their application in the language, and reported that it resulted in “shorter code and a faster development cycle”. They did identify places where the implementation could be faster, an issue we discuss in greater detail below. They found that one fringe benefit of using Flapjax was that it insulated developers “from most of the cross-browser issues”, which are considerable in JavaScript, due to notoriously poor standardization
of Web technologies.

Conclusion

Flapjax is designed with the needs of Ajax developers in mind. It provides a unified framework for programming with events, both within the program and when communicating with Web services. Flapjax is a dataflow-based reactive language wherein values are automatically updated to be consistent, relieving developers of this burden. Through examples and discussion, we have shown that the mechanisms of Flapjax collaborate to make programs more declarative, and to achieve valuable separations of concerns. Flapjax is currently only a client-side programming language. This is because of the relative uniformity of Web clients: estimates are that over 90% of browsers have JavaScript enabled, making it the Web’s other lingua franca besides HTML. In contrast, there is a much greater range of technologies in use on servers, making it harder to target one platform. Nevertheless, it would be valuable to build support for reactivity and events for server applications as well, and to make these consistent with Flapjax clients to enable the establishment of properties such as glitch-freedom across an entire distributed system.








Added on January 24, 2017 Comment

Comments

#1

Balaji K commented on January 26, 2017 at 12:43 p.m.

This is great. I probably won't use the framework, but hopefully this will help me understand FRP :)

Post a comment