Bobylito

09 Feb 2013

Binding worlds using Scala/Play/Javascript

Today’s sample project is all about using existing bricks, adding glue and letting the magic happen. But be careful, I’m using cool stuff and wonderful pieces of tech but I don’t intend to explain them by the menu.

So ok, I have a project, it is wonderful and awesome and… Let’s stop there and explain a bit further. It is a server which reads its standard input and streams that to a webpage it is also serving. The idea of this POC server is to bind command line tool and a web server, in order to do something like this :

$> tail -f apache_access_log | webPipe

This outputs the content of the Apache access log to the web through server sent events as the file log grows. By the way, the webPipe command is an alias to the command that launches the server.

Webpipe

The code is in Scala and uses Play 2.1. It also contains just the right amount of javascript code (for the size of this project).

The project code is here (we’ll go directly to the specific parts during the article).

The root of all things

Reading the standard input of the application is straightforward, thanks to scala.io and especially scala.io.Source, both included in the standard library.

val stdIn = scala.io.Source.stdin

Returns the standard input as a scala.io.BufferedSource. We can then use its api to read line by line :

val lines = stdIn.getLines

This returns a brand new scala.collection.Iterator[String] . Now we can have some fun and start using the Play Framework .

Enumerate the input

The internals of Play2 don’t play along with Iterators. They use other data structures for having non blocking, reactive processing. For more infos on this subject, I would recommend that you read Mandubian’s article (it is not mandatory to read it to understand the rest of this post though)

Long story short, we introduced a new function to the Enumerator object that actually give the ability to take any TraversableOnce (which represents any kinds of stuff you might want to iterate on) and transform it to an Enumerator, which is what Play understands.

For example :

val enumLines = Enumerator.enumerate( lines ) // returns Enumerator[String]

To be real life compliant, we can actually want json data structure (makes life easier afterward) :

val enumLines = Enumerator.enumerate( lines.map( { l => 
  play.api.libs.json.Json.toJson(l) 
} ) )// returns Enumerator[JsonValue]

Actually this isn’t quite enough to send stuff to many clients. In order to do that, we need to use play.api.iteratee.Concurrent.broadcast.

val (broadcast, _) = play.api.libs.Concurrent.broadcast(enumLines)

This gives us an enumerator we can use for streaming for any client (the underscore is an object used to control the broadcast itself, useless here).

Check how it looks in the app

Feed me!

With an Enumerator in our pocket, we are already half way through this example. Actually, now we want to serve it and it’ll be quite fast. Be ready :

ok.feed(enumLines &> play.api.libs.EventSource).as("text/event-stream")

BAMM, the controller now returns the feed as EventSource!! Bravo!

Not your usual client

The feed is feeding but we are not yet eating.

new EventSource("/log")
EventSource.addEventListener("message", function(){
 ... 
});

Click here to see how it is used in the project.

Conclusion

In less than 20 lines of actual code, this demonstrates we can push the command line to the web and use the play2 web framework as a command line tool. Hope you’ll have fun playing with all these stuff. If you have any questions, ping me on twitter


Comments