Roy van Kaathoven

Translating Urls in Zend Framework 2

Recently i needed to build a multi language website using Zend Framework 2. The framework has a lot options to translate the website, but one thing that was lacking in the documentation was a method to translate URL’s.

After some research i found a Pull Request which adds the ability to translate routes. The current version only supports the Segment route for now, which is fine for basic routing.

In order to start translating routes the router needs to be aware of the translator, do this by injecting the translator into the router during the module bootstrap.

module/Module.php
1
2
3
4
5
6
7
8
9
10
11
12
13
14
namespace Application;

class Module
{
    public function onBootstrap(MvcEvent $e)
    {
        $app = $e->getApplication();
        $sm = $app->getServiceManager();
        $translator = $sm->get('translator');

        // Attach the translator to the router
        $e->getRouter()->setTranslator($translator);
    }
}

Now the routes can be translated, it requires some modifications to the URL’s which are configured in the 'router.routes' key in the module.config.ph

By default the router object does not support translating so it has to be replaced by a router which is translator aware. Replace the router with TranslatorAwareTreeRouteStack by setting the router_class configuration.

The segments can be translated by wrapping the parts which needs to translated within curly braces. If the route is called 'home' then it must be replaced with '{home}'

module/config/module.config.php
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
return [
    'router' => [
        // Replace the default router with the Translator aware TreeRouteStack
        'router_class' => 'Zend\Mvc\Router\Http\TranslatorAwareTreeRouteStack',
        'routes' => [
            'home' => [
                'type' => 'Segment',
                'options' => [
                    'route' => '/{home}', // The parts within curly braces will be translated
                    'defaults' => [
                        'controller' => '...',
                        'action' => 'index',
                    ],
                ],
                'may_terminate' => true,
                'child_routes' => [
                    'profile' => [
                        'type' => 'Segment',
                        'options' => [
                            'route' => '/{profile}',
                            'defaults' => [
                                'controller' => '...',
                                'action' => 'profile',
                            ]
                        ]
                    ]
                ]
            ]
        ]
    ]
];

Using Akka Event Bus

I’ve been working on a new project for a while which is based on Scala, Play Framework and Akka. The goal of the project is to create a library which can communicate with the Steam Servers, and using the library write multiple bots which perform various actions, like starting trades with other Steam users and exchange items.

There can be any number of bots online and their events can be handled by a varying number of services. The services should not have a hard dependency on eachother or other bots, they all have to communicate using a EventBus.

What is a Event Bus?

The Event Bus Pattern is a publish-subscribe-style communication pattern where services or objects can subscribe to the event bus and listen to specific types of events. Other services can publish events to the Event Bus, which then will be propagated to subscribers which are interested in the specific event type.

This pattern is a flexible loosely coupled design which allows for decoupled components that are not aware of eachother.

When to use

There are a lot of use cases for an event bus, a few common are:

  • GUI’s: In a traditional application, multiple sources of events can exist across the application. A menu or toolbar button, a mouse click, window resizing, or some other external data source could feasibly produce the same desired result in the application. Selecting a certain row in a JTable, for example, might result in a toolbar button being deactivated, a database record updated, and a new window spawned. A double click of a JTree item might invoke another long chain of events. The event bus helps solve this problem by simply allowing each component to subscribe to the event bus, and when an interesting action occurs (such as the row selection in a table), an event should be generated to the bus. Each subscriber to the event type will be notified and can react accordingly.
  • Service Oriented Architecture: In a SOA environment there can be a lot of services which perform various actions. The Orderservice will handle an order and notify that an order has succesfully been completed. The newsletter service listens and subscribes the customers to its newsletter, the reporting service adds the informations to its database for later BI, and the inventory registers that the items are substracted from the warehouse. This business logic can be designed in a loosely coupled way using the EventBus.

Implementation

In order to construct a valid EventBus you need to inherit the EventBus class and mix a Classification trait. There are 3 classifications distributed with Akka which you can used in your application.

  • Lookup
  • SubChannel
  • Scanning

The default eventbus provides the following methods which are used to interact with actors:

1
2
3
4
5
6
7
8
// subscribes the given subscriber to events with the given classifier
def subscribe(subscriber: Subscriber, classifier: Classifier): Boolean
// undoes a specific subscription
def unsubscribe(subscriber: Subscriber, classifier: Classifier): Boolean
// undoes all subscriptions for the given subscriber
def unsubscribe(subscriber: Subscriber): Unit
// // publishes an event, which first is classified according to the specific bus (see Classifiers) and then published to all subscribers for the obtained classifier
def publish(event: Event): Unit

In order to build a valid event bus you will need to implement the following types:

Event specifies the type of messages the Eventbus will accept Classifier specifies the type of the classifier Subscriber specifies the type of the subscribers

Examples

The 3 classifications that are already available suit most cases so lets start with explaining how they can and should be used, and which fits your use case.

Lookup classification

Lookup classification is the most basic and provides a simple string matcher. If the published string is exactly the same as a subscriber has subscribed to then it will receive a message.

Lookup Classification
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
case class MessageEvent(val channel: String, val message: Any)
case class Message(text: String)

class LookupEventBus extends ActorEventBus with LookupClassification {
  type Event = MessageEvent
  type Classifier = String

  protected def mapSize(): Int = 10

  protected def classify(event: Event): Classifier = {
    event.channel
  }

  protected def publish(event: Event, subscriber: Subscriber): Unit = {
    subscriber ! event.message
  }
}

The example below shows a subcriber which listens to events send to the “/events” channel, it then publishes a new messages on the “/events” channel and the message is received and printed.

The second publish/subscribe shows exactly the same but on a more specific channel “/events/10”. The message is only received by one subscriber because the Lookup classification does not support subchannels.

Lookup Classification Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
object LookupClassificationApp extends App {

  val system = ActorSystem()
  val eventBus = new LookupEventBus

  val subscriber = system.actorOf(Props(new Actor {
    def receive = {
      case Message(text) => println(text)
    }
  }))

  eventBus.subscribe(subscriber, "/events")
  eventBus.publish(MessageEvent("/events", Message(text = "hello world!")))
  // Output: hello world!

  eventBus.subscribe(subscriber, "/events/10")
  eventBus.publish(MessageEvent("/events/10", Message(text = "hello world!")))
  // Output: hello world!
}

Subclass classification

The subclass specification is a EventBus which supports a hierachy of channels or classifiers. This first example focuses on a simple string based approach which allows to listen to a top level channel and their leafs.

Subclass Classification
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
case class MessageEvent(channel: String, message: Any)
case class Message(text: String)

object SubChannelClassificationEventBus extends EventBus with SubchannelClassification {
  type Event = MessageEvent
  type Classifier = String
  type Subscriber = ActorRef

  protected def classify(event: Event): Classifier = event.channel

  protected def subclassification = new Subclassification[Classifier] {
    def isEqual(x: Classifier, y: Classifier) = x == y
    def isSubclass(x: Classifier, y: Classifier) = x.startsWith(y)
  }

  protected def publish(event: Event, subscriber: Subscriber): Unit = {
    subscriber ! event.message
  }
}

The example below shows subcribers and publishers which become increasingly more specific, and that previous subscribers which are higher up the hierachy will catch the upcoming more specific publishers.

Subclass Classification Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
object ChannelClassification extends App {

  val system = ActorSystem()
  val eventBus = new SubChannelClassificationEventBus

  val subscriber = system.actorOf(Props(new Actor {
    def receive = {
      case Message(text) => println(text)
    }
  }))

  // Top level
  eventBus.subscribe(subscriber, "/")
  eventBus.publish(MessageEvent("/", Message(text = "hello world!")))
  // Output: hello world!

  // Only events
  eventBus.subscribe(subscriber, "/events")
  eventBus.publish(MessageEvent("/events", Message(text = "hello world!")))
  // Output: hello world!
  // Output: hello world!

  // Specific events
  eventBus.subscribe(subscriber, "/events/10")
  eventBus.publish(MessageEvent("/events/10", Message(text = "hello world!")))
  // Output: hello world!
  // Output: hello world!
  // Output: hello world!
}

Downfalls

The sender is not preserved when receiving a message which passed through a EventBus, to solve this you need to pass the ActorRef manually

Scraping With Scala

A few months ago i got asked if i could scrape data from various websites, store it in a database and output some nice reports. Back then i’ve written a simple scraper with Ruby and Nokogiri, saved everything to a Microsoft SQL database and created reports using Excel. Because of the tight budget the script became a quick and dirty solution, but it did the job and the client was happy.

Now that the scrape requests are becoming more common i wanted to design a framework in Scala which should make scrape jobs a lot easier, faster and fun to write. But why should i develop a new webscraping framework when there are already plenty of alternatives in various languages?

Firstly because i like learning new things in Scala, and webscraping is something i haven’t tried out in Scala yet. Secondly because after some research the available frameworks all lack something, the following things i was not happy with:

  • Most frameworks are single-threaded and can’t download pages in parallel
  • They do not scale horizontally
  • The amount of code it requires do something simple is to much

Akka solves point 1 with Round Robin Routing, and point 2 with Remote Actors. Points 3 can be solved with Scala alone.

So far i came up with the following DSL to search for php elephants on Google and read the results

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import org.rovak.scraper.ScrapeManager._

object Google {
  val results = "#res li.g h3.r a"
  def search(term: String) = {
    "http://www.google.com/search?q=" + term.replace(" ", "+")
  }
}

// Open the search results page for the query "php elephant"
scrape from Google.search("php elephant") open { implicit page =>

  // Iterate through every result link
  Google.results each { x: Element =>

    val link = x.select("a[href]").attr("abs:href").substring(28)
    if (link.isValidURL) {
      // every found link in the found page
      scrape from link each (x => println("found: " + x))
    }
  }
}

To make common tasks easier there are spiders available which search through a website, clicking on every link with the allowed domains and open the webpage for reading. A simple spider reads the entire domain and stop when it has nothing left to do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
new Spider with EmailSpider {
  startUrls ::= "http://events.stanford.edu/"
  allowedDomains ::= "events.stanford.edu"

  onEmailFound ::= { email: String =>
    // Email found
  }

  onReceivedPage ::= { page: WebPage =>
    // Page received
  }

  onLinkFound ::= { link: Href =>
    println(s"Found link ${link.url} with name ${link.name}")
  }
}.start()

The project is far from finished, but hopefully it gives you an idea of the upcoming features. The project can be found on Github

Scalext: Ext JS Module for Play Framework

I’ve been working on a project called Scalext for a while and want to share the current version and upcoming ideas. Scalext is a module for Play Framework 2.1 which makes working with Ext JS a lot easier.

I’ve already implemented a Ext JS Module for Zend Framework 2 in the past which made writing a similair module in Scala a way to compare the language with PHP. The current version of the module has most features from KJSencha implemented and took a lot less code than the PHP version.

The most notable improvements are:

  • Parallel execution of Direct API actions
  • Compiler checked annotations
  • DSL for javascript objects

Direct API

The api generator scans a project for classes which have a @Remotable annotation (in later versions a trait) configured and generates Ext JS Direct configuration which can be called in a Ext JS Project.

When the Direct API calls a javascript method then it will be routed to the Scalext module which then dispatches the request to the correct Direct action. The incoming requests are JSON objects which will be deserialized by Google GSON to Scala values. This makes working with javascript requests a lot easier. Any return values given by Direct action are again automatically serialized by Google GSON again and returned as JSON.

The combination of generating the API plus handling serialization and deserialization makes working with the Ext JS Direct API a lot more enjoyable.

Component DSL

A common issue for newcomers to Ext JS is configuring Ext JS classes. Writing classes is not a problem, but the massive amount of available options overwhelm most users. Only a few IDE’s have plugins available to support Ext JS syntax, and most do not support the Ext.define('Classname', { key: value}) syntax which prevents the IDE to build a autocompletion list for available classes or properties. Ofcourse there is the Ext JS documentation which is a pleasure to work with but it may take some time to find what you need, and you have to leave the IDE. A second issue which newcomers often encounter is case-sensitive properties.

To tackle these issues in future versions of Scalext there needs to be autocompletion and syntax checking available. Both can be achieved by implementing Ext JS classes in Scala and output them as javascript files. Although this is perfect to configure class and their properties, there is still the question how can inline functions be defined? Something like the following snippet will be hard to implement:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Ext.create('Ext.Window', {
  title: 'Popup',
  html:'This is a test popup',
  buttons: [
      {
          text: 'OK',
          // How to handle inline functions?
          handler: function() {
              alert('Confirm!');
              this.up('window').close();
          }
      }
  ]
});

Hopefully Scala.js solves this problem, it is still an early version but the Reversi example shows that a simple game can already be build.

Apart from the syntax checking and autocompletion it also tackles the following problems:

  • Building highly dynamic components
  • Components which are restricted by ACL rules, unauthorized parts will never reach the client-side
  • i18n

The current version only supports Play Framework 2.1, upcoming versions will depend on other modules which can easily be used in different frameworks.

More info can be found by checking out Scalext on Github or running the Example Application

Managing Assets in Zend Framework 2

When you are developing a module for Zend Framework 2 in combination with assets you will most likely run into the question on how to serve the files to your public folder.

There are quite a few methods to do this, i will explain the 3 most commonly used methods.

Github Attachments

Github recently added a new feature in which you can add images inside a ordinary textarea by simply dragging an image inside it, or pasting (only in chrome) an image. While i was working on the markdown editor for Socialog i needed a similar way to add images to a blog post. Therefore i wrote a javascript plugin which does the exact same thing. The result is a plugin which can be used with jQuery, CodeMirror or standalone, so you can easily use it in your project.

During this project i’ve gained experience with Grunt.js, structuring a jQuery plugin and the HTML 5 File API which i will briefly describe.

While writing out some quick requirements for the plugin i quickly noticed that there had to be a way to build my javascript files into 3 different versions. The standalone version should be compiled inside the jQuery and CodeMirror version without having to copy the code manually, and ideally there should also be a minified production version.

Meet Grunt.js, a JS command line build tool

Grunt.js has proven itself in the jQuery project and became my weapon of choice to combine and minify the required files.

Like any other build tool you need a script which tells it what to do, create a grunt.js in your project directory and setup some tasks. Here are the tasks which i use in my plugin, for simplicity i removed the codemirror version.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
grunt.initConfig({

  // Grunt can read json files, here we load the package.json values inside the pkg property so it can be used inside the banner
    pkg: grunt.file.readJSON('package.json'),

    meta: {
        banner: '/*! <%= pkg.name %> - v<%= pkg.version %> - ' +
            '<%= grunt.template.today("yyyy-mm-dd") %> */'
    },

    // Concat action provided by UglifyJS, it automatically runs every child object
    concat: {
        normal: {
            src: ['<banner>', 'src/inline-attach.js' ],
            dest: 'dist/inline-attach.js'
        },
        jquery: {
            src: ['<banner>', 'src/inline-attach.js', 'src/jquery.inline-attach.js' ],
            dest: 'dist/jquery.inline-attach.js'
        }
    },

    // Minify action provided by UglifyJS, automatically runs every child object
    min: {
        normal: {
            src: [ '<banner>', 'dist/inline-attach.js' ],
            dest: 'dist/inline-attach.min.js',
            separator: ';'
        },
        jquery: {
            src: [ '<banner>', 'dist/jquery.inline-attach.js' ],
            dest: 'dist/jquery.inline-attach.min.js',
            separator: ';'
        }
    }
});

// Load the plugin that provides the "concat" and "min" tasks
grunt.loadNpmTasks('grunt-contrib-uglify');

// Register the default task
grunt.registerTask('default', ['concat', 'min']);

The config is defined first, it contains data like the pkg variable which loads package.json so it can be used in for example the banner. The meta.banner property contains the string which can be placed above every generated file. It can be called with the <banner> tag.

Secondly the tasks are loaded using grunt.loadNpmTasks('grunt-contrib-uglify')

And last the default task is defined when grunt is called without arguments, if you are familiar with Apache Ant then the equivelant is:

1
2
3
<project name="Grunt" default="build">
  <target name="build" depends="concat,min"/>
</project>

When everything is done you can run grunt and it will generate everything in the dist/ folder, that saves a lot of time!

jQuery plugin

Writing your code as a jQuery plugin saves you a lot of time when reusing the plugin and makes it easy for other developers to integrate your code into their own projects.

Setting up the structure is really easy and a tutorial can be found at the jQuery plugin tutorial

HTML 5 File API

Handling files with the latest HTML 5 File API provides you with a lot of new options to handle files, like handling files which are dragged from the desktop, listening to a paste event inside a textbox and showing a preview of the just pasted image, or resizing a picture before uploading it to save time and data.

The end result is an easy image attachment system in which you can paste or drop images inside a textbox and include an image without a WYSIWYG editor.

The project can be found at Github

Razko.nl

Welcome on my new blog, here i will post my thoughts on web development, personal projects and code that i’m working on.

There isn’t much to see right now as i am working on some upcoming features like the code sandbox, portfolio and Github integration.