I've been playing around with Scala quite a bit this week. So far I'm digging the extreme utility and breadth of the language. As my day-to-day work is typically centered around web development, I figured I should check out what Scala had to offer in that direction. Quite a bit, it turns out. There are several web frameworks for Scala, not including all the Java ones that are available.

Scalatra was very accessible, and I've been able to get useable things working in a short amount of time with it. I was able to get Squeryl, a Scala ORM, integrated with Scalatra by following the advice of the very helpful Scalatra User Google Group. With all my newfound knowledge, I figured I might provide an overview of how to get it all working.

I'm used to PHP programming with Drupal and have not had much need to utilize connection pooling, so when I got my small app deployed on CloudBees (fast & automated JVM hosting) I was quite surprised the next day to find I was getting connection errors to the provided MySQL database. It turns out that a connection pool is often used with JVM web apps to limit the amount of connections an app can make to the database server.

For connection pooling I went with C3P0 as it's often recommended on Stack Overflow and other sites I visited. If you are using SBT you can add C3P0 by inserting the following in your SBT project file.

val c3p0 = "c3p0" % "c3p0" % "0.9.1.2"

C3P0 integrates with Squeryl by providing an object that Squeryl can use to request connections. You will need to instantiate a C3P0 ComboPooledDataSource, configure it, and tell Squeryl's singleton SessionFactory about it.

var cpds = new ComboPooledDataSource
cpds.setDriverClass("com.mysql.jdbc.Driver")
cpds.setJdbcUrl(databaseConnection)
cpds.setUser(databaseUsername)
cpds.setPassword(databasePassword)

SessionFactory.concreteFactory = Some(() => connection)

def connection = {
  Session.create(cpds.getConnection, new MySQLAdapter)
}

Then for each request you need to make sure to use Session.bindToCurrentThread before running any code that accesses the database.

C3P0 helped with limiting the amount of connections my code was making to the server, but I still had another problem: C3P0 would only let the application request a configured maximum number of connections. If I increased the MaxPoolSize I could make up to that many requests, and then the app would hang. When using a connection pool, the connections need to be cleaned up after the response has been generated. This allows the connection pool to provide connections for other responses to be generated. The code wasn't cleaning up the Squeryl Session after the response was generated and therefore was using up the connection pool.

Session.close
Session.unbindFromCurrentThread

The app was working and not eating connections for breakfast, but I had a mess of code to deal with. Scalatra typically has a single file for the app that extends either ScalatraFilter or ScalatraServlet, and at first I had all of the database authentication and connection pooling in that file. This is not ideal from a maintainability or reusability perspective. Using Scala Traits I was able to clean up the source code. Scala traits are like a cross between Java Interfaces and Ruby mixins. With Scala traits you can provide a partially implemented interface that another class can use to extend itself. This is very useful for database configuration and database Session handling. Take all the code that connects to the database and add it into a trait.

trait DatabaseInit extends Initializable {
  val databaseUsername = "username"
  val databasePassword = "password"
  val databaseConnection = "jdbc:mysql://example.com:3306/database"

  var cpds = new ComboPooledDataSource
  abstract override def initialize(config: Config) {
    cpds.setDriverClass("com.mysql.jdbc.Driver")
    cpds.setJdbcUrl(databaseConnection)
    cpds.setUser(databaseUsername)
    cpds.setPassword(databasePassword)

    cpds.setMinPoolSize(1)
    cpds.setAcquireIncrement(1)
    cpds.setMaxPoolSize(50)

    SessionFactory.concreteFactory = Some(() => connection)

    def connection = {
      Session.create(cpds.getConnection, new MySQLAdapter)
    }
  }
}

Then extend your ScalatraFilter/ScalatraServlet with your new trait.

class AppFilter extends ScalatraFilter with DatabaseInit

This same pattern can be used for the database session.

trait DatabaseSessionSupport extends Handler {
  abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) {
    Session.bindToCurrentThread
    try {
      super.handle(req, res)
    } finally {
      Session.close
      Session.unbindFromCurrentThread
    }
  }
}

A few members of the Scalatra User Group pointed out some concurrency issues. I had overlooked the fact that the Session variable I had been using would be accessed by multiple threads. The session could potentially be wiped out while one thread is serving a request if another thread is serving a concurrent request. To resolve this, it was recommended that I bind the Session to a DynamicVariable/ThreadLocal variable. Scala makes this really easy to do. Create a new DynamicVariable of type Session, then, whenever you need a new Session, you wrap the Session code inside of a block called by the DynamicVariable's withValue method.

trait DatabaseSessionSupport extends Handler {
  val dbSession = new DynamicVariable[Session](null)
  abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) {
    dbSession.withValue(SessionFactory.newSession) {
      dbSession.value.bindToCurrentThread
      try {
        super.handle(req, res)
      } finally {
        dbSession.value.close
        dbSession.value.unbindFromCurrentThread
      }
    }
  }
}

Coming from a PHP background with web development, some of the basics of the way JVM apps are written are completely foreign to me. I think many users new to Scala are probably coming from a Java background and come equipped with knowledge of connection pooling libraries and multithreaded web servers. Hopefully this summary has been helpful to someone looking for ways to get Scalatra talking to a traditional SQL database. If you'd like to get started with this setup I've posted all the relevant code to GitHub.

https://github.com/fadeddata/Scalatra-Squeryl-C3P0-Base

On the recommendation from Dries and some prodding from Jeremy I decided that maybe I should post something regarding some work I did a few months ago.

Drupal is amazing with how it's able to provide information about its runtime and environment even though PHP has little of the functionality to provide it. Specifically the Forms API layer allows you to programmatically access all the "metadata" related to any form.

$form = array();
$form['form_id'] = 'system_site_information_settings';
$form_state = array('storage' => NULL, 'submitted' => FALSE);
$form['form'] = drupal_retrieve_form($form['form_id'], $form_state);
drupal_prepare_form($form['form_id'], $form['form'], $form_state);

This bit of code will provide an array that has everything related to the system_site_information_settings form. All fields on the form including their types are provided within the array.

Using this with a remote system or a rich environment (flash, javascript) it's possible to recreate an interface for the end user to work with. Once all the information is collected the fields could be sent back to Drupal to then be used with drupal_execute.

I've developed a bit of code to try this out. It's very alpha and did work at one time. Basically there are a few Services modules that provide the needed form functionality, and another that provides some basic node listing functionality. The node listing functionality should probably be provided by the built in node service.

If there is interest in this project I may press on. Right now I'm holding back because I feel it would be hard to get either of the interdependent projects up to their respective central locations (Apples App Store and drupal.org). I suspect getting a project for drupal.org would not be that hard but getting the App side of the project past Apples app requirements maybe impossible.

The relavant repositories are publicly accessible from our git server. I'll be updating the licensing information soon and putting all the code under the GNU GPLv2.

git://git.7sudos.com/ipad_service.git
git://git.7sudos.com/form_service.git
git://git.7sudos.com/drupal_ipad.git