An external module is one way to share lumps of code with the Lift community.  The word "external" indicates that these are distinct from the modules in the core Lift repository maintained by Lift comitters.  But the pattern to create a module is the same, as Peter outlined on the Lift Wiki and in one or two posts on the Lift mailing list.

Anyone can create one, anywhere they like, and people do. So there's nothing to talk about.

Except... I'd like to have modules build automtically when Lift changes, with reports on problems, and have an easy way to find modules and add them to Lift projects.  If you buy into that as something useful, there are lots of ways to achieve it.  

Are we there yet? Are we there yet? Are we...

Today, we're nowhere near, but we've started by experimenting with a build system and a common repository for external modules. 

We want to track Scala and Lift releases, including Lift milestone releases, and publish these builds so they can be available quickly to anyone wanting to use them via Maven or SBT.  That's the plan.  To explore this we're using CloudBees. Under their open source programme we're given a hosted Jenkins instance and repository for build artifacts.  In other words, we go to the CloudBees web site, create a new Jenkins job, point it at the appropriate GitHub repo and press "Build now". Nice and easy, but all manual.

Currently we're building just a few modules, which you can see on the Jenkins dashboard:

•  Google Analytics support
•  IMAP IDLE, providing push-like email into a Lift app
•  JqPlot module for graphs

...but none of it is automatic yet.  We know we can do better.

The developer experience

This is what the current development process is like for getting a new release out.  Let's say, Lift 2.4-M4 becomes available (which it has; this post has been in draft too long).

The module versioning number I'm using is the Lift version with the module version appended.  So the 0.91 version of the IMAP module for Lift 2.4-M3 is: 2.4-M3-0.91. Sometimes, that's all you need to change, making me wonder if this is a wasteful system.  Anway, let's publish a version of 2.4-M4:

$ cd liftmodules-imap-idle
$ git checkout -b 2.4-M4
$ vi build.sbt
// modify version, liftVersion, test
$ git push origin 2.4-M4
$ sbt 
 > +publish   

...then merge back to master:

$ git checkout master
$ git merge 2.4-M4
$ git push origin master   

Where next?

Are we making a meal out of this?  Is this something worth doing?  Are there better ways?  Feedback is what we need, and if you're part of the Lift community, the place for that is on the mailing list please.

Otherwise we'll be investigating: 

• adding triggers or polling to trigger builds of modules. 
• possibly a directory of modules - but that seems like the easiest part now that we've started to push out notifications to the fantastic implicit.ly space.
• scratching our heads about how to track SNAPSHOP.
• can we automate module changes to track Lift milestone releases?
• automatically publishing builds into the repo.

Or something like that.

If you want your module included in this build process, hassle me or Jono on the Lift mailing list.

If you want to send email from your Lift app using SendGrid you need to do two things: set an authenticator and enable the javax.mail authentication flag. That second part isn't obvious.

Step by step...

Assuming your Lift props file contains the following:

# Sendgrid.com configuration: 
mail.smtp.host=smtp.sendgrid.net 
mail.user=you@example.com 
mail.password=letme1n 
 

In Boot you need to set the authenticator:

import javax.mail._ 
def optionalAuth: Box[Authenticator] = for {
    user <- Props.get("mail.user")
    pass <- Props.get("mail.password")
  } yield new Authenticator {
    override def getPasswordAuthentication = new PasswordAuthentication(user,pass)
}
Mailer.authenticator = optionalAuth

And the part that I forgot:

optionalAuth foreach { a => System.setProperty("mail.smtp.auth", "true") } 

(download)

Click here to download:

sbt-terminal-in-eclipse-FHeHugqxpArwkhHxipkt.zip (209 KB)

There are a few mentioned on the SBT site, but I realized all I needed was a shell prompt as a view in Eclipse and I'd be happy.  And of course it exists already as "Target Management Terminal", part of the Galileo update site.

1. Eclipse > Install New Software....
2. From "Work with:" select Galileo (or whatever's appropriate for your version of Eclipse).
3. Type "terminal" in the search box, and select Target Management Terminal.
4. Go through the download, restart process.
5. Eclipse > Preferences and search for Terminal if you want to invert the colours (I did).
6. Window > Show View > Other and select Terminal to show the Terminal view in Eclipse.
7. Click the green connect button and set up the type of terminal you want (ssh to localhost in my case, and I'm prompted for a password).
8. Login, you have a shell, do what you want, such as cd into your project and run SBT.

Probably totally obvious, but I missed this trick of having access to a shell from Eclipse.

Lift development has moved on from the depressingly long Maven commands to start a new project.  I made the above screencast to show the simple build tool that I'm growing fond of, and an add-on for SBT called Lifty that makes it easy to create Lift projects.

It took me a couple of attempts to make the video, so of course many of the automatically downloaded libraries were locally cached.  It'll take longer for you if you're starting from scratch, but not as long as I'd expected. For some reason (possibly imaginary) the SBT downloads for Lift seemed much smaller, and consequently faster, than I'm used to with Maven. YMMV.

If you want to know more about Lifty, check out Mads Hartmann's talk from the Scala LiftOff London.  There's also an "Up-close and personal with SBT" workshop from the London Scala User Group in the evening on 10 November 2010 (free, just sign up).

Going beyond the screencast

I don't use SBT projects exactly as I've shown in the screencast: I use JRebel and the Scala IDE for Eclipse.

"JRebel maps your project workspace directly to your running application. When a developer makes a change to any class or resource in their IDE the change is immediately reflected in the application, skipping the build and redeploy phases".  And the Scala IDE compiles on save, so the combination gives me "press save in the IDE, hit reload in the browser" development.

I've paid for the enterprise version of JRebel as I have legacy Java apps to live with, but for pure Scala/Lift development you can pick up a free Scala license of JRebel — and you should. 

The configuration is just in two places.  For SBT itself, modify the sbt launch script to include the JRebel magic.  I have:

export REBEL_HOME=/opt/zeroturnaround/jrebel
export REBEL_JAR=$REBEL_HOME/jrebel.jar 
export REBEL_LIC=/opt/zeroturnaround/javarebel.lic 
export WITH_REBEL="-Drebel.license=$REBEL_LIC -noverify -javaagent:$REBEL_JAR"
java $JAVA_OPTS -XX:MaxPermSize=256m -Xmx512M $WITH_REBEL -jar `dirname $0`/sbt-launch.jar "$@"

The other change is to prevent Jetty from restarting itself on change.  That's an edit to your project/build/Project.scala file (or whatever you called it):

override def jettyWebappPath = webappPath
override def scanDirectories = Nil

That's all, and when you fire up SBT and run the jetty command, you have a pretty efficient working environment.

Oh, to generate the Eclipse project files, there are a few options listed on the SBT wiki, but I'm currently trying SbtEclipsify.

All the code that matters is in one file.

(download)

Click here to download:

scala-lift-off-london-was-wonderful-gaIwmpBCBiJfdFjgymzs.zip (3.82 MB)

I just spent two days at the Scala Lift Off unconference, meeting and chatting with 100+ old and new friends, hearing what everyone is interested in or learning about or doing. Plus the luminaries of the field were there to join in, answer questions, discuss what can be changed and how.  Think about that for a second: it's pretty much the best possible world, and it's really your fault if you can't get something good out of that :-)

I'm not going to give an account of the sessions.  I'm sure others will do a good job of that, and also there's the unconference wiki, photos tagged with scalaliftoffoct78, tweets tagged with #scalalol and videos on the Skillsmatter Vimeo site.    A few small factoids I picked up...

• Happy to see many hands go up in answer to the "who has Scala in production?" question. 
• Scala Solutions are adding their effort to the Scala IDE: making Eclipse+Scala better is seen as a good way to help Java developers over to Scala. 
• Parallel collections to ship in Jan 2011.  Making use of those multicores.
• I should look at Vector and HashMap and not just List, as they are efficient and persistent in 2.8
• sbt and lifty give a very nice way to start and build Lift projects.
• There was lots of interest in Akka.  I didn't hit any of those sessions, but this Wednesday the London Scala User Group is hosting "Simpler Scalability, Fault-tolerance, concurrency and remoting through Actors" at 6:30pm.
• There's going to be a Lift training course in London.

For me the most impressive session was the one where all the scary functional words were listed on a white board ("applicative functor", "monads"...)  and then the group tried to define them and give examples. 

If any of this sounds appealing, keep an eye on the Skillsmatter Scala page, because there's plenty happening you might like.   

See you at Scala Liftoff London 2011 (Oct 6 and 7, registration now open). 

We're all looking forward to Tim Perrett's Lift in Action book, but until Chapter 6, Testing, is done the main resource for testing in Lift is Vassil Dichev's blog post and the entry in the Lift wiki.  If all you need to do is test Mapper, though, an in-memory database works just fine.

Here's what we've used:

$ cat InMemoryDB.scala

package com.spiralarm.model

import net.liftweb.mapper._
import net.liftweb.common._
import net.liftweb.util._
import java.sql._

object InMemoryDB {

 val vendor = new StandardDBVendor("org.h2.Driver", 
    "jdbc:h2:mem:lift;DB_CLOSE_DELAY=-1", Empty, Empty)

 Logger.setup = Full(net.liftweb.util.LoggingAutoConfigurer())
 Logger.setup.foreach { _.apply() }

 def init {
  DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
  Schemifier.destroyTables_!!(Schemifier.infoF _, Your, Classes, Here)
  Schemifier.schemify(true, Schemifier.infoF _, Your, Classes, Here)
 }

 def shutdown {
  // TODO: figure out if anything goes here
 }
}

Effectively we create an in-memory H2 database, and use Schemifier to drop our tables and recreate them with no data.  You can use something like the above in tests pretty easily...

import org.specs._
import org.specs.matcher._
import net.liftweb.mapper._
import com.spiralarm.model._

class MyTest extends SpecificationWithJUnit {

 "My thing" should {
  "store stuff in the database" {
   InMemoryDB.init
   var person = Person.create.first("Billy").last("NoMates").saveMe
   var friends = Friends.find(By(Friends.person, person))
   friends must_== Empty
  }

 }
}

...or something like that, with a less fake example that actually compiles.  If you're doing a lot of this, you may want to have the in-memory initialized (and possibly shutdown) automatically around each test, and generalize to pass in the classes you care about to be Schemifier-ed.  But you get the idea.

I have my ticket. How about you?  It's £150 at the moment.

What we're looking at here is a two day unconference, meaning we get a chance to learn about things we're interested in by figuring it out on the day, talking to people, rather than being lectured at.  I thoroughly enjoyed the Scala Lift Off last year in San Francisco, so I'm looking forward to the London one.  Especially as it close by, rather than 8,720km away.

(download)

Click here to download:

scaling-web-apps-with-akka-vHotFvgaDFtkrCIuHpbB.zip (284 KB)

There were a couple of things I particularly liked: getting coverage of the various styles of actors available in Scala; and seeing a different development style. I won't be switching back to emacs any time soon, but I do now see the appeal of SBT.

Do go check out the talk: it goes from zero through actors and a
complete Lift web app talking to Akka actors.

Last night I was asked about the aliases I used during my Lift talk.  Here they are...

$ cat ~/.profile
export REBEL_JAR=/Users/richard/Developer/libs/javarebel/jrebel-2.1a/jrebel.jar 
export MAVEN_HOME=/Users/richard/Applications/maven/apache-maven-2.2.1
export MAVEN_OPTS="-Xmx256M -XX:MaxPermSize=256m -Dfile.encoding=utf8"
export JAVA_OPTS="-Dfile.encoding=utf8"
export PATH=$MAVEN_HOME/bin:$PATH

# E.g., "m package" to run the Maven package target in offline mode:
alias m='mvn -o'


# Run Maven in a mode that fetches sources and documentation for libraries:
alias mm='mvn -DdownloadJavadocs=true -DdownloadSources=true'


# Start your Lift application in the REPL
alias liftconsole='mvn -o scala:console -DmainConsole=LiftConsole'


# Start your Lift application using JRebel
alias mmm='MAVEN_OPTS="-Dfile.encoding=utf8 -XX:MaxPermSize=256m -Djetty.scanIntervalSeconds=0 -noverify -javaagent:$REBEL_JAR" mvn -o jetty:run'


# Create a new 2.0-M2 Lift application
alias liftcreate="mvn archetype:generate -DarchetypeGroupId=net.liftweb -DarchetypeArtifactId=lift-archetype-basic -DarchetypeVersion=2.0-M2 -DremoteRepositories=http://scala-tools.org/repo-releases"