The Functional Brighton meetup for May was a set of short demos and talks on the subject of "What functional programming means to us".  Kit kicked off the evening with an F# show-and-tell of a 3D flocking simulator (complete wth 3D glasses); Eric spoke about why he uses Haskell; Andy showed us some Scheme code and made it progressively more functional; and Tom jumped up with a code tour of a Haskell project of his.  A good evening with plenty of Q&A.

I had every intention of recording what I said about Scala, but failed.  So here goes with a reconstruction of what I said or intended to say.

"What functional programming means to me" has been a fun assignment. I have no background in functional programming—and the funky languages I have used for any period of time were never described as functional.   So the way I've approached this is to report on three things I've noticed during my transition from mostly-Java to Scala, and consequently I'm not sure how much of this is really about functional programming.  

Approach to problem solving

The first of the three things is very general. It's about how I seem to approach problems.  It's also the hardest one for me to explain, and probably the most waffly.

Here (slide 2) is an example of the kind of problem I had to deal with last year, and what we're doing is taking a tweet and adding a hashtag to it if it's not already there.  I'm not sure how I used to solve these problems, but I think it would have dived in with a loop and maybe a string buffer and I'd test and append.

Now what I seem to find myself doing is thinking about what I have, and what I want to get (slide 3).  What do I have? Some tags and a tweet.  What do I want to get back? A tweet. Which really means (slide 4) I have a list of strings and a string, and I want a string back.  By thinking about what types are involved, I find myself asking what kinds of transformations will get me the result I want.

So in code (slide 5) if we have a List[String] and a String, and we want to get a String back, then a fold is a reasonable choice.   This is Scala, we're defining appendTags which takes our tweet which is of type String, and the tags of type List of String, and it returns a String (although I've left that off as it's being inferred).  

AppendOne solves the simple case: it expects a string and a tag and does the obvious thing.  As luck would have it, the two parameters appendOne wants are the parameters that foldLeft gives it for each of the tags and the current tweet. 

(At this point I think we dropped to the REPL and tried this out)

I find these solutions preferable in terms of clarity, LOC, also happiness in not writting the same old loop I've written many times before (slide 6).

That was the first thing: thinking in terms of transformations, and how it sort of suggests solutions in terms of functional programming constructs.

Concurrency

The second thing is about concurrency.  From what I recall, in C it was just plain hard to write concurrent code at all.  In Java, it's easy, but even easier to get it wrong.   It looks to me that the functional world want to address this problem, being both correct and easy to use.  Maybe this quote (slide 7) alludes to why, as concurrent problems boil down to coordinated access to shared mutable state.  As functional tends to point you away from mutable state, part of the battle is over.  

These are facilities I take advantage of today, and it makes it easy to bring not-insubstantial improvements to applications.   I have a code base (slide 8) that makes multiple HTTP requests to web APIs.  In this example the function yahoo, if you call it, hits Yahoo's site and brings back address book entries (let's say).  And the google one does the same to Google's API site.   They don't take very long, but they absolutely can happen concurrently.

I have a list of two functions, the underscore telling us the functions aren't actually being called yet.  I turn them into a parallel collection and then call the apply function on them.  They are executing in parallel and the result will be a List containing two lists: one for the results from each site.

This is one-liner stuff.  There's no barrier to going concurrent.

Actually, that example is from Scala 2.9, and I don't use Scala 2.9 yet, so I do something a little more involved (slide 9).

Useful concepts

The third thing that "functional is" to me, is the set of constructs functional programming assumes.  Facilities like Option, comprehensions, flatMap, pattern matching which, as I've said here (slide 10), are astonishingly useful every day and yet I'd never heard of them before using Scala.

Initially they seem like esoteric, academic, hard to understand.  They're not hard: they're rich. It takes time to appreciate them.  But it's time well spent.

I've blogged about this example (slides 11 and 12), but it's accessing the Google contacts API.  Comparing the Google Java code and the the corresponding use of Option and for comprehensions are much easier to read, and easier to write.  They make a big impact in day-to-day code.

Summary

Those were the three things (slide 13) that functional programming appears to mean to me at the moment.

1. Thinking in terms of transforamtions changed my approach to problem solving for the better.
2. Concurrency without any barriers to use.
3. The facilities in functional programming that let you write better programmes.

@wfaler asked:

Doing a presentation soonish on "Why #Scala" - have a truckload of arguments, but what are your arguments for Scala over Java? Give me ammo!

Ok, I'll bite.  Here's one of those every-day things I stop and notice from time-to-time. It's not new, it's not huge, it's just one of the things that makes a difference when you're trying to ship stuff.

The Google Java APIs for accessing Google contacts has you write code like this:

for (int i = 0; i < resultFeed.getEntries().size(); i++) {
   ContactEntry entry = resultFeed.getEntries().get(i);
   if (entry.hasName()) {
     Name name = entry.getName();
     if (name.hasFullName()) { 
    ...

I recently had the need to go through contacts to extract birthdays.  I only care about contacts with a name and a birthday, but I use Scala so don't have to do the if/get dance because Option does the right thing in a for-comprehension:

for { entry <- contacts
      name <- Option(entry.getName)
      full_name <- Option(name.getFullName)
      birthday <- Option(entry.getBirthday)
      when = birthday.getWhen 
}
     yield "%s: %s".format(full_name.getValue, when)

The above evaluates to a List[String] (because contacts starts as a List) containing those contacts with a name and a birthday.  So even when using regular Java libraries, you end with benefits (less code, more readable code, less-likely-to-cock-up-the-logic code) just by using Scala. IMHO.

Graham Tackley spoke at a packed LSUG about how guardian.co.uk implemented their first Scala project, the content API.  It's well worth watching the video to see the timeline of how the team switched their brains from Java to Scala (what the team initially liked, what they liked after a year, grokking flatMap, tools, and more).  

The result after the transition: more fun, better code, more productive and happier developers.  Regarding the cost of switching, the comment from the team during the Q&A was: there's a learning curve to Scala, but providing you have someone to lead the way, the switch doesn't materially effect delivery in the first three months, and post three months the team becomes more productive.

Go watch that video, and if you're close by, sign up to the London Scala User Group.  I had a sneak peek at some of the talks for 2011, and there are some great ones in there.

Related content: 

You might also like David Copeland's Sneaking Scala Into Your Organization video from Scala Days 2010.

(download)

Click here to download:

sbt-terminal-in-eclipse-FHeHugqxpArwkhHxipkt.zip (209 KB)

There are a few mentioned on the SBT site, but I realized all I needed was a shell prompt as a view in Eclipse and I'd be happy.  And of course it exists already as "Target Management Terminal", part of the Galileo update site.

1. Eclipse > Install New Software....
2. From "Work with:" select Galileo (or whatever's appropriate for your version of Eclipse).
3. Type "terminal" in the search box, and select Target Management Terminal.
4. Go through the download, restart process.
5. Eclipse > Preferences and search for Terminal if you want to invert the colours (I did).
6. Window > Show View > Other and select Terminal to show the Terminal view in Eclipse.
7. Click the green connect button and set up the type of terminal you want (ssh to localhost in my case, and I'm prompted for a password).
8. Login, you have a shell, do what you want, such as cd into your project and run SBT.

Probably totally obvious, but I missed this trick of having access to a shell from Eclipse.

John at the London Scala User Group ran "LSug Workshop: Up-close and personal with SBT" yesterday at Skillsmatter.  It was a kind of mini-unconference, and I contributed an introduction to SBT.  The video above is the first half of a screen recording I made, and it continues into part 2 on Youtube.

I learnt a lot from the evening on how others use SBT, and from Maciej Matyjas talk on giter8 and plugins (including what looks like a wonderful plugin to Posterous). Phil Wills showed us how to create a SBT plugin, and that doesn't look too scary now.

Skillsmatter should post video of the evening at some point, and Chris Tuner live blogged the event.

There were a couple of things I was particularly interested in.  First was Ross' description of how Thatcham have moved to Scala and Akka for the services they supply.  There's a write up, but highlights include the stability they have, on just a couple of servers, and the minimal amounts of code they have.  

I loved the module to integrate with Camel, meaning you can do things like wait for a file to be dropped in a directory and have the contents appear as a message to your actor. Or listen for a POST to a URL, meaning the content is delivered to your actor as a message.  Or JMS queue, which turns into...? Yes, a message to your actor.  You get the idea.  Goodbye integration, hello getting stuff done.

Impressive, rich open source product, with commercial support if you need it.

Links:

• The video and slides are online.  
• Actors in Scala by Philipp Haller and Frank Sommers published by Artima.
• http://akkasource.org/

(download)

Click here to download:

scala-lift-off-london-was-wonderful-gaIwmpBCBiJfdFjgymzs.zip (3.82 MB)

I just spent two days at the Scala Lift Off unconference, meeting and chatting with 100+ old and new friends, hearing what everyone is interested in or learning about or doing. Plus the luminaries of the field were there to join in, answer questions, discuss what can be changed and how.  Think about that for a second: it's pretty much the best possible world, and it's really your fault if you can't get something good out of that :-)

I'm not going to give an account of the sessions.  I'm sure others will do a good job of that, and also there's the unconference wiki, photos tagged with scalaliftoffoct78, tweets tagged with #scalalol and videos on the Skillsmatter Vimeo site.    A few small factoids I picked up...

• Happy to see many hands go up in answer to the "who has Scala in production?" question. 
• Scala Solutions are adding their effort to the Scala IDE: making Eclipse+Scala better is seen as a good way to help Java developers over to Scala. 
• Parallel collections to ship in Jan 2011.  Making use of those multicores.
• I should look at Vector and HashMap and not just List, as they are efficient and persistent in 2.8
• sbt and lifty give a very nice way to start and build Lift projects.
• There was lots of interest in Akka.  I didn't hit any of those sessions, but this Wednesday the London Scala User Group is hosting "Simpler Scalability, Fault-tolerance, concurrency and remoting through Actors" at 6:30pm.
• There's going to be a Lift training course in London.

For me the most impressive session was the one where all the scary functional words were listed on a white board ("applicative functor", "monads"...)  and then the group tried to define them and give examples. 

If any of this sounds appealing, keep an eye on the Skillsmatter Scala page, because there's plenty happening you might like.   

See you at Scala Liftoff London 2011 (Oct 6 and 7, registration now open). 

We're all looking forward to Tim Perrett's Lift in Action book, but until Chapter 6, Testing, is done the main resource for testing in Lift is Vassil Dichev's blog post and the entry in the Lift wiki.  If all you need to do is test Mapper, though, an in-memory database works just fine.

Here's what we've used:

$ cat InMemoryDB.scala

package com.spiralarm.model

import net.liftweb.mapper._
import net.liftweb.common._
import net.liftweb.util._
import java.sql._

object InMemoryDB {

 val vendor = new StandardDBVendor("org.h2.Driver", 
    "jdbc:h2:mem:lift;DB_CLOSE_DELAY=-1", Empty, Empty)

 Logger.setup = Full(net.liftweb.util.LoggingAutoConfigurer())
 Logger.setup.foreach { _.apply() }

 def init {
  DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
  Schemifier.destroyTables_!!(Schemifier.infoF _, Your, Classes, Here)
  Schemifier.schemify(true, Schemifier.infoF _, Your, Classes, Here)
 }

 def shutdown {
  // TODO: figure out if anything goes here
 }
}

Effectively we create an in-memory H2 database, and use Schemifier to drop our tables and recreate them with no data.  You can use something like the above in tests pretty easily...

import org.specs._
import org.specs.matcher._
import net.liftweb.mapper._
import com.spiralarm.model._

class MyTest extends SpecificationWithJUnit {

 "My thing" should {
  "store stuff in the database" {
   InMemoryDB.init
   var person = Person.create.first("Billy").last("NoMates").saveMe
   var friends = Friends.find(By(Friends.person, person))
   friends must_== Empty
  }

 }
}

...or something like that, with a less fake example that actually compiles.  If you're doing a lot of this, you may want to have the in-memory initialized (and possibly shutdown) automatically around each test, and generalize to pass in the classes you care about to be Schemifier-ed.  But you get the idea.

I have my ticket. How about you?  It's £150 at the moment.

What we're looking at here is a two day unconference, meaning we get a chance to learn about things we're interested in by figuring it out on the day, talking to people, rather than being lectured at.  I thoroughly enjoyed the Scala Lift Off last year in San Francisco, so I'm looking forward to the London one.  Especially as it close by, rather than 8,720km away.

Play is a MVC, convention-based, stateless web framework for Java with growing support for Scala too.

It's not for me as I can't face going back to MVC and the kinds of presentation languages they use. Having said that, if you like MVC, and you're not already using Grails or Rails or a similar framework, I strongly urge you to look at Play as there's some nice technology in there. 

Everything I know about Play comes from Rustem Suniev's talk for the London Scala User Group at Skilsmatter on Wednesday. The slides and video are already on-line and they contain a really nice live-coding demo of Play which gave me a good sense of what the framework is about.  Nice work—and pizza courtesy of autoquake.com (who are hiring).

One comment I will make is that Play pushes it's stateless-ness prominently. For many of us I suspect our default position is that stateful=bad and stateless=good.  That sounds sane, but you probably do need some state in your application, and you have to deal with it somehow, or push the issue somewhere, which leaves me feeling that state v. stateless thing is all a bit more complicated that we often think it is.  It certainly does not automatically mean better scaling or performance, but there are definite positives to it.  I'm glad to see the Play community discussing state and exploring some nice ideas.  Just don't assume a label of "stateless" solves all your scaling problems—if you're lucky enough to have any :-)