Wednesday, April 21, 2010

Scala 2.8 Maven and continuous compilation

Scala looks very interesting, and I wanted to give a try to akka, hoping to be able to push erlang style practices in a JVM oriented workplace.

I one of the key features I want is to be able to use jrebel to hot update by running scala code during development. Another important requirement is to run scala 2.8.0 (beta or rc1) because akka 0.8.x requires it and I don't want to suffer later on from migration issues; I hope it's safe since for the time I've learned scala, 2.8.x will probably be mainstream.

I invested some time to learn sbt (looks really promising and fast) but at the end I was not feeling comfortable using it because of a couple of issues regarding unit test detection and dependency management issues. So for now, let's start with maven.

But with maven I had to:
MAVEN_OPTS="$MAVEN_OPTS -noverify -javaagent:/home/marko/bin/ZeroTurnaround/JRebel/jrebel.jar" mvn scala:console
because the documented method doesn't work (because the scala maven plugin doesn't fork a new virtual machine, see issue ...)

Ok, now whenever my scala sources are recompiled I can see hot updates in my scala console! Great stuff.

Eclipse can recompile each source file as soon as it's saved but, unfortunately the eclipse scala plugin for 2.8 (I tried both snapshot and -rc1) is not very stable and there are tons of issues which will be fixed soon, but I want to start to focus on scala and reduce the impact of immature tooling.

So I had to delegate this continuous recompilation to command line tools and use my programmer's editor of choice.

Both sbt and maven support continuous compilation. Sbt worked great out of box with "~ compile" but I preferred being able to use only maven to do the job.

Maven scala:cc goal has some issues with the compilation daemon:


[INFO] [scala:cc {execution: default-cli}]
[INFO] Checking for multiple versions of scala
[INFO] use fsc for compilation
[INFO] stop server(s)...
[No compilation server running.]
[INFO] start server...
[INFO] wait for files to compile...
[INFO] /home/marko/tmp/scala/scalatest/src/main/scala:-1: info: compiling
It turned out that it happens because there is an executable /usr/bin/fsc from the ubuntu scala 2.7.5 installation. If I remove that:

[INFO] [scala:cc {execution: default-cli}]
[INFO] Checking for multiple versions of scala
[INFO] use fsc for compilation
[INFO] stop server(s)...
[No compilation server running.]
[INFO] start server...
[INFO] wait for files to compile...
[INFO] /home/marko/tmp/scala/scalatest/src/main/scala:-1: info: compiling
[INFO] Compiling 1 source files to /home/marko/tmp/scala/scalatest/target/classes at 1271863673692
[INFO] Cannot start compilation daemon.
[INFO] tried command: List(scala, scala.tools.nsc.CompileServer)
[INFO] prepare-compile in 0 s
[INFO] compile in 0 s
[INFO] wait for files to compile...

In the meantime I tried to start my own fsc daemon (see gist.github.com/373965)

java -cp ... scala.tools.nsc.CompileServer 

and it worked, and it's also incredibly fast at compiling (mvn scala:cc -Dfsc=false on the other hand works but it's very slow, it takes 3-4 seconds to compile)

So, basically all we need it provide an executable called "scala" which gets invoked by maven scala plugin with the scala.tools.nsc.CompileServer, parameter:

gist.github.com/373981

Now maven scala:cc works fine, and spawns a compile server as needed.

Next, I will try to do some mixed java/scala development, thrift stubs building, subprojects etc.

Tuesday, April 13, 2010

reverse proxy overriding content-type

I had some links to images hosted on http servers which reported application/octet-stream instead of an 'image' mime type.

My application relied on that and I had to find a quick way to fix this by putting a reverse proxy in front of that particular server and override the content type.

I tried with no luck to do it with apache2 but it appears that there is no way to set the content-type response header after it's generated by the mod_proxy module.

So I tried out nginx, but it turns out that even nginx cannot force headers, but only add new headers

However the nginx docs point to a 3rd party module . Unfortunately the ubuntu nginx distribution didn't have this module compiled in, so I had to recompile nginx with this module built in. It turned out to be a simple operation.

I managed to reuse the same ubuntu nginx configuration files, by simply installing the custom nginx in /opt/nginx and symlinking ubuntu /etc/nginx to /opt/nginx/conf and changing the /etc/init.d/nginx script to point to /opt/nginx/sbin/nginx instead of /usr/sbin/nginx

then I simply added this line in my 'server' section:

   more_set_headers 'Content-Type: image/jpeg';