Cold Code

python winpdb automation

2012-01-25T03:33:00.000-08:00

I wasted some time trying to automate the launching of winpdb without asking the user for the password; hoping that what I found out will help somebody:

Passing the password to winpdb with -p is unsupported on unix, because of security issues (the command line parameters can be seen by anyone). However there is an undocumented --rid parameter which takes an integer which is used to construct a file name which will contain the password (beware: no newline after the password). winpdb will delete this password file after reading it. You should create this file it with appropriate permissions.

Here's a bash haiku to show what I mean:

 RID=$RANDOM; echo -n 'test' >/home/marko/.rpdb2_settings/passwords/$RID; winpdb --rid $RID --attach yourscript.py

golang interface is not nil even if 'containing' a nil pointer

2011-06-06T14:12:00.000-07:00

Today I was struggling with a weird issue with "err != nil" during a refactoring of the go9p library. Here is a simplified situation:

This prints:

"Test *main.Test 0x0 dummy"

here 'err' is not nil, even if nil is actually returned.

The problem arised because go9p has an inner API returning some custom *Error, which is then 'casted' to a os.Error interface in the public API layer, so that it can implement the io.Reader,... interfaces. All code using this standard API was getting errors even if no error was returned.

From what I understood, it goes this way: *Test implements os.Error. *Test can be nil since it's a pointer.
os.Error is an interface value. Any interface value can also be nil.

When a value of type X is put* in a variable (or return value) declared as interface Y, then a new value is created holding X. In our case a value of type os.Error contains a *Test, both of which can be nil.

"if err != nil" checks whether 'err', the interface value, is nil, which is not. "%T" fetches the runtime type of err, which skips (confusingly) the interface envelope and prints *main.Test.

"%p" also skips the interface value, probably because it's not really a pointer (or perhaps because it sounded more natural to follow what the underlying real value points to).
Lastly "%s" invokes the String() method, which is implemented for *Test; here "self != nil" is true because here we have the real underlying pointer.

So, assuming I understood correctly what's happening, we can generalize this issue to:

This is because an interface value has to have a "zero" value, and it's nil. If you assign a pointer which implements that interface, that pointer can point somewhere or be nil, it doesn't matter, it's a valid pointer; and if you assign this pointer to the interface value, that interface value now has a value which is not nil.

I imagine that otherwise it would be impossible to have a zero value for the interfaces:

So, in order to fix this code you have to catch and "rethrow" the error:

This might seem a big issue, but usually this kind of code already checks for errors and returns so you just have to pay attention to not assigning to a previously declared interface value like os.Error, for example here "err" was inferred as os.Error in a previous ":=" assignement and then used to hold a *Error:

scala generics and eclipse autocompletion

2011-01-25T07:52:00.000-08:00

The Scala 2.8.1 compiler doesn't create correctly the generics signature in some cases

https://lampsvn.epfl.ch/trac/scala/ticket/4067

Eclipse 3.6 is very strict about the signatures and it's autocompletion feature throws an exception when you work on a java project using scala code:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=332423

The issue is being fixed in both projects, but in the meantime I needed to work in eclipse with the akka library, and this issue is annoying.

I wrote a small tool which tries to fix the scala classes. Currently I tested it only on scala-library.jar, which contained the "orElse" method whose signature affected my setup. I don't know if it fixes all the problems, but I tried it on my project and eclipse doesn't complain.

For now I only tried to put that jar in the eclipse IDE classpath, while keeping the original scala-library.jar in the executable classpath.

You can get the code at:

https://github.com/mmikulicic/scala-fix-generics

Ballerina

2011-01-13T10:34:00.000-08:00

A colleague of mine told me today that that his girlfriend woudln't believe that there was an optical illusion in the animated dancer image that you can find at: http://dl.dropbox.com/u/2520452/ballerina/ballerina.gif

I was just about to play with html5 canvas drawing, and so I wanted to try out to pilot the illusion with a small red ball which can rotate in both directions, following an elliptical path (an illusion of perspective) and appearing to disappear when going "behind" the dancer.

I don't know if it works for all, but for me this simple (and unprecise) animation, combined with the psycologial effect introduced by clicking on a "change direction" button, swaps the dancer's direction immediately in my head, or at least within one rotation.

Click here to se a demo

Source code on github

scala specs functional matchers

2010-12-13T15:28:00.000-08:00

I was reading http://code.google.com/p/specs/wiki/MatchersGuide and wondered if new matchers could be defined more tersely than:

So here it is, with a couple of implicits:

The example shows how to promote a function to a matcher. Currently I didn't find how to define
a function literal with call-by-name semantics, but for simple matchers where a call-by-value semantic is acceptable this trick works fine.

Haml macro for clojure

2010-10-10T15:45:00.001-07:00

http://github.com/mmikulicic/haml-macro

I made a small project which implements a clojure macro that reads a haml file and generates hiccup markup.

The parser is more or less compatible with the ruby haml parser, except the clojure expressions instead of ruby.

An example:

  %table.pane(cellspacing="0")
   %tr
   - (for [side-name ["left" "right" ]])
   %td(class="side-#{side-name}")
   %table

TracAutoSessionMail

2010-08-04T08:36:00.000-07:00

http://github.com/mmikulicic/tracautosessionmail

This is a trac plugin which automatically adds the anonymous reporter email into the user's session.

This is useful when you want to make a helpdesk trac instance, open to anonymous users. They will put their email in the "reporter" field, pass a captcha test, and
submit tickets. The problem is that, whenever they want to comment or make another ticket, they are required to type their email over and over again, unless they
go to 'preferences' and set their session's email address, which is unlikely to happen for 'normal' users.

This plugin simply adds the reporter email address into the session email field, when an anonymous user creates a ticket, and it doesn't already have a session email address.

nvidia auto display

2010-05-10T19:14:00.000-07:00

I've created a little shell script which allows me to avoid continuously struggling with nvidia-settings in order to manually switch monitor resolution on my MacBook pro 5,3 running ubuntu linux.

http://github.com/mmikulicic/nvidia-auto-display

you plug your external screen and it automatically switches to it, changing resolution and resizing the desktop
useful on laptops with nvidia graphic card and nvidia proprietary drivers
behaves more or less like on MacOS X
works also after standby

Hope you find it useful.

Scala 2.8 Maven and continuous compilation

2010-04-21T08:39:00.000-07:00

Scala looks very interesting, and I wanted to give a try to akka, hoping to be able to push erlang style practices in a JVM oriented workplace.

I one of the key features I want is to be able to use jrebel to hot update by running scala code during development. Another important requirement is to run scala 2.8.0 (beta or rc1) because akka 0.8.x requires it and I don't want to suffer later on from migration issues; I hope it's safe since for the time I've learned scala, 2.8.x will probably be mainstream.

I invested some time to learn sbt (looks really promising and fast) but at the end I was not feeling comfortable using it because of a couple of issues regarding unit test detection and dependency management issues. So for now, let's start with maven.

But with maven I had to:

MAVEN_OPTS="$MAVEN_OPTS -noverify -javaagent:/home/marko/bin/ZeroTurnaround/JRebel/jrebel.jar" mvn scala:console

because the documented method doesn't work (because the scala maven plugin doesn't fork a new virtual machine, see issue ...)

Ok, now whenever my scala sources are recompiled I can see hot updates in my scala console! Great stuff.

Eclipse can recompile each source file as soon as it's saved but, unfortunately the eclipse scala plugin for 2.8 (I tried both snapshot and -rc1) is not very stable and there are tons of issues which will be fixed soon, but I want to start to focus on scala and reduce the impact of immature tooling.

So I had to delegate this continuous recompilation to command line tools and use my programmer's editor of choice.

Both sbt and maven support continuous compilation. Sbt worked great out of box with "~ compile" but I preferred being able to use only maven to do the job.

Maven scala:cc goal has some issues with the compilation daemon:

[INFO] [scala:cc {execution: default-cli}]

[INFO] Checking for multiple versions of scala

[INFO] use fsc for compilation

[INFO] stop server(s)...

[No compilation server running.]

[INFO] start server...

[INFO] wait for files to compile...

[INFO] /home/marko/tmp/scala/scalatest/src/main/scala:-1: info: compiling

It turned out that it happens because there is an executable /usr/bin/fsc from the ubuntu scala 2.7.5 installation. If I remove that:

[INFO] [scala:cc {execution: default-cli}]

[INFO] Checking for multiple versions of scala

[INFO] use fsc for compilation

[INFO] stop server(s)...

[No compilation server running.]

[INFO] start server...

[INFO] wait for files to compile...

[INFO] /home/marko/tmp/scala/scalatest/src/main/scala:-1: info: compiling

[INFO] Compiling 1 source files to /home/marko/tmp/scala/scalatest/target/classes at 1271863673692

[INFO] Cannot start compilation daemon.

[INFO] tried command: List(scala, scala.tools.nsc.CompileServer)

[INFO] prepare-compile in 0 s

[INFO] compile in 0 s

[INFO] wait for files to compile...

In the meantime I tried to start my own fsc daemon (see gist.github.com/373965)

java -cp ... scala.tools.nsc.CompileServer

and it worked, and it's also incredibly fast at compiling (mvn scala:cc -Dfsc=false on the other hand works but it's very slow, it takes 3-4 seconds to compile)

So, basically all we need it provide an executable called "scala" which gets invoked by maven scala plugin with the scala.tools.nsc.CompileServer, parameter:

gist.github.com/373981

Now maven scala:cc works fine, and spawns a compile server as needed.

Next, I will try to do some mixed java/scala development, thrift stubs building, subprojects etc.

reverse proxy overriding content-type

2010-04-13T08:10:00.000-07:00

I had some links to images hosted on http servers which reported application/octet-stream instead of an 'image' mime type.

My application relied on that and I had to find a quick way to fix this by putting a reverse proxy in front of that particular server and override the content type.

I tried with no luck to do it with apache2 but it appears that there is no way to set the content-type response header after it's generated by the mod_proxy module.

So I tried out nginx, but it turns out that even nginx cannot force headers, but only add new headers

However the nginx docs point to a 3rd party module . Unfortunately the ubuntu nginx distribution didn't have this module compiled in, so I had to recompile nginx with this module built in. It turned out to be a simple operation.

I managed to reuse the same ubuntu nginx configuration files, by simply installing the custom nginx in /opt/nginx and symlinking ubuntu /etc/nginx to /opt/nginx/conf and changing the /etc/init.d/nginx script to point to /opt/nginx/sbin/nginx instead of /usr/sbin/nginx

then I simply added this line in my 'server' section:

more_set_headers 'Content-Type: image/jpeg';

Deploying a Git subdirectory in Capistrano

2010-03-24T09:11:00.000-07:00

Many suggest applying a patch to capistrano but for me this isn't an option.

I found this solution which works:

stackoverflow

Perhaps this can be refactored in vendor plugin receipt

apache CXF and .NET

2010-03-15T07:26:00.000-07:00

I had some troubles finding out why .NET (3.5 RC1) svcutil wasn't able to invoke my java CXF web service. Hope this may help you.

The error was:

exception = {"There was an error reflecting 'return'."} InnerException = {"The Form property may not be 'Unqualified' when an explicit Namespace property is present."}

It turned out that (besides mandatory namespaces) it was also caused by the fact svcutil doesn't like encoding arrays/lists as:

<xs:element form="qualified" maxoccurs="unbounded" minoccurs="0" name="linkedGroups" type="tns:group">

but it requires a nested element:

<xs:element form="qualified" minOccurs="0" name="linkedGroups">
  <xs:complexType>
     <xs:sequence>
        <xs:element form="qualified" maxOccurs="unbounded" minOccurs="0" name="item" type="tns:group"/>
     </xs:sequence>
  </xs:complexType>
</xs:element>

This can be accomplished with the following jaxws annotations:

@XmlElementWrapper(namespace = "http://rmi.r2d2.dnetlib.eu/", name = "linkedGroups")
@XmlElement(namespace = "http://rmi.r2d2.dnetlib.eu/", name = "item")
public List<Profile> getLinkedGroups() {
  return linkedGroups;
}

This blog helped me http://kjellsj.blogspot.com/2008/03/wcf-datacontractserializer-schema-rules.html

lazy clojure couchdb

2010-03-01T08:10:00.000-08:00

lazy paged clojure couch db interface:

http://gist.github.com/318486

Clojure lazy sequence

2010-01-26T09:08:00.000-08:00

I want to share my experience with lazy chunked sequences of clojure 1.1 and threads.

I wrote some code that used 'futures' to execute parallel IO, potentially in a large number of background threads (about 200).

I noticed that my code wasn't behaving like I expected and I was wondering if clojure executed the futures in some unbounded thread pool, or if it had some fixed maximum. So I wanted to try it out:

(map deref (map #(future-call (fn [] (Thread/sleep 1000) %)) (range 20)))

this actually created 20 threads (I saw it with jconsole), and returned to the REPL within 1 second. So far so good. However:

(map deref (map #(future (Thread/sleep 1000) %) (range 200)))

this took about 6 times more to execute. I also noticed a strange behaviour in thread creation. New threads were only created in ... chunks of 32....

Well now it seems obvious for me, but I didn't realize that I stuck upon the new clojure chunked lazy evaluation feature. The correct code is:

(map deref (doall (map #(future (Thread/sleep 1000) %) (range 200))))

Without the doall, only the first 32 futures are evaluated and actually submitted to the cachingThreadExecutor that sits behind the "future-call" core function.

In order to avoid this kind of errors in future, I created a simple helper:

(defn future-map [f seq]
  (doall (map #(future-call (fn [] (f %))) seq)))

to be used as:

 (future-map do-something asequence)

This sounds like a 'pmap' parallel map, but AFAIK the 'pmap' stuff was intended as a performance enhancement, and as such it tries to use a reasonable number of threads in order to exploit the available CPUs, a different kind of requirement from what I needed.

clarsec

2010-01-08T09:43:00.000-08:00

I attempted to port Haskell monadic monadic parsing library Parsec to clojure

http://github.com/mmikulicic/clarsec

for now it's very basic but I already for work.