Tag Archives: functional programming

Open source project breakdown.

I realised today that I actually have a fairly large number of open source projects published online (all on google code. Another thing I realised is that I should fix that).

I also realised that some of these are totally defunct.

I thought this would be a good time to do a quick breakdown of them, explaining what’s there, what they do and what their current status is.

JTypebuilder

Code generator for creating immutable data structures in Java. The idea was to define simple datatypes with a record notation and get an immutable class from them with correct equality, hash code and toString implementations, getters for the properties and a builder class for generating instances.

Status: Very very dead. It was at best a weak idea, and I have no interest in pursuing it. Use Scala’s case classes instead.

Generators4j

A brief foray into functional programming in Java. Lazy generators with functions like map, filter, etc.

Status: So dead. I didn’t get very far before concluding that trying to do this in Java was unusably awful.

Lazy Strings

Experiments in efficient representation of String types, with the aim being to provide a drop in replacement for java.lang.String with a different set of performance characteristics. Started in Java, moved to Scala.

Status: Just resting its eyes. I’m not doing much with this at the moment, but I occasionally peek at it and will probably factor out some of the ideas and turn it into a more tightly focused library.

Ranged Types

Very small Scala library for statically checked numeric ranges.

Status: Awaiting a use case. I occasionally think about picking it up again, but then I wonder why. It’s a fun idea, but I don’t actually have anything I need to use it for and as far as I can tell neither does anyone else.

SBinary

A small library for binary serialization and deserialization of Scala data types, based on Haskell’s Data.Binary.

Status: Very much alive. I’ve just released version 0.1 RC1, am using it as a dependency in other things and am continuing to tinker with it to improve its usability.

Prefer Scala

A wrapper around the Java preferences API designed to be nicer to use from within Scala and support a wider variety of preferences in a typesafe way. Uses SBinary to serialize Scala types to and from the preference backing store. It’s been factored out of the code for Hector’s Reminder Service.

Status: Fledgling. I’ve only just released it. It’s very small, and I intend it to remain so, so I expect to push it towards a 1.0 fairly quickly and then have it enter maintenance mode where future updates are just to fix bugs and bring it into line with the latest versions of its dependencies.

Hector’s Reminder Service

Unlike the other ones, this one is an application. It’s a small cross platform status bar application based on QT which gives you reminder messages on a semi-regular basis. Designed to be unobtrusive and simple and intended for the occasional casual reminder rather than of specific events. Uses “Prefer Scala” for persisting of state between application runs.

Status: Again, quite recent. I have a semi-official version released which works and more or less does what I want. I’m intending to polish that, add a very small number of new features (currently planned are a more expressive way of specifying message intervals, the ability to temporarily suppress a message group and possibly a simple API for other programs to interact with him) and then declare it to be feature complete. Once it’s reached that point it will enter a similar state of “Updates are only to fix bugs and match new dependency versions”.

Learning Scala

Some questions for people who are learning / have learned Scala: What languages did you know beforehand, and how easy did you find learning Scala in comparison to these? Are there any languages which you found knowing particularly helpful when picking up Scala?

An explanation follows:

Scala seems to be a relatively hard language to learn for some people, not so much for others. Part of this is its complexity – it really does have a lot of little features – but I’m wondering if more of it might be its approach. It’s a language with two major inspirations – object orientation (in the peculiar flavour of it Java practices) and statically typed functional programming, and I’m not sure how easy it is to understand the language unless you understand where it’s coming from in this regard.

In particular one thing we’ve observed in #scala from people learning the language is that if you know both Java and Haskell (I presume an ML would work as well?), learning Scala becomes significantly easier. I had almost no trouble picking it up, but I know both. Ricky Clarkson seems to be in a similar boat in terms of Haskell + Java having helped. I presume others are too. On the other hand, people with Java background but not much FP seem to have more trouble and people coming from a predominantly ruby or python background have a harder time yet. (I don’t know what happens to people coming from a Haskell with no Java background. I’d expect a similar degree of confusion to the Java with no Haskell background).

Some of this is probably in terms of material – a lot of Scala tutorials, etc. out there seem to assume you already know Java. This is probably largely accurate but seems like a mistake in the long-term to me. On the other hand, I’d be really uncomfortable teaching Scala as a first language, so what languages *should* they be learning to prepare the way? Anyone tried learning it on the basis of, say, Ruby + OCaml?

So, what do we want people’s path into Scala to be? Should we suggest they learn Java first if they don’t want a bit of a rough start, or is there a better way?

This entry was posted in programming and tagged , , , on by .

Why not Scala?

I thought I’d follow up on my previous post on why one would want to use Scala with one on why you wouldn’t. I’m definitely planning to continue using it, but it would be dishonest of me to pretend it was a perfect language.

I’m not going to cover the usual ones – weak tool support, difficulty of hiring Scala programmers, etc. These are pretty standard and will be true in most ‘esoteric’ languages you care to name. They’re certainly important, but not the point of this post. I’m just going to focus on language (and implementation) issues.

You’re looking for a functional language

Scala is not a functional programming language. It has pretensions of being so, and it has adequate support for functional programming, but it only goes so far. It’s got better support for functional programming than C#, Ruby, etc. but if you compare its functional aspects to ML, Haskell, OCaml, etc. you’ll find it sadly lacking. Problems include:

  • Its pattern matching is really rather cumbersome.
  • An annoying distinction between methods and functions. Scala’s first class functions are really no more than a small amount of syntactic sugar around its objects. Because Scala’s scoping is sane this isn’t particularly an issue, but it occasionally shows up.
  • The handling of multiple arguments is annoying. It doesn’t have the pleasant feature of Haskell or ML that every function has a single argument (multiple arguments are encoded as either tuples or via currying). Admittedly this isn’t a prerequisite of a functional language – e.g. Scheme doesn’t do it – but it’s a very big deal in terms of typing and adds a nice consistency to the language. I’m not aware of any statically typed functional languages which *don’t* do this (although the emphasis between tupling and currying varies from language to language).
  • Almost no tail call elimination worth mentioning. A very small subset of tail calls (basically self tail calls – the ones you can obviously turn into loops) are eliminated. This is more the JVM’s fault than Scala’s, but Martin Odersky himself has shown that you can do better (although admittedly it comes with a performance hit).
  • The type inference is embarrassingly weak. e.g. recursive methods won’t have their return type inferred. Even what type inference is there is less than reliable.

Compiler stability

The compiler is buggy. It’s not as buggy as I sometimes get the impression it is – I’ve definitely claimed a few things to be bugs which turned out to be me misunderstanding features – but it’s buggy enough that you’ll definitely run into issues. They’re rarely blockers (although sometimes they are. Jan Kristen has run into a few with his recent experiments with wicket + scala), but more importantly the bugginess means you really can’t trust the compiler as much as you’d like to. When something goes wrong it’s not always certain whether it’s your fault or the compiler’s. This is a big deal when one of the selling points is supposed to be a type system which helps you catch a wide class of errors.

Language consistency

The language has a lot of edge cases. These can be really difficult to wrap your head around, and can be really annoying to remember.

Let’s take an example. Variables. Simple, eh? Well, no.

A variable (local or field) can be a function (or constructor) parameter, a val, or a var. A val is a definition – it can’t be assigned to after the definition is made. A var is a normal mutable variable like in Java. A function parameter is almost like a val, except for the parts where it isn’t. Additionally, a function parameter can also be a var or a val. But it doesn’t have to be. Variables can be call by value (normal), call by name (the expression is evaluated each time you reference its value) or lazy (the expression is evaluated the first time you need its value and never again). But only vals can be lazy. And function parameters can’t be lazy, even if they’re also vals (I don’t understand this one. It seems obviously stupid to me). Meanwhile, only function parameters can be call by name – you can’t assign them to vars or vals (a no argument def is the equivalent of a call by name val).

Clear as mud, eh? Now, granted I wrote the above to make it sound deliberately confusing (it’s probably owed a blog post later to make it seem deceptively simple), but it’s a fairly accurate representation of the state of affairs.

Here’s another one (it’s related to the arguments issue). Consider the following snippet of code:

def foo = "Hello world";
println(foo());

def bar() = "Goodbye world";
println(bar);

Pop quiz: Does this code compile? If not, which bit breaks? No cheating and running it through the compiler!

Answer: No, it doesn’t. Because foo was defined without an argument list, it can’t be invoked as foo(). However, despite bar being defined with an (empty) argument list we can invoke it without one.

I could keep going, but I won’t. The short of it is that there are a lot of these little annoying edge cases. It seems to give beginners to the language a lot of grief.

Too much sugar

Scala has a lot of syntactic sugar. Too much in my opinion. There’s the apply/update sugar, unary operators by prefixing with unary_, general overloaded assignment (which, as I discovered when testing, only works in the presence of an associated def to go with it. Another edge case). Operators ending in : are left associative. Constructors are infixed in pattern matching case classes but not in application. etc. It’s hard to keep track of it all, and most of it is annoyingly superfluous.

Lack of libraries

Yes, yes, I know. It has all of the Java libraries to play with. And this is great. Except… well, they’re Java libraries. They’re designed with a Java mindset, and they can’t take advantage of Scala’s advanced features. Implicit conversions, and a number other tricks, are quite useful for making an API more palatable, but there’s a strong danger that what you end up with isn’t much more than Java with funny syntax. Much more than that requires a reasonable amount of porting work to get a good API for your use.

All in all, I find these add up to just a bunch of annoyances. It’s still my preferred language for the JVM, but depending on how you wait your priorities they might be more significant for you. Even for me I occasionally find myself getting *very* irritated with some of these.

This entry was posted in programming and tagged , , , , on by .

No, seriously, why Scala?

Recently an article called Why Scala? was posted on reddit. It’s an ok introduction to the language, but the very fair observation was made that it’s much more of a “What is Scala?” than a “Why Scala?”. I thought I’d share my thoughts on the subject. Mainly because I like hearing (reading) myself talk (write). :-)

Quick background: I initially learned to program in standard ML while at university (self taught, mostly, with the help of some friends doing computer science. I was doing maths). On graduating I then switched tracks entirely and started doing Java web development in a small software firm in London (I’ve since switched again, but I’m still doing Java professionally). I’ve also dabbled and read a lot with computer science and programming languages in my spare time since then, filling in the gaps that not having done any real computer science at university left me.

My train of thought on the language switch from ML to Java was basically:

a) Wow, this is different.
b) Ugh. Where are my higher order functions?
c) Where are all these random exceptions coming from?? I’ve compiled the code successfully, isn’t it supposed to work now?
d) Hmm. But there’s some useful stuff here too.

Scala’s a nice way to scratch both itches, and adds some very interesting features and functionality of its own. It’s not my favourite language (I don’t really have one. All languages suck. It’s just that some of them suck less in interesting ways), but it has a lot I like. Here’s a brain dump of some of it.

Things I like:

Object oriented programming

I know, it’s so entrenched it’s past even bothering with its buzzword status. But object oriented programming has a lot of advantages for medium to large scale composition. It has some disadvantages, and frankly sucks at small scale composition of functionality (which is where functional programming shines), but it allows for some very nice pluggability.

Module oriented programming

ML has higher order modules. I never really used them much when I was programming it more often (mostly because I was only writing enough code to do some simple maths projects. I never wrote anything large scale), but having looked into them in more details since they’re really powerful. They’re essentially a different take on the composition that object orientation provides. Where object orientation resolves everything dynamically, ML’s higher order modules resolve everything statically. This introduces some limitations in flexibility but makes up for them in power and type safety – they provide a much more flexible and interesting abstraction over a type than mere subclassing and interfaces can.

Scala has both. Further, it has both and lo and behold they are the same thing. Objects are modules, and can declare their own types (note: This is much more than just declaring an inner class in Java is), imported, etc. Modules are objects and can be instantiated at runtime, extended, etc. You lose a bit of the static guarantees that ML modules but you gain a lot of flexibility from both sides.

Static Typing

I’ve written too much Java to not like static typing.

Wait, I know that sounds like a non sequitur, but read on.

I’ve written too much Java and seen flagrantly stupid and really subtle runtime errors that should never have made it past the compiler coming out of it to not like static typing. NullPointerException, ClassCastException, argh.

If you’ve written enough code in a language like ML, OCaml or Haskell you will know that the compiler is your friend. And, like all good friends, it will yell at you if you do something stupid and then help you pick up the pieces.

Scala doesn’t quite manage that. If you write code in just the right way you can achieve that level of guarantee (and in some cases, more. But that tends to be the result of abuse of the type system by deranged maniacs), but the combination of subtyping and some Java interoperability decisions mean that it’s not quite as good. It’s not bad though.

So: I like object oriented programming, I like static typing. It logically follows that I must like statically typed object oriented languages, right? Well, in principle, yes. But Scala is the first one I’ve met with a type system that didn’t suck. Scala’s traits (a sort of mixin) are so much better to work with than interfaces, the generics work properly, provide variance annotations, etc. A reasonable subset of the types are inferred. Compared to the type systems of Java, C# and C++ it’s a dream (it’s not as nice as the type systems of the statically typed functional languages I know of. Subtyping seems to cause issues, with a lot of research still needed to make it work well, and Scala seems to have largely ignored what prior work there was Hindley-Milner style type systems with subtyping)

Functional programming

You’ve all been dreading this section. “Oh no. Now he’s going to enthuse about how marvelous functional programming is and how it’s going to cure cancer”. Nope. Can’t be bothered. Functional programming is nice. If you don’t believe that, I’m not going to try to convince you of it. Scala’s support for functional programming is ok. It has some warts, but it also has some nice points, and it generally works well and isn’t too verbose. I’m not going to get any more excited about its presence than I am about the fact that my bike has wheels (but I’d be pretty pissed off if my bike didn’t have wheels). Higher order functions, pattern matching, etc. It’s all there. It works. Moving on swiftly…

Implicits

Scala offers a bag of features under the keyword ‘implicit’. This is one of those things that makes you go “Oh, that’s cute” when you first see it and then go “Wow, that’s powerful” six months later.

Essentially implicits give you statically guaranteed and provided dynamic scoping. You say “I need a Foo. I don’t care where it comes from”, the compiler says “Here you go” or “Sorry, no Foos today”. These can be objects, implicit conversions between types (You know the way Ints get implicitly converted to longs, double, etc in Java? Scala does that too, but it’s all programmer definable. They’re just library functions in scala.Predefined). If you remember what I said about Scala objects being modules and you’ve read this paper a little light might just have gone on in your brain. If you haven’t read it and don’t want to, here’s the summary version: Implicit function arguments + first class modules gives you something that looks and quacks very much like Haskell type classes (yes, I know this isn’t actually what the paper says, but it follows from it). Mmm.

These are the big things to like about Scala. Here are a few little things:

  • Sane constructor/class semantics. If you’ve written a lot of Java there’s a good chance you hate its constructor system. Scala’s is much nicer.
  • Expression oriented code. Everything is an expression. You can form compound expressions trivially – { val foo = bar(); baz(foo, foo); } is an expression which evaluates to baz(foo, foo).
  • Sanely uniform scope. Pretty much anything you can do inside a method you can do inside an object and vice versa. Things are for the most part lexically scoped in the right way.
  • The primitive/object divide is much less irritating. Primitives get a few special treatments at the language level, but mostly they’re just objects. When things should compile to use primitives, they do. When the primitives need to be boxed, they will be. It’s almost entirely transparent.
  • Performance. Scala generates very good (well. ‘good’. Java-like) bytecode, which means it gets to take advantage of most of the optimizations the JVM is willing to throw its way. Further it puts a reasonable amount of its own work into performing optimisations on the bytecode, etc so you get those nice juicy abstractions without much overhead. There’s essentiall y no performance penalty for choosing Scala over Java

etc.

Scala’s far from perfect. It has some syntactic weirdnesses, a few issues carried over from Java, a moderately buggy compiler and a host of little features and edge cases that are really hard to keep in your head. However, I find that these issues don’t actually do more than annoy you from time to time. The core language is powerful and very useful for just sitting down and writing good code in.

Don’t forget to fly

I saw this comic a while ago and it occurred to me during today’s HUG.

I think the analogy is obvious, but I’m going to spell it out anyway because I feel the need to rant about it. (As a side note, I know I’ve occasionally been guilty of what I’m ranting about. Hopefully I’ve stopped…)

You like functional programming. That’s great. You write C#/Java/C++/Brainfuck/PL-SQL/Malbolge during your day job. That’s a shame, but oh well. You and everyone else. I bet you really wish you could use folds/lazy evaluation/lightweight threading/COMEFROM statements in your work code. Great. Me too.

There’s a lot of neat stuff in functional programming (and in the better OO languages. And in logic programming. And in a wide variety of other things). So, use it. Go wild. Write code.

DON’T waste time posting endless blog posts about how closures are awesome and wonderful and here’s an example of how they might work in Java. “Here’s a nice bit of code I wrote” is one thing. “Here’s how to implement a for loop. Isn’t it awesome!!!”? Not so much. If you’re interested in something, use it. Don’t waste time thinking about how to shoehorn its features into a language you know far too well.

You can fly. Stop thinking about how great it would be to do so and go out and do it.

This entry was posted in programming and tagged , on by .