Dereferencing operators

I’m writing a small library for mutable reference cells. This has spawned a heated debate about what to call the dereferencing operator. Possible options for dereferencing foo are:

One of the big questions is whether it should be postfix or prefix. If it’s postfix, using them as properties becomes much more readable. foo.bar! vs. !(foo.bar). But it also runs into weird precedence issues. On the other hand, the set of characters which can be used in a prefixy manner is really limited and they all seem to have significant meaning.

!foo

Pros: Historical precedent. It’s what ML uses.
Cons: Very easy to confuse with negation. Suppose foo is a reference to a boolean. if (!foo) { } is potentially really confusing.

foo!

Pros: Same as !foo. Less confusing – it’s not currently used by anything major.
Cons: Retains misleading association with negation, although less easy to write confusing code.

foo&

Pros: Historical precedent. Looks almost like C (prefix & isn’t legal).
Cons: Similar confusion to !. & more normally means and. On the other hand, C programmers seem to have gotten used to it.

@foo
Pros: Nice distinctive character. Easy to get used to.
Cons: It isn’t legal Scala (this is kinda a big one :-) ).

~foo
Pros: Same as @. Legal Scala. :-)
Cons: Prefix operator, so doesn’t work well with properties. Somewhat non-obvious.

foo<> (credit to Bob Jones… err. I mean Jan Kriesten for this one)
Pros: Visually distinctive and appealing.
Cons: Looks vaguely directional.

foo^ (credit to Martin Odersky)
Pros: Um. Beats me.
Cons: Confusion with xor. Looks weird.

foo deref

Pros: Fewer weird precedence issues because it’s not an operator. Some people seem to like wordy operator names.
Cons: Visually distracting, overly verbose. Scatters meaningless words throughout the code. Core operations should have nice symbolic notation.
Additional cons: Over my dead body.

foo() (credit to Eric Willigers)

Pros: Interacts much better with precedence rules than any of the others. You can write foo() == “Bar” whereas you’d have to write (foo!) == “Bar”. It seems intuitively obvious what invoking a reference should mean.
Cons: I don’t really have a good argument against this except that it feels wrong. It looks a little weird when you have a reference to a function. e.g. if you had a Ref[() => Unit] it would be potentially easy to write myRef() and think you’d invoked it, when in fact you’d merely returned a function.

Any of the above with an implicit conversion from references to their contents
Pros: The mainline case is syntax free.
Cons: No no no no no no no. This creates *exactly* the sort of confusion between reference cells and their values that I’m trying to avoid, and opens up the possibility of huge classes of subtle bugs where you passed a reference to an object and meant to pass the object. I initially thought it was a good idea, and it has a strong intuitive appeal to it, but I’m convinced it would be disastrous. A slight conciseness advantage in no way offsets the introduction of perniciously evil bugs.

On balance I think foo() is going to win. The precedence issues seem to prohibit the use of any sort of postfix operator. This seems to leave ~foo as the only good alternative, and I think it’s less obviously meaningful and the prefix nature would annoy the properties people.

Why not Scala?

I thought I’d follow up on my previous post on why one would want to use Scala with one on why you wouldn’t. I’m definitely planning to continue using it, but it would be dishonest of me to pretend it was a perfect language.

I’m not going to cover the usual ones – weak tool support, difficulty of hiring Scala programmers, etc. These are pretty standard and will be true in most ‘esoteric’ languages you care to name. They’re certainly important, but not the point of this post. I’m just going to focus on language (and implementation) issues.

You’re looking for a functional language

Scala is not a functional programming language. It has pretensions of being so, and it has adequate support for functional programming, but it only goes so far. It’s got better support for functional programming than C#, Ruby, etc. but if you compare its functional aspects to ML, Haskell, OCaml, etc. you’ll find it sadly lacking. Problems include:

Its pattern matching is really rather cumbersome.
An annoying distinction between methods and functions. Scala’s first class functions are really no more than a small amount of syntactic sugar around its objects. Because Scala’s scoping is sane this isn’t particularly an issue, but it occasionally shows up.
The handling of multiple arguments is annoying. It doesn’t have the pleasant feature of Haskell or ML that every function has a single argument (multiple arguments are encoded as either tuples or via currying). Admittedly this isn’t a prerequisite of a functional language – e.g. Scheme doesn’t do it – but it’s a very big deal in terms of typing and adds a nice consistency to the language. I’m not aware of any statically typed functional languages which *don’t* do this (although the emphasis between tupling and currying varies from language to language).
Almost no tail call elimination worth mentioning. A very small subset of tail calls (basically self tail calls – the ones you can obviously turn into loops) are eliminated. This is more the JVM’s fault than Scala’s, but Martin Odersky himself has shown that you can do better (although admittedly it comes with a performance hit).
The type inference is embarrassingly weak. e.g. recursive methods won’t have their return type inferred. Even what type inference is there is less than reliable.

Compiler stability

The compiler is buggy. It’s not as buggy as I sometimes get the impression it is – I’ve definitely claimed a few things to be bugs which turned out to be me misunderstanding features – but it’s buggy enough that you’ll definitely run into issues. They’re rarely blockers (although sometimes they are. Jan Kristen has run into a few with his recent experiments with wicket + scala), but more importantly the bugginess means you really can’t trust the compiler as much as you’d like to. When something goes wrong it’s not always certain whether it’s your fault or the compiler’s. This is a big deal when one of the selling points is supposed to be a type system which helps you catch a wide class of errors.

Language consistency

The language has a lot of edge cases. These can be really difficult to wrap your head around, and can be really annoying to remember.

Let’s take an example. Variables. Simple, eh? Well, no.

A variable (local or field) can be a function (or constructor) parameter, a val, or a var. A val is a definition – it can’t be assigned to after the definition is made. A var is a normal mutable variable like in Java. A function parameter is almost like a val, except for the parts where it isn’t. Additionally, a function parameter can also be a var or a val. But it doesn’t have to be. Variables can be call by value (normal), call by name (the expression is evaluated each time you reference its value) or lazy (the expression is evaluated the first time you need its value and never again). But only vals can be lazy. And function parameters can’t be lazy, even if they’re also vals (I don’t understand this one. It seems obviously stupid to me). Meanwhile, only function parameters can be call by name – you can’t assign them to vars or vals (a no argument def is the equivalent of a call by name val).

Clear as mud, eh? Now, granted I wrote the above to make it sound deliberately confusing (it’s probably owed a blog post later to make it seem deceptively simple), but it’s a fairly accurate representation of the state of affairs.

Here’s another one (it’s related to the arguments issue). Consider the following snippet of code:

def foo = "Hello world";
println(foo());

def bar() = "Goodbye world";
println(bar);

Pop quiz: Does this code compile? If not, which bit breaks? No cheating and running it through the compiler!

Answer: No, it doesn’t. Because foo was defined without an argument list, it can’t be invoked as foo(). However, despite bar being defined with an (empty) argument list we can invoke it without one.

I could keep going, but I won’t. The short of it is that there are a lot of these little annoying edge cases. It seems to give beginners to the language a lot of grief.

Too much sugar

Scala has a lot of syntactic sugar. Too much in my opinion. There’s the apply/update sugar, unary operators by prefixing with unary_, general overloaded assignment (which, as I discovered when testing, only works in the presence of an associated def to go with it. Another edge case). Operators ending in : are left associative. Constructors are infixed in pattern matching case classes but not in application. etc. It’s hard to keep track of it all, and most of it is annoyingly superfluous.

Lack of libraries

Yes, yes, I know. It has all of the Java libraries to play with. And this is great. Except… well, they’re Java libraries. They’re designed with a Java mindset, and they can’t take advantage of Scala’s advanced features. Implicit conversions, and a number other tricks, are quite useful for making an API more palatable, but there’s a strong danger that what you end up with isn’t much more than Java with funny syntax. Much more than that requires a reasonable amount of porting work to get a good API for your use.

All in all, I find these add up to just a bunch of annoyances. It’s still my preferred language for the JVM, but depending on how you wait your priorities they might be more significant for you. Even for me I occasionally find myself getting *very* irritated with some of these.

Variance of type parameters in Scala

This is just a quick introduction to one of the features of Scala’s generics. I realised earlier on IRC that they’re probably quite unfamiliar looking to people new to the language, so thought I’d do a quick writeup.

What does the following mean?

  trait Function1[-T1, +R]

It’s saying that the trait Function1 is contravariant in the parameter T1 and covariant in the parameter R.

Err. Eek. Scary words!

Lets try that again.

A Function1[Any, T] is safe to use as a Function1[String, T]. If I can apply f to anything I can certainly apply it to a String. This is contravariance. Similarly, a Function1[T, String] can be quite happily treated as a Function1[String, Any] – if it returns a String, it certainly returns an Any.

So, Foo[+T] means that if S <: T then Foo[S] <: Foo[T]. Foo[-T] means that if S <: T then Foo[T] <: Foo[S] (note the swapped the direction). The default Foo[T], called invariant, is that Foo[S] is not a subtype of Foo[T] unless S == T.

Examples of this sort of behaviour abound. Covariance is more common than contravariance, because immutable collections are almost always covariant in their type parameters. An immutable.List[String] can equally well be treated as an immutable.List[Any] – all the operations are concerned with what values you can get out of the list, so can easily be widened to some supertype.

However, a mutable.List is *not* covariant in its type parameter. You might be familiar with the problems that result from treating it as such from Java. Suppose I have a mutable.List[String], upcast it to a mutable.List[Any] and now do myList += 3. I’ve now added an integer to a list of Strings. Oops! For this reason, mutable objects tend to be invariant in their type parameters.

So, we have three types of type parameter: Covariant, contravariant, invariant. All three crop up and are quite useful.

But there are safe ways to treat mutable objects invariable. Suppose I want someone to pass me an array of Foos, and I have no intention of mutating it. It’s perfectly safe for them to pass me an array of Bars where Bar extends Foo. Can I do this?

Well, this can indeed be done. We could start by doing this:

  def doStuff[T <: Foo](arg : Array[T]) = stuff;

So we introduce a type parameter for the array. Because T will be inferred in most cases, this isn't too painful to use, but it can quickly cause the number of type parameters to explode (and you don't seem to be able to let some type parameters be inferred and some be explicitly provided). Further, we only care about the type parameter in one place. So, let's move it there.

  def doStuff(arg : Array[T forSome { type T <: Foo }]) = stuff;

This uses Scala's existential types to specify that there's an unknown type for which this holds true. This is effectively equivalent to the previous code, but narrows the scope of the type parameter.

The equivalent using Java style wildcards would be:

  def doStuff(arg : Array[? <: Foo ]) = stuff;

But this isn't legal Scala. This is unfortunately a case of Scala being more verbose than the Java equivalent. However, it's not all bad - because of the explicitly named type inside the forSome, you can express more complicated type relationships than wildcards allow for. For example the following:

  def doStuff(arg : Array[T forSome { type T <: Comparable[T]}]) = stuff;

And that's about it for variance in Scala. Hope you found it useful.

Amusing discovery of the day

Java bytecode lets you overload method names based on the return type as well as the argument types, even if the language doesn’t.

Hardware vendor shit list

In keeping with my recent stance on blog content, this has nothing whatsoever to do with programming. It won’t educate you about esoteric languages, you won’t learn good development practices from it, and frankly you probably won’t care about it.

This is a post about hardware companies who suck. This post has two purposes:

a) In the hopes that it will prevent someone else from having to deal with these issues.
b) A small measure of petty revenge against those who have SERIOUSLY PISSED ME OFF. It’s not like they’ll notice the loss of a few sales, but if this persuades even one person not to give them money for their crap it will make me feel better.

There’s a tentative c) that if anyone could give me a suggestion for fixing my laptop I’d be eternally grateful. :-) But I don’t really expect that.

The past six months have not been happy ones for hardware. My work machine has never worked right, and my personal laptop has mysteriously died. Twice. The incompetence of Mesh in dealing with the desktop’s problems has been rivaled only by the astoundingly poor communication skills and unreliable hardware of Asus.

First, Mesh.

The problem with the desktop is a very annoying one: The screen would intermittently go black for a second or two. At first it did this once every few hours and I wasn’t sure if it wasn’t just power saving mode or something. Eventually I verified that no, it really was happening, and it was happening in both vista and ubuntu, so it most likely wasn’t a software problem. As time went on the problem got progressively worse – these days the rate is several times per minute.

I eventually narrowed it down. The problem would only manifest in the exact following circumstances:

a) This exact PC. Any other one worked fine.
b) The exact brand and make of monitor. Brand and make you say? Want to know how I know this? It’s because they sent us three. Sending us the second one was quite reasonable – it looked like a monitor problem. After the second we pointed out that maybe this wasn’t such a productive use (however we didn’t complain too hard, as the monitor had developed several dead pixels. But they wouldn’t replace it for the dead pixels sitting smack in the middle of the screen. You need at least 5 before it’s a replaceable condition. More on this later.).
c) Only on DVI input.

c) was the fortunate point which meant that I’ve actually been able to get work done in the last six months. I dislike VGA, but it’s at least usable.

Speaking of the brand of monitors, here’s another shit list entry: Mirai, the brand of this monitor. Quite possibly the worst monitor I’ve ever used. Despite my best efforts and tinkering the colour has never been anything less than washed out. They’ve developed dead pixels, random image problems (my favourite was the “yellow line down the middle of the screen”. That was a fun one. The flickering effect was another nice touch).

Anyway, back to mesh and this peculiarly specific hardware problem. Very random, isn’t it? What’s the chance that two pieces of hardware picked out of the blue would demonstrate such a specific incompatibility?

Wait, picked out of the blue? That doesn’t sound right. Let’s try again.

What’s the chance that two pieces of hardware picked and provided specifically to work together would show such a specific incompatibility?

Having exhausted monitor options they concluded that maybe we should replace the motherboard (it had onboard video). And so they did.

Eventually.

It took more than a little trying and them sending no less than four people out (it might even have been five. I forget if we had one or two no shows). Between these four, they managed the following problems:

a) A no show
b) someone who came out with a very handy motherboard box. No motherboard mind you, but one out of two isn’t bad, right?
c) Someone who in his truly inspiring competence managed to bring not only a motherboard box but an actual motherboard contained within. Rapture! What service! Oops. Wrong motherboard. What service?
d) Finally someone who managed the simultaneous combination of box, motherboard and correct brand of motherboard. He didn’t arrive until 7:00 in the evening and seemed to think he was going to be visiting a home address, but oh well. Why be picky? He managed to replace the motherboard successfully and everything, we booted up the machine and the display appeared stable. Could it be that after all this time the problem was finally resolved??

For context, I should clarify that I was now working on a different machine. We’d cut our losses and moved me to another desktop (a Dell. I’m not a big fan of Dell, but I have to admit their customer service is great). This meant that I wasn’t directly observing the other machine, but we booted it up with the intent of testing it and once we were sure the problem was resolved putting it to another use.

At some point during the following day I got a question from someone who sits next to where I had the computer set up.

“David, did Mesh come to fix this last night?”

I moved around to talk to them. As I did I said something along the lines of “Yeah, they replaced the motherboard and it seems to have solved the… oh shit”

Flicker. Flicker flicker. Blink.

Which is about where we stand currently. I don’t know what we’re going to do now.

Now, Asus.

I went with Asus because I’d been told their laptops were very reliable. Yeah… not so much. Also their website is awful, their customer support is awful, and their communication skills lie somewhere between “awful” and “nonexistent”.

I placed the order on a weekend. I got a generic “thank you for placing an order” with a note about sending confirmation when they were ready to deliver. On tuesday I still hadn’t heard from them so I made a mental note to call them in the morning to ask when they were likely to deliver.

And arrived to find my laptop sitting on my doorstep. On the one hand, yay laptop. On the other hand, RANDOMLY LEAVING EXPENSIVE HARDWARE ON ONE’S DOORSTEP IS NOT COOL. Granted it was internal to my building, not on the street, so clearly someone had to have let them in and signed for it, but this is still not acceptable behaviour. But even setting that aside, the key point here is that there was no communication, no advance warning, nothing. If they’d just done exactly what they should have done and said they were going to and let me know that they were planning to deliver that day I would have planned accordingly.

Anyway, rather than tear them a new one, I decided to let it pass. After all, I had a laptop (and a reasonably nice one). No harm done.

A week and a half later the laptop turned into a brick. The power light came on when you pressed the power button, and some vague whirring noises, but nothing more than that. I called up asus technical support, reading their support procedures on the card with growing dread (it involved sending you a snail mail form to fill out if they decided that yes really you needed replacement hardware). Fortunately their procedures were 71% less insane than claimed, and the dreaded form was in fact an excel spreadsheet. Hurray.

Upon filling this out and getting confirmation I had to call an entirely different branch in order to arrange a pickup. When I did they asked me to call back later as the details had yet to come through. Apparently their inability to communicate extends to eachother as well as their customers. It’s good to know I’m not being singled out.

Eventually I persuaded them to pick up the laptop. In fairness it didn’t require *that* much persuading. They picked it up promptly once arranged.

And then silence.

Two and a half weeks later I finally found time to call them up and say “Soo… laptop. What’s going on with that?”. “Oh, yeah, we sent it out this morning. You should be getting it today”. Fine, great, I’m glad they sent it back. But again with the total non-communication. And slowness. Two and a half weeks is not a nice length of time to be laptopless. For additional failure to communicate, they gave no indication whatsoever of what the fault was. I should have enquired, but I couldn’t be bothered at the time. This was a mistake.

That was a couple of weeks ago – I forget how many, but definitely not longer than a month – and the absence of this knowledge is now highly relevant, because the laptop has just done it again. This time it appears to have been the result of leaving it on until it went into power saving mode – I fell asleep with it on, woke up four hours later to discover that I no longer possessed a functioning laptop. If anything it’s brickier than last time – I’m not even getting a power light when pressing the switch.

This is especially upsetting as it means I will have no laptop over the christmas period. This means no working on personal projects I’ve been putting off due to work and, more importantly, a very limited amount of communication – in particular I’ll have to borrow my parents’ computer if I want to use Skype or IM (and no video at all. Sigh). It also probably means that you get a respite from my blogging. :-)

For added irritation (and admittedly I can’t blame this on hardware vendors, although if I could I’d try at this point), I seem to be running a moderately high fever. All in all this does not result in a happy David.