Improving Compiler Error Messages

written by Walter Bright

May 1, 2010

Compiler error messages are often an afterthought of compiler writers, focusing instead on compilation speed and the performance of compiled code. But the error messages have a strong influence on the day to day usability of a compiler, and are often a source of derision and frustration. Compilers are often built like Ferraris, with great attention to performance and relatively little for driver comfort.

Lately, the clang compiler project has revived interest in improving compiler error diagnostic messages, so I thought it would be fun to talk a bit about my experience with them, what works and what doesn’t.

Presentation

At a minimum, the error message should contain the file name, line number, and an explanatory message. Back in the 1980’s, I thought I’d do this one better and have the compiler also emit the offending source code line with a ^ under where it went wrong, like:

int x = y + 3;
          ^
test.cpp(2) : Error: undefined identifier 'y'

Great idea, right? I certainly thought so, and waited for the accolades to roll in. And waited, and waited... Turns out, it’s not so great. First off, most people just open up their editor and jump to the line number (or have their editor/IDE do this automatically). Having the source line as part of the message added no value to them. Secondly, it ate up 3 lines of valuable vertical real estate rather than one, meaning that other useful messages got scrolled off the screen. And lastly, although the examples look good, many errors require much more code context to understand and fix. One line is worthless.

The Digital Mars C and C++ compilers still do this, but I dropped it for the D programming language compiler. Nobody remarked upon the absence of this long standing feature, so I figured it was the right decision to abandon it.

Redundancy

Spend some time looking at the grammar for a programming language, and you’ll soon notice that there’s redundancy in there. It’s natural to think that redundancy is wasteful. Programmer keystrokes could be reduced if such redundancy were removed. But redundancy is what makes error messages even possible. If there were no redundancy in a language, then any random sequence of characters would form a valid program. Redundancy acts like having a parity bit for memory bytes, if the parity is off then you know that you have a memory error. Even more redundancy means that the error can be corrected (known as error recovery for compilers).

The most obvious example of that is the terminating semicolon for statements and declarations in C, C++ and D. It’s redundant, and a common question is why have it there at all, as certainly the compiler can figure out where it must be and so it’s not necessary for the programmer to insert it. The more enterprising of these folks will build their own language to prove their point. The result is that it works great as long as there are no mistakes in the source code. As soon as there is one, the error diagnostics tend to be gibberish, the location of the error is often off by several lines, and one mistake results in a cascade of increasingly obscure messages. And so the semicolon persists.

Memory

Years ago, memory was so tight that very little could be spared for verbose diagnostic messages. My all time favorite was in a Basic interpreter Hal Finney (who is not only a freakin’ genius, he’s one of the nicest fellows you’ll ever meet) wrote and had to squeeze into a tiny EPROM. The interpreter had only one error message, which he reduced to simply:

EH?

thereby occupying only 3 precious bytes!

When I started working on compilers a few years later, things weren’t that bad, but on a 64K IBM PC, not a lot could be spared for messages. I even resorted at one point to storing the messages in a separate file, loaded only when necessary.

Memory constraints aren’t much of an issue these days (though compiler performance certainly is), but sometimes tradition, as in short, relatively unhelpful, error messages is slow to change. For example, we might see:

Error: type mismatch

which can be awfully frustrating because although the compiler knows which types are mismatched, it can be very hard for the programmer to figure out which ones they are and where they went wrong. A much better message might be:

  Error: saw type "int" where type "double" was expected

Another thoughtless message that appears far too often is along the lines of:

Error: maximum string length exceeded

instead of:

Error: string length of 123 exceeds maximum string length of 100

In general, a good message should explain what is expected and what it saw instead, hopefully in enough detail that the user shouldn’t have to consult the reference manual too often.

Recovery

Earlier I pointed out that more redundancy in a language can lead not just to error detection, but even error correction. Error correction is desirable in a compiler so that ideally the programmer can, in one pass, see all the errors and can edit the source code and fix them all at once. Unfortunately, there is rarely enough redundancy in a language to do that successfully, so various strategies are employed to reduce the incidence of cascaded error messages and get the compiler back in sync with the source code:

quit on first error
This certainly prevents any cascaded messages, and is adequate for a hobby compiler. Professional users expect much better.
quit after N errors
The idea is that after a certain number of error messages, the compiler has probably failed to recover and the rest is just useless spew anyway, so might as well quit. The first message is most likely to be the only accurate one, and scrolling it off the top of the screen would be most frustrating. This is a common technique today.
scan till a ; is found
Not only does ; provide useful redundancy in detecting errors, it also works as what’s known as a “synchronizing token”. A parser can just throw away any partial parse from a bad expression or statement, and chew up source code till it finds a synchronizing token, then restart the parse after that. This is reasonably effective.
only show first error message for any particular source line
This simple and hackish approach works surprisingly well. If an expression is malformed and recovery is poor, just don’t print error messages past the first one for any particular source line.
guess what the user intended, patch up the parse results, and continue
This is the classic approach. It works great on examples the compiler developer tries it on, and looks like genius in an article about it. But it fails miserably on real user errors. There just isn’t enough redundancy in the language to guess what was intended. The result is cascaded, nonsensical error messages.
propagate error productions
In modern floating point computations, when a divide by zero or other illegal operation is performed, and exception is raised, and the result of the computation is a special NaN (Not a Number) value. Then, every operation that has a NaN as an operand has a NaN as a result, without raising another exception. As such, the errors propagate so that any result that depends on an error is a NaN, and one can easily distinguish good results from bad.

This can be done in a compiler by replacing every production that produces an error with an error production (i.e. a NaN) and then propagate them. For example, for the expression:
```
z = x + y;
```
If y is undefined, print the error message about y, then replace y in the syntax tree with:
```
z = x + _error_;
```
Then, when analyzing the + operator, note that its right operand is an error, so do no further analysis and replace the + expression as:
```
z = _error_;
```
and so on to replace the assignment to z with:
```
_error_;
```
Hence, this should virtually eliminate most cascaded errors, while still allowing a full semantic check on constructs that do not rely on the results of any constructs that failed to compile. It doesn’t involve any attempts at guessing what the programmer might have intended.

I’ve got this about half implemented in the D compiler now, and the results are looking promising.

Spell Checking

This appears to have been first tried by the Clang group. At first blush, it seemed like a dumb idea to me, but I thought I’d implement it and give it a try. Turns out, it’s surprisingly cool, even awesome. It’s only a handful of lines of code to implement, and I put it in both the C++ and D compilers. In everyday coding, it’s just nice to have it pop up for things like:

import std.stdio;

void main() {
    wrietln("hello world!");
}

the message:

test.d(4): Error: undefined identifier wrietln, did you mean template writeln(T...)?

In real coding, it gets it right well over half the time. For non-trivial code, it’s a significant time saver.

Conclusion

It’s surprising, but even after all these decades with compilers, there are still better ways of doing error diagnostics being developed.

Acknowledgements

Thanks to Jason House for reviewing a draft of this article.

Articles