Skip to content
A scene from Ireland

Language Design

Lexical Structure

It is traditional to divide the parsing of a program into two conceptually separate stages - the lexical analysis which extracts a stream of tokens from a stream of characters, and the grammar matching stage which gathers those tokens into grammatical units such a declarations, statements, and expressions. Recognising the value of tradition, Ocean will similarly have separate lexical and grammatical stages. This note will focus on the lexical stage.

Future language design decisions will refine may details of the lexical structure of the language, however there is a lot of commonality among current languages and we can build on that commonality to establish a base approach to lexical analysis which can then be fine tuned when the details of the language are known.

Literate programming?

I was going to start out by describing the lexical structure of Ocean, but as I thought more about it, I realised there was something that needed to come first. And that something is literate programming.

Literate programming is the idea - popularised by Donald Knuth - of writing programs like pieces of literature. They can be structured as a story or an exposition, and presented in a way that makes sense to the human rather than just to a computer.

The naming of a language

"The naming of cats is a difficult matter, It isn't just one of your holiday games." The naming of programming languages is also important. As with any project a name is needed to be able to refer to, and it inevitably will set expectations and flavour to some degree.

I've had a few different thoughts about names. My first idea was "plato". Plato was a philosopher and is particularly known for drawing a distinction between the real and the ideal. All things in the real world, circles and squares and so forth, are just poor shadows of the perfect circles and the perfect squares that exist in the ideal, or "Platonic" plane.

An exercise in Language Design

When I was doing my honours year in Computer Science (UNSW, 1986) I wanted to design a new programming language. That would be a rather large project for an honours year and naturally it didn't happen. I have remained interested in languages, though for most of the time that interest has been idle.

I recently wrote some articles about languages for LWN and that has re-awoken my interest in language design. While I had scribbled down (or typed out) various notes about different ideas in the past, this time I seem have have progressed much further than ever before. It probably won't ever amount to much but I've decided to try to continue with the project this time and create as concrete a design and implementation as I can ... in my spare time.

operators

The core operators in this language are the period, used for substructure access, and the parentheses, used to pass parameters to a function call.

Every object embodies a namespace of components and methods. These are accessed by following the object by a period and the name.

Methods can be called by following the name by a (possibly empty) list of parameters enclosed in parentheses. All operations can be called this way, though there are often easier means, such as as infix operators. So for example

Integer.add(a, b)

might be the same as

a + b

Note that this is not necessarily a.add(b). Infix operators are defined as operation in a type, not methods of an object.

When finding which operation to use for an infix operator we look through the operations in the class of the left hand operand which have been associated with the given operator, and choose one for which the types of both sides are correct. Of these, the operation for which the first parameter is lowest in the type lattice is perferred. If choices still remain, the lowest inf the second parameter is chosen.

Thus the virtual class "Number" might declare the "+" infix operator. Then Integer32 might refine Number to a concrete class and define "add(Interger32, Interger32)" which adds two integers. Also Float might refine Number and define "add(Float, Float)" and also "addint(Float, Int)" and "addtoint(Int, Float)" all of which return Float. Then a + b would clearly resolve to one of these providing each of a and b were either Int or Float.

No automatic coersion is done. If another class "BigNum" were defined that defined addition within bignums and between bignums and integers, then an addition of a bignum and a float would fail.

In general, automatic coersion is frowned upon as lack of precission is likely.

Note that classes can define operations and methods for other classes. For example, Float might define a method for the Integer32 class which converts the integer into a floating point number. This is done by declaring the name in the Number class, and defining it for various subclasses.

For this reason, it is good for an abstract class to declare a number of representatons for subclasses to work with. Thus "Number" might declare the names Integer32, Integer64, Float, Double, BigNum, BigInt, Complex, Guassian (Is that what complex integers are called?) Duplex (Complex doubles?) so that different classes can make use of each other representations.

Question: Is this only useful for numbers? Numbers are clearly a fairly special case, but the less special we can make them, the cleaner the language.

Syntax

Having recently read up on python, I quite like it's syntax. It is clean and not noisy. However I do have some problems with it.

Tuples

Tuples are a useful construct in a programming language, as long as we know what they are...

A tuple is a fixed collection of other objects. It is different from a list in that it is not extensible: once it has been created, it stays that size. Also, unlike lists, the elements can be of different types. Thus it is more substantially like a Pascal record of C struct.

The value of a tuple over a struct is anonymity. It is sometimes useful to have a struct that is not explicitly typed, but still have enough type information to be passed around safely. Effectively there are standard labels "first" and "second" etc, and type matching is based on substructure matching.

In some languages, such as python, the arguments to a function can be passed positionally or by name. This could be seen as the arguments being a tuple which is addressed either by standard labels (first, second, etc) or by pre-defined names. However argument lists can usefully be of varying length, which is at odds with a tuples pre-defined structure.

UNFINISHED