David Crisp-Hihn 1950-2013

David Crisp-Hihn died on December 27th 2013, after being admitted to hospital with a stroke on the 15th. He leaves behind his wife, Sandra, four children and a grandchild.

David Crisp was a pupil at King Edward VI School, Southampton when I knew him first. We became friends as a result of belonging to the same Christian Society and both attending Above Bar Interdenominational Church, Southampton. When David obtained a place at New College, Oxford to study Mathematics, he inspired me to apply also, and I was able to attend two years later. We remained firm friends throughout my university years and long after, and he was Best Man at my wedding in 1977, though later we lost touch.

About three and a half years ago we renewed contact (the internet is a wonderful thing) and arranged to go on a walking weekend together. This was so enjoyable we repeated it twice more, walking in Cornwall, the Welsh borders and Norfolk in the last three years. In Norfolk, David was less able to manage the longer walks.

I was blessed to spend these times with my old friend but I did not know how precious and important these last few days would become.

I miss him terribly.

I hate make

Actually, I hate make and shell (all forms) and sed and basically, the *nix command structure.

The main reason: escapes. I just can’t seem to ever get them right.  In Makefiles they are a nightmare!

Just try issuing a sed -e command after a dependency — AFAICT§ there are three levels of escape to consider: 

1. the make level (where $ is special, and you double it to get a real one), 
2. the shell level (where $ is special outside of single quotes) and 
3. the sed pattern level, where $ is special and you “escape” it with a backslash to make it not special again.

Here is the line (almost) I had to get right:

sed -e 's:^ABC=$:ABC=${VAR}:' file >file.tmp

Now, execute this on the command line (bash shell) and it nearly works — it should replace a complete line “ABC=” by the complete line “ABC=${VAR}” in the file called file.

It doesn’t actually work because of the second $.  I have used (single) quotes so that the shell doesn’t try to substitute the (apparent) shell variable ${VAR} but I forgot that $ means ‘end-of-line’ to sed (even though I used it to mean that in the first part of the substitution string). So actually, it ought to be:

sed -e 's:^ABC=$:ABC=\${VAR}:' file >file.tmp

using a backslash escape to tell sed to treat the second $ literally.

So this second version goes in the Makefile, and of course it fails.  (And, it takes me some time to find this out, since the make is executed in the heart of some long-winded build process, and if I don’t execute it there it will not have the context to make it work — bórza!)

The reason is that single quotes don’t protect expressions in make, so it tries to substitute the ${VAR}.  “Aha,” I cried, “I’ll have to escape the backslash and the $ for make.”  Like this:

sed -e 's:^ABC=$:ABC=\\$${VAR}:' file >file.tmp

But I was wrong, the backslash isn’t sufficiently special in make — well it is, but double backslash is not “escaped” to backslash: instead it leaves it as double backslash, and the single quotes still offer no protection.  So we end up with “ABC=\${VAR}” in the output.

Round I go again:

sed -e 's:^ABC=$:ABC=\$${VAR}:' file >file.tmp

and this works.  Finally.

Why all this fuss?

Because *nix systems and their utilities have internalized all this escapery, inherited from an old-fashioned and stupid macro-language-style command structure, and we are now stuck with it.  Everywhere.  And of course, *nix being the democratic organisation it is, everyone does it differently, with different rules and with different exceptions.  Ugh.

So... I think we should start to unravel it.  We could start with make, for example. Let’s create a version of make where all the expressions are evaluated with referential transparency, which is to say that a literal string, once established, is never rescanned and re-interpreted again, even if I pass it around in a variable and it appears in another expression.  Substitutions are only made once if I say they should happen.  If I write an expression with a $ in it, it cannot be interpreted as a variable indicator anywhere else.

Then, the utilities that we call from make should have an ‘uninterpreted’ interface. That is, it should be possible to pass them a literal string without worrying about the contents of the string.  If we are a utility passing strings we haven’t checked, we no longer have to worry about what they might contain and scan them for things to escape.  That way, we also don’t have to worry about what the utility is that we are calling, nor where it might send this string.

I guess we would have to tackle the shell, too.

There would be consequences: some of our tricks (for example: escaping the right number of times so that the cascade of string passings remove the escapes and trigger interpretation at just the right level; or, varying the name of a reference using the value of another) would be harder to achieve — but let’s face it, who needs these really? And when you do, how long does it take to get it right?  And have you ever got it (really) right?

Yup, a proper referentially transparent shell would be a first step.  Or maybe a grep....

hmm...

§ I also hate all these cheesy acronyms — it’s just a linguistic barrier to ensure the prols can’t easily join the club.

PS: Did you notice that the ‘footnote’ didn’t link properly? I don’t know how to do this in this form — how do I link to an anchor in the same page?

Spec thought for the day

Browsing around StackOverflow I came across a question about specifying binary files. One of the answers said that the Java spec for class files was a good example of how to do it.

So I went there to look and found this:

  ClassFile {
   …
   u2 constant_pool_count;
   cp_info constant_pool[constant_pool_count-1];
   …
  }

and the following description:

constant_pool_count

The value of the constant_pool_count item is equal to the number of entries in the constant_pool table plus one. A constant_pool index is considered valid if it is greater than zero and less than constant_pool_count, with the exception for constants of type long and double noted in §4.4.5.

constant_pool[]

The constant_pool is a table of structures (§4.4) representing various string constants, class and interface names, field names, and other constants that are referred to within the ClassFile structure and its substructures. The format of each constant_pool table entry is indicated by its first "tag" byte. The constant_pool table is indexed from 1 to constant_pool_count-1.

and I wondered if this is a good spec what does a horrible one look like?

Granted, this is quite chatty, but when you read it for sense you get more and more confused.

It looks like a C structure, so let's read it like one… 

Why is the count one more than the number of elements in the array?  The array index starts at 0 when I declare one.  Oh yeah: the valid indexes are greater than zero (why omit the zero index?)—except for some long and double stuff, life's too short—and then the valid indexes are from 1 to count-1.

Whoa! where do they go? The declaration a[3] normally allocates three elements (indexes 0, 1 and 2), but here we allocate count-1 and index from 1 to count-1.  The last index would be invalid!??!

Aha!  We must be basing our arrays from one and not zero!! 

…and the count entry isn't a count really.

Ugh!

If this is a good spec my name is Edsgar Dijkstra.