Thursday, January 10. 2008

Why I don't use scsh as a scripting language anymore

I learned Scheme in my first semester at university and I liked it right away for its clearity and flexibility. However, I stopped using Scheme for my everyday tasks some long time ago. Recently, I wanted to practice some Scheme again a little bit and I thought, I could come back to scsh, which we used in the lectures, to write some small script that does something similar to rsync. Given that I wrote some fairly large applications in scsh, I thought practicing will be smooth.

However, turning back to scsh reminded me of why I stopped using scsh as a scripting language (as opposed to programming language or Scheme implementation).

The biggest show stopper is that errors don't tell you where they come from:

$ ./rsync.scm /tmp /tmp

Error: wrong number of arguments
       ('#{Procedure 14743 start} '("./rsync.scm" "/tmp" "/tmp"))

You have to load the script into an interactive session to actually get the location of the bug. Beware, that "location" does not imply any line number, just the name of a procedure where the error happened and the context. I hope you don't do higher-order programming and use anonymous functions... This is worse than in C, where I can at least load the core dump into an external debugger or use tinycc, which emits the line number along with the error message. Other scripting languages also tell me the line number right away.

Using the interactive session in a terminal is painful, as scsh comes without readline support, so you can't really use the interactive session in a terminal, unless you like typing a lot. For example, it is not possible to edit a previously entered line or even go to the beginning of the line to fix a typo. The solution is some external program with which you start your scsh and which mimics readline support, such as Xemacs M-x run-scheme

For care-free scripting, you must use an ugly starting line with caveats:

#! /usr/local/bin/scsh \
-e main -s
!#

Why not just

#! /usr/local/bin/scsh

you may ask? Turns out, "#" does not start a comment in Scheme, so scsh would try to read the first line of the script as a Scheme form and fail.

Instead of extending some standard to allow a scsh script to load modules or set search paths from within the script, the scsh author introduced "#!" as the beginning of a multi-line block comment. You can use the "#!...!#" multi-line block comment anywhere in your program, but the closing "!#" must stand by itself at the beginning of a line (how's that for a block comment, hu?). So, in scsh scripts, you always have to "close" the hash-bang at the beginning of your script like this

#! /usr/local/bin/scsh -s
!#

Ah, and you can't leave the "-s" at the end as otherwise the scsh interpreter thinks, your script name is a switch:

$ cat test.scm
#! /usr/local/bin/scsh
!#

(display "Hooray!\n")

$ ./test.scm
Unknown switch ./test.scm 
Usage: scsh [...]

In case your script has some errors (and while developing it sure will have), the starting line above is not enough. As soon as you want to use some module, say srfi-9 to define records, you have to add switches like this:

#! /usr/local/bin/scsh -o srfi-9 -s
!#

Of course, that would be too easy. scsh now thinks "-o srfi-9 -s" is a single switch:

$ cat ./test.scm
#! /usr/local/bin/scsh -o srfi-9 -s
!#

(display "Hooray!\n")

(define-record-type :foo
  (foo a)
  foo?
  (a foo-a))
$ ./test.scm
Unknown switch -o srfi-9 -s 
Usage: scsh [meta-arg] [switch ..] [end-option arg ...]

And by the way, who told scsh to clobber my screen with a 33 line help message when I just mistype some switch argument? Ah, I see, the scsh authors did not see any reason to include a "--help" or "-h" switch, that would probably be to GNU-like or user-friendly:

$ scsh --help
Unknown switch --help 
Usage: scsh [meta-arg] [switch ..] [end-option arg ...]
[...]

Back to how to load some module. As you can't put it on the starting line, the scsh authors did not think about some mechanism to load a module from within your script (such as "(open-module srfi-9)"), as every other reasonable script language does, but they (or must I say "he", Olin?) invented the "meta argument"! The "meta argument" is a backslash which tells the scsh interpreter to scan the second line of the script for additional arguments. It looks like this:

#! /usr/local/bin/scsh \
-o srfi-9 -s
!#

Although this looks neat, beware! First, all arguments (and thus all module loading) must be on that second line. No third line, no newline-quoting. Second, scsh has a very dump "second-line"-reader which splits the arguments at the space character. Two consecutive space characters will result in an empty switch which in turn will result in a not so meaningful error message:

$ cat ./test.scm
#! /usr/local/bin/scsh \
-o srfi-9  -s
!#

(display "Hooray!\n")
(format #t "Command line args are: ~a~%" command-line-arguments)

$ ./test.scm
Unknown switch  
Usage: scsh [meta-arg] [switch ..] [end-option arg ...]

Thus, type that second line very carefully. In particular, don't add spaces after the switches, otherwise you will get that weird error message:

$ head -3 test.scm
#!/usr/local/bin/scsh \
-o srfi-9 -s 
!#

$ ./test.scm 

Error: exception
       cannot-open-channel
       (open-channel "" 1)

The single extra space after the "-s" switch makes scsh want to load the script named like the empty string.

However, this does still not explain, why the starting lines should look like the one mentioned above:

#! /usr/local/bin/scsh \
-e main -s
!#

The reason is debugging. Remember the error in the script "rsync.scm" mentioned at the beginning?

$ ./rsync.scm /tmp /tmp

Error: wrong number of arguments
       ('#{Procedure 14743 start} '("./rsync.scm" "/tmp" "/tmp"))

As said previously, in order to locate a bug in your script, you have to load it into an interactive session. So I fire up scsh and load "rsync.scm":

$ scsh
Welcome to scsh 0.6.7 (R6RS)
Type ,? for help.
> ,load rsync.scm
rsync.scm

Error: undefined variable
       :file
       (package user)

Of course, that would be too easy. Instead of locating the "wrong number of arguments" error from above, I have a new error. The reason is that the script uses the srfi-9 module to create records ("structs" in C lingo). The script uses a switch (instead of some import-statement like any other language implementation) to load that module, but the interactive scsh interpreter does not care about the switches, because they are in the comments.

$ head -3 rsync.scm
#!/usr/local/bin/scsh \
-o srfi-9 -s  
!#

OK, to find the location of the bug, I have to open the module myself in the scsh session:

$ scsh
Welcome to scsh 0.6.7 (R6RS)
Type ,? for help.
> ,open srfi-9
> ,load rsync.scm
rsync.scm
Usage: rsync.scm src-dir dest-dir

The scsh session exited. The reason is that scsh not just "loads" rsync.scm, but evaluates it. And one of the Scheme forms checks for the number of arguments. As I haven't provided any, the script quits with a usage message.

Of course, just adding the command-line arguments to the scsh interpreter is too easy and results in an error message:

$ scsh /tmp /tmp
Unknown switch /tmp

The solution is to tell scsh there are no switches:

$ scsh -- /tmp /tmp
Welcome to scsh 0.6.7 (R6RS)
Type ,? for help.
> ,open srfi-9
> ,load rsync.scm
rsync.scm

Error: wrong number of arguments
       ('#{Procedure 14745 start} '("scsh" "/tmp" "/tmp"))

1>

OK, that looks much better now. Unfortunately, the debugger can't help me in this case:

1> ,debug
'#{Exception-continuation (pc 15) 14743}

inspect: d
'#{Continuation (pc 34) (loop in for-each1 in scheme-level-1)}

 [0] '("rsync.scm")
 [1] '#{Procedure 716 (loop in for-each1 in scheme-level-1)}
 [2] '#{Procedure 5982 (unnamed in unnamed in load ---)}
 [3] '("rsync.scm")
inspect:

No line number, no context information. From my experience I know that scheme-level-1 refers to the top-level, so I check for an error somewhere at top-level where procedure start is called. Finally, I found the error and fixed it.

So here's the reason for that weird starting lines from the beginning: I avoid the hassles with debugging by providing a procedure that takes the command-line arguments and calls the rest of the script, like this:

(define (main command-line-arguments)
  (if (not (= (length command-line-arguments) 3))
      (begin
    (println "Usage: rsync.scm src-dir dest-dir")
    (exit 1)
    ))

  (let ((src (cadr command-line-arguments))
        (dest (caddr command-line-arguments)))
    (start src dest)))

The -e main tells scsh to start the script by calling main with the command-line arguments. This has two advantages. First, when loading the script into the scsh interactive environment for debugging, it does not get executed, as the starting code is wrapped into the procedure main. Second, I can provide any command-line arguments I like when calling main from within the interactive environment. However, I must be aware that command-line-arguments does not only contain the arguments of the command line, but also the script's name as the first element of the list.

With all that hassle to just write the first lines of my script, the inability to specify the modules to load from within the script (and don't tell me to use define-structure, I want to write a script, not a module), and the missing line numbers in error messages, I refrain from using scsh and turn to some other program for scripting, where I can concentrate more on my script instead of peculiarities of the language implementation. If a future version of scsh tackles the aforementioned points, I would consider again.