7.2 @ Reader Internals

► |  | Scribble: Racket Documentation Tool

1 | Getting Started
2 | @ Syntax
3 | High-Level Scribble API
4 | Scribbling Documentation
5 | Literate Programming
6 | Text Preprocessing
7 | Low-Level Scribble API
| Index

▼ | 7 | Low-Level Scribble API

7.1 | Scribble Layers
7.2 | @ Reader Internals
7.3 | Structures And Processing
7.4 | Renderer
7.5 | Decoding Text
7.6 | Document Language
7.7 | Document Reader
7.8 | Cross-Reference Utilities
7.9 | Extending and Configuring Scribble Output

► | 7.2 | @ Reader Internals

7.2.1 | Using the @ Reader
7.2.2 | Syntax Properties
7.2.3 | Adding @-expressions to a Language
7.2.4 | Interface

On this page:

7.2.1`` Using the @ Reader
7.2.2`` Syntax Properties
7.2.3`` Adding @-expressions to a Language
7.2.4`` Interface
read
read-syntax
read-inside
read-syntax-inside
make-at-readtable
make-at-reader
use-at-readtable

top← prev up next →

#### 7.2`` @ Reader Internals

##### 7.2.1`` Using the @ Reader

You can use the reader via Racket’s #reader form:

#reader scribble/reader @foo{This is free-form text!}

or use the at-exp meta-language as described in Adding @-expressions to a Language.

Note that the Scribble reader reads @-forms as S-expressions. This means that it is up to you to give meanings for these expressions in the usual way: use Racket functions, define your functions, or require functions. For example, typing the above into racket is likely going to produce a “reference to undefined identifier” error, unless foo is defined. You can use string-append instead, or you can define foo as a function (with variable arity).

A common use of the Scribble @-reader is when using Scribble as a documentation system for producing manuals. In this case, the manual text is likely to start with

#lang scribble/doc

which installs the @ reader starting in “text mode,” wraps the file content afterward into a Racket module where many useful Racket and documentation related functions are available, and parses the body into a document using scribble/decode. See Document Reader for more information.

Another way to use the reader is to use the use-at-readtable function to switch the current readtable to a readtable that parses @-forms. You can do this in a single command line:

racket -ile scribble/reader "(use-at-readtable)"

##### 7.2.2`` Syntax Properties

The Scribble reader attaches properties to syntax objects. These properties might be useful in some rare situations.

Forms that Scribble reads are marked with a 'scribble property, and a value of a list of three elements: the first is 'form, the second is the number of items that were read from the datum part, and the third is the number of items in the body part (strings, sub-forms, and escapes). In both cases, a 0 means an empty datum/body part, and #f means that the corresponding part was omitted. If the form has neither parts, the property is not attached to the result. This property can be used to give different meanings to expressions from the datum and the body parts, for example, implicitly quoted keywords:

( define-syntax (foo stx)
( let ([p ( syntax-property stx 'scribble)])
( printf ">>> ~s\n" ( syntax->datum stx))
( syntax-case stx ()
[(_ x ...)
( and ( pair? p) ( eq? ( car p) 'form) ( even? ( cadr p)))
( let loop ([n (/ ( cadr p) 2)]
[as '()]
[xs ( syntax->list #'(x ...))])
( if ( zero? n)
( with-syntax ([attrs ( reverse as)]
[(x ...) xs])
#'( list 'foo `attrs x ...))
(loop ( sub1 n)
( cons ( with-syntax ([key ( car xs)]
[val ( cadr xs)])
#'(key ,val))
as)
( cddr xs))))])))

> @foo[x 1 y (* 2 3)]{blah}
>>> (foo x 1 y (* 2 3) "blah")
'(foo ((x 1) (y 6)) "blah")

In addition, the Scribble parser uses syntax properties to mark syntax items that are not physically in the original source – indentation spaces and newlines. Both of these will have a 'scribble property; an indentation string of spaces will have 'indentation as the value of the property, and a newline will have a '(newline S) value where S is the original newline string including spaces that precede and follow it (which includes the indentation for the following item). This can be used to implement a verbatim environment: drop indentation strings, and use the original source strings instead of the single-newline string. Here is an example of this.

( define-syntax (verb stx)
( syntax-case stx ()
[(_ cmd item ...)
#`(cmd
#,@( let loop ([items ( syntax->list #'( item ...))])
( if ( null? items)
'()
( let* ([fst ( car items)]
[prop ( syntax-property fst 'scribble)]
[rst (loop ( cdr items))])
( cond [( eq? prop 'indentation) rst]
[( not ( and ( pair? prop)
( eq? ( car prop) 'newline)))
( cons fst rst)]
[ else ( cons (datum->syntax-object
fst ( cadr prop) fst)
rst)])))))]))

> @verb[string-append]{
foo
bar
}

"foo\n bar"

##### 7.2.3`` Adding @-expressions to a Language

#lang at-exp

The at-exp language installs @-reader support in the readtable, and then chains to the reader of another language that is specified immediately after at-exp.

For example, #lang at-exp racket/base adds @-reader support to racket/base, so that

#lang at-exp racket/base
( define (greet who) @ string-append{Hello, @|who|.})
(greet "friend")

reports "Hello, friend.".

##### 7.2.4`` Interface

( require scribble/reader)

The scribble/reader module provides direct Scribble reader functionality for advanced needs.

( read [in]) → any
in : input-port? = ( current-input-port)

( read-syntax [source-name in]) → ( or/c syntax? eof-object?)
source-name : any/c = ( object-name in)
in : input-port? = ( current-input-port)

These procedures implement the Scribble reader. They do so by constructing a reader table based on the current one, and using that for reading.

( read-inside [in]) → any
in : input-port? = ( current-input-port)

( read-syntax-inside [source-name in]) → ( or/c syntax? eof-object?)
source-name : any/c = ( object-name in)
in : input-port? = ( current-input-port)

These -inside variants parse as if starting inside a @{...}, and they return a (syntactic) list. Useful for implementing languages that are textual by default (see "docreader.ss" for example).

( make-at-readtable | [ | #:readtable readtable |
|  | #:command-char command-char |
|  | #:datum-readtable datum-readtable |
|  | #:syntax-post-processor syntax-post-proc]) |

→ readtable?
readtable : readtable? = ( current-readtable)
command-char : character? = #\@

datum-readtable

:

( or/c readtable? boolean?
( readtable? . -> . readtable?))

|  |  | = |  | #t

syntax-post-proc : ( syntax? . -> . syntax?) = values

Constructs an @-readtable. The keyword arguments can customize the resulting reader in several ways:

- readtable – a readtable to base the @-readtable on.

- command-char – the character used for @-forms.

- datum-readtable – determines the readtable used for reading the datum part. A #t values uses the @-readtable, otherwise it can be a readtable, or a readtable-to-readtable function that will construct one from the @-readtable. The idea is that you may want to have completely different uses for the datum part, for example, introducing a convenient key=val syntax for attributes.

- syntax-post-proc – function that is applied on each resulting syntax value after it has been parsed (but before it is wrapped quoting punctuations). You can use this to further control uses of @-forms, for example, making the command be the head of a list:

  ( use-at-readtable
  #:syntax-post-processor
  ( lambda (stx)
  ( syntax-case stx ()
  [(cmd rest ...) #'( list 'cmd rest ...)]
  [else ( error "@ forms must have a body")])))

( make-at-reader |  | #:syntax? syntax? |  |  |  |
|  | #:inside? inside? ...) |  | → |  | procedure?

syntax? : #t
inside? : #f

Constructs a variant of a @-readtable. The arguments are the same as in make-at-readtable, with two more that determine the kind of reader function that will be created: syntax? chooses between a read- or read-syntax-like function, and inside? chooses a plain reader or an -inside variant.

The resulting function has a different contract and action based on these inputs. The expected inputs are as in read or read-syntax depending on syntax?; the function will read a single expression or, if inside? is true, the whole input; it will return a syntactic list of expressions rather than a single one in this case.

Note that syntax? defaults to #t, as this is the more expected common case when you’re dealing with concrete-syntax reading.

Note that if syntax? is true, the read-like function is constructed by simply converting a syntax result back into a datum.

( use-at-readtable ...) → void?

Passes all arguments to make-at-readtable, and installs the resulting readtable using current-readtable. It also enables line counting for the current input-port via port-count-lines!.

This is mostly useful for playing with the Scribble syntax on the REPL.

top← prev up next →