LingDoc Transform

 

A tool for document and language engineering

 

 

 

Reference Manual

 

Version 3.2

 

 

 

 

 

 

 

Palstar

Uffelte

The Netherlands

www.lingdoc.eu

www.palstar.nl

 

 

 

1........... Introduction. 6

1.1      Hands‑on. 6

1.1.1      Example on Dutch license plates. 6

1.2      From grammar to program.. 7

1.2.1      Preparation phase. 7

1.2.2      Compilation phase. 8

1.2.3      Execution phase. 8

1.3      Grammars 8

1.3.1      Meta language. 9

1.3.2      Types of rules and types of grammars 10

1.4      The compiler and the run time system.. 11

1.4.1      Finite state automata. 11

1.4.2      Technical remarks 12

1.5      Miscellaneous concepts 12

1.5.1      Recursion. 12

1.5.2      Ambiguity. 13

1.5.3      Unit rules and empty rules 13

1.6      Contents of this manual 13

2........... Basic Syntax. 15

2.1      Meta symbols 15

2.2      Terminal symbols 16

2.2.1      Characters and strings 17

2.2.2      Ranges 18

2.2.3      Wildcards 19

2.3      Non-terminal symbols 19

2.4      Regular expressions 20

3........... Extended Features. 22

3.1      Actions 22

3.1.1      Assignment and tests 22

3.1.1.1    Variables, literals and expressions 22

3.1.1.2    Assignment 23

3.1.1.3    Types of tests 24

3.1.1.4    Combinations of tests and/or assignments 25

3.1.2      Messages 26

3.1.2.1    Simple messages 26

3.1.2.2    Messages with expressions 27

3.1.3      Places of actions in rules 28

3.1.3.1    Actions following symbols 28

3.1.3.2    Actions following regular expressions 29

3.1.3.3    Actions as part of the repetition of regular expressions 29

3.1.3.4    Actions in empty units 30

3.2      Symbol type extensions 31

3.2.1      Non-terminals with parameters 31

3.2.2      Lexicon symbols 33

3.2.3      Cover non-terminals 34

3.2.4      Intermediate symbols 35

3.3      Negation. 36

3.3.1      Negation of terminal characters, ranges and intermediates 37

3.3.2      Negation of non-terminal symbols 37

3.3.3      Negation of regular expressions 37

3.3.4      Negation of a string of terminal symbols 38

3.4      Error recovery. 38

4........... Context Sensitive Grammars. 40

4.1      Description of context sensitive rules 40

4.2      Context sensitive grammars for recognition or parsing. 40

4.2.1      Example of a context sensitive grammar used for recognition. 40

4.2.2      (Cover) non-terminals in context sensitive grammars 41

4.2.3      Intermediates in context sensitive grammars 42

4.3      Transduction grammars 43

4.3.1      Description of transduction rules 43

4.3.2      Non-terminals in transduction rules 45

4.3.3      Intermediates in transduction rules 45

4.3.4      Ambiguity and "looping" problems in transduction grammars 46

4.3.5      Compiling and running a transduction grammar 47

4.3.6      Cascaded grammars 47

5........... Run Time Input and Output files. 48

5.1      Input files 48

5.2      Appearance of symbols in output files 49

5.3      Parse tree output 50

5.4      Report output 51

5.4.1      Report of messages 51

5.4.2      Report of output parameters 52

5.5      Error output 52

6........... Switches (Directives) 52

6.1      Compiler Directives 53

6.1.1      GUI: Advanced Mode, Standard fields: Compiler 53

6.1.1.1    Switch: FINITE. 54

6.1.1.2    Switch: ONE_CS_REDUCE. 54

6.1.1.3    Switch: ONE_ALTERNATIVE. 54

6.1.1.4    Switch: RECURSION_MESSAGE. 55

6.1.1.5    Switch: SHARED_UNIVERSE. 55

6.1.1.6    Switch: UNIVERSE. 55

6.1.1.7    Switch: TRANSDUCE. 55

6.1.1.8    Switch: PREPARE_ PARSETREE. 56

6.1.1.9    Switch: INTERM_INPUT. 56

6.1.1.10  Switch: CHECK_VARIABLES. 56

6.1.1.11  Switch: COMP_BASE. 56

6.2      Runtime Directives 56

6.2.1      GUI: Advanced Mode, Standard fields: Parser 56

6.2.1.1    Switch: ECHO.. 57

6.2.1.2    Switch: IGNORE_CR. 57

6.2.1.3    Switch: SPLIT_OUTPUT. 57

6.2.1.4    Switch: BUILD_PARSETREE. 57

6.2.1.5    Switch: SKIP_CHARS. 58

6.2.1.6    Switch: SKIP_ON_ERROR. 58

6.2.1.7    Switch: OUTPUT_PARAMETERS. 58

6.2.1.8    Switch: LEXICON.. 58

6.2.1.9    Switch: LEX_CHARS. 58

6.2.1.10  Switch: SINGLE_LINES. 58

6.2.1.11  Switch: IGNORE_BLANKS. 59

6.2.1.12  Switch: FLAT_OUTPUT. 59

6.2.1.13  Switch: CASE_CONVERT. 59

6.2.1.14  Switch: FIND_ON_ERROR. 59

6.2.1.15  Switch: PARSE_BASE. 59

6.2.1.16  Switch: LEX_INCREMENT. 60

6.3      Switches for error recovery. 60

6.3.1.1    Switch: CPU_LOG.. 60

7........... Operation. 60

7.1      Compiler 60

7.1.1      Commands for compilation in batch. 61

7.1.2      Screen output in the compilation phase. 61

7.1.3      Files related to compilation. 62

7.1.4      Error handling during compilation. 62

7.2      Run time system.. 63

7.2.1      Commands for the run time system in batch. 63

7.2.2      Screen output and keyboard input during the execution phase. 64

7.2.3      Files related to the run time system.. 64

7.2.4      Error handling during run time. 65

8........... Problems and current limitations. 66

8.1      Current limitations 66

8.1.1      Upper limits for the size of a grammar 66

8.1.2      Finite versus non-finite. 66

8.1.3      Limitations concerning context sensitive rules 67

8.1.4      Limitations concerning actions 67

8.1.5      Miscellaneous limitations 67

8.1.6      Known effects of "bugs" 68

8.2      Debugging Grammars 68

9........... Tools. 69

9.1      Lexicon. 70

9.1.1      GUI: Advanced Mode: Lexicon-Compiler 71

9.2      Drawtree. 71

9.3      Drawgraph. 72

9.4      Plink. 72

9.4.1      GUI: Advanced Mode: Cascade Linker 73

9.5      Xgram.. 73

9.6      Disassembler 74

9.6.1      GUI: Advanced Mode: Disassembler 74

10.......... Overview. 75

10.1     Meta language of Transform grammars 75

10.2     Rules for the construction of user defined symbol names 76

10.3     Checklist for switches 77

10.4     Batch-Commands 78

11.......... APPENDIX A: SWITCHES FOR PROGRAMMERS. 79