# std/lexbase

Theme:

- Manual
- Standard library
- Index
- Compiler docs
- Fusion docs
- devel, stable

Search:
Source Edit

This module implements a base object of a lexer with efficient buffer handling. Only at line endings checks are necessary if the buffer needs refilling.

# Imports

strutils, streams, assertions

# Types

    BaseLexer = object of RootObj
      bufpos*: int               ## the current position within the buffer
      buf*: string               ## the buffer itself
      lineNumber*: int           ## the current line number
      offsetBase*: int

  the base lexer. Inherit your lexer from this object. Source Edit

# Consts

    EndOfFile = '\x00'

  end of file marker Source Edit

    NewLines = {'\r', '\n'}

  Source Edit

# Procs

    proc close(L: var BaseLexer) {....raises: [IOError, OSError],
                                   tags: [WriteIOEffect], forbids: [].}

  closes the base lexer. This closes `L`'s associated stream too. Source Edit

    proc getColNumber(L: BaseLexer; pos: int): int {....raises: [], tags: [],
        forbids: [].}

  retrieves the current column. Source Edit

    proc getCurrentLine(L: BaseLexer; marker: bool = true): string {....raises: [],
        tags: [], forbids: [].}

  retrieves the current line. Source Edit

    proc handleCR(L: var BaseLexer; pos: int): int {....raises: [IOError, OSError],
        tags: [ReadIOEffect], forbids: [].}

  Call this if you scanned over `'\c'` in the buffer; it returns the position to continue the scanning from. `pos` must be the position of the `'\c'`. Source Edit

    proc handleLF(L: var BaseLexer; pos: int): int {....raises: [IOError, OSError],
        tags: [ReadIOEffect], forbids: [].}

  Call this if you scanned over `'\L'` in the buffer; it returns the position to continue the scanning from. `pos` must be the position of the `'\L'`. Source Edit

    proc handleRefillChar(L: var BaseLexer; pos: int): int {.
        ...raises: [IOError, OSError], tags: [ReadIOEffect], forbids: [].}

  Call this if a terminator character other than a new line is scanned at `pos`; it returns the position to continue the scanning from. Source Edit

    proc open(L: var BaseLexer; input: Stream; bufLen: int = 8192;
              refillChars: set[char] = NewLines) {....raises: [IOError, OSError],
        tags: [ReadIOEffect], forbids: [].}

  inits the BaseLexer with a stream to read from. Source Edit