# ic/rodfiles

Theme:

- Index

Search:
Group by:
Source Edit

Low level binary format used by the compiler to store and load various AST and related data.

NB: this is incredibly low level and if you're interested in how the compiler works and less a storage format, you're probably looking for the `ic` or `packed_ast` modules to understand the logical format.

# Overview

`RodFile` represents a Rod File (versioned binary format), and the associated data for common interactions such as IO and error tracking (`RodFileError`). The file format broken up into sections (`RodSection`) and preceded by a header (see: `cookie`). The precise layout, section ordering and data following the section are determined by the user. See `ic.loadRodFile`.

## A basic but "wrong" example of the lifecycle:

1. `create` or `open` - create a new one or open an existing
2. `storeHeader` - header info
3. `storePrim` or `storeSeq` - save your stuff
4. `close` - and we're done

Now read the bits below to understand what's missing.

### Issues with the Example

Missing Sections: This is a low level API, so headers and sections need to be stored and loaded by the user, see `storeHeader` & `loadHeader` and `storeSection` & `loadSection`, respectively.

No Error Handling: The API is centered around IO and prone to error, each operation checks or sets the `RodFile.err` field. A user of this API needs to handle these appropriately.

# API Notes

## Valid inputs for Rod files

ASTs, hopes, dreams, and anything as long as it and any children it may have support `copyMem`. This means anything that is not a pointer and that does not contain a pointer. At a glance these are:

- string
- objects & tuples (fields are recursed)
- sequences AKA `seq[T]`

## Note on error handling style

A flag based approach is used where operations no-op in case of a preexisting error and set the flag if they encounter one.

## Misc

- 'Prim' is short for 'primitive', as in a non-sequence type

# Types

    RodFile = object
      f*: File
      currentSection*: RodSection
      err*: RodFileError

  Source Edit

    RodFileError = enum
      ok, tooBig, cannotOpen, ioFailure, wrongHeader, wrongSection, configMismatch,
      includeFileChanged

  Source Edit

    RodSection = enum
      versionSection, configSection, stringsSection, checkSumsSection, depsSection,
      numbersSection, exportsSection, hiddenSection, reexportsSection,
      compilerProcsSection, trmacrosSection, convertersSection, methodsSection,
      pureEnumsSection, toReplaySection, topLevelSection, bodiesSection,
      symsSection, typesSection, typeInstCacheSection, procInstCacheSection,
      attachedOpsSection, methodsPerGenericTypeSection, enumToStringProcsSection,
      methodsPerTypeSection, dispatchersSection, typeInfoSection,
      backendFlagsSection, aliveSymsSection, sideChannelSection, namespaceSection,
      symnamesSection

  Source Edit

# Procs

    proc close(f: var RodFile) {....raises: [IOError], tags: [], forbids: [].}

  Source Edit

    proc create(filename: string): RodFile {....raises: [], tags: [], forbids: [].}

  create the file and open it for writing Source Edit

    proc loadHeader(f: var RodFile; cookie = defaultCookie) {....raises: [IOError],
        tags: [ReadIOEffect], forbids: [].}

  Loads the header which is described by `cookie`. Source Edit

    proc loadOrderedTable[K, T](f: var RodFile; s: var OrderedTable[K, T])

  `T` must be compatible with `copyMem`, see `loadPrim` Source Edit

    proc loadPrim(f: var RodFile; s: var string) {....raises: [IOError],
        tags: [ReadIOEffect], forbids: [].}

  Read a string, the length was stored as a prefix Source Edit

    proc loadPrim[T](f: var RodFile; x: var T)

  Load a non-sequence/string `T`. Source Edit

    proc loadSection(f: var RodFile; expected: RodSection) {....raises: [IOError],
        tags: [ReadIOEffect], forbids: [].}

  read the bytes value of s, sets and error if the section is incorrect. Source Edit

    proc loadSeq[T](f: var RodFile; s: var seq[T])

  `T` must be compatible with `copyMem`, see `loadPrim` Source Edit

    proc open(filename: string): RodFile {....raises: [], tags: [], forbids: [].}

  open the file for reading Source Edit

    proc storeHeader(f: var RodFile; cookie = defaultCookie) {....raises: [IOError],
        tags: [WriteIOEffect], forbids: [].}

  stores the header which is described by `cookie`. Source Edit

    proc storeOrderedTable[K, T](f: var RodFile; s: OrderedTable[K, T])

  Source Edit

    proc storePrim(f: var RodFile; s: string) {....raises: [IOError],
        tags: [WriteIOEffect], forbids: [].}

  Stores a string. The len is prefixed to allow for later retreival. Source Edit

    proc storePrim[T](f: var RodFile; x: T)

  Stores a non-sequence/string `T`. If `T` doesn't support `copyMem` and is an object or tuple then the fields are written -- the user from context will need to know which `T` to load. Source Edit

    proc storeSection(f: var RodFile; s: RodSection) {....raises: [IOError],
        tags: [WriteIOEffect], forbids: [].}

  update `currentSection` and writes the bytes value of s. Source Edit

    proc storeSeq[T](f: var RodFile; s: seq[T])

  Stores a sequence of `T`s, with the len as a prefix for later retrieval. Source Edit