diff --git a/versioned_docs/version-v1.16.0/README.md b/versioned_docs/version-v1.16.0/README.md
new file mode 100644
index 00000000..765b6c15
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/README.md
@@ -0,0 +1,108 @@
+---
+sidebar_position: 1
+sidebar_label: Introduction
+---
+
+# The Zed Project
+
+Zed offers a new approach to data that makes it easier to manipulate and manage
+your data.
+
+With Zed's new [super-structured data model](formats/README.md#2-zed-a-super-structured-pattern),
+messy JSON data can easily be given the fully-typed precision of relational tables
+without giving up JSON's uncanny ability to represent eclectic data.
+
+## Getting Started
+
+Trying out Zed is easy: just [install](install.md) the command-line tool
+[`zq`](commands/zq.md) and run through the [zq tutorial](tutorials/zq.md).
+
+`zq` is a lot like [`jq`](https://stedolan.github.io/jq/)
+but is built from the ground up as a search and analytics engine based
+on the [Zed data model](formats/zed.md). Since Zed data is a
+proper superset of JSON, `zq` also works natively with JSON.
+
+While `zq` and the Zed data formats are production quality, the Zed project's
+[Zed data lake](commands/zed.md) is a bit [earlier in development](commands/zed.md#status).
+
+For a non-technical user, Zed is as easy to use as web search
+while for a technical user, Zed exposes its technical underpinnings
+in a gradual slope, providing as much detail as desired,
+packaged up in the easy-to-understand
+[ZSON data format](formats/zson.md) and
+[Zed language](language/README.md).
+
+## Terminology
+
+"Zed" is an umbrella term that describes
+a number of different elements of the system:
+* The [Zed data model](formats/zed.md) is the abstract definition of the data types and semantics
+that underlie the Zed formats.
+* The [Zed formats](formats/README.md) are a family of
+[sequential (ZNG)](formats/zng.md), [columnar (VNG)](formats/vng.md),
+and [human-readable (ZSON)](formats/zson.md) formats that all adhere to the
+same abstract Zed data model.
+* A [Zed lake](commands/zed.md) is a collection of Zed data stored
+across one or more [data pools](commands/zed.md#data-pools) with ACID commit semantics and
+accessed via a [Git](https://git-scm.com/)-like API.
+* The [Zed language](language/README.md) is the system's dataflow language for performing
+queries, searches, analytics, transformations, or any of the above combined together.
+* A [Zed query](language/overview.md) is a Zed script that performs
+search and/or analytics.
+* A [Zed shaper](language/shaping.md) is a Zed script that performs
+data transformation to _shape_
+the input data into the desired set of organizing Zed data types called "shapes",
+which are traditionally called _schemas_ in relational systems but are
+much more flexible in the Zed system.
+
+## Digging Deeper
+
+The [Zed language documentation](language/README.md)
+is the best way to learn about `zq` in depth.
+All of its examples use `zq` commands run on the command line.
+Run `zq -h` for a list of command options and online help.
+
+The [Zed lake documentation](commands/zed.md)
+is the best way to learn about `zed`.
+All of its examples use `zed` commands run on the command line.
+Run `zed -h` or `-h` with any subcommand for a list of command options
+and online help. The same language query that works for `zq` operating
+on local files or streams also works for `zed query` operating on a lake.
+
+## Design Philosophy
+
+The design philosophy for Zed is based on composable building blocks
+built from self-describing data structures. Everything in a Zed lake
+is built from Zed data and each system component can be run and tested in isolation.
+
+Since Zed data is self-describing, this approach makes stream composition
+very easy. Data from a Zed query can trivially be piped to a local
+instance of `zq` by feeding the resulting Zed stream to stdin of `zq`, for example,
+```
+zed query "from pool | ...remote query..." | zq "...local query..." -
+```
+There is no need to configure the Zed entities with schema information
+like [protobuf configs](https://developers.google.com/protocol-buffers/docs/proto3)
+or connections to
+[schema registries](https://docs.confluent.io/platform/current/schema-registry/index.html).
+
+A Zed lake is completely self-contained, requiring no auxiliary databases
+(like the [Hive metastore](https://cwiki.apache.org/confluence/display/hive/design))
+or other third-party services to interpret the lake data.
+Once copied, a new service can be instantiated by pointing a `zed serve`
+at the copy of the lake.
+
+Functionality like [data compaction](commands/zed.md#manage) and retention are all API-driven.
+
+Bite-sized components are unified by the Zed data, usually in the ZNG format:
+* All lake meta-data is available via meta-queries.
+* All like operations available through the service API are also available
+directly via the `zed` command.
+* Lake management is agent-driven through the API. For example, instead of complex policies
+like data compaction being implemented in the core with some fixed set of
+algorithms and policies, an agent can simply hit the API to obtain the meta-data
+of the objects in the lake, analyze the objects (e.g., looking for too much
+key space overlap) and issue API commands to merge overlapping objects
+and delete the old fragmented objects, all with the transactional consistency
+of the commit log.
+* Components are easily tested and debugged in isolation.
diff --git a/versioned_docs/version-v1.16.0/commands/README.md b/versioned_docs/version-v1.16.0/commands/README.md
new file mode 100644
index 00000000..4c6b608e
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/commands/README.md
@@ -0,0 +1,18 @@
+# Command Tooling
+
+The Zed system is managed and queried with the [`zed` command](zed.md),
+which is organized into numerous subcommands like the familiar command patterns
+of `docker` or `kubectrl`.
+Built-in help for the `zed` command and all of its subcommands is always
+accessible with the `-h` flag.
+
+The [`zq` command](zq.md) offers a convenient slice of `zed` for running
+stand-alone, command-line queries on inputs from files, HTTP URLs, or [S3](../integrations/amazon-s3.md).
+`zq` is like [`jq`](https://stedolan.github.io/jq/) but is easier and faster, utilizes the richer
+Zed data model, and interoperates with a number of other formats beyond JSON.
+If you don't need a Zed lake, you can install just the
+slimmer `zq` command which omits lake support and dev tools.
+
+`zq` is always installed alongside `zed`. You might find yourself mixing and
+matching `zed` lake queries with `zq` local queries and stitching them
+all together with Unix pipelines.
diff --git a/versioned_docs/version-v1.16.0/commands/_category_.yaml b/versioned_docs/version-v1.16.0/commands/_category_.yaml
new file mode 100644
index 00000000..e5c770ba
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/commands/_category_.yaml
@@ -0,0 +1,2 @@
+position: 3
+label: Commands
diff --git a/versioned_docs/version-v1.16.0/commands/zed.md b/versioned_docs/version-v1.16.0/commands/zed.md
new file mode 100644
index 00000000..980a42e0
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/commands/zed.md
@@ -0,0 +1,806 @@
+---
+sidebar_position: 2
+sidebar_label: zed
+---
+
+# zed
+
+> **TL;DR** `zed` is a command-line tool to manage and query Zed data lakes.
+> You can import data from a variety of formats and `zed` will automatically
+> commit the data in the Zed data model's [super-structured](../formats/README.md)
+> format, providing full fidelity of the original format and the ability
+> to reconstruct the original data without loss of information.
+>
+> Zed lakes provide an easy-to-use substrate for data discovery, preparation,
+> and transformation as well as serving as a queryable and searchable store
+> for super-structured data both for online and archive use cases.
+
+
+
+:::tip Status
+While [`zq`](zq.md) and the [Zed formats](../formats/README.md)
+are production quality, the Zed lake is still fairly early in development
+and alpha quality.
+That said, Zed lakes can be utilized quite effectively at small scale,
+or at larger scales when scripted automation
+is deployed to manage the lake's data layout via the
+[lake API](../lake/api.md).
+
+Enhanced scalability with self-tuning configuration is under development.
+:::
+
+## The Lake Model
+
+A Zed lake is a cloud-native arrangement of data, optimized for search,
+analytics, ETL, data discovery, and data preparation
+at scale based on data represented in accordance
+with the [Zed data model](../formats/zed.md).
+
+A lake is organized into a collection of data pools forming a single
+administrative domain. The current implementation supports
+ACID append and delete semantics at the commit level while
+we have plans to support CRUD updates at the primary-key level
+in the near future.
+
+The semantics of a Zed lake loosely follows the nomenclature and
+design patterns of [`git`](https://git-scm.com/). In this approach,
+* a _lake_ is like a GitHub organization,
+* a _pool_ is like a `git` repository,
+* a _branch_ of a _pool_ is like a `git` branch,
+* the _use_ command is like a `git checkout`, and
+* the _load_ command is like a `git add/commit/push`.
+
+A core theme of the Zed lake design is _ergonomics_. Given the Git metaphor,
+our goal here is that the Zed lake tooling be as easy and familiar as Git is
+to a technical user.
+
+Since Zed lakes are built around the Zed data model,
+getting different kinds of data into and out of a lake is easy.
+There is no need to define schemas or tables and then fit
+semi-structured data into schemas before loading data into a lake.
+And because Zed supports a large family of formats and the load endpoint
+automatically detects most formats, it's easy to just load data into a lake
+without thinking about how to convert it into the right format.
+
+### CLI-First Approach
+
+The Zed project has taken a _CLI-first approach_ to designing and implementing
+the system. Any time a new piece of functionality is added to the lake,
+it is first implemented as a `zed` command. This is particularly convenient
+for testing and continuous integration as well as providing intuitive,
+bite-sized chunks for learning how the system works and how the different
+components come together.
+
+While the CLI-first approach provides these benefits,
+all of the functionality is also exposed through [an API](../lake/api.md) to
+a Zed service. Many use cases involve an application like
+[Zui](https://zui.brimdata.io/) or a
+programming environment like Python/Pandas interacting
+with the service API in place of direct use with the `zed` command.
+
+### Storage Layer
+
+The Zed lake storage model is designed to leverage modern cloud object stores
+and separates compute from storage.
+
+A lake is entirely defined by a collection of cloud objects stored
+at a configured object-key prefix. This prefix is called the _storage path_.
+All of the meta-data describing the data pools, branches, commit history,
+and so forth is stored as cloud objects inside of the lake. There is no need
+to set up and manage an auxiliary metadata store.
+
+Data is arranged in a lake as a set of pools, which are comprised of one
+or more branches, which consist of a sequence of data commit objects
+that point to cloud data objects.
+
+Cloud objects and commits are immutable and named with globally unique IDs,
+based on the [KSUIDs](https://github.com/segmentio/ksuid), and many
+commands may reference various lake entities by their ID, e.g.,
+* _Pool ID_ - the KSUID of a pool
+* _Commit object ID_ - the KSUID of a commit object
+* _Data object ID_ - the KSUID of a committed data object
+
+Data is added and deleted from the lake only with new commits that
+are implemented in a transactionally consistent fashion. Thus, each
+commit object (identified by its globally-unique ID) provides a completely
+consistent view of an arbitrarily large amount of committed data
+at a specific point in time.
+
+While this commit model may sound heavyweight, excellent live ingest performance
+can be achieved by micro-batching commits.
+
+Because the Zed lake represents all state transitions with immutable objects,
+the caching of any cloud object (or byte ranges of cloud objects)
+is easy and effective since a cached object is never invalid.
+This design makes backup/restore, data migration, archive, and
+replication easy to support and deploy.
+
+The cloud objects that comprise a lake, e.g., data objects,
+commit history, transaction journals, partial aggregations, etc.,
+are stored as Zed data, i.e., either as [row-based ZNG](../formats/zng.md)
+or [columnar VNG](../formats/vng.md).
+This makes introspection of the lake structure straightforward as many key
+lake data structures can be queried with metadata queries and presented
+to a client as Zed data for further processing by downstream tooling.
+
+Zed's implementation also includes a storage abstraction that maps the cloud object
+model onto a file system so that Zed lakes can also be deployed on standard file systems.
+
+### Zed Command Personalities
+
+The `zed` command provides a single command-line interface to Zed lakes, but
+different personalities are taken on by `zed` depending on the particular
+sub-command executed and the [lake location](#locating-the-lake).
+
+To this end, `zed` can take on one of three personalities:
+
+* _Direct Access_ - When the lake is a storage path (`file` or `s3` URI),
+then the `zed` commands (except for `serve`) all operate directly on the
+lake located at that path.
+* _Client Personality_ - When the lake is an HTTP or HTTPS URL, then the
+lake is presumed to be a Zed lake service endpoint and the client
+commands are directed to the service managing the lake.
+* _Server Personality_ - When the [`zed serve`](#serve) command is executed, then
+the personality is always the server personality and the lake must be
+a storage path. This command initiates a continuous server process
+that serves client requests for the lake at the configured storage path.
+
+Note that a storage path on the file system may be specified either as
+a fully qualified file URI of the form `file://` or be a standard
+file system path, relative or absolute, e.g., `/lakes/test`.
+
+Concurrent access to any Zed lake storage, of course, preserves
+data consistency. You can run multiple `zed serve` processes while also
+running any `zed` lake command all pointing at the same storage endpoint
+and the lake's data footprint will always remain consistent as the endpoints
+all adhere to the consistency semantics of the Zed lake.
+
+> One caveat here: data consistency is not fully implemented yet for
+> the S3 endpoint so only single-node access to S3 is available right now,
+> though support for multi-node access is forthcoming.
+> For a shared file system, the close-to-open cache consistency
+> semantics of NFS should provide the necessary consistency guarantees needed by
+> a Zed lake though this has not been tested. Multi-process, single-node
+> access to a local file system has been thoroughly tested and should be
+> deemed reliable, i.e., you can run a direct-access instance of `zed` alongside
+> a server instance of `zed` on the same file system and data consistency will
+> be maintained.
+
+### Locating the Lake
+
+At times you may want the Zed CLI tools to access the same lake storage
+used by other tools such as [Zui](https://zui.brimdata.io/). To help
+enable this by default while allowing for separate lake storage when desired,
+`zed` checks each of the following in order to attempt to locate an existing
+lake.
+
+1. The contents of the `-lake` option (if specified)
+2. The contents of the `ZED_LAKE` environment variable (if defined)
+3. A Zed lake service running locally at `http://localhost:9867` (if a socket
+ is listening at that port)
+4. A `zed` subdirectory below a path in the
+ [`XDG_DATA_HOME`](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html)
+ environment variable (if defined)
+5. A default file system location based on detected OS platform:
+ - `%LOCALAPPDATA%\zed` on Windows
+ - `$HOME/.local/share/zed` on Linux and macOS
+
+### Data Pools
+
+A lake is made up of _data pools_, which are like "collections" in NoSQL
+document stores. Pools may have one or more branches and every pool always
+has a branch called `main`.
+
+A pool is created with the [create command](#create)
+and a branch of a pool is created with the [branch command](#branch).
+
+A pool name can be any valid UTF-8 string and is allocated a unique ID
+when created. The pool can be referred to by its name or by its ID.
+A pool may be renamed but the unique ID is always fixed.
+
+#### Commit Objects
+
+Data is added into a pool in atomic units called _commit objects_.
+
+Each commit object is assigned a global ID.
+Similar to Git, Zed commit objects are arranged into a tree and
+represent the entire commit history of the lake.
+
+> Technically speaking, Git can merge from multiple parents and thus
+Git commits form a directed acyclic graph instead of a tree;
+Zed does not currently support multiple parents in the commit object history.
+
+A branch is simply a named pointer to a commit object in the Zed lake
+and like a pool, a branch name can be any valid UTF-8 string.
+Consistent updates to a branch are made by writing a new commit object that
+points to the previous tip of the branch and updating the branch to point at
+the new commit object. This update may be made with a transaction constraint
+(e.g., requiring that the previous branch tip is the same as the
+commit object's parent); if the constraint is violated, then the transaction
+is aborted.
+
+The _working branch_ of a pool may be selected on any command with the `-use` option
+or may be persisted across commands with the [use command](#use) so that
+`-use` does not have to be specified on each command-line. For interactive
+workflows, the `use` command is convenient but for automated workflows
+in scripts, it is good practice to explicitly specify the branch in each
+command invocation with the `-use` option.
+
+#### Commitish
+
+Many `zed` commands operate with respect to a commit object.
+While commit objects are always referenceable by their commit ID, it is also convenient
+to refer to the commit object at the tip of a branch.
+
+The entity that represents either a commit ID or a branch is called a _commitish_.
+A commitish is always relative to the pool and has the form:
+* `@` or
+* `@`
+
+where `` is a pool name or pool ID, `` is a commit object ID,
+and `` is a branch name.
+
+In particular, the working branch set by the [use command](#use) is a commitish.
+
+A commitish may be abbreviated in several ways where the missing detail is
+obtained from the working-branch commitish, e.g.,
+* `` - When just a pool name is given, then the commitish is assumed to be
+`@main`.
+* `@` or ``- When an ID is given (optionally with the `@` prefix), then the commitish is assumed to be `@` where `` is obtained from the working-branch commitish.
+* `@` - When a branch name is given with the `@` prefix, then the commitish is assumed to be `@` where `` is obtained from the working-branch commitish.
+
+An argument to a command that takes a commit object is called a _commitish_
+since it can be expressed as a branch or as a commit ID.
+
+#### Pool Key
+
+Each data pool is organized according to its configured _pool key_,
+which is the sort key for all data stored in the lake. Different data pools
+can have different pool keys but all of the data in a pool must have the same
+pool key.
+
+As pool data is often comprised of Zed records (analogous to JSON objects),
+the pool key is typically a field of the stored records.
+When pool data is not structured as records/objects (e.g., scalar or arrays or other
+non-record types), then the pool key would typically be configured
+as the [special value `this`](../language/dataflow-model.md#the-special-value-this).
+
+Data can be efficiently scanned if a query has a filter operating on the pool
+key. For example, on a pool with pool key `ts`, the query `ts == 100`
+will be optimized to scan only the data objects where the value `100` could be
+present.
+
+> The pool key will also serve as the primary key for the forthcoming
+> CRUD semantics.
+
+A pool also has a configured sort order, either ascending or descending
+and data is organized in the pool in accordance with this order.
+Data scans may be either ascending or descending, and scans that
+follow the configured order are generally more efficient than
+scans that run in the opposing order.
+
+Scans may also be range-limited but unordered.
+
+Any data loaded into a pool that lacks the pool key is presumed
+to have a null value with regard to range scans. If large amounts
+of such "keyless data" are loaded into a pool, the ability to
+optimize scans over such data is impaired.
+
+### Time Travel
+
+Because commits are transactional and immutable, a query
+sees its entire data scan as a fixed "snapshot" with respect to the
+commit history. In fact, Zed's [from operator](../language/operators/from.md)
+allows a commit object to be specified with the `@` suffix to a
+pool reference, e.g.,
+```
+zed query 'from logs@1tRxi7zjT7oKxCBwwZ0rbaiLRxb | ...'
+```
+In this way, a query can time-travel through the commit history. As long as the
+underlying data has not been deleted, arbitrarily old snapshots of the Zed
+lake can be easily queried.
+
+If a writer commits data after and while a reader is scanning, then the reader
+does not see the new data since it's scanning the snapshot that existed
+before these new writes occurred.
+
+Also, arbitrary metadata can be committed to the log as described below,
+e.g., to associate derived analytics to a specific
+journal commit point potentially across different data pools in
+a transactionally consistent fashion.
+
+While time travel through commit history provides one means to explore
+past snapshots of the commit history, another means is to use a timestamp.
+Because the entire history of branch updates is stored in a transaction journal
+and each entry contains a timestamp, branch references can be easily
+navigated by time. For example, a list of branches of a pool's past
+can be created by scanning the internal "pools log" and stopping at the largest
+timestamp less than or equal to the desired timestamp. Then using that
+historical snapshot of the pools, a branch can be located within the pool
+using that pool's "branches log" in a similar fashion, then its corresponding
+commit object can be used to construct the data of that branch at that
+past point in time.
+
+ > Note that time travel using timestamps is a forthcoming feature.
+
+## Zed Commands
+
+The `zed` command is structured as a primary command
+consisting of a large number of interrelated sub-commands, similar to the
+[`docker`](https://docs.docker.com/engine/reference/commandline/cli/)
+or [`kubectl`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands)
+commands.
+
+The following sections describe each of the available commands and highlight
+some key options. Built-in help shows the commands and their options:
+
+* `zed -h` with no args displays a list of `zed` commands.
+* `zed command -h`, where `command` is a sub-command, displays help
+for that sub-command.
+* `zed command sub-command -h` displays help for a sub-command of a
+sub-command and so forth.
+
+### Auth
+```
+zed auth login|logout|method|verify
+```
+Access to a Zed lake can be secured with [Auth0 authentication](https://auth0.com/).
+Please reach out to us on our [community Slack](https://www.brimdata.io/join-slack/)
+if you'd like help setting this up and trying it out.
+
+### Branch
+```
+zed branch [options] [name]
+```
+The `branch` command creates a branch with the name `name` that points
+to the tip of the working branch or, if the `name` argument is not provided,
+lists the existing branches of the selected pool.
+
+For example, this branch command
+```
+zed branch -use logs@main staging
+```
+creates a new branch called "staging" in pool "logs", which points to
+the same commit object as the "main" branch. Once created, commits
+to the "staging" branch will be added to the commit history without
+affecting the "main" branch and each branch can be queried independently
+at any time.
+
+Supposing the `main` branch of `logs` was already the working branch,
+then you could create the new branch called "staging" by simply saying
+```
+zed branch staging
+```
+Likewise, you can delete a branch with `-d`:
+```
+zed branch -d staging
+```
+and list the branches as follows:
+```
+zed branch
+```
+
+### Create
+```
+zed create [-orderby key[,key...][:asc|:desc]]
+```
+The `create` command creates a new data pool with the given name,
+which may be any valid UTF-8 string.
+
+The `-orderby` option indicates the pool key that is used to sort
+the data in lake, which may be in ascending or descending order.
+
+If a pool key is not specified, then it defaults to
+the [special value `this`](../language/dataflow-model.md#the-special-value-this).
+
+A newly created pool is initialized with a branch called `main`.
+
+> Zed lakes can be used without thinking about branches. When referencing a pool without
+> a branch, the tooling presumes the "main" branch as the default, and everything
+> can be done on main without having to think about branching.
+
+### Delete
+```
+zed delete [options] [...]
+zed delete [options] -where
+```
+The `delete` command removes one or more data objects indicated by their ID from a pool.
+This command
+simply removes the data from the branch without actually deleting the
+underlying data objects thereby allowing time travel to work in the face
+of deletes. Permanent deletion of underlying data objects is handled by the
+separate [`vacuum`](#vacuum) command.
+
+If the `-where` flag is specified, delete will remove all values for which the
+provided filter expression is true. The value provided to `-where` must be a
+single filter expression, e.g.:
+
+```
+zed delete -where 'ts > 2022-10-05T17:20:00Z and ts < 2022-10-05T17:21:00Z'
+```
+
+### Drop
+```
+zed drop [options] |
+```
+The `drop` command deletes a pool and all of its constituent data.
+As this is a DANGER ZONE command, you must confirm that you want to delete
+the pool to proceed. The `-f` option can be used to force the deletion
+without confirmation.
+
+### Init
+```
+zed init [path]
+```
+A new lake is initialized with the `init` command. The `path` argument
+is a [storage path](#storage-layer) and is optional. If not present, the path
+is [determined automatically](#locating-the-lake).
+
+If the lake already exists, `init` reports an error and does nothing.
+
+Otherwise, the `init` command writes the initial cloud objects to the
+storage path to create a new, empty lake at the specified path.
+
+### Load
+```
+zed load [options] input [input ...]
+```
+The `load` command commits new data to a branch of a pool.
+
+Run `zed load -h` for a list of command-line options.
+
+Note that there is no need to define a schema or insert data into
+a "table" as all Zed data is _self describing_ and can be queried in a
+schema-agnostic fashion. Data of any _shape_ can be stored in any pool
+and arbitrary data _shapes_ can coexist side by side.
+
+As with `zq`,
+the [input arguments](zq.md#usage) can be in
+any [supported format](zq.md#input-formats) and
+the input format is auto-detected if `-i` is not provided. Likewise,
+the inputs may be URLs, in which case, the `load` command streams
+the data from a Web server or [S3](../integrations/amazon-s3.md) and into the lake.
+
+When data is loaded, it is broken up into objects of a target size determined
+by the pool's `threshold` parameter (which defaults 500MiB but can be configured
+when the pool is created). Each object is sorted by the pool key but
+a sequence of objects is not guaranteed to be globally sorted. When lots
+of small or unsorted commits occur, data can be fragmented. The performance
+impact of fragmentation can be eliminated by regularly [compacting](#manage)
+pools.
+
+For example, this command
+```
+zed load sample1.json sample2.zng sample3.zson
+```
+loads files of varying formats in a single commit to the working branch.
+
+An alternative branch may be specified with a branch reference with the
+`-use` option, i.e., `@`. Supposing a branch
+called `live` existed, data can be committed into this branch as follows:
+```
+zed load -use logs@live sample.zng
+```
+Or, as mentioned above, you can set the default branch for the load command
+via `use`:
+```
+zed use logs@live
+zed load sample.zng
+```
+During a `load` operation, a commit is broken out into units called _data objects_
+where a target object size is configured into the pool,
+typically 100MB-1GB. The records within each object are sorted by the pool key.
+A data object is presumed by the implementation
+to fit into the memory of an intake worker node
+so that such a sort can be trivially accomplished.
+
+Data added to a pool can arrive in any order with respect to the pool key.
+While each object is sorted before it is written,
+the collection of objects is generally not sorted.
+
+Each load operation creates a single commit object, which includes:
+* an author and message string,
+* a timestamp computed by the server, and
+* an optional metadata field of any Zed type expressed as a ZSON value.
+This data has the Zed type signature:
+```
+{
+ author: string,
+ date: time,
+ message: string,
+ meta:
+}
+```
+where `` is the type of any optionally attached metadata .
+For example, this command sets the `author` and `message` fields:
+```
+zed load -user user@example.com -message "new version of prod dataset" ...
+```
+If these fields are not specified, then the Zed system will fill them in
+with the user obtained from the session and a message that is descriptive
+of the action.
+
+The `date` field here is used by the Zed lake system to do time travel
+through the branch and pool history, allowing you to see the state of
+branches at any time in their commit history.
+
+Arbitrary metadata expressed as any [ZSON value](../formats/zson.md)
+may be attached to a commit via the `-meta` flag. This allows an application
+or user to transactionally commit metadata alongside committed data for any
+purpose. This approach allows external applications to implement arbitrary
+data provenance and audit capabilities by embedding custom metadata in the
+commit history.
+
+Since commit objects are stored as Zed, the metadata can easily be
+queried by running the `log -f zng` to retrieve the log in ZNG format,
+for example, and using [`zq`](zq.md) to pull the metadata out
+as in:
+```
+zed log -f zng | zq 'has(meta) | yield {id,meta}' -
+```
+
+### Log
+```
+zed log [options] [commitish]
+```
+The `log` command, like `git log`, displays a history of the commit objects
+starting from any commit, expressed as a [commitish](#commitish). If no argument is
+given, the tip of the working branch is used.
+
+Run `zed log -h` for a list of command-line options.
+
+To understand the log contents, the `load` operation is actually
+decomposed into two steps under the covers:
+an "add" step stores one or more
+new immutable data objects in the lake and a "commit" step
+materializes the objects into a branch with an ACID transaction.
+This updates the branch pointer to point at a new commit object
+referencing the data objects where the new commit object's parent
+points at the branch's previous commit object, thus forming a path
+through the object tree.
+
+The `log` command prints the commit ID of each commit object in that path
+from the current pointer back through history to the first commit object.
+
+A commit object includes
+an optional author and message, along with a required timestamp,
+that is stored in the commit journal for reference. These values may
+be specified as options to the `load` command, and are also available in the
+API for automation.
+
+> Note that the branchlog meta-query source is not yet implemented.
+
+### Manage
+```
+zed manage [options]
+```
+The `manage` command performs maintenance tasks on a lake.
+
+Currently the only supported task is _compaction_, which reduces fragmentation
+by reading data objects in a pool and writing their contents back to large,
+non-overlapping objects.
+
+If the `-monitor` option is specified and the lake is [located](#locating-the-lake)
+via network connection, `zed manage` will run continuously and perform updates
+as needed. By default a check is performed once per minute to determine if
+updates are necessary. The `-interval` option may be used to specify an
+alternate check frequency in [duration format](../formats/zson.md#23-primitive-values).
+
+If `-monitor` is not specified, a single maintenance pass is performed on the
+lake.
+
+The output from `manage` provides a per-pool summary of the maintenance
+performed, including a count of `objects_compacted`.
+
+As an alternative to running `manage` as a separate command, the `-manage`
+option is also available on the [`serve`](#serve) command to have maintenance
+tasks run at the specified interval by the service process.
+
+### Merge
+
+Data is merged from one branch into another with the `merge` command, e.g.,
+```
+zed merge -use logs@updates main
+```
+where the `updates` branch is being merged into the `main` branch
+within the `logs` pool.
+
+A merge operation finds a common ancestor in the commit history then
+computes the set of changes needed for the target branch to reflect the
+data additions and deletions in the source branch.
+While the merge operation is performed, data can still be written concurrently
+to both branches and queries performed and everything remains transactionally
+consistent. Newly written data remains in the
+branch while all of the data present at merge initiation is merged into the
+parent.
+
+This Git-like behavior for a data lake provides a clean solution to
+the live ingest problem.
+For example, data can be continuously ingested into a branch of main called `live`
+and orchestration logic can periodically merge updates from branch `live` to
+branch `main`, possibly [compacting](#manage) data after the merge
+according to configured policies and logic.
+
+### Query
+```
+zed query [options]
+```
+The `query` command runs a Zed program with data from a lake as input.
+A query typically begins with a [from operator](../language/operators/from.md)
+indicating the pool and branch to use as input. If `from` is not present, then the
+query reads from the working branch.
+
+The pool/branch names are specified with `from` at the beginning of the Zed
+query.
+
+As with `zq`, the default output format is ZSON for
+terminals and ZNG otherwise, though this can be overridden with
+`-f` to specify one of the various supported output formats.
+
+If a pool name is provided to `from` without a branch name, then branch
+"main" is assumed.
+
+This example reads every record from the full key range of the `logs` pool
+and sends the results to stdout.
+
+```
+zed query 'from logs'
+```
+
+We can narrow the span of the query by specifying a filter on the pool key:
+```
+zed query 'from logs | ts >= 2018-03-24T17:36:30.090766Z and ts <= 2018-03-24T17:36:30.090758Z'
+```
+Filters on pool keys are efficiently implemented as the data is laid out
+according to the pool key and seek indexes keyed by the pool key
+are computed for each data object.
+
+Lake queries also can refer to HEAD (i.e., the branch context set in the most
+recent `use` command) either implicitly by omitting the `from` operator:
+```
+zed query '*'
+```
+or by referencing `HEAD`:
+```
+zed query 'from HEAD'
+```
+
+When querying data to the ZNG output format,
+output from a pool can be easily piped to other commands like `zq`, e.g.,
+```
+zed query -f zng 'from logs' | zq -f table 'count() by field' -
+```
+Of course, it's even more efficient to run the query inside of the pool traversal
+like this:
+```
+zed query -f table 'from logs | count() by field'
+```
+By default, the `query` command scans pool data in pool-key order though
+the Zed optimizer may, in general, reorder the scan to optimize searches,
+aggregations, and joins.
+An order hint can be supplied to the `query` command to indicate to
+the optimizer the desired processing order, but in general, `sort` operators
+should be used to guarantee any particular sort order.
+
+Arbitrarily complex Zed queries can be executed over the lake in this fashion
+and the planner can utilize cloud resources to parallelize and scale the
+query over many parallel workers that simultaneously access the Zed lake data in
+shared cloud storage (while also accessing locally- or cluster-cached copies of data).
+
+#### Meta-queries
+
+Commit history, metadata about data objects, lake and pool configuration,
+etc. can all be queried and
+returned as Zed data, which in turn, can be fed into Zed analytics.
+This allows a very powerful approach to introspecting the structure of a
+lake making it easy to measure, tune, and adjust lake parameters to
+optimize layout for performance.
+
+These structures are introspected using meta-queries that simply
+specify a metadata source using an extended syntax in the `from` operator.
+There are three types of meta-queries:
+* `from :` - lake level
+* `from pool:` - pool level
+* `from pool[@]<:meta>` - branch level
+
+`` is the name of the metadata being queried. The available metadata
+sources vary based on level.
+
+For example, a list of pools with configuration data can be obtained
+in the ZSON format as follows:
+```
+zed query -Z "from :pools"
+```
+This meta-query produces a list of branches in a pool called `logs`:
+```
+zed query -Z "from logs:branches"
+```
+Since this is all just Zed, you can filter the results just like any query,
+e.g., to look for particular branch:
+```
+zed query -Z "from logs:branches | branch.name=='main'"
+```
+
+This meta-query produces a list of the data objects in the `live` branch
+of pool `logs`:
+```
+zed query -Z "from logs@live:objects"
+```
+
+You can also pretty-print in human-readable form most of the metadata Zed records
+using the "lake" format, e.g.,
+```
+zed query -f lake "from logs@live:objects"
+```
+
+The `main` branch is queried by default if an explicit branch is not specified,
+e.g.,
+
+```
+zed query -f lake "from logs:objects"
+```
+
+### Rename
+```
+zed rename
+```
+The `rename` command assigns a new name `` to an existing
+pool ``, which may be referenced by its ID or its previous name.
+
+### Serve
+```
+zed serve [options]
+```
+The `serve` command implements Zed's server personality to service requests
+from instances of Zed's client [personality](#zed-command-personalities).
+It listens for Zed lake API requests on the interface and port
+specified by the `-l` option, executes the requests, and returns results.
+
+The `-log.level` option controls log verbosity. Available levels, ordered
+from most to least verbose, are `debug`, `info` (the default), `warn`,
+`error`, `dpanic`, `panic`, and `fatal`. If the volume of logging output at
+the default `info` level seems too excessive for production use, `warn` level
+is recommended.
+
+The `-manage` option enables the running of the same maintenance tasks
+normally performed via the separate [`manage`](#manage) command.
+
+### Use
+```
+zed use []
+```
+The `use` command sets the working branch to the indicated commitish.
+When run with no argument, it displays the working branch and [lake](#locating-the-lake).
+
+For example,
+```
+zed use logs
+```
+provides a "pool-only" commitish that sets the working branch to `logs@main`.
+
+If a `@branch` or commit ID are given without a pool prefix, then the pool of
+the commitish previously in use is presumed. For example, if you are on
+`logs@main` then run this command:
+```
+zed use @test
+```
+then the working branch is set to `logs@test`.
+
+To specify a branch in another pool, simply prepend
+the pool name to the desired branch:
+```
+zed use otherpool@otherbranch
+```
+This command stores the working branch in `$HOME/.zed_head`.
+
+### Vacuum
+```
+zed vacuum [options]
+```
+
+The `vacuum` command permanently removes underlying data objects that have
+previously been subject to a [`delete`](#delete) operation. As this is a
+DANGER ZONE command, you must confirm that you want to remove
+the objects to proceed. The `-f` option can be used to force removal
+without confirmation. The `-dryrun` option may also be used to see a summary
+of how many objects would be removed by a `vacuum` but without removing them.
diff --git a/versioned_docs/version-v1.16.0/commands/zq.md b/versioned_docs/version-v1.16.0/commands/zq.md
new file mode 100644
index 00000000..2e04cba2
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/commands/zq.md
@@ -0,0 +1,695 @@
+---
+sidebar_position: 1
+sidebar_label: zq
+description: A command-line tool that uses the Zed Language for pipeline-style search and analytics.
+---
+
+# zq
+
+> **TL;DR** `zq` is a command-line tool that uses the [Zed language](../language/README.md)
+for pipeline-style search and analytics. `zq` can query a variety
+of data formats in files, over HTTP, or in [S3](../integrations/amazon-s3.md) storage.
+It is particularly fast when operating on data in the Zed-native [ZNG](../formats/zng.md) format.
+>
+> The `zq` design philosophy blends the query/search-tool approach
+of `jq`, `awk`, and `grep` with the command-line, embedded database approach
+of `sqlite` and `duckdb`.
+
+## Usage
+
+```
+zq [ options ] [ query ] input [ input ... ]
+zq [ options ] query
+```
+`zq` is a command-line tool for processing data in diverse input
+formats, providing search, analytics, and extensive transformations
+using the [Zed language](../language/README.md). A query typically applies Boolean logic
+or keyword search to filter the input, then transforms or analyzes
+the filtered stream. Output is written to one or more files or to
+standard output.
+
+Each `input` argument must be a file path, an HTTP or HTTPS URL,
+an S3 URL, or standard input specified with `-`.
+
+For built-in command help and a listing of all available options,
+simply run `zq` with no arguments.
+
+`zq` supports a [number of formats](#input-formats) but [ZNG](../formats/zng.md)
+tends to be the most space-efficient and most performant. ZNG has efficiency similar to
+[Avro](https://avro.apache.org/docs/current/spec.html)
+and [Protocol Buffers](https://developers.google.com/protocol-buffers)
+but its comprehensive [Zed type system](../formats/zed.md) obviates
+the need for schema specification or registries.
+Also, the [ZSON](../formats/zson.md) format is human-readable and entirely one-to-one with ZNG
+so there is no need to represent non-readable formats like Avro or Protocol Buffers
+in a clunky JSON encapsulated form.
+
+`zq` typically operates on ZNG-encoded data and when you want to inspect
+human-readable bits of output, you merely format it as ZSON, which is the
+default format when output is directed to the terminal. ZNG is the default
+when redirecting to a non-terminal output like a file or pipe.
+
+When run with input arguments, each input's format is [automatically inferred](#auto-detection)
+and each input is scanned
+in the order appearing on the command line forming the input stream.
+
+A query expressed in the [Zed language](../language/README.md)
+may be optionally specified and applied to the input stream.
+
+If no query is specified, the inputs are scanned without modification
+and output in the desired format as described below. This latter approach
+provides a convenient means to convert files from one format to another.
+
+To determine whether the first argument is a query or an input,
+`zq` checks the local file system for the existence of a file by that name
+or whether the name is an URL.
+If no such file or URL exists, it attempts to parse the text as a Zed program.
+If both checks fail, then an error is reported and `zq` exits.
+This heuristic is convenient but can result in a rare surprise when a simple
+Zed query (like a keyword search) happens to correspond with a file of the
+same name in the local directory.
+
+When `zq` is run with a query and no input arguments, then the query must
+begin with
+* a [from, file, or get operator](../language/operators/from.md), or
+* an explicit or implied [yield operator](../language/operators/yield.md).
+
+In the case of a `yield` with no inputs, the query is run with
+a single input value of `null`. This provides a convenient means to run in a
+"calculator mode" where input is produced by the yield and can be operated upon
+by the Zed query, e.g.,
+```mdtest-command
+zq -z '1+1'
+```
+emits
+```mdtest-output
+2
+```
+Note here that the query `1+1` [implies](../language/dataflow-model.md#implied-operators)
+`yield 1+1`.
+
+## Input Formats
+
+`zq` currently supports the following input formats:
+
+| Option | Auto | Specification |
+|-----------|------|------------------------------------------|
+| `arrows` | yes | [Arrow IPC Stream Format](https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format) |
+| `json` | yes | [JSON RFC 8259](https://www.rfc-editor.org/rfc/rfc8259.html) |
+| `csv` | yes | [CSV RFC 4180](https://www.rfc-editor.org/rfc/rfc4180.html) |
+| `line` | no | One string value per input line |
+| `parquet` | yes | [Apache Parquet](https://github.com/apache/parquet-format) |
+| `tsv` | yes | [TSV - Tab-Separated Values](https://en.wikipedia.org/wiki/Tab-separated_values) |
+| `vng` | yes | [VNG - Binary Columnar Format](../formats/vng.md) |
+| `zson` | yes | [ZSON - Human-readable Format](../formats/zson.md) |
+| `zng` | yes | [ZNG - Binary Row Format](../formats/zson.md) |
+| `zjson` | yes | [ZJSON - Zed over JSON](../formats/zjson.md) |
+| `zeek` | yes | [Zeek Logs](https://docs.zeek.org/en/master/logs/index.html) |
+
+The input format is typically detected automatically and the formats for which
+`Auto` is `yes` in the table above support _auto-detection_.
+Formats without auto-detection require the `-i` option.
+
+### Hard-wired Input Format
+
+The input format is specified with the `-i` flag.
+
+When `-i` is specified, all of the inputs on the command-line must be
+in the indicated format.
+
+### Auto-detection
+
+When using _auto-detection_, each input's format is independently determined
+so it is possible to easily blend different input formats into a unified
+output format.
+
+For example, suppose this content is in a file `sample.csv`:
+```mdtest-input sample.csv
+a,b
+1,foo
+2,bar
+```
+and this content is in `sample.json`
+```mdtest-input sample.json
+{"a":3,"b":"baz"}
+```
+then the command
+```mdtest-command
+zq -z sample.csv sample.json
+```
+would produce this output in the default ZSON format
+```mdtest-output
+{a:1.,b:"foo"}
+{a:2.,b:"bar"}
+{a:3,b:"baz"}
+```
+
+### ZSON-JSON Auto-detection
+
+Since ZSON is a superset of JSON, `zq` must be careful in whether it
+interprets input as ZSON as JSON. While you can always clarify your intent
+with the `-i zson` or `-i json`, `zq` attempts to "just do the right thing"
+when you run it with JSON vs. ZSON.
+
+While `zq` can parse any JSON using its built-in ZSON parser this is typically
+not desirable because (1) the ZSON parser is not particularly performant and
+(2) all JSON numbers are floating point but the ZSON parser will parse as
+JSON any number that appears without a decimal point as an integer type.
+
+> The reason `zq` is not particularly performant for ZSON is that the ZNG or
+> VNG formats are semantically equivalent to ZSON but much more efficient and
+> the design intent is that these efficient binary formats should be used in
+> use cases where performance matters. ZSON is typically used only when
+> data needs to be human-readable in interactive settings or in automated tests.
+
+To this end, `zq` uses a heuristic to select between ZSON in JSON when the
+`-i` option is not specified. Specifically, JSON is selected when the first values
+of the input are parsable as valid JSON and includes a JSON object either
+as an outer object or as a value nested somewhere within a JSON array.
+
+This heuristic almost always works in practice because ZSON records
+typically omit quotes around field names.
+
+## Output Formats
+
+The output format defaults to either ZSON or ZNG and may be specified
+with the `-f` option. The supported output formats include all of
+the input formats along with text and table formats, which are useful
+for displaying data. (They do not capture all the information required
+to reconstruct the original data so they are not supported input formats.)
+
+Since ZSON is a common format choice, the `-z` flag is a shortcut for
+`-f zson.` Also, `-Z` is a shortcut for `-f zson` with `-pretty 4` as
+described below.
+
+And since JSON is another common format choice, the `-j` flag is a shortcut for
+`-f json.`
+
+### Output Format Selection
+
+When the format is not specified with `-f`, it defaults to ZSON if the output
+is a terminal and to ZNG otherwise.
+
+While this can cause an occasional surprise (e.g., forgetting `-f` or `-z`
+in a scripted test that works fine on the command line but fails in CI),
+we felt that the design of having a uniform default had worse consequences:
+* If the default format were ZSON, it would be very easy to create pipelines
+and deploy to production systems that were accidentally using ZSON instead of
+the much more efficient ZNG format because the `-f zng` had been mistakenly
+omitted from some command. The beauty of Zed is that all of this "just works"
+but it would otherwise perform poorly.
+* If the default format were ZNG, then users would be endlessly annoyed by
+binary output to their terminal when forgetting to type `-f zson`.
+
+In practice, we have found that the output defaults
+"just do the right thing" almost all of the time.
+
+### ZSON Pretty Printing
+
+ZSON text may be "pretty printed" with the `-pretty` option, which takes
+the number of spaces to use for indentation. As this is a common option,
+the `-Z` option is a shortcut for `-f zson -pretty 4`.
+
+For example,
+```mdtest-command
+echo '{a:{b:1,c:[1,2]},d:"foo"}' | zq -Z -
+```
+produces
+```mdtest-output
+{
+ a: {
+ b: 1,
+ c: [
+ 1,
+ 2
+ ]
+ },
+ d: "foo"
+}
+```
+and
+```mdtest-command
+echo '{a:{b:1,c:[1,2]},d:"foo"}' | zq -f zson -pretty 2 -
+```
+produces
+```mdtest-output
+{
+ a: {
+ b: 1,
+ c: [
+ 1,
+ 2
+ ]
+ },
+ d: "foo"
+}
+```
+
+When pretty printing, colorization is enabled by default when writing to a terminal,
+and can be disabled with `-color false`.
+
+### Pipeline-friendly ZNG
+
+Though it's a compressed binary format, ZNG data is self-describing and stream-oriented
+and thus is pipeline friendly.
+
+Since data is self-describing you can simply take ZNG output
+of one command and pipe it to the input of another. It doesn't matter if the value
+sequence is scalars, complex types, or records. There is no need to declare
+or register schemas or "protos" with the downstream entities.
+
+In particular, ZNG data can simply be concatenated together, e.g.,
+```mdtest-command
+zq -f zng 'yield 1,[1,2,3]' > a.zng
+zq -f zng 'yield {s:"hello"},{s:"world"}' > b.zng
+cat a.zng b.zng | zq -z -
+```
+produces
+```mdtest-output
+1
+[1,2,3]
+{s:"hello"}
+{s:"world"}
+```
+And while this ZSON output is human readable, the ZNG files are binary, e.g.,
+```mdtest-command
+zq -f zng 'yield 1,[1,2,3]' > a.zng
+hexdump -C a.zng
+```
+produces
+```mdtest-output
+00000000 02 00 01 09 1b 00 09 02 02 1e 07 02 02 02 04 02 |................|
+00000010 06 ff |..|
+00000012
+```
+
+### Schema-rigid Outputs
+
+Certain data formats like Arrow and Parquet are "schema rigid" in the sense that
+they require a schema to be defined before values can be written into the file
+and all the values in the file must conform to this schema.
+
+Zed, however, has a fine-grained type system instead of schemas and a sequence
+of data values are completely self-describing and may be heterogeneous in nature.
+This creates a challenge converting the type-flexible Zed formats to a schema-rigid
+format like Arrow and Parquet.
+
+For example, this seemingly simple conversion:
+```mdtest-command fails
+echo '{x:1}{s:"hello"}' | zq -o out.parquet -f parquet -
+```
+causes this error
+```mdtest-output
+parquetio: encountered multiple types (consider 'fuse'): {x:int64} and {s:string}
+```
+
+#### Fusing Schemas
+
+As suggested by the error above, the Zed `fuse` operator can merge different record
+types into a blended type, e.g., here we create the file and read it back:
+```mdtest-command
+echo '{x:1}{s:"hello"}' | zq -o out.parquet -f parquet fuse -
+zq -z out.parquet
+```
+but the data was necessarily changed (by inserting nulls):
+```mdtest-output
+{x:1,s:null(string)}
+{x:null(int64),s:"hello"}
+```
+
+#### Splitting Schemas
+
+Another common approach to dealing with the schema-rigid limitation of Arrow and
+Parquet is to create a separate file for each schema.
+
+`zq` can do this too with the `-split` option, which specifies a path
+to a directory for the output files. If the path is `.`, then files
+are written to the current directory.
+
+The files are named using the `-o` option as a prefix and the suffix is
+`-.` where the `` is determined from the output format and
+where `` is a unique integer for each distinct output file.
+
+For example, the example above would produce two output files,
+which can then be read separately to reproduce the original data, e.g.,
+```mdtest-command
+echo '{x:1}{s:"hello"}' | zq -o out -split . -f parquet -
+zq -z out-*.parquet
+```
+produces the original data
+```mdtest-output
+{x:1}
+{s:"hello"}
+```
+
+While the `-split` option is most useful for schema-rigid formats, it can
+be used with any output format.
+
+## Query Debugging
+
+If you are ever stumped about how the `zq` compiler is parsing your query,
+you can always run `zq -C` to compile and display your query in canonical form
+without running it.
+This can be especially handy when you are learning the language and
+[its shortcuts](../language/dataflow-model.md#implied-operators).
+
+For example, this query
+```mdtest-command
+zq -C 'has(foo)'
+```
+is an implied [where operator](../language/operators/where.md), which matches values
+that have a field `foo`, i.e.,
+```mdtest-output
+where has(foo)
+```
+while this query
+```mdtest-command
+zq -C 'a:=x+1'
+```
+is an implied [put operator](../language/operators/put.md), which creates a new field `a`
+with the value `x+1`, i.e.,
+```mdtest-output
+put a:=x+1
+```
+
+## Error Handling
+
+Fatal errors like "file not found" or "file system full" are reported
+as soon as they happen and cause the `zq` process to exit.
+
+On the other hand,
+runtime errors resulting from the Zed query itself
+do not halt execution. Instead, these error conditions produce
+[first-class Zed errors](../language/data-types.md#first-class-errors)
+in the data output stream interleaved with any valid results.
+Such errors are easily queried with the
+[is_error function](../language/functions/is_error.md).
+
+This approach provides a robust technique for debugging complex query pipelines,
+where errors can be wrapped in one another providing stack-trace-like debugging
+output alongside the output data. This approach has emerged as a more powerful
+alternative to the traditional technique of looking through logs for errors
+or trying to debug a halted program with a vague error message.
+
+For example, this query
+```
+echo '1 2 0 3' | zq '10.0/this' -
+```
+produces
+```
+10.
+5.
+error("divide by zero")
+3.3333333333333335
+```
+and
+```
+echo '1 2 0 3' | zq '10.0/this' - | zq 'is_error(this)' -
+```
+produces just
+```
+error("divide by zero")
+```
+
+## Examples
+
+As you may have noticed, many examples of the [Zed language](../language/README.md)
+are illustrated using this pattern
+```
+echo | zq -
+```
+which is used throughout the [language documentation](../language/README.md)
+and [operator reference](../language/operators/README.md).
+
+The language documentation and [tutorials directory](../tutorials/README.md)
+have many examples, but here are a few more simple `zq` use cases.
+
+_Hello, world_
+```
+echo '"hello, world"' | zq -z 'yield this' -
+```
+produces this ZSON output
+```
+"hello, world"
+```
+
+_Some values of available data types_
+```
+echo '1 1.5 [1,"foo"] |["apple","banana"]|' | zq -z 'yield this' -
+```
+produces
+```
+1
+1.5
+[1,"foo"]
+|["apple","banana"]|
+```
+_The types of various data_
+```
+echo '1 1.5 [1,"foo"] |["apple","banana"]|' | zq -z 'yield typeof(this)' -
+```
+produces
+```
+
+
+<[(int64,string)]>
+<|[string]|>
+```
+_A simple aggregation_
+```
+echo '{key:"foo",val:1}{key:"bar",val:2}{key:"foo",val:3}' | zq -z 'sum(val) by key | sort key' -
+```
+produces
+```
+{key:"bar",sum:2}
+{key:"foo",sum:4}
+```
+_Convert CSV to Zed and cast a to an integer from default float_
+```
+printf "a,b\n1,foo\n2,bar\n" | zq 'a:=int64(a)' -
+```
+produces
+```
+{a:1,b:"foo"}
+{a:2,b:"bar"}
+```
+_Convert JSON to Zed and cast to an integer from default float_
+```
+echo '{"a":1,"b":"foo"}{"a":2,"b":"bar"}' | zq 'a:=int64(a)' -
+```
+produces
+```
+{a:1,b:"foo"}
+{a:2,b:"bar"}
+```
+_Make a schema-rigid Parquet file using fuse and turn it back into Zed_
+```
+echo '{a:1}{a:2}{b:3}' | zq -f parquet -o tmp.parquet fuse -
+zq -z tmp.parquet
+```
+produces
+```
+{a:1,b:null(int64)}
+{a:2,b:null(int64)}
+{a:null(int64),b:3}
+```
+
+## Performance
+
+Your mileage may vary, but many new users of `zq` are surprised by its speed
+compared to tools like `jq`, `grep`, `awk`, or `sqlite` especially when running
+`zq` over files in the ZNG format.
+
+### Fast Pattern Matching
+
+One important technique that helps `zq` run fast is to take advantage of queries
+that involve fine-grained searches.
+
+When a query begins with a logical expression containing either a search
+or a predicate match with a constant value, and presuming the input data format
+is ZNG, then the runtime optimizes the query by performing an efficient,
+byte-oriented "pre-search" of the values required in the predicate. This pre-search
+scans the bytes that comprise a large buffer of values and looks for these values
+and, if they are not present, the entire buffer is discarded knowing no individual
+value in that buffer could match because the required serialized
+values were not present in the buffer.
+
+For example, if the Zed query is
+```
+"http error" and ipsrc==10.0.0.1 | count()
+```
+then the pre-search would look for the string "http error" and the Zed encoding
+of the IP address 10.0.0.1 and unless both those values are present, then the
+buffer is discarded.
+
+Moreover, ZNG data is compressed and arranged into frames that can be decompressed
+and processed in parallel. This allows the decompression and pre-search to
+run in parallel very efficiently across a large number of threads. When searching
+for sparse results, many frames are discarded without their uncompressed bytes
+having to be processed any further.
+
+### Efficient JSON Processing
+
+While processing data in the ZNG format is far more efficient than JSON,
+there is substantial JSON data in the world and it is important for JSON
+input to perform well.
+
+This proved a challenge as `zq` is written in Go and Go's JSON package
+is not particularly performant. To this end, `zq` has its own lean and simple
+[JSON tokenizer](https://pkg.go.dev/github.com/brimdata/zed/pkg/jsonlexer),
+which performs quite well,
+and is
+[integrated tightly](https://github.com/brimdata/zed/blob/main/zio/jsonio/reader.go)
+with Zed's internal data representation.
+Moreover, like `jq`,
+`zq`'s JSON parser does not require objects to be newline delimited and can
+incrementally parse the input to minimize memory overhead and improve
+processor cache performance.
+
+The net effect is a JSON parser that is typically a bit faster than the
+native C implementation in `jq`.
+
+### Performance Comparisons
+
+To provide a rough sense of the performance tradeoffs between `zq` and
+other tooling, this section provides results of a few simple speed tests.
+
+#### Test Data
+
+These tests are easy to reproduce. The input data comes from the
+[Zed sample data repository](https://github.com/brimdata/zed-sample-data),
+where we used a semi-structured Zeek "conn" log from the `zeek-default` directory.
+
+It is easy to convert the Zeek logs to a local ZNG file using
+zq's built-in `get` operator:
+```
+zq -o conn.zng 'get https://raw.githubusercontent.com/brimdata/zed-sample-data/main/zeek-default/conn.log.gz'
+```
+This creates a new file `conn.zng` from the Zeek log file fetched from GitHub.
+
+Note that this data is a gzip'd file in the Zeek format and `zq`'s auto-detector
+figures out both that it is gzip'd and that the uncompressed format is Zeek.
+There's no need to specify flags for this.
+
+Next, a JSON file can be converted from ZNG using:
+```
+zq -f json conn.zng > conn.json
+```
+Note here that we lose information in this conversion because the rich data types
+of Zed (that were [translated from the Zeek format](../integrations/zeek/data-type-compatibility.md) are lost.
+
+We'll also make a SQLite database in the file `conn.db` as the table named `conn`.
+One easy way to do this is to install
+[sqlite-utils](https://sqlite-utils.datasette.io/en/stable/)
+and run
+```
+sqlite-utils insert conn.db conn conn.json --nl
+```
+(If you need a cup of coffee, a good time to get it would be when
+loading the JSON into SQLite.)
+
+#### File Sizes
+
+Note the resulting file sizes:
+```
+% du -h conn.json conn.db conn.zng
+416M conn.json
+192M conn.db
+ 38M conn.zng
+```
+Much of the performance of ZNG derives from an efficient, parallelizable
+structure where frames of data are compressed
+(currently with [LZ4](http://lz4.github.io/lz4/) though the
+specification supports multiple algorithms) and the sequence of values
+can be processed with only partial deserialization.
+
+That said, there are quite a few more opportunities to further improve
+the performance of `zq` and the Zed system and we have a number of projects
+forthcoming on this front.
+
+#### Tests
+
+We ran three styles of tests on a Mac quad-core 2.3GHz i7:
+* `count` - compute the number of values present
+* `search` - find a value in a field
+* `agg` - sum a field grouped by another field
+
+Each test was run for `jq`, `zq` on JSON, `sqlite3`, and `zq` on ZNG.
+
+We used the Bash `time` command to measure elapsed time.
+
+The command lines for the `count` test were:
+```
+jq -s length conn.json
+sqlite3 conn.db 'select count(*) from conn'
+zq 'count()' conn.zng
+zq 'count()' conn.json
+```
+The command lines for the `search` test were:
+```
+jq 'select(.id.orig_h=="10.47.23.5")' conn.json
+sqlite3 conn.db 'select * from conn where json_extract(id, "$.orig_h")=="10.47.23.5"'
+zq 'id.orig_h==10.47.23.5' conn.zng
+zq 'id.orig_h==10.47.23.5' conn.json
+```
+Here, we look for an IP address (10.47.23.5) in a specific
+field `id.orig_h` in the semi-structured data. Note when using ZNG,
+the IP is a native type whereas for `jq` and SQLite it is a string.
+Note that `sqlite` must use its `json_extract` function since nested JSON objects
+are stored as minified JSON text.
+
+The command lines for the `agg` test were:
+```
+jq -n -f agg.jq conn.json
+sqlite3 conn.db 'select sum(orig_bytes),json_extract(id, "$.orig_h") as orig_h from conn group by orig_h'
+zq "sum(orig_bytes) by id.orig_h" conn.zng
+zq "sum(orig_bytes) by id.orig_h" conn.json
+```
+where the `agg.jq` script is:
+```
+def adder(stream):
+ reduce stream as $s ({}; .[$s.key] += $s.val);
+adder(inputs | {key:.id.orig_h,val:.orig_bytes})
+| to_entries[]
+| {orig_h: (.key), sum: .value}
+```
+
+#### Results
+
+The following table summarizes the results of each test as a column and
+each tool as a row with the speed-up factor (relative to `jq`)
+shown in parentheses:
+
+| | `count` | `search` | `agg` |
+|------|---------------|---------------|---------------|
+| `jq` | 11,540ms (1X) | 10,730ms (1X) | 20,175ms (1X) |
+| `zq-json` | 7,150ms (1.6X) | 7,230ms (1.5X) | 7,390ms (2.7X) |
+| `sqlite` | 100ms (115X) | 620ms (17X) | 1,475ms (14X) |
+| `zq-zng` | 110ms (105X) | 135ms (80X) | 475ms (42X) |
+
+To summarize, `zq` with ZNG is consistently fastest though `sqlite`
+was a bit faster counting rows.
+
+In particular, `zq` is substantially faster (40-100X) than `jq` with the efficient
+ZNG format but more modestly faster (50-170%) when processing the bulky JSON input.
+This is expected because parsing JSON becomes the bottleneck.
+
+While SQLite is much faster than `jq`, it is not as fast as `zq`. The primary
+reason for this is that SQLite stores its semi-structured columns as minified JSON text,
+so it must scan and parse the JSON when executing the _where_ clause above
+as well as the aggregated fields.
+
+Also, note that the inferior performance of `sqlite` is in areas where databases
+perform extraordinarily well if you do the work to
+(1) transform semi-structured columns to relational columns by flattening
+nested JSON objects (which are not indexable by `sqlite`) and
+(2) configuring database indexes.
+
+In fact, if you implement these changes, `sqlite` performs better than `zq` on these tests.
+
+However, the benefit of Zed is that no flattening is required. And unlike `sqlite`,
+`zq` is not intended to be a database. That said, there is no reason why database
+performance techniques cannot be applied to the Zed model and this is precisely what the
+open-source Zed project intends to do.
+
+Stay tuned!
diff --git a/versioned_docs/version-v1.16.0/formats/README.md b/versioned_docs/version-v1.16.0/formats/README.md
new file mode 100644
index 00000000..fd33a5f2
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/README.md
@@ -0,0 +1,284 @@
+# Zed Formats
+
+> **TL;DR** The Zed data model defines a new and easy way to manage, store,
+> and process data utilizing an emerging concept called
+[super-structured data](#2-zed-a-super-structured-pattern).
+> The [data model specification](zed.md) defines the high-level model that is realized
+> in a [family of interoperable serialization formats](#3-the-data-model-and-formats),
+> providing a unified approach to row, columnar, and human-readable formats.
+> Zed is a superset of both the dataframe/table model of relational systems and the
+> semi-structured model that is used ubiquitously in development as JSON and by NOSQL
+> data stores. The ZSON spec has [a few examples](zson.md#3-examples).
+
+## 1. Background
+
+Zed offers a new and improved way to think about and manage data.
+
+Modern data models are typically described in terms of their _structured-ness_:
+* _tabular-structured_, often simply called _"structured"_,
+where a specific schema is defined to describe a table and values are enumerated that conform to that schema;
+* _semi-structured_, where arbitrarily complex, hierarchical data structures
+define the data and values do not fit neatly into tables, e.g., JSON and XML; and
+* _unstructured_, where arbitrary text is formatted in accordance with
+external, often vague, rules for its interpretation.
+
+### 1.1 The Tabular-structured Pattern
+
+CSV is arguably the simplest but most frustrating format that follows the tabular-structured
+pattern. It provides a bare bones schema consisting of the names of the columns as the
+first line of a file followed by a list of comma-separated, textual values
+whose types must be inferred from the text. The lack of a universally adopted
+specification for CSV is an all too common source of confusion and frustration.
+
+The traditional relational database, on the other hand,
+offers the classic, comprehensive example of the tabular-structured pattern.
+The table columns have precise names and types.
+Yet, like CSV, there is no universal standard format for relational tables.
+The [_SQLite file format_](https://sqlite.org/fileformat.html)
+is arguably the _de facto_ standard for relational data,
+but this format describes a whole, specific database --- indexes and all ---
+rather than a stand-alone table.
+
+Instead, file formats like Avro, ORC, and Parquet arose to represent tabular data
+with an explicit schema followed by a sequence of values that conform to the schema.
+While Avro and Parquet schemas can also represent semi-structured data, all of the
+values in a given Avro or Parquet file must conform to the same schema.
+The [Iceberg specification](https://iceberg.apache.org/spec/)
+defines data types and metadata schemas for how large relational tables can be
+managed as a collection of Avro, ORC, and/or Parquet files.
+
+### 1.2 The Semi-structured Pattern
+
+JSON, on the other hand, is the ubiquitous example of the semi-structured pattern.
+Each JSON value is self-describing in terms of its
+structure and types, though the JSON type system is limited.
+
+When a sequence of JSON objects is organized into a stream
+(perhaps [separated by newlines](https://en.wikipedia.org/wiki/JSON_streaming#NDJSON))
+each value can take on any form.
+When all the values have the same form, the JSON sequence
+begins to look like a relational table, but the lack of a comprehensive type system,
+a union type, and precise semantics for columnar layout limits this interpretation.
+
+[BSON](https://bsonspec.org/)
+and [Ion](https://amzn.github.io/ion-docs/)
+were created to provide a type-rich elaboration of the
+semi-structured model of JSON along with performant binary representations
+though there is no mechanism for precisely representing the type of
+a complex value like an object or an array other than calling it
+type "object" or type "array", e.g., as compared to "object with field s
+of type string" or "array of number".
+
+[JSON Schema](https://json-schema.org/)
+addresses JSON's lack of schemas with an approach to augment
+one or more JSON values with a schema definition itself expressed in JSON.
+This creates a parallel type system for JSON, which is useful and powerful in many
+contexts, but introduces schema-management complexity when simply trying to represent
+data in its natural form.
+
+### 1.3 The Hybrid Pattern
+
+As the utility and ease of the semi-structured design pattern emerged,
+relational system design, originally constrained by the tabular-structured
+design pattern, has embraced the semi-structured design pattern
+by adding support for semi-structured table columns.
+"Just put JSON in a column."
+
+[SQL++](https://asterixdb.apache.org/docs/0.9.7.1/sqlpp/manual.html)
+pioneered the extension of SQL to semi-structured data by
+adding support for referencing and unwinding complex, semi-structured values,
+and most modern SQL query engines have adopted variations of this model
+and have extended the relational model with a semi-structured column type.
+
+But once you have put a number of columns of JSON data into a relational
+table, is it still appropriately called "structured"?
+Instead, we call this approach the hybrid tabular-/semi-structured pattern,
+or more simply, _"the hybrid pattern"_.
+
+## 2. Zed: A Super-structured Pattern
+
+The insight in Zed is to remove the tabular and schema concepts from
+the underlying data model altogether and replace them with a granular and
+modern type system inspired by general-purpose programming languages.
+Instead of defining a single, composite schema to
+which all values must conform, the Zed type system allows each value to freely
+express its type in accordance with the type system.
+
+In this approach,
+Zed is neither tabular nor semi-structured. Zed is "super-structured".
+
+In particular, the Zed record type looks like a schema but when
+serializing Zed data, the model is very different. A Zed sequence does not
+comprise a record-type declaration followed by a sequence of
+homogeneously-typed record values, but instead,
+is a sequence of arbitrarily typed Zed values, which may or may not all
+be records.
+
+Yet when a sequence of Zed values _in fact conforms to a uniform record type_,
+then such a collection of Zed records looks precisely like a relational table.
+Here, the record type
+of such a collection corresponds to a well-defined schema consisting
+of field names (i.e, column names) where each field has a specific Zed type.
+Zed also has named types, so by simply naming a particular record type
+(i.e., a schema), a relational table can be projected from a pool of Zed data
+with a simple type query for that named type.
+
+But unlike traditional relational tables, these Zed-constructed tables can have arbitrary
+structure in each column as Zed allows the fields of a record
+to have an arbitrary type. This is very different compared to the hybrid pattern:
+all Zed data at all levels conforms to the same data model. Here, both the
+tabular-structured and semi-structured patterns are representable in a single model.
+Unlike the hybrid pattern, systems based on Zed have
+no need to simultaneously support two very different data models.
+
+In other words, Zed unifies the relational data model of SQL tables
+with the document model of JSON into a _super-structured_
+design pattern enabled by the Zed type system.
+An explicit, uniquely-defined type of each value precisely
+defines its entire structure, i.e., its super-structure. There is
+no need to traverse each hierarchical value --- as with JSON, BSON, or Ion ---
+to discover each value's structure.
+
+And because Zed derives it design from the vast landscape
+of existing formats and data models, it was deliberately designed to be
+a superset of --- and thus interoperable with --- a broad range of formats
+including JSON, BSON, Ion, Avro, ORC, Parquet, CSV, JSON Schema, and XML.
+
+As an example, most systems that are based on semi-structured data would
+say the JSON value
+```
+{"a":[1,"foo"]}
+```
+is of type object and the value of key `a` is type array.
+In Zed, however, this value's type is type `record` with field `a`
+of type `array` of type `union` of `int64` and `string`,
+expressed succinctly in ZSON as
+```
+{a:[(int64,string)]}
+```
+This is super-structuredness in a nutshell.
+
+### 2.1 Zed and Schemas
+
+While the Zed data model removes the schema constraint,
+the implication here is not that schemas are unimportant;
+to the contrary, schemas are foundational. Schemas not only define agreement
+and semantics between communicating entities, but also serve as the cornerstone
+for organizing and modeling data for data engineering and business intelligence.
+
+That said, schemas often create complexity in system designs
+where components might simply want to store and communicate data in some
+meaningful way. For example, an ETL pipeline should not break when upstream
+structural changes prevent data from fitting in downstream relational tables.
+Instead, the pipeline should continue to operate and the data should continue
+to land on the target system without having to fit into a predefined table,
+while also preserving its super-structure.
+
+This is precisely what Zed enables. A system layer above and outside
+the scope of the Zed data layer can decide how to adapt to the structural
+changes with or without administrative intervention.
+
+To this end, whether all the values must conform to a schema and
+how schemas are managed, revised, and enforced is all outside the scope of Zed;
+rather, the Zed data model provides a flexible and rich foundation
+for schema interpretation and management.
+
+### 2.2 Type Combinatorics
+
+A common objection to using a type system to represent schemas is that
+diverse applications generating arbitrarily structured data can produce
+a combinatorial explosion of types for each shape of data.
+
+In practice, this condition rarely arises. Applications generating
+"arbitrary" JSON data generally conform to a well-defined set of
+JSON object structures.
+
+A few rare applications carry unique data values as JSON object keys,
+though this is considered bad practice.
+
+Even so, this is all manageable in the Zed data model as types are localized
+in scope. The number of types that must be defined in a stream of values
+is linear in the input size. Since data is self-describing and there is
+no need for a global schema registry in Zed, this hypothetical problem is moot.
+
+### 2.3 Analytics Performance
+
+One might think that removing schemas from the Zed data model would conflict
+with an efficient columnar format for Zed, which is critical for
+high-performance analytics.
+After all, database
+tables and formats like Parquet and ORC all require schemas to organize values
+and then rely upon the natural mapping of schemas to columns.
+
+Super-structure, on the other hand, provides an alternative approach to columnar structure.
+Instead of defining a schema and then fitting a sequence of values into their appropriate
+columns based on the schema, Zed values self-organize into columns based on their
+super-structure. Here columns are created dynamically as data is analyzed
+and each top-level type induces a specific set of columns. When all of the
+values have the same top-level type (i.e., like a schema), then the Zed columnar
+object is just as performant as a traditional schema-based columnar format like Parquet.
+
+### 2.4 First-class Types
+
+With first-class types, any type can also be a value, which means that in
+a properly designed query and analytics system based on Zed, a type can appear
+anywhere that a value can appear. In particular, types can be aggregation keys.
+
+This is very powerful for data discovery and introspection. For example,
+to count the different shapes of data, you might have a SQL-like query,
+operating on each input value as `this`, that has the form:
+```
+ SELECT count(), typeof(this) as shape GROUP by shape, count
+```
+Likewise, you could select a sample value of each shape like this:
+```
+ SELECT shape FROM (
+ SELECT any(this) as sample, typeof(this) as shape GROUP by shape,sample
+ )
+```
+The Zed language is exploring syntax so that such operations are tighter
+and more natural given the super-structure of Zed. For example, the above
+two SQL-like queries could be written as:
+```
+ count() by shape:=typeof(this)
+ any(this) by typeof(this) | cut any
+```
+
+### 2.5 First-class Errors
+
+In SQL based systems, errors typically
+result in cryptic messages or null values offering little insight as to the
+actual cause of the error.
+
+Zed however includes first-class errors. When combined with the super-structured
+data model, error values may appear anywhere in the output and operators
+can propagate or easily wrap errors so complicated analytics pipelines
+can be debugged by observing the location of errors in the output results.
+
+## 3. The Data Model and Formats
+
+The concept of super-structured data and first-class types and errors
+is solidified in the [Zed data model specification](zed.md),
+which defines the model but not the serialization formats.
+
+A set of companion documents define a family of tightly integrated
+serialization formats that all adhere to the same Zed data model,
+providing a unified approach to row, columnar, and human-readable formats:
+
+* [ZSON](zson.md) is a JSON-like, human readable format for Zed data. All JSON
+documents are Zed values as the ZSON format is a strict superset of the JSON syntax.
+* [ZNG](zng.md) is a row-based, binary representation of Zed data somewhat like
+Avro but with Zed's more general model to represent a sequence of arbitrarily-typed
+values.
+* [VNG](vng.md) is a columnar version of ZNG like Parquet or ORC but also
+embodies Zed's more general model for heterogeneous and self-describing schemas.
+* [Zed over JSON](zjson.md) defines a JSON format for encapsulating Zed data
+in JSON for easy decoding by JSON-based clients, e.g.,
+the [zed-js JavaScript library](https://github.com/brimdata/zui/tree/main/packages/zed-js)
+and the [Zed Python library](../libraries/python.md).
+
+Because all of the formats conform to the same Zed data model, conversions between
+a human-readable form, a row-based binary form, and a row-based columnar form can
+be trivially carried out with no loss of information. This is the best of both worlds:
+the same data can be easily expressed in and converted between a human-friendly
+and easy-to-program text form alongside efficient row and columnar formats.
diff --git a/versioned_docs/version-v1.16.0/formats/_category_.yaml b/versioned_docs/version-v1.16.0/formats/_category_.yaml
new file mode 100644
index 00000000..6464e66a
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/_category_.yaml
@@ -0,0 +1,2 @@
+position: 6
+label: Formats
diff --git a/versioned_docs/version-v1.16.0/formats/compression.md b/versioned_docs/version-v1.16.0/formats/compression.md
new file mode 100644
index 00000000..a478e5c1
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/compression.md
@@ -0,0 +1,17 @@
+---
+sidebar_position: 6
+sidebar_label: Compression
+---
+
+# ZNG Compression Types
+
+This document specifies values for the `` byte of a
+[ZNG compressed value message block](zng.md#2-the-zng-format)
+and the corresponding algorithms for the `` byte sequence.
+
+As new compression algorithms are specified, they will be documented
+here without any need to change the ZNG specification.
+
+Of the 256 possible values for the `` byte, only type `0` is currently
+defined and specifies that `` contains an
+[LZ4 block](https://github.com/lz4/lz4/blob/master/doc/lz4_Block_format.md).
diff --git a/versioned_docs/version-v1.16.0/formats/vng.md b/versioned_docs/version-v1.16.0/formats/vng.md
new file mode 100644
index 00000000..f358bbdc
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/vng.md
@@ -0,0 +1,399 @@
+---
+sidebar_position: 4
+sidebar_label: VNG
+---
+
+# VNG Specification
+
+VNG, pronounced "ving", is a file format for columnar data based on
+[the Zed data model](zed.md).
+VNG is the "stacked" version of Zed, where the fields from a stream of
+Zed records are stacked into vectors that form columns.
+Its purpose is to provide for efficient analytics and search over
+bounded-length sequences of [ZNG](zng.md) data that is stored in columnar form.
+
+Like [Parquet](https://github.com/apache/parquet-format),
+VNG provides an efficient columnar representation for semi-structured data,
+but unlike Parquet, VNG is not based on schemas and does not require
+a schema to be declared when writing data to a file. Instead,
+VNG exploits the super-structured nature of Zed data: columns of data
+self-organize around their type structure.
+
+## VNG Files
+
+A VNG file encodes a bounded, ordered sequence of Zed values.
+To provide for efficient access to subsets of VNG-encoded data (e.g., columns),
+the VNG file is presumed to be accessible via random access
+(e.g., range requests to a cloud object store or seeks in a Unix file system)
+and VNG is therefore not intended as a streaming or communication format.
+
+A VNG file can be stored entirely as one storage object
+or split across separate objects that are treated
+together as a single VNG entity. While the VNG format provides much flexibility
+for how data is laid out, it is left to an implementation to lay out data
+in intelligent ways for efficient sequential read accesses of related data.
+
+## Column Streams
+
+The VNG data abstraction is built around a collection of _column streams_.
+
+There is one column stream for each top-level type encountered in the input where
+each column stream is encoded according to its type. For top-level complex types,
+the embedded elements are encoded recursively in additional column streams
+as described below. For example,
+a record column encodes a "presence" vector encoding any null value for
+each field then encodes each non-null field recursively, whereas
+an array column encodes a "lengths" vector and encodes each
+element recursively.
+
+Values are reconstructed one by one from the column streams by picking values
+from each appropriate column stream based on the type structure of the value and
+its relationship to the various column streams. For hierarchical records
+(i.e., records inside of records, or records inside of arrays inside of records, etc),
+the reconstruction process is recursive (as described below).
+
+## The Physical Layout
+
+The overall layout of a VNG file is comprised of the following sections,
+in this order:
+* the data section,
+* the reassembly section, and
+* the trailer.
+
+This layout allows an implementation to buffer metadata in
+memory while writing column data in a natural order to the
+data section (based on the volume statistics of each column),
+then write the metadata into the reassembly section along with the trailer
+at the end. This allows a ZNG stream to be converted to a VNG file
+in a single pass.
+
+> That said, the layout is
+> flexible enough that an implementation may optimize the data layout with
+> additional passes or by writing the output to multiple files then
+> merging them together (or even leaving the VNG entity as separate files).
+
+### The Data Section
+
+The data section contains raw data values organized into _segments_,
+where a segment is a seek offset and byte length relative to the
+data section. Each segment contains a sequence of
+[primitive-type Zed values](zed.md#1-primitive-types),
+encoded as counted-length byte sequences where the counted-length is
+variable-length encoded as in the [ZNG specification](zng.md).
+Segments may be compressed.
+
+There is no information in the data section for how segments relate
+to one another or how they are reconstructed into columns. They are just
+blobs of ZNG data.
+
+> Unlike Parquet, there is no explicit arrangement of the column chunks into
+> row groups but rather they are allowed to grow at different rates so a
+> high-volume column might be comprised of many segments while a low-volume
+> column must just be one or several. This allows scans of low-volume record types
+> (the "mice") to perform well amongst high-volume record types (the "elephants"),
+> i.e., there are not a bunch of seeks with tiny reads of mice data interspersed
+> throughout the elephants.
+>
+> TBD: The mice/elephants model creates an interesting and challenging layout
+> problem. If you let the row indexes get too far apart (call this "skew"), then
+> you have to buffer very large amounts of data to keep the column data aligned.
+> This is the point of row groups in Parquet, but the model here is to leave it
+> up to the implementation to do layout as it sees fit. You can also fall back
+> to doing lots of seeks and that might work perfectly fine when using SSDs but
+> this also creates interesting optimization problems when sequential reads work
+> a lot better. There could be a scan optimizer that lays out how the data is
+> read that lives under the column stream reader. Also, you can make tradeoffs:
+> if you use lots of buffering on ingest, you can write the mice in front of the
+> elephants so the read path requires less buffering to align columns. Or you can
+> do two passes where you store segments in separate files then merge them at close
+> according to an optimization plan.
+
+### The Reassembly Section
+
+The reassembly section provides the information needed to reconstruct
+column streams from segments, and in turn, to reconstruct the original Zed values
+from column streams, i.e., to map columns back to composite values.
+
+> Of course, the reassembly section also provides the ability to extract just subsets of columns
+> to be read and searched efficiently without ever needing to reconstruct
+> the original rows. How well this performs is up to any particular
+> VNG implementation.
+>
+> Also, the reassembly section is in general vastly smaller than the data section
+> so the goal here isn't to express information in cute and obscure compact forms
+> but rather to represent data in an easy-to-digest, programmer-friendly form that
+> leverages ZNG.
+
+The reassembly section is a ZNG stream. Unlike Parquet,
+which uses an externally described schema
+(via [Thrift](https://thrift.apache.org/)) to describe
+analogous data structures, we simply reuse ZNG here.
+
+#### The Super Types
+
+This reassembly stream encodes 2*N+1 Zed values, where N is equal to the number
+of top-level Zed types that are present in the encoded input.
+To simplify terminology, we call a top-level Zed type a "super type",
+e.g., there are N unique super types encoded in the VNG file.
+
+These N super types are defined by the first N values of the reassembly stream
+and are encoded as a null value of the indicated super type.
+A super type's integer position in this sequence defines its identifier
+encoded in the super column (defined below). This identifier is called
+the super ID.
+
+> Change the first N values to type values instead of nulls?
+
+The next N+1 records contain reassembly information for each of the N super types
+where each record defines the column streams needed to reconstruct the original
+Zed values.
+
+#### Segment Maps
+
+The foundation of column reconstruction is based on _segment maps_.
+A segment map is a list of the segments from the data area that are
+concatenated to form the data for a column stream.
+
+Each segment map that appears within the reassembly records is represented
+with a Zed array of records that represent seek ranges conforming to this
+type signature:
+```
+[{offset:uint64,length:uint32,mem_length:uint32,compression_format:uint8}]
+```
+
+In the rest of this document, we will refer to this type as `` for
+shorthand and refer to the concept as a "segmap".
+
+> We use the type name "segmap" to emphasize that this information represents
+> a set of byte ranges where data is stored and must be read from *rather than*
+> the data itself.
+
+#### The Super Column
+
+The first of the N+1 reassembly records defines the "super column", where this column
+represents the sequence of super types of each original Zed value, i.e., indicating
+which super type's column stream to select from to pull column values to form
+the reconstructed value.
+The sequence of super types is defined by each type's super ID (as defined above),
+0 to N-1, within the set of N super types.
+
+The super column stream is encoded as a sequence of ZNG-encoded `int32` primitive values.
+While there are a large number entries in the super column (one for each original row),
+the cardinality of super IDs is small in practice so this column
+will compress very significantly, e.g., in the special case that all the
+values in the VNG file have the same super ID,
+the super column will compress trivially.
+
+The reassembly map appears as the next value in the reassembly section
+and is of type ``.
+
+#### The Reassembly Records
+
+Following the root reassembly map are N reassembly maps, one for each unique super type.
+
+Each reassembly record is a record of type ``, as defined below,
+where each reassembly record appears in the same sequence as the original N schemas.
+Note that there is no "any" type in Zed, but rather this terminology is used
+here to refer to any of the concrete type structures that would appear
+in a given VNG file.
+
+In other words, the reassembly record of the super column
+combined with the N reassembly records collectively define the original sequence
+of Zed data values in the original order.
+Taken in pieces, the reassembly records allow efficient access to sub-ranges of the
+rows, to subsets of columns of the rows, to sub-ranges of columns of the rows, and so forth.
+
+This simple top-down arrangement, along with the definition of the other
+column structures below, is all that is needed to reconstruct all of the
+original data.
+
+> Note that each row reassembly record has its own layout of columnar
+> values and there is no attempt made to store like-typed columns from different
+> schemas in the same physical column.
+
+The notation `` refers to any instance of the five column types:
+* ``,
+* ``,
+* ``,
+* ``, or
+* ``.
+
+Note that when decoding a column, all type information is known
+from the super type in question so there is no need
+to encode the type information again in the reassembly record.
+
+#### Record Column
+
+A `` is defined recursively in terms of the column types of
+its fields, i.e., other types that represent arrays, unions, or primitive types
+and has the form:
+```
+{
+ :{column:,presence:},
+ :{column:,presence:},
+ ...
+ :{column:,presence:}
+}
+```
+where
+* `` through `` are the names of the top-level fields of the
+original row record,
+* the `column` fields are column stream definitions for each field, and
+* the `presence` columns are `int32` ZNG column streams comprised of a
+run-length encoding of the locations of column values in their respective rows,
+when there are null values (as described below).
+
+If there are no null values, then the `presence` field contains an empty ``.
+If all of the values are null, then the `column` field is null (and the `presence`
+contains an empty ``). For an empty ``, there is no
+corresponding data stored in the data section. Since a `` is a Zed
+array, an empty `` is simply the empty array value `[]`.
+
+#### Array Column
+
+An `` has the form:
+```
+{values:,lengths:}
+```
+where
+* `values` represents a continuous sequence of values of the array elements
+that are sliced into array values based on the length information, and
+* `lengths` encodes a Zed `int32` sequence of values that represent the length
+ of each array value.
+
+The `` structure is used for both Zed arrays and sets.
+
+#### Map Column
+
+A `` has the form:
+```
+{key:,value:}
+```
+where
+* `key` encodes the column of map keys and
+* `value` encodes the column of map values.
+
+#### Union Column
+
+A `` has the form:
+```
+{columns:[],tags:}
+```
+where
+* `columns` is an array containing the reassembly information for each tagged union value
+in the same column order implied by the union type, and
+* `tags` is a column of `int32` values where each subsequent value encodes
+the tag of the union type indicating which column the value falls within.
+
+> TBD: change code to conform to columns array instead of record{c0,c1,...}
+
+The number of times each value of `tags` appears must equal the number of values
+in each respective column.
+
+#### Primitive Column
+
+A `` is a `` that defines a column stream of
+primitive values.
+
+#### Presence Columns
+
+The presence column is logically a sequence of booleans, one for each position
+in the original column, indicating whether a value is null or present.
+The number of values in the encoded column is equal to the number of values
+present so that null values are not encoded.
+
+Instead the presence column is encoded as a sequence of alternating runs.
+First, the number of values present is encoded, then the number of values not present,
+then the number of values present, and so forth. These runs are then stored
+as Zed `int32` values in the presence column (which may be subject to further
+compression based on segment compression).
+
+### The Trailer
+
+After the reassembly section is a ZNG stream with a single record defining
+the "trailer" of the VNG file. The trailer provides a magic field
+indicating the "vng" format, a version number,
+the size of the segment threshold for decomposing segments into frames,
+the size of the skew threshold for flushing all segments to storage when
+the memory footprint roughly exceeds this threshold,
+and an array of sizes in bytes of the sections of the VNG file.
+
+This type of this record has the format
+```
+{magic:string,type:string,version:int64,sections:[int64],meta:{skew_thresh:int64,segment_thresh:int64}
+```
+The trailer can be efficiently found by scanning backward from the end of the
+VNG file to find a valid ZNG stream containing a single record value
+conforming to the above type.
+
+## Decoding
+
+To decode an entire VNG file into rows, the trailer is read to find the sizes
+of the sections, then the ZNG stream of the reassembly section is read,
+typically in its entirety.
+
+Since this data structure is relatively small compared to all of the columnar
+data in the VNG file,
+it will typically fit comfortably in memory and it can be very fast to scan the
+entire reassembly structure for any purpose.
+
+> For example, for a given query, a "scan planner" could traverse all the
+> reassembly records to figure out which segments will be needed, then construct
+> an intelligent plan for reading the needed segments and attempt to read them
+> in mostly sequential order, which could serve as
+> an optimizing intermediary between any underlying storage API and the
+> VNG decoding logic.
+
+To decode the "next" row, its schema index is read from the root reassembly
+column stream.
+
+This schema index then determines which reassembly record to fetch
+column values from.
+
+The top-level reassembly fetches column values as a ``.
+
+For any ``, a value from each field is read from each field's column,
+accounting for the presence column indicating null,
+and the results are encoded into the corresponding ZNG record value using
+ZNG type information from the corresponding schema.
+
+For a `` a value is determined by reading the next
+value from its segmap.
+
+For an ``, a length is read from its `lengths` segmap as an `int32`
+and that many values are read from its the `values` sub-column,
+encoding the result as a ZNG array value.
+
+For a ``, a value is read from its `tags` segmap
+and that value is used to select the corresponding column stream
+`c0`, `c1`, etc. The value read is then encoded as a ZNG union value
+using the same tag within the union value.
+
+## Examples
+
+### Hello, world
+
+Start with this ZNG data (shown as human-readable [ZSON](zson.md)):
+```
+{a:"hello",b:"world"}
+{a:"goodnight",b:"gracie"}
+```
+
+To convert to VNG format:
+```
+zq -f vng hello.zson > hello.vng
+```
+
+Segments in the VNG format would be laid out like this:
+```
+=== column for a
+hello
+goodnight
+=== column for b
+world
+gracie
+=== column for schema IDs
+0
+0
+===
+```
diff --git a/versioned_docs/version-v1.16.0/formats/zed.md b/versioned_docs/version-v1.16.0/formats/zed.md
new file mode 100644
index 00000000..60187042
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/zed.md
@@ -0,0 +1,240 @@
+---
+sidebar_position: 1
+sidebar_label: Data Model
+---
+
+# Zed Data Model
+
+Zed data is defined as an ordered sequence of one or more typed data values.
+Each value's type is either a "primitive type", a "complex type", the "type type",
+a "named type", or the "null type".
+
+## 1. Primitive Types
+
+Primitive types include signed and unsigned integers, IEEE binary and decimal
+floating point, string, byte sequence, Boolean, IP address, IP network,
+null, and a first-class type _type_.
+
+There are 30 types of primitive values with syntax defined as follows:
+
+| Name | Definition |
+|------------|-------------------------------------------------|
+| `uint8` | unsigned 8-bit integer |
+| `uint16` | unsigned 16-bit integer |
+| `uint32` | unsigned 32-bit integer |
+| `uint64` | unsigned 64-bit integer |
+| `uint128` | unsigned 128-bit integer |
+| `uint256` | unsigned 256-bit integer |
+| `int8` | signed 8-bit integer |
+| `int16` | signed 16-bit integer |
+| `int32` | signed 32-bit integer |
+| `int64` | signed 64-bit integer |
+| `int128` | signed 128-bit integer |
+| `int256` | signed 256-bit integer |
+| `duration` | signed 64-bit integer as nanoseconds |
+| `time` | signed 64-bit integer as nanoseconds from epoch |
+| `float16` | IEEE-754 binary16 |
+| `float32` | IEEE-754 binary32 |
+| `float64` | IEEE-754 binary64 |
+| `float128` | IEEE-754 binary128 |
+| `float256` | IEEE-754 binary256 |
+| `decimal32` | IEEE-754 decimal32 |
+| `decimal64` | IEEE-754 decimal64 |
+| `decimal128` | IEEE-754 decimal128 |
+| `decimal256` | IEEE-754 decimal256 |
+| `bool` | the Boolean value `true` or `false` |
+| `bytes` | a bounded sequence of 8-bit bytes |
+| `string` | a UTF-8 string |
+| `ip` | an IPv4 or IPv6 address |
+| `net` | an IPv4 or IPv6 address and net mask |
+| `type` | a Zed type value |
+| `null` | the null type |
+
+The _type_ type provides for first-class types and even though a type value can
+represent a complex type, the value itself is a singleton.
+
+Two type values are equivalent if their underlying types are equal. Since
+every type in the Zed type system is uniquely defined, type values are equal
+if and only if their corresponding types are uniquely equal.
+
+The _null_ type is a primitive type representing only a `null` value.
+A `null` value can have any type.
+
+> Note that `time` values correspond to 64-bit epoch nanoseconds and thus
+> not every valid RFC 3339 date and time string represents a valid Zed time.
+> In addition, nanosecond epoch times overflow on April 11, 2262.
+> For the world of 2262, a new epoch can be created well in advance
+> and the old time epoch and new time epoch can live side by side with
+> the old using a named type for the new epoch time referring to the old `time`.
+> An app that wants more than 64 bits of timestamp precision can always use
+> a named type of a `bytes` type and do its own conversions to and from the
+> corresponding bytes values. A time with a local time zone can be represented
+> as a Zed record of a time field and a zone field
+
+## 2. Complex Types
+
+Complex types are composed of primitive types and/or other complex types.
+The categories of complex types include:
+* _record_ - an ordered collection of zero or more named values called fields,
+* _array_ - an ordered sequence of zero or more values called elements,
+* _set_ - a set of zero or more unique values called elements,
+* _map_ - a collection of zero or more key/value pairs where the keys are of a
+uniform type called the key type and the values are of a uniform type called
+the value type,
+* _union_ - a type representing values whose type is any of a specified collection of two or more unique types,
+* _enum_ - a type representing a finite set of symbols typically representing categories, and
+* _error_ - any value wrapped as an "error".
+
+The type system comprises a total order:
+* The order of primitive types corresponds to the order in the table above.
+* All primitive types are ordered before any complex types.
+* The order of complex type categories corresponds to the order above.
+* For complex types of the same category, the order is defined below.
+
+### 2.1 Record
+
+A record comprises an ordered set of zero or more named values
+called "fields". The field names must be unique in a given record
+and the order of the fields is significant, e.g., type `{a:string,b:string}`
+is distinct from type `{b:string,a:string}`.
+
+A field name is any UTF-8 string.
+
+A field value is any Zed value.
+
+In contrast to many schema-oriented data formats, Zed has no way to specify
+a field as "optional" since any field value can be a null value.
+
+If an instance of a record value omits a value
+by dropping the field altogether rather than using a null, then that record
+value corresponds to a different record type that elides the field in question.
+
+A record type is uniquely defined by its ordered list of field-type pairs.
+
+The type order of two records is as follows:
+* Record with fewer columns than other is ordered before the other.
+* Records with the same number of columns are ordered as follows according to:
+ * the lexicographic order of the field names from left to right,
+ * or if all the field names are the same, the type order of the field types from left to right.
+
+### 2.2 Array
+
+An array is an ordered sequence of zero or more Zed values called "elements"
+all conforming to the same Zed type.
+
+An array value may be empty. An empty array may have element type `null`.
+
+An array type is uniquely defined by its single element type.
+
+The type order of two arrays is defined as the type order of the
+two array element types.
+
+> Note that mixed-type JSON arrays are representable as a Zed array with
+> elements of type union.
+
+### 2.3 Set
+
+A set is an unordered sequence of zero or more Zed values called "elements"
+all conforming to the same Zed type.
+
+A set may be empty. An empty set may have element type `null`.
+
+A set of mixed-type values is representable as a Zed set with
+elements of type union.
+
+A set type is uniquely defined by its single element type.
+
+The type order of two sets is defined as the type order of the
+two set element types.
+
+### 2.4 Map
+
+A map represents a list of zero or more key-value pairs, where the keys
+have a common Zed type and the values have a common Zed type.
+
+Each key across an instance of a map value must be a unique value.
+
+A map value may be empty.
+
+A map type is uniquely defined by its key type and value type.
+
+The type order of two map types is as follows:
+* the type order of their key types,
+* or if they are the same, then the order of their value types.
+
+### 2.5 Union
+
+A union represents a value that may be any one of a specific enumeration
+of two or more unique Zed types that comprise its "union type".
+
+A union type is uniquely defined by an ordered set of unique types (which may be
+other union types) where the order corresponds to the Zed type system's total order.
+
+Union values are tagged in that
+any instance of a union value explicitly conforms to exactly one of the union's types.
+The union tag is an integer indicating the position of its type in the union
+type's ordered list of types.
+
+The type order of two union types is as follows:
+* The union type with fewer types than other is ordered before the other.
+* Two union types with the same number of types are ordered according to
+the type order of the constituent types in left to right order.
+
+### 2.6 Enum
+
+An enum represents a symbol from a finite set of one or more unique symbols
+referenced by name. An enum name may be any UTF-8 string.
+
+An enum type is uniquely defined by its ordered set of unique symbols,
+where the order is significant, e.g., two enum types
+with the same set of symbols but in different order are distinct.
+
+The type order of two enum types is as follows:
+* The enum type with fewer symbols than other is ordered before the other.
+* Two enum types with the same number of symbols are ordered according to
+the type order of the constituent types in left to right order.
+
+### 2.7 Error
+
+An error represents any value designated as an error.
+
+The type order of an error is the type order of the type of its contained value.
+
+## 3. Named Type
+
+A _named type_ is a name for a specific Zed type.
+Any value can have a named type and the named type is a distinct type
+from the underlying type. A named type can refer to another named type.
+
+The binding between a named type and its underlying type is local in scope
+and need not be unique across a sequence of values.
+
+A type name may be any UTF-8 string exclusive of primitive type names.
+
+For example, if "port" is a named type for `uint16`, then two values of
+type "port" have the same type but a value of type "port" and a value of type `uint16`
+do not have the same type.
+
+The type order of a named type is the type order of its underlying type with two
+exceptions:
+* A named type is ordered after its underlying type.
+* Named types sharing an underlying type are ordered lexicographically by name.
+
+> While the Zed data model does not include explicit support for schema versioning,
+> named types provide a flexible mechanism to implement versioning
+> on top of the Zed serialization formats. For example, a Zed-based system
+> could define a naming convention of the form `.`
+> where `` is the type name of a record representing the schema
+> and `` is a decimal string indicating the version of that schema.
+> Since types need only be parsed once per stream
+> in the Zed binary serialization formats, a Zed type implementation could
+> efficiently support schema versioning using such a convention.
+
+## 4. Null Values
+
+All Zed types have a null representation. It is up to an
+implementation to decide how external data structures map into and
+out of values with nulls. Typically, a null value is either the
+zero value or, in the case of record fields, an optional field whose
+value is not present, though these semantics are not explicitly
+defined by the Zed data model.
diff --git a/versioned_docs/version-v1.16.0/formats/zjson.md b/versioned_docs/version-v1.16.0/formats/zjson.md
new file mode 100644
index 00000000..af8ff2ad
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/zjson.md
@@ -0,0 +1,476 @@
+---
+sidebar_position: 5
+sidebar_label: ZJSON
+---
+
+# ZJSON Specification
+
+## 1. Introduction
+
+The [Zed data model](zed.md)
+is based on richly typed records with a deterministic field order,
+as is implemented by the [ZSON](zson.md), [ZNG](zng.md), and [VNG](vng.md) formats.
+Given the ubiquity of JSON, it is desirable to also be able to serialize
+Zed data into the JSON format. However, encoding Zed data values
+directly as JSON values would not work without loss of information.
+
+For example, consider this Zed data as [ZSON](zson.md):
+```
+{
+ ts: 2018-03-24T17:15:21.926018012Z,
+ a: "hello, world",
+ b: {
+ x: 4611686018427387904,
+ y: 127.0.0.1
+ }
+}
+```
+A straightforward translation to JSON might look like this:
+```
+{
+ "ts": 1521911721.926018012,
+ "a": "hello, world",
+ "b": {
+ "x": 4611686018427387904,
+ "y": "127.0.0.1"
+ }
+}
+```
+But, when this JSON is transmitted to a JavaScript client and parsed,
+the result looks something like this:
+```
+{
+ "ts": 1521911721.926018,
+ "a": "hello, world",
+ "b": {
+ "x": 4611686018427388000,
+ "y": "127.0.0.1"
+ }
+}
+```
+The good news is the `a` field came through just fine, but there are
+a few problems with the remaining fields:
+* the timestamp lost precision (due to 53 bits of mantissa in a JavaScript
+IEEE 754 floating point number) and was converted from a time type to a number,
+* the int64 lost precision for the same reason, and
+* the IP address has been converted to a string.
+
+As a comparison, Python's `json` module handles the 64-bit integer to full
+precision, but loses precision on the floating point timestamp.
+Also, it is at the whim of a JSON implementation whether
+or not the order of object keys is preserved.
+
+While JSON is well suited for data exchange of generic information, it is not
+so appropriate for a [super-structured data model](./README.md#2-zed-a-super-structured-pattern)
+like Zed. That said, JSON can be used as an encoding format for Zed by mapping Zed data
+onto a JSON-based protocol. This allows clients like web apps or
+Electron apps to receive and understand Zed and, with the help of client
+libraries like [zed-js](https://github.com/brimdata/zui/tree/main/packages/zed-js),
+to manipulate the rich, structured Zed types that are implemented on top of
+the basic JavaScript types.
+
+In other words,
+because JSON objects do not have a deterministic field order nor does JSON
+in general have typing beyond the basics (i.e., strings, floating point numbers,
+objects, arrays, and booleans), we decided to encode Zed data with
+its embedded type model all in a layer above regular JSON.
+
+## 2. The Format
+
+The format for representing Zed in JSON is called ZJSON.
+Converting ZSON, ZNG, or VNG to ZJSON and back results in a complete and
+accurate restoration of the original Zed data.
+
+A ZJSON stream is defined as a sequence of JSON objects where each object
+represents a Zed value and has the form:
+```
+{
+ "type": ,
+ "value":
+}
+```
+The type and value fields are encoded as defined below.
+
+### 2.1 Type Encoding
+
+The type encoding for a primitive type is simply its [Zed type name](zed.md#1-primitive-types)
+e.g., "int32" or "string".
+
+Complex types are encoded with small-integer identifiers.
+The first instance of a unique type defines the binding between the
+integer identifier and its definition, where the definition may recursively
+refer to earlier complex types by their identifiers.
+
+For example, the Zed type `{s:string,x:int32}` has this ZJSON format:
+```
+{
+ "id": 123,
+ "kind": "record",
+ "fields": [
+ {
+ "name": "s",
+ "type": {
+ "kind": "primitive",
+ "name": "string"
+ }
+ },
+ {
+ "name": "x",
+ "type": {
+ "kind": "primitive",
+ "name": "int32"
+ }
+ }
+ ]
+}
+```
+
+A previously defined complex type may be referred to using a reference of the form:
+```
+{
+ "kind": "ref",
+ "id": 123
+}
+```
+
+#### 2.1.1 Record Type
+
+A record type is a JSON object of the form
+```
+{
+ "id": ,
+ "kind": "record",
+ "fields": [ , , ... ]
+}
+```
+where each of the fields has the form
+```
+{
+ "name": ,
+ "type": ,
+}
+```
+and `` is a string defining the field name and `` is a
+recursively encoded type.
+
+#### 2.1.2 Array Type
+
+An array type is defined by a JSON object having the form
+```
+{
+ "id": ,
+ "kind": "array",
+ "type":
+}
+```
+where `` is a recursively encoded type.
+
+#### 2.1.3 Set Type
+
+A set type is defined by a JSON object having the form
+```
+{
+ "id": ,
+ "kind": "set",
+ "type":
+}
+```
+where `` is a recursively encoded type.
+
+#### 2.1.4 Map Type
+
+A map type is defined by a JSON object of the form
+```
+{
+ "id": ,
+ "kind": "map",
+ "key_type": ,
+ "val_type":
+}
+```
+where each `` is a recursively encoded type.
+
+#### 2.1.5 Union type
+
+A union type is defined by a JSON object having the form
+```
+{
+ "id": ,
+ "kind": "union",
+ "types": [ , , ... ]
+}
+```
+where the list of types comprise the types of the union and
+and each ``is a recursively encoded type.
+
+#### 2.1.6 Enum Type
+
+An enum type is a JSON object of the form
+```
+{
+ "id": ,
+ "kind": "enum",
+ "symbols": [ , , ... ]
+}
+```
+where the unique `` values define a finite set of symbols.
+
+#### 2.1.7 Error Type
+
+An error type is a JSON object of the form
+```
+{
+ "id": ,
+ "kind": "error",
+ "type":
+}
+```
+where `` is a recursively encoded type.
+
+#### 2.1.8 Named Type
+
+A named type is encoded as a binding between a name and a Zed type
+and represents a new type so named. A type definition type has the form
+```
+{
+ "id": ,
+ "kind": "named",
+ "name": ,
+ "type": ,
+}
+```
+where `` is a JSON string representing the newly defined type name
+and `` is a recursively encoded type.
+
+### 2.2 Value Encoding
+
+The primitive values comprising an arbitrarily complex Zed data value are encoded
+as a JSON array of strings mixed with nested JSON arrays whose structure
+conforms to the nested structure of the value's schema as follows:
+* each record, array, and set is encoded as a JSON array of its composite values,
+* a union is encoded as a string of the form `:` where `tag`
+is an integer string representing the positional index in the union's list of
+types that specifies the type of ``, which is a JSON string or array
+as described recursively herein,
+* a map is encoded as a JSON array of two-element arrays of the form
+`[ , ]` where `key` and `value` are recursively encoded,
+* a type value is encoded [as above](#21-type-encoding),
+* each primitive that is not a type value
+is encoded as a string conforming to its ZSON representation, as described in the
+[corresponding section of the ZSON specification](zson.md#23-primitive-values).
+
+For example, a record with three fields --- a string, an array of integers,
+and an array of union of string, and float64 --- might have a value that looks like this:
+```
+[ "hello, world", ["1","2","3","4"], ["1:foo", "0:10" ] ]
+```
+
+## 3. Object Framing
+
+A ZJSON file is composed of ZJSON objects formatted as
+[newline delimited JSON (NDJSON)](https://en.wikipedia.org/wiki/JSON_streaming#NDJSON).
+e.g., the [zq](../commands/zq.md) CLI command
+writes its ZJSON output as lines of NDJSON.
+
+## 4. Example
+
+Here is an example that illustrates values of a repeated type,
+nesting, records, array, and union. Consider the file `input.zson`:
+
+```mdtest-input input.zson
+{s:"hello",r:{a:1,b:2}}
+{s:"world",r:{a:3,b:4}}
+{s:"hello",r:{a:[1,2,3]}}
+{s:"goodnight",r:{x:{u:"foo"((string,int64))}}}
+{s:"gracie",r:{x:{u:12((string,int64))}}}
+```
+
+This data is represented in ZJSON as follows:
+
+```mdtest-command
+zq -f zjson input.zson | jq .
+```
+
+```mdtest-output
+{
+ "type": {
+ "kind": "record",
+ "id": 31,
+ "fields": [
+ {
+ "name": "s",
+ "type": {
+ "kind": "primitive",
+ "name": "string"
+ }
+ },
+ {
+ "name": "r",
+ "type": {
+ "kind": "record",
+ "id": 30,
+ "fields": [
+ {
+ "name": "a",
+ "type": {
+ "kind": "primitive",
+ "name": "int64"
+ }
+ },
+ {
+ "name": "b",
+ "type": {
+ "kind": "primitive",
+ "name": "int64"
+ }
+ }
+ ]
+ }
+ }
+ ]
+ },
+ "value": [
+ "hello",
+ [
+ "1",
+ "2"
+ ]
+ ]
+}
+{
+ "type": {
+ "kind": "ref",
+ "id": 31
+ },
+ "value": [
+ "world",
+ [
+ "3",
+ "4"
+ ]
+ ]
+}
+{
+ "type": {
+ "kind": "record",
+ "id": 34,
+ "fields": [
+ {
+ "name": "s",
+ "type": {
+ "kind": "primitive",
+ "name": "string"
+ }
+ },
+ {
+ "name": "r",
+ "type": {
+ "kind": "record",
+ "id": 33,
+ "fields": [
+ {
+ "name": "a",
+ "type": {
+ "kind": "array",
+ "id": 32,
+ "type": {
+ "kind": "primitive",
+ "name": "int64"
+ }
+ }
+ }
+ ]
+ }
+ }
+ ]
+ },
+ "value": [
+ "hello",
+ [
+ [
+ "1",
+ "2",
+ "3"
+ ]
+ ]
+ ]
+}
+{
+ "type": {
+ "kind": "record",
+ "id": 38,
+ "fields": [
+ {
+ "name": "s",
+ "type": {
+ "kind": "primitive",
+ "name": "string"
+ }
+ },
+ {
+ "name": "r",
+ "type": {
+ "kind": "record",
+ "id": 37,
+ "fields": [
+ {
+ "name": "x",
+ "type": {
+ "kind": "record",
+ "id": 36,
+ "fields": [
+ {
+ "name": "u",
+ "type": {
+ "kind": "union",
+ "id": 35,
+ "types": [
+ {
+ "kind": "primitive",
+ "name": "int64"
+ },
+ {
+ "kind": "primitive",
+ "name": "string"
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ ]
+ },
+ "value": [
+ "goodnight",
+ [
+ [
+ [
+ "1",
+ "foo"
+ ]
+ ]
+ ]
+ ]
+}
+{
+ "type": {
+ "kind": "ref",
+ "id": 38
+ },
+ "value": [
+ "gracie",
+ [
+ [
+ [
+ "0",
+ "12"
+ ]
+ ]
+ ]
+ ]
+}
+```
diff --git a/versioned_docs/version-v1.16.0/formats/zng.md b/versioned_docs/version-v1.16.0/formats/zng.md
new file mode 100644
index 00000000..1c2a3b46
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/zng.md
@@ -0,0 +1,646 @@
+---
+sidebar_position: 2
+sidebar_label: ZNG
+---
+
+# ZNG Specification
+
+## 1. Introduction
+
+ZNG (pronounced "zing") is an efficient, sequence-oriented serialization format for any data
+conforming to the [Zed data model](zed.md).
+
+ZNG is "row oriented" and
+analogous to [Apache Avro](https://avro.apache.org) but does not
+require schema definitions as it instead utilizes the fine-grained type system
+of the Zed data model.
+This binary format is based on machine-readable data types with an
+encoding methodology inspired by Avro,
+[Parquet](https://en.wikipedia.org/wiki/Apache_Parquet), and
+[Protocol Buffers](https://developers.google.com/protocol-buffers).
+
+To this end, ZNG embeds all type information
+in the stream itself while having a binary serialization format that
+allows "lazy parsing" of fields such that
+only the fields of interest in a stream need to be deserialized and interpreted.
+Unlike Avro, ZNG embeds its "schemas" in the data stream as Zed types and thereby admits
+an efficient multiplexing of heterogeneous data types by prepending to each
+data value a simple integer identifier to reference its type.
+
+Since no external schema definitions exist in ZNG, a "type context" is constructed
+on the fly by composing dynamic type definitions embedded in the ZNG format.
+ZNG can be readily adapted to systems like
+[Apache Kafka](https://kafka.apache.org/) which utilize schema registries,
+by having a connector translate the schemas implied in the
+ZNG stream into registered schemas and vice versa. Better still, Kafka could
+be used natively with ZNG obviating the need for the schema registry.
+
+Multiple ZNG streams with different type contexts are easily merged because the
+serialization of values does not depend on the details of
+the type context. One or more streams can be merged by simply merging the
+input contexts into an output context and adjusting the type reference of
+each value in the output ZNG sequence. The values need not be traversed
+or otherwise rewritten to be merged in this fashion.
+
+## 2. The ZNG Format
+
+A ZNG stream comprises a sequence of frames where
+each frame contains one of three types of data:
+_types_, _values_, or externally-defined _control_.
+
+A stream is punctuated by the end-of-stream value `0xff`.
+
+Each frame header includes a length field
+allowing an implementation to easily skip from frame to frame.
+
+Each frame begins with a single-byte "frame code":
+```
+ 7 6 5 4 3 2 1 0
+ +-+-+-+-+-+-+-+-+
+ |V|C| T| L|
+ +-+-+-+-+-+-+-+-+
+
+ V: 1 bit
+
+ Version number. Must be zero.
+
+ C: 1 bit
+
+ Indicates compressed frame data.
+
+ T: 2 bits
+
+ Type of frame data.
+
+ 00: Types
+ 01: Values
+ 10: Control
+ 11: End of stream
+
+ L: 4 bits
+
+ Low-order bits of frame length.
+```
+
+Bit 7 of the frame code must be zero as it defines version 0
+of the ZNG stream format. If a future version of ZNG
+arises, bit 7 of future ZNG frames will be 1.
+ZNG version 0 readers must ignore and skip over such frames using the
+`len` field, which must survive future versions.
+Any future versions of ZNG must be able to integrate version 0 frames
+for backward compatibility.
+
+Following the frame code is its encoded length followed by a "frame payload"
+of bytes of said length:
+```
+
+```
+The length encoding utilizes a variable-length unsigned integer called herein a `uvarint`:
+
+> Inspired by Protocol Buffers,
+> a `uvarint` is an unsigned, variable-length integer encoded as a sequence of
+> bytes consisting of N-1 bytes with bit 7 clear and the Nth byte with bit 7 set,
+> whose value is the base-128 number composed of the digits defined by the lower
+> 7 bits of each byte from least-significant digit (byte 0) to
+> most-significant digit (byte N-1).
+
+The frame payload's length is equal to the value of the `uvarint` following the
+frame code times 16 plus the low 4-bit integer value `L` field in the frame code.
+
+If the `C` bit is set in the frame code, then the frame payload following the
+frame length is compressed and has the form:
+```
+
+```
+where
+* `` is a single byte indicating the compression format of the the compressed payload,
+* `` is a `uvarint` encoding the size of the uncompressed payload, and
+* `` is a bytes sequence whose length equals
+the outer frame length less 1 byte for the compression format and the encoded length
+of the `uvarint` size field.
+
+The `compressed payload` is compressed according to the compression algorithm
+specified by the `format` byte. Each frame is compressed independently
+such that the compression algorithm's state is not carried from frame to frame
+(thereby enabling parallel decoding).
+
+The `` value is redundant with the compressed payload
+but is useful to an implementation to deterministically
+size decompression buffers in advance of decoding.
+
+Values for the `format` byte are defined in the
+[ZNG compression format specification](./compression.md).
+
+> This arrangement of frames separating types and values allows
+> for efficient scanning and parallelization. In general, values depend
+> on type definitions but as long as all of the types are known when
+> values are used, decoding can be done in parallel. Likewise, since
+> each block is independently compressed, the blocks can be decompressed
+> in parallel. Moreover, efficient filtering can be carried out over
+> uncompressed data before it is deserialized into native data structures,
+> e.g., allowing entire frames to be discarded based on
+> heuristics, e.g., knowing a filtering predicate can't be true based on a
+> quick scan of the data perhaps using the Boyer-Moore algorithm to determine
+> that a comparison with a string constant would not work for any
+> value in the buffer.
+
+Whether the payload was originally uncompressed or was decompressed, it is
+then interpreted according to the `T` bits of the frame code as a
+* [types frame](#21-types-frame),
+* [values frame](#22-values-frame), or
+* [control frame](#23-control-frame).
+
+### 2.1 Types Frame
+
+A _types frame_ encodes a sequence of type definitions for complex Zed types
+and establishes a "type ID" for each such definition.
+Type IDs for the "primitive types"
+are predefined with the IDs listed in the [Primitive Types](#3-primitive-types) table.
+
+Each definition, or "typedef",
+consists of a typedef code followed by its type-specific encoding as described below.
+Each type must be decoded in sequence to find the start of the next type definition
+as there is no framing to separate the typedefs.
+
+The typedefs are numbered in the order encountered starting at 30
+(as the largest primary type ID is 29). Types refer to other types
+by their type ID. Note that the type ID of a typedef is implied by its
+position in the sequence and is not explicitly encoded.
+
+The typedef codes are defined as follows:
+
+| Code | Complex Type |
+|------|--------------------------|
+| 0 | record type definition |
+| 1 | array type definition |
+| 2 | set type definition |
+| 3 | map type definition |
+| 4 | union type definition |
+| 5 | enum type definition |
+| 6 | error type definition |
+| 7 | named type definition |
+
+Any references to a type ID in the body of a typedef are encoded as a `uvarint`,
+
+#### 2.1.1 Record Typedef
+
+A record typedef creates a new type ID equal to the next stream type ID
+with the following structure:
+```
+---------------------------------------------------------
+|0x00||...|
+---------------------------------------------------------
+```
+Record types consist of an ordered set of fields where each field consists of
+a name and its type. Unlike JSON, the ordering of the fields is significant
+and must be preserved through any APIs that consume, process, and emit ZNG records.
+
+A record type is encoded as a count of fields, i.e., `` from above,
+followed by the field definitions,
+where a field definition is a field name followed by a type ID, i.e.,
+`` followed by `` etc. as indicated above.
+
+The field names in a record must be unique.
+
+The `` value is encoded as a `uvarint`.
+
+The field name is encoded as a UTF-8 string defining a "ZNG identifier".
+The UTF-8 string
+is further encoded as a "counted string", which is the `uvarint` encoding
+of the length of the string followed by that many bytes of UTF-8 encoded
+string data.
+
+N.B.: As defined by [ZSON](zson.md), a field name can be any valid UTF-8 string much like JSON
+objects can be indexed with arbitrary string keys (via index operator)
+even if the field names available to the dot operator are restricted
+by language syntax for identifiers.
+
+The type ID follows the field name and is encoded as a `uvarint`.
+
+#### 2.1.2 Array Typedef
+
+An array type is encoded as simply the type code of the elements of
+the array encoded as a `uvarint`:
+```
+----------------
+|0x01||
+----------------
+```
+
+#### 2.1.3 Set Typedef
+
+A set type is encoded as the type ID of the
+elements of the set, encoded as a `uvarint`:
+```
+----------------
+|0x02||
+----------------
+```
+
+#### 2.1.4 Map Typedef
+
+A map type is encoded as the type code of the key
+followed by the type code of the value.
+```
+--------------------------
+|0x03|||
+--------------------------
+```
+Each `` is encoded as `uvarint`.
+
+#### 2.1.5 Union Typedef
+
+A union typedef creates a new type ID equal to the next stream type ID
+with the following structure:
+```
+-----------------------------------------
+|0x04||...|
+-----------------------------------------
+```
+A union type consists of an ordered set of types
+encoded as a count of the number of types, i.e., `` from above,
+followed by the type IDs comprising the types of the union.
+The type IDs of a union must be unique.
+
+The `` and the type IDs are all encoded as `uvarint`.
+
+`` cannot be 0.
+
+#### 2.1.6 Enum Typedef
+
+An enum type is encoded as a `uvarint` representing the number of symbols
+in the enumeration followed by the names of each symbol.
+```
+--------------------------------
+|0x05||...|
+--------------------------------
+```
+`` is encoded as `uvarint`.
+The names have the same UTF-8 format as record field names and are encoded
+as counted strings following the same convention as record field names.
+
+#### 2.1.7 Error Typedef
+
+An error type is encoded as follows:
+```
+----------------
+|0x06||
+----------------
+```
+which defines a new error type for error values that have the underlying type
+indicated by ``.
+
+#### 2.1.8 Named Type Typedef
+
+A named type defines a new type ID that binds a name to a previously existing type ID.
+
+A named type is encoded as follows:
+```
+----------------------
+|0x07||
+----------------------
+```
+where `` is an identifier representing the new type name with a new type ID
+allocated as the next available type ID in the stream that refers to the
+existing type ID ``. `` is encoded as a `uvarint` and ``
+is encoded as a `uvarint` representing the length of the name in bytes,
+followed by that many bytes of UTF-8 string.
+
+As indicated in the [data model](zed.md),
+it is an error to define a type name that has the same name as a primitive type,
+and it is permissible to redefine a previously defined type name with a
+type that differs from the previous definition.
+
+### 2.2 Values Frame
+
+A _values frame_ is a sequence of Zed values each encoded as the value's type ID,
+encoded as a `uvarint`, followed by its tag-encoded serialization as described below.
+
+Since a single type ID encodes the entire value's structure, no additional
+type information is needed. Also, the value encoding follows the structure
+of the type explicitly so the type is not needed to parse the structure of the
+value, but rather only its semantics.
+
+It is an error for a value to reference a type ID that has not been
+previously defined by a typedef scoped to the stream in which the value
+appears.
+
+The value is encoded using a "tag-encoding" scheme
+that captures the structure of both primitive types and the recursive
+nature of complex types. This structure is encoded
+explicitly in every value and the boundaries of each value and its
+recursive nesting can be parsed without knowledge of the type or types of
+the underlying values. This admits an efficient implementation
+for traversing the values, inclusive of recursive traversal of complex values,
+whereby the inner loop need not consult and interpret the type ID of each element.
+
+#### 2.2.1 Tag-Encoding of Values
+
+Each value is prefixed with a "tag" that defines:
+* whether it is the null value, and
+* its encoded length in bytes.
+
+The tag is 0 for the null value and `length+1` for non-null values where
+`length` is the encoded length of the value. Note that this encoding
+differentiates between a null value and a zero-length value. Many data types
+have a meaningful interpretation of a zero-length value, for example, an
+empty array, the empty record, etc.
+
+The tag itself is encoded as a `uvarint`.
+
+#### 2.2.2 Tag-Encoded Body of Primitive Values
+
+Following the tag encoding is the value encoded in N bytes as described above.
+A typed value with a `value` of length `N` is interpreted as described in the
+[Primitive Types](#3-primitive-types) table. The type information needed to
+interpret all of the value elements of a complex type are all implied by the
+top-level type ID of the values frame. For example, the type ID could indicate
+a particular record type, which recursively provides the type information
+for all of the elements within that record, including other complex types
+embedded within the top-level record.
+
+Note that because the tag indicates the length of the value, there is no need
+to use varint encoding of integer values. Instead, an integer value is encoded
+using the full 8 bits of each byte in little-endian order. Signed values,
+before encoding, are shifted left one bit, and the sign bit stored as bit 0.
+For negative numbers, the remaining bits are negated so that the upper bytes
+tend to be zero-filled for small integers.
+
+#### 2.2.3 Tag-Encoded Body of Complex Values
+
+The body of a length-N container comprises zero or more tag-encoded values,
+where the values are encoded as follows:
+
+| Type | Value |
+|----------|-----------------------------------------|
+| `array` | concatenation of elements |
+| `set` | normalized concatenation of elements |
+| `record` | concatenation of elements |
+| `map` | concatenation of key and value elements |
+| `union` | concatenation of tag and value |
+| `enum` | position of enum element |
+| `error` | wrapped element |
+
+Since N, the byte length of any of these container values, is known,
+there is no need to encode a count of the
+elements present. Also, since the type ID is implied by the typedef
+of any complex type, each value is encoded without its type ID.
+
+For sets, the concatenation of elements must be normalized so that the
+sequence of bytes encoding each element's tag-counted value is
+lexicographically greater than that of the preceding element.
+
+A union value is encoded as a container with two elements. The first
+element, called the tag, is the `uvarint` encoding of the
+positional index determining the type of the value in reference to the
+union's list of defined types, and the second element is the value
+encoded according to that type.
+
+An enumeration value is represented as the `uvarint` encoding of the
+positional index of that value's symbol in reference to the enum's
+list of defined symbols.
+
+A map value is encoded as a container whose elements are alternating
+tag-encoded keys and values, with keys and values encoded according to
+the map's key type and value type, respectively.
+
+The concatenation of elements must be normalized so that the
+sequence of bytes encoding each tag-counted key (of the key/value pair) is
+lexicographically greater than that of the preceding key (of the preceding
+key/value pair).
+
+### 2.3 Control Frame
+
+A _control frame_ contains an application-defined control message.
+
+Control frames are available to higher-layer protocols and are carried
+in ZNG as a convenient signaling mechanism. A ZNG implementation
+may skip over all control frames and is guaranteed by
+this specification to decode all of the data as described herein even if such
+frames provide additional semantics on top of the base ZNG format.
+
+The body of a control frame is a control message and may be JSON,
+ZSON, ZNG, binary, or UTF-8 text. The serialization of the control
+frame body is independent of the ZNG stream containing the control
+frame.
+
+Any control message not known by a ZNG data receiver shall be ignored.
+
+The delivery order of control messages with respect to the delivery
+order of values of the ZNG stream should be preserved by an API implementing
+ZNG serialization and deserialization.
+In this way, system endpoints that communicate using ZNG can embed
+protocol directives directly into the ZNG stream as control payloads
+in an order-preserving semantics rather than defining additional
+layers of encapsulation and synchronization between such layers.
+
+A control frame has the following form:
+```
+-------------------------
+||||
+-------------------------
+```
+where
+* `` is a single byte indicating whether the body is encoded
+as ZNG (0), JSON (1), ZSON (2), an arbitrary UTF-8 string (3), or arbitrary binary data (4),
+* `` is a `uvarint` encoding the length in bytes of the body
+(exclusive of the length 1 encoding byte), and
+* `` is a control message whose semantics are outside the scope of
+the base ZNG specification.
+
+If the encoding type is ZNG, the embedded ZNG data
+starts and ends a single ZNG stream independent of the outer ZNG stream.
+
+### 2.4 End of Stream
+
+A ZNG stream must be terminated by an end-of-stream marker.
+A new ZNG stream may begin immediately after an end-of-stream marker.
+Each such stream has its own, independent type context.
+
+In this way, the concatenation of ZNG streams (or ZNG files containing
+ZNG streams) results in a valid ZNG data sequence.
+
+For example, a large ZNG file can be arranged into multiple, smaller streams
+to facilitate random access at stream boundaries.
+This benefit comes at the cost of some additional overhead --
+the space consumed by stream boundary markers and repeated type definitions.
+Choosing an appropriate stream size that balances this overhead with the
+benefit of enabling random access is left up to implementations.
+
+End-of-stream markers are also useful in the context of sending ZNG over Kafka,
+as a receiver can easily resynchronize with a live Kafka topic by
+discarding incomplete frames until a frame is found that is terminated
+by an end-of-stream marker (presuming the sender implementation aligns
+the ZNG frames on Kafka message boundaries).
+
+A end-of-stream marker is encoded as follows:
+```
+------
+|0xff|
+------
+```
+
+After this marker, all previously read
+typedefs are invalidated and the "next available type ID" is reset to
+the initial value of 30. To represent subsequent values that use a
+previously defined type, the appropriate typedef control code must
+be re-emitted
+(and note that the typedef may now be assigned a different ID).
+
+## 3. Primitive Types
+
+For each ZNG primitive type, the following table describes:
+* its type ID, and
+* the interpretation of a length `N` [value frame](#22-values-frame).
+
+All fixed-size multi-byte sequences representing machine words
+are serialized in little-endian format.
+
+| Type | ID | N | ZNG Value Interpretation |
+|--------------|---:|:--------:|------------------------------------------------|
+| `uint8` | 0 | variable | unsigned int of length N |
+| `uint16` | 1 | variable | unsigned int of length N |
+| `uint32` | 2 | variable | unsigned int of length N |
+| `uint64` | 3 | variable | unsigned int of length N |
+| `uint128` | 4 | variable | unsigned int of length N |
+| `uint256` | 5 | variable | unsigned int of length N |
+| `int8` | 6 | variable | signed int of length N |
+| `int16` | 7 | variable | signed int of length N |
+| `int32` | 8 | variable | signed int of length N |
+| `int64` | 9 | variable | signed int of length N |
+| `int128` | 10 | variable | signed int of length N |
+| `int256` | 11 | variable | signed int of length N |
+| `duration` | 12 | variable | signed int of length N as ns |
+| `time` | 13 | variable | signed int of length N as ns since epoch |
+| `float16` | 14 | 2 | 2 bytes of IEEE 64-bit format |
+| `float32` | 15 | 4 | 4 bytes of IEEE 64-bit format |
+| `float64` | 16 | 8 | 8 bytes of IEEE 64-bit format |
+| `float128` | 17 | 16 | 16 bytes of IEEE 64-bit format |
+| `float256` | 18 | 32 | 32 bytes of IEEE 64-bit format |
+| `decimal32` | 19 | 4 | 4 bytes of IEEE decimal format |
+| `decimal64` | 20 | 8 | 8 bytes of IEEE decimal format |
+| `decimal128` | 21 | 16 | 16 bytes of IEEE decimal format |
+| `decimal256` | 22 | 32 | 32 bytes of IEEE decimal format |
+| `bool` | 23 | 1 | one byte 0 (false) or 1 (true) |
+| `bytes` | 24 | variable | N bytes of value |
+| `string` | 25 | variable | UTF-8 byte sequence |
+| `ip` | 26 | 4 or 16 | 4 or 16 bytes of IP address |
+| `net` | 27 | 8 or 32 | 8 or 32 bytes of IP prefix and subnet mask |
+| `type` | 28 | variable | type value byte sequence [as defined below](#4-type-values) |
+| `null` | 29 | 0 | No value, always represents an undefined value |
+
+## 4. Type Values
+
+As the ZSON data model supports first-class types and because the ZNG design goals
+require that value serializations cannot change across type contexts, type values
+must be encoded in a fashion that is independent of the type context.
+Thus, a serialized type value encodes the entire type in a canonical form
+according to the recursive definition in this section.
+
+The type value of a primitive type (include type `type`) is its primitive ID,
+serialized as a single byte.
+
+The type value of a complex type is serialized recursively according to the
+complex type it represents as described below.
+
+#### 4.1 Record Type Value
+
+A record type value has the form:
+```
+---------------------------------------------------
+|30||...|
+---------------------------------------------------
+```
+where `` is the number of fields in the record encoded as a `uvarint`,
+`` etc. are the field names encoded as in the
+record typedef, and each `` is a recursive encoding of a type value.
+
+#### 4.2 Array Type Value
+
+An array type value has the form:
+```
+--------------
+|31||
+--------------
+```
+where `` is a recursive encoding of a type value.
+
+#### 4.3 Set Type Value
+
+An set type value has the form:
+```
+--------------
+|32||
+--------------
+```
+where `` is a recursive encoding of a type value.
+
+#### 4.4 Map Type Value
+
+A map type value has the form:
+```
+--------------------------
+|33|||
+--------------------------
+```
+where `` and `` are recursive encodings of type values.
+
+#### 4.5 Union Type Value
+
+A union type value has the form:
+```
+-----------------------------------
+|34||...|
+-----------------------------------
+```
+where `` is the number of types in the union encoded as a `uvarint`
+and each `` is a recursive definition of a type value.
+
+#### 4.6 Enum Type Value
+
+An enum type value has the form:
+```
+------------------------------
+|35||...|
+------------------------------
+```
+where `` and each symbol name is encoded as in an enum typedef.
+
+#### 4.7 Error Type Value
+
+An error type value has the form:
+```
+-----------
+|36||
+-----------
+```
+where `` is the type value of the error.
+
+#### 4.8 Named Type Type Value
+
+A named type type value may appear either as a definition or a reference.
+When a named type is referenced, it must have been previously
+defined in the type value in accordance with a left-to-right depth-first-search (DFS)
+traversal of the type.
+
+A named type definition has the form:
+```
+--------------------
+|37||
+--------------------
+```
+where `` is encoded as in an named type typedef
+and `` is a recursive encoding of a type value. This creates
+a binding between the given name and the indicated type value only within the
+scope of the encoded value and does not affect the type context.
+This binding may be changed by another named type definition
+of the same name in the same type value according to the DFS order.
+
+An named type reference has the form:
+```
+-----------
+|38||
+-----------
+```
+It is an error for an named type reference to appear in a type value with a name
+that has not been previously defined according to the DFS order.
diff --git a/versioned_docs/version-v1.16.0/formats/zson.md b/versioned_docs/version-v1.16.0/formats/zson.md
new file mode 100644
index 00000000..42452771
--- /dev/null
+++ b/versioned_docs/version-v1.16.0/formats/zson.md
@@ -0,0 +1,582 @@
+---
+sidebar_position: 3
+sidebar_label: ZSON
+---
+
+# ZSON Specification
+
+## 1. Introduction
+
+ZSON is the human-readable, text-based serialization format of
+the super-structured [Zed data model](zed.md).
+
+ZSON builds upon the elegant simplicity of JSON with "type decorators".
+Where the type of a value is not implied by its syntax, a parenthesized
+type decorator is appended to the value thus establishing a well-defined
+type for every value expressed in ZSON text.
+
+ZSON is also a superset of JSON in that all JSON documents are valid ZSON values.
+
+## 2. The ZSON Format
+
+A ZSON text is a sequence of UTF-8 characters organized either as a bounded input
+or an unbounded stream.
+
+The input text is organized as a sequence of one or more Zed values optionally
+separated by and interspersed with whitespace.
+Single-line (`//`) and multi-line (`/* ... */`) comments are
+treated as whitespace and ignored.
+
+All subsequent references to characters and strings in this section refer to
+the Unicode code points that result when the stream is decoded.
+If a ZSON input includes data that is not valid UTF-8, the input is invalid.
+
+### 2.1 Names
+
+ZSON _names_ encode record fields, enum symbols, and named types.
+A name is either an _identifier_ or a [quoted string](#231-strings).
+Names are referred to as `` below.
+
+An _identifier_ is case-sensitive and can contain Unicode letters, `$`, `_`,
+and digits (0-9), but may not start with a digit. An identifier cannot be
+`true`, `false`, or `null`.
+
+### 2.2 Type Decorators
+
+A value may be explicitly typed by tagging it with a type decorator.
+The syntax for a decorator is a parenthesized type:
+```
+ ( )
+```
+For union values, multiple decorators might be
+required to distinguish the union-member type from the possible set of
+union types when there is ambiguity, as in
+```
+123. (float32) ((int64,float32,float64))
+```
+In contrast, this union value is unambiguous:
+```
+123. ((int64,float64))
+```
+
+The syntax of a union value decorator is
+```
+ ( ) [ ( ) ...]
+```
+where the rightmost type must be a union type if more than one decorator
+is present.
+
+A decorator may also define a [named type](#258-named-type):
+```
+ ( = )
+```
+which declares a new type with the indicated type name using the
+implied type of the value. Type names may not be numeric, where a
+numeric is a sequence of one or more characters in the set `[0-9]`.
+
+A decorator may also defined a temporary numeric reference of the form:
+```
+ ( = )
+```
+Once defined, this numeric reference may then be used anywhere a named type
+is used but a named type is not created.
+
+It is an error for the decorator to be type incompatible with its referenced value.
+
+Note that the `=` sigil here disambiguates between the case that a new
+type is defined, which may override a previous definition of a different type with the
+same name, from the case that an existing named type is merely decorating the value.
+
+### 2.3 Primitive Values
+
+The type names and format for
+[Zed primitive](zed.md#1-primitive-types) values is as follows:
+
+| Type | Value Format |
+|------------|---------------------------------------------------------------|
+| `uint8` | decimal string representation of any unsigned, 8-bit integer |
+| `uint16` | decimal string representation of any unsigned, 16-bit integer |
+| `uint32` | decimal string representation of any unsigned, 32-bit integer |
+| `uint64` | decimal string representation of any unsigned, 64-bit integer |
+| `uint128` | decimal string representation of any unsigned, 128-bit integer |
+| `uint256` | decimal string representation of any unsigned, 256-bit integer |
+| `int8` | decimal string representation of any signed, 8-bit integer |
+| `int16` | decimal string representation of any signed, 16-bit integer |
+| `int32` | decimal string representation of any signed, 32-bit integer |
+| `int64` | decimal string representation of any signed, 64-bit integer |
+| `int128` | decimal string representation of any signed, 128-bit integer |
+| `int256` | decimal string representation of any signed, 256-bit integer |
+| `duration` | a _duration string_ representing signed 64-bit nanoseconds |
+| `time` | an RFC 3339 UTC date/time string representing signed 64-bit nanoseconds from epoch |
+| `float16` | a _non-integer string_ representing an IEEE-754 binary16 value |
+| `float32` | a _non-integer string_ representing an IEEE-754 binary32 value |
+| `float64` | a _non-integer string_ representing an IEEE-754 binary64 value |
+| `float128` | a _non-integer string_ representing an IEEE-754 binary128 value |
+| `float256` | a _non-integer string_ representing an IEEE-754 binary256 value |
+| `decimal32` | a _non-integer string_ representing an IEEE-754 decimal32 value |
+| `decimal64` | a _non-integer string_ representing an IEEE-754 decimal64 value |
+| `decimal128` | a _non-integer string_ representing an IEEE-754 decimal128 value |
+| `decimal256` | a _non-integer string_ representing an IEEE-754 decimal256 value |
+| `bool` | the string `true` or `false` |
+| `bytes` | a sequence of bytes encoded as a hexadecimal string prefixed with `0x` |
+| `string` | a double-quoted or backtick-quoted UTF-8 string |
+| `ip` | a string representing an IP address in [IPv4 or IPv6 format](https://tools.ietf.org/html/draft-main-ipaddr-text-rep-02#section-3) |
+| `net` | a string in CIDR notation representing an IP address and prefix length as defined in RFC 4632 and RFC 4291. |
+| `type` | a string in canonical form as described in [Section 2.5](#25-types) |
+| `null` | the string `null` |
+
+The format of a _duration string_
+is an optionally-signed concatenation of decimal numbers,
+each with optional fraction and a unit suffix,
+such as "300ms", "-1.5h" or "2h45m", representing a 64-bit nanosecond value.
+Valid time units are
+"ns" (nanosecond),
+"us" (microsecond),
+"ms" (millisecond),
+"s" (second),
+"m" (minute),
+"h" (hour),
+"d" (day),
+"w" (7 days), and
+"y" (365 days).
+Note that each of these time units accurately represents its calendar value,
+except for the "y" unit, which does not reflect leap years and so forth.
+Instead, "y" is defined as the number of nanoseconds in 365 days.
+
+The format of floating point values is a _non-integer string_
+conforming to any floating point representation that cannot be
+interpreted as an integer, e.g., `1.` or `1.0` instead of
+`1` or `1e3` instead of `1000`. Unlike JSON, a floating point number can
+also be one of:
+`Inf`, `+Inf`, `-Inf`, or `Nan`.
+
+A floating point value may be expressed with an integer string provided
+a type decorator is applied, e.g., `123 (float64)`.
+
+Decimal values require type decorators.
+
+A string may be backtick-quoted with the backtick character `` ` ``.
+None of the text between backticks is escaped, but by default, any newlines
+followed by whitespace are converted to a single newline and the first
+newline of the string is deleted. To avoid this automatic deletion and
+preserve indentation, the backtick-quoted string can be preceded with `=>`.
+
+Of the 30 primitive types, eleven of them represent _implied-type_ values:
+`int64`, `time`, `duration`, `float64`, `bool`, `bytes`, `string`, `ip`, `net`, `type`, and `null`.
+Values for these types are determined by the format of the value and
+thus do not need decorators to clarify the underlying type, e.g.,
+```
+123 (int64)
+```
+is the same as `123`.
+
+Values that do not have implied types must include a type decorator to clarify
+its type or appear in a context for which its type is defined (i.e., as a field
+value in a record, as an element in an array, etc.).
+
+While a `type` value may represent a complex type, the value itself is a singleton
+and thus always a primitive type. A `type` value is encoded as:
+* a left angle bracket `<`, followed by
+* a type as [encoded below](#25-types), followed by
+* a right angle bracket `>`.
+
+A `time` value corresponds to 64-bit Unix epoch nanoseconds and thus
+not all possible RFC 3339 date/time strings are valid. In addition,
+nanosecond epoch times overflow on April 11, 2262.
+For the world of 2262, a new epoch can be created well in advance
+and the old time epoch and new time epoch can live side by side with
+the old using a named type for the new epoch time defined as the old `time` type.
+An app that requires more than 64 bits of timestamp precision can always use
+a typedef of a `bytes` type and do its own conversions to and from the
+corresponding `bytes` values.
+
+#### 2.3.1 Strings
+
+Double-quoted `string` syntax is the same as that of JSON as described
+in [RFC 8259](https://tools.ietf.org/html/rfc8259#section-7). Notably,
+the following escape sequences are recognized:
+
+| Sequence | Unicode Character |
+|----------|------------------------|
+| `\"` | quotation mark U+0022 |
+| `\\` | reverse solidus U+005C |
+| `\/` | solidus U+002F |
+| `\b` | backspace U+0008 |
+| `\f` | form feed U+000C |
+| `\n` | line feed U+000A |
+| `\r` | carriage return U+000D |
+| `\t` | tab U+0009 |
+| `\uXXXX` | U+XXXX |
+
+In `\uXXXX` sequences, each `X` is a hexadecimal digit, and letter
+digits may be uppercase or lowercase.
+
+The behavior of an implementation that encounters an unrecognized escape
+sequence in a `string` type is undefined.
+
+`\u` followed by anything that does not conform to the above syntax
+is not a valid escape sequence. The behavior of an implementation
+that encounters such invalid sequences in a `string` type is undefined.
+
+These escaping rules apply also to quoted field names in record values and
+record types as well as enum symbols.
+
+### 2.4 Complex Values
+
+Complex values are built from primitive values and/or other complex values
+and conform to the Zed data model's complex types:
+[record](zed.md#21-record),
+[array](zed.md#22-array),
+[set](zed.md#23-set),
+[map](zed.md#24-map),
+[union](zed.md#25-union),
+[enum](zed.md#26-enum), and
+[error](zed.md#27-error).
+
+Complex values have an implied type when their constituent values all have
+implied types.
+
+#### 2.4.1 Record Value
+
+A record value has the form:
+```
+{ : , : , ... }
+```
+where `` is a [ZSON name](#21-names) and `` is
+any optionally-decorated ZSON value inclusive of other records.
+Each name/value pair is called a _field_.
+There may be zero or more fields.
+
+#### 2.4.2 Array Value
+
+An array value has the form:
+```
+[ , , ... ]
+```
+If the elements of the array are not of uniform type, then the implied type of
+the array elements is a union of the types present.
+
+An array value may be empty. An empty array value without a type decorator is
+presumed to be an empty array of type `null`.
+
+#### 2.4.3 Set Value
+
+A set value has the form:
+```
+|[ , , ... ]|
+```
+where the indicated values must be distinct.
+
+If the elements of the set are not of uniform type, then the implied type of
+the set elements is a union of the types present.
+
+A set value may be empty. An empty set value without a type decorator is
+presumed to be an empty set of type `null`.
+
+#### 2.4.4 Map Value
+
+A map value has the form:
+```
+|{ : , : , ... }|
+```
+where zero or more comma-separated, key/value pairs are present.
+
+Whitespace around keys and values is generally optional, but to
+avoid ambiguity, whitespace must separate an IPv6 key from the colon
+that follows it.
+
+An empty map value without a type decorator is
+presumed to be an empty map of type `|{null: null}|`.
+
+#### 2.4.5 Union Value
+
+A union value is a value that conforms to one of the types within a union type.
+If the value appears in a context in which the type is unknown or ambiguous,
+then the value must be decorated as [described above](#22-type-decorators).
+
+#### 2.4.6 Enum Value
+
+An enum type represents a symbol from a finite set of symbols
+referenced by name.
+
+An enum value is indicated with the sigil `%` and has the form
+```
+%
+```
+where the `` is [ZSON name](#21-names).
+
+An enum value must appear in a context where the enum type is known, i.e.,
+with an explicit enum type decorator or within a complex type where the
+contained enum type is defined by the complex type's decorator.
+
+A sequence of enum values might look like this:
+```
+%HEADS (flip=(enum(HEADS,TAILS)))
+%TAILS (flip)
+%HEADS (flip)
+```
+
+#### 2.4.7 Error Value
+
+An error value has the form:
+```
+error()
+```
+where `` is any ZSON value.
+
+### 2.5 Types
+
+A primitive type is simply the name of the primitive type, i.e., `string`,
+`uint16`, etc. Complex types are defined as follows.
+
+#### 2.5.1 Record Type
+
+A _record type_ has the form:
+```
+{ : , : , ... }
+```
+where `` is a [ZSON name](#21-names) and
+`` is any type.
+
+The order of the record fields is significant,
+e.g., type `{a:int32,b:int32}` is distinct from type `{b:int32,a:int32}`.
+
+#### 2.5.2 Array Type
+
+An _array type_ has the form:
+```
+[ ]
+```
+
+#### 2.5.3 Set Type
+
+A _set type_ has the form:
+```
+|[ ]|
+```
+
+#### 2.5.4 Map Type
+
+A _map type_ has the form:
+```
+|{ : }|
+```
+where `` is the type of the keys and `` is the
+type of the values.
+
+#### 2.5.5 Union Type
+
+A _union type_ has the form:
+```
+( , , ... )
+```
+where there are at least two types in the list.
+
+#### 2.5.6 Enum Type
+
+An _enum type_ has the form:
+```
+enum( ,