13. Working with hql¶
The language is half of what you use; the other half is the command and what
it says back. This chapter is about the tools: the commands, the diagnostics,
and the messages hql writes when a program is wrong.
The commands¶
| Command | What it does |
|---|---|
hql eval <program> |
check and evaluate a program given as an argument |
hql run <file> |
check and evaluate a program in a file; - reads standard input |
hql check <file> |
write the program's type, and the signature of every alias, without evaluating |
hql repl |
read, evaluate and print, keeping bindings between lines |
hql render <file> |
build a document from its cells (chapter 24) |
hql builtins |
list every step, by the provider that implements it, with its signature |
hql config |
the settings in force, and where each came from |
hql context [file] |
the execution context: the mode, the project, the vault, the configuration files and module roots, and where each came from |
hql doctor |
check the semantic-search chain and name the first missing part |
--vault <dir> names the vault, --to <format> the output format, and
--envelope json wraps the answer for another program. --project <dir>,
--bare, --import <module> and --context <name> set the execution
context, below.
The REPL¶
hql repl keeps its bindings from one input to the next. The arrow keys edit a
line and recall earlier inputs, from a history kept between sessions. An input
goes on over several lines while a bracket is open, or while the next line
opens with |, so every program in this book can be pasted in as it is
written. :type asks for the type of an expression, or, given a function's
name, prints the standard library's declarations of it, one per overload.
:quit leaves, as do quit, exit and Ctrl-D:
$ hql repl
hql 0.5.1. `:help` for commands, `:quit` to leave.
hql> shelf = [412, 280]
()
hql> shelf | sum
692
hql> :type shelf
List<Int>
hql> :type range
fn range(start: Int, end: Int, step: Int = 1) : List<Int> // std/seq
hql> :quit
What a step takes¶
hql builtins lists every step with its signature, one line per form,
grouped by the module declaring it. Each heading names the module, the
package providing it and whether a program has it without an import:
$ hql builtins
collections (std 0.6.0, always available)
size
Collection<T> | size<T>() -> Int
Window<T> | size<T>() -> Int
pure, reduction
The number of element occurrences in `self`, an `Int`: `[1, 2, 2, 3] | size` is `4`, and `Set([1, 2, 2]) | size` is `2`.
contains
Collection<T> | contains<T>(value: T (positional)) -> Bool
pure, reduction
`true` when `value` occurs in `self`, and `false` otherwise: `[1, 2, 3] | contains(2)` is `true`.
…
A function of a module that the program has not imported is listed too, so
the import it needs can be looked up. import stats binds the module's
name, so its functions are called stats.mean(…); from stats import *
binds mean and the rest by their own names (chapter 5). The modules of
lib/std/prelude.hql, core, collections, string, seq, math,
option, result, graph, format and io among them, are imported both
ways in every program unless the project's .hql/config.toml says
prelude = false under [modules].
Diagnostics¶
A run produces a value and the diagnostics its phases wrote: warnings that change nothing, errors that make a result wrong, failures that make one impossible. Each is headed by the phase that wrote it, its severity and its kind, and names where it points, by line and column. Chapter 12 is the whole error model.
By default hql stops at the first error. --report collect reports every
error it can find:
$ hql --report collect eval 'a = nosuchthing
b = alsomissing'
check error[unknown-name]: `nosuchthing` is not bound
--> <expression>:1:5
|
1 | a = nosuchthing
| ^^^^^^^^^^^
check error[unknown-name]: `alsomissing` is not bound
--> <expression>:2:5
|
2 | b = alsomissing
| ^^^^^^^^^^^
2 diagnostics: 0 failure, 2 error, 0 warning
A misspelled step suggests the nearest one, and a retired name says what
replaced it, on a help line under the caret:
$ hql eval '[1, 2] | mapp(n => n)'
check error[unknown-step]: `mapp` is not a pipeline step
--> <expression>:1:10
|
1 | [1, 2] | mapp(n => n)
| ^^^^^^^^^^^^
= help: did you mean `map`?
A warning never changes the exit status. Names have one casing per kind —
snake_case for bindings, functions and fields, PascalCase for types — and
a name off its kind's convention is a warning where it is written:
$ hql eval 'bookCount = 7
bookCount'
7
check warning[naming-convention]: `bookCount` is a binding, written in snake_case
--> <expression>:1:1
|
1 | bookCount = 7
| ^^^^^^^^^
= help: write it `book_count`
--envelope json carries every diagnostic, whatever the level, as data, and
--diagnostics-format json writes each one to standard error as one JSON
document per line. A program that wants to report on itself, apart from its
result, writes to a logger: the logging page of chapter 9.
The execution context¶
Every invocation runs in an execution context: a project or none, a vault
or none, and the configuration gathered for it. The same program text checks
and evaluates the same way in every context; the context determines the
module roots, packages, endpoints and settings it is given, and, where a run
imports hmd, the documents it reads as cards.
Projects and vaults¶
A project is HQL's: a directory holding .hql/config.toml, which
configures the programs run in it. A vault is HyperMarkdown's: a
directory holding .hmd/, whose documents are cards. The two are found
apart, and one directory may be both.
The anchor is the directory discovery starts from: the source directory
of the program, the directory of its file for hql run, check, plan,
boundary and context <file>, and the working directory for an expression
or standard input. hql walks from the anchor upward to the nearest
directory holding .hql/config.toml, the project, and separately to the
nearest holding .hmd/, the vault. The walk for a project skips the
directory whose .hql/ is the HQL home, so ~/.hql/config.toml is never a
project. The mode is project where a project was found, and bare where
none was.
The working directory is not consulted when a file is the anchor, so a
program finds its project wherever hql is run from. A relative path in a
program resolves against its source directory too, never against the working
directory (the files and folders page of chapter 9).
--project <dir> names the project and replaces its walk; the directory must
hold .hql/config.toml. --vault <dir> names the vault the hmd package
reads, and the directory must hold .hmd/: it is the vault root, not the
directory its cards start in, which the vault's wiki setting names
(chapter 21). A directory inside a vault that holds no .hmd/
is refused, naming the vault root above it. Where no project is named, the
project is the nearest at or above the vault. --bare runs with no project
and no discovery at all, and --bare with --project is a usage error.
hql render finds the vault of each document from that document's
directory, and under --bare renders none.
hql context writes the context, each part with the layer it came from:
$ cd ~/tmp
$ hql context ../notes/scripts/report.hql
mode: project
context: work (user configuration)
project: /home/ada/notes (discovered from the anchor)
vault: /home/ada/notes (discovered from the anchor)
anchor: /home/ada/notes/scripts
cwd: /home/ada/tmp
hql-home: /home/ada/.hql (the default)
prelude: on (the default)
config:
/etc/hql/config.toml (system configuration, absent)
/home/ada/.hql/config.toml (user configuration)
/home/ada/notes/.hql/config.toml (the project's .hql/config.toml)
module roots:
/home/ada/.hql/lib (user configuration)
imports:
hmd (the project's .hql/config.toml)
--envelope json writes the same as one JSON document.
Configuration layers¶
Bare means no project, not no configuration. These layers contribute in every mode, each read in turn:
| Layer | Where |
|---|---|
| system | /etc/hql/config.toml on Unix, %ProgramData%\hql\config.toml on Windows; $HQL_SYSTEM_CONFIG names another file |
| user | config.toml in the HQL home, ~/.hql/; $HQL_HOME names another directory |
| named context | the selected [context.<name>] table of the system or user file |
| HQL home modules | ~/.hql/modules/, a module root when it exists |
| project | the project's .hql/config.toml |
$HQL_PATH |
further module roots, separated as $PATH is |
| command line | --modules <dir> and --import <module> |
The system and user files take the keys a project's .hql/config.toml
takes: [modules] roots, import and prelude, [packages] dependencies
and paths, [report] mode and level, and [execution] backend. A
relative path in a project's file is read against the project root, the
directory holding .hql/; in the system or user file, against the directory
holding that file. A configuration file that does not parse, or that states a
key its layer does not take, fails every run, naming the file and the key.
Module roots and imports gather across the layers, each once. A gathered root
is a place to look, not a priority: a module that two roots provide is still
an error. prelude is the last layer's that states it. The HQL home is the
user's environment, not a project, and never a vault.
Packages come from the project and from the HQL home, in that order, each
holding its own packages/ (chapter 36).
Named contexts¶
A named context is a [context.<name>] table of the system or user file:
settings grouped under a name and reused across projects. It takes the keys
above, and project, which binds a project, read against the directory of
the declaring file. A user file's definition of a name replaces the system
file's; a project's .hql/config.toml defines no context.
current-context = "work"
[modules]
roots = ["lib"]
[context.work.report]
level = "info"
[context.archive]
project = "../archive"
One context is selected: --context <name>, else $HQL_CONTEXT, else
current-context, else a context named default when one is defined. A
selected name that no file defines fails every run, naming the layer that
selected it. A context that binds a project replaces the walk for one; a
context without one leaves discovery as it is. A context selected by
--context or $HQL_CONTEXT that binds a project, together with --bare or
a --project naming another directory, is an error; under a context
selected by current-context, the command line wins.
hql config lists and changes the selection; use-context writes
current-context into the user file and keeps every other line of it:
$ hql config get-contexts
CURRENT NAME PROJECT DEFINED IN
archive /home/ada/archive /home/ada/.hql/config.toml
* work - /home/ada/.hql/config.toml
$ hql --context archive context
mode: project
context: archive (--context)
project: /home/ada/archive (context `archive`)
…
$ hql --context nope eval 1
hql: no context is named `nope`, which --context selects; the defined contexts are `archive`, `work`
hql config current-context writes the persistent current-context,
which --context and $HQL_CONTEXT leave unchanged, hql config
use-context <name> sets it, and hql config view writes the system and
user files merged, a table key by key and any other value from the user
file.
Settings¶
The report mode and level come from the command line, then the project's
.hql/config.toml, then the environment ($HQL_REPORT, $HQL_REPORT_LEVEL), then the
selected context, the user file and the system file, then the defaults. The
backend follows the same order with --backend, [execution] backend and
$HQL_BACKEND. hql config says which applies, and why:
$ hql config
report mode: strict (from the default)
report level: warning (from the default)
backend: localhost (from the default)
vault: none
backend selects the execution strategy described in
Logical steps and physical plans, and
localhost is the only one. Run inside a vault, or with --vault, the last
line names it and counts its cards.
What is next¶
Every chapter so far was about a step. Chapter 14 is about
the pipeline the steps form: what | does, when a program is checked, where
it runs, and the systems a pipeline will connect.