Skip to content

31. Keys and cryptography

The module crypto holds keys: the algorithms they belong to, the types that hold them, the text formats other tools write them in, and the rules that keep a private key out of every output. This chapter reads, writes and generates keys, signs and verifies with them, computes MACs and hashes passwords, and then describes the operations that are not in the current release: encryption and the key vault. Digests and text encodings, which need no key, are chapter 30, and the octets a key is made of are the Bytes type of chapter 2.

Algorithms and key types

crypto is not in the prelude: a program binds its functions, from crypto import *, or writes import crypto and calls crypto.read_private_key(…). An algorithm is named by a type, as a format is, and a key's type carries its algorithm:

Algorithm Kind What it is
Ed25519 signing RFC 8032, pure Ed25519
P256 signing ECDSA over NIST P-256 with SHA-256
Rsa signing RSASSA-PKCS1-v1_5 with SHA-256, 2048 to 8192 bits
X25519 agreement RFC 7748; the key of an age recipient
Aes256Gcm, ChaCha20Poly1305 cipher a 256-bit shared key
HmacSha256, HmacSha512 MAC a shared key of any length

The first four are asymmetric, each key a pair of a public and a private half. The last four are symmetric, one shared key. Four types hold them:

Type What it holds
PublicKey<A> the public half of a key of algorithm A
PrivateKey<A> the private half
KeyPair<A> both halves, as the fields public and private
SecretKey<A> a symmetric key of a cipher or a MAC

PrivateKey, KeyPair and SecretKey are secret types, the subject of the section Private keys below.

Reading a public key

read_public_key(file, format, algorithm) reads the key a file holds. The key here is the published test key 1 of RFC 8032, kept in the repository's examples:

from crypto import *

key = read_public_key(file("examples/crypto/keys/rfc8032-test-1.pub.pem"), Pem, Ed25519)
key
PublicKey<Ed25519>(kPrK_qmxVWaYVA9wwBF6Iuo3vVzz7TxHCTwXBygrS4k)

A public key prints as its type and its thumbprint: the RFC 7638 JWK thumbprint, the base64url SHA-256 of the key's canonical JWK. thumbprint() gives it as a String. It names a key in one short line, and is the same whichever format the key was read from.

A key format is named by a type:

Format Text
Pem RFC 7468 text around DER: SPKI for a public key, PKCS #8 for a private one
Jwk the RFC 7517 JSON text of one key
OpenSsh an authorized_keys line, or an OpenSSH private key file
Age age's Bech32 recipient and identity text, for X25519 only

PublicKey(text, format, algorithm) reads a key from a String. A public key is not a secret, so its text may be written in a program. The same key as an OpenSSH line and as a JWK is one value:

from crypto import *

line = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINdamAGCsQq31Uv+08lkBzoO4XLz2qYjJa8CGmj3B1Ea"
jwk = "{\"kty\": \"OKP\", \"crv\": \"Ed25519\", \"x\": \"11qYAYKxCrfVS_7TyWQHOg7hcvPapiMlrwIaaPcHURo\"}"
a = PublicKey(line, OpenSsh, Ed25519)
b = PublicKey(jwk, Jwk, Ed25519)
[a == b, a.thumbprint()]
[true, "kPrK_qmxVWaYVA9wwBF6Iuo3vVzz7TxHCTwXBygrS4k"]

Public keys compare by their octets, so == holds across formats.

Text that is not a key of the format and the algorithm is an evaluation error. to_public_key gives none instead, for text whose validity is not known in advance:

from crypto import *

"ssh-ed25519 AAAA" | to_public_key(OpenSsh, Ed25519)
none

A key of one algorithm read as another is an evaluation error naming both:

from crypto import *

read_public_key(file("examples/crypto/keys/rfc8032-test-1.pub.pem"), Pem, P256)
execute failure[invalid-argument]: examples/crypto/keys/rfc8032-test-1.pub.pem: the DER text holds an Ed25519 key, and it is read as P256
 --> <expression>:3:1
  |
3 | read_public_key(file("examples/crypto/keys/rfc8032-test-1.pub.pem"), Pem, P256)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A format that holds no key of the algorithm at all is a check error, found before anything is read. Age holds X25519 keys alone, and X25519 has no OpenSsh text:

from crypto import *

PublicKey("age1qyqszqgpqyqszqgpqyqszqgp", Age, Ed25519)
check error[type-mismatch]: Age text holds no Ed25519 key: `Age` holds `X25519` keys alone
 --> <expression>:3:1
  |
3 | PublicKey("age1qyqszqgpqyqszqgpqyqszqgp", Age, Ed25519)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Writing a public key

to_text(format) writes a public key in a format another tool reads. OpenSsh is an authorized_keys line with no comment, Jwk the canonical JSON of the required members, and Pem 64-column lines with a final line break:

from crypto import *

key = read_public_key(file("examples/crypto/keys/rfc8032-test-1.pub.pem"), Pem, Ed25519)
[key.to_text(OpenSsh), key.to_text(Jwk)]
["ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINdamAGCsQq31Uv+08lkBzoO4XLz2qYjJa8CGmj3B1Ea", "{\"crv\":\"Ed25519\",\"kty\":\"OKP\",\"x\":\"11qYAYKxCrfVS_7TyWQHOg7hcvPapiMlrwIaaPcHURo\"}"]

to_der() gives the key's SPKI DER as Bytes, and PublicKey(der, algorithm) reads it back:

from crypto import *

key = read_public_key(file("examples/crypto/keys/rfc8032-test-1.pub.pem"), Pem, Ed25519)
der = key.to_der()
[str(der.size()), str(der), str(PublicKey(der, Ed25519) == key)]
["44", "0x302a300506032b6570032100d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a", "true"]

The last 32 octets are the Ed25519 key itself; the 12 before them name the algorithm.

Private keys

A secret type is a type whose values hold private key material: PrivateKey, KeyPair and SecretKey. Four rules hold for every value of one:

  • it prints redacted;
  • no format writes it;
  • == does not compare it;
  • its text does not appear in a program's source.

read_private_key(file, format, algorithm) reads a private key. It prints as its type and its public key's thumbprint, which names the key and discloses nothing of it:

from crypto import *

key = read_private_key(file("examples/crypto/keys/rfc8032-test-1.pem"), Pem, Ed25519)
[str(key), key.public().thumbprint()]
["PrivateKey<Ed25519>(kPrK_qmxVWaYVA9wwBF6Iuo3vVzz7TxHCTwXBygrS4k)", "kPrK_qmxVWaYVA9wwBF6Iuo3vVzz7TxHCTwXBygrS4k"]

public() gives a private key's public key. On a Unix host, a private key file that its group or others may read gives a warning naming the path and its mode; chmod 600 the file to clear it. The warning is a diagnostic, and the result is unchanged.

A secret reaches no format. Writing one is a check error that names the operation that turns a private key into text:

from crypto import *

pair = generate_key_pair(Ed25519)
pair.private | format(Json)
check error[type-mismatch]: PrivateKey<Ed25519> holds private key material, which no format writes; `export_private` turns a private key into text where a program means to
 --> <expression>:4:16
  |
4 | pair.private | format(Json)
  |                ^^^^^^^^^^^^

A secret type implements no Eq. Two private keys are the same key when their public keys are equal, and that is the comparison to write:

from crypto import *

pair = generate_key_pair(Ed25519)
pair.private == pair.private
check error[type-mismatch]: PrivateKey<Ed25519> is a secret and implements no `Eq`: `==` does not compare key material; compare public keys, or a MAC with `verify_mac`
 --> <expression>:4:1
  |
4 | pair.private == pair.private
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A string literal that holds private key material is a check error. A private key is read from a file, the environment or a key vault, or generated:

import crypto

text = "-----BEGIN OPENSSH PRIVATE KEY-----"
text
check error[type-mismatch]: this literal holds a PEM private key: private key material never lives in a program's source; read it from a file (`read_private_key`), the environment (`env.var`), a key vault (`keyvault.key`), or generate it
 --> <expression>:3:8
  |
3 | text = "-----BEGIN OPENSSH PRIVATE KEY-----"
  |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exporting a private key

export_private(format) writes a private key's material as a String, the one operation that does. The result is no longer a secret type: it prints, and a format writes it. With passphrase, the text is encrypted: PKCS #8 under scrypt and AES-256-CBC for Pem, bcrypt-pbkdf and AES-256-CTR for OpenSsh. PrivateKey(text, format, algorithm, passphrase = …) reads it back, and to_private_key gives none for a wrong passphrase:

from crypto import *

pair = generate_key_pair(Ed25519)
sealed = pair.private.export_private(OpenSsh, passphrase = "correct horse")
[PrivateKey(sealed, OpenSsh, Ed25519, passphrase = "correct horse").public() == pair.public,
 sealed | to_private_key(OpenSsh, Ed25519, passphrase = "wrong") | is_none]
[true, true]

PrivateKey with a wrong passphrase is an evaluation error that says so:

from crypto import *

pair = generate_key_pair(Ed25519)
sealed = pair.private.export_private(OpenSsh, passphrase = "correct horse")
PrivateKey(sealed, OpenSsh, Ed25519, passphrase = "wrong")
execute failure[invalid-argument]: the passphrase does not decrypt the OpenSsh private key
 --> <expression>:5:1
  |
5 | PrivateKey(sealed, OpenSsh, Ed25519, passphrase = "wrong")
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Generating keys

generate_key_pair(algorithm) gives a new KeyPair from the host's cryptographically secure generator, and is effectful. bits sets the size of an Rsa key, 2048 to 8192 and 3072 by default. A generated key differs on every evaluation, so the programs here show what holds of every one:

from crypto import *

pair = generate_key_pair(X25519)
[pair.public == pair.private.public(),
 pair.public.to_text(Age) | starts_with("age1"),
 str(pair) | starts_with("KeyPair<X25519>(")]
[true, true, true]

An X25519 public key in Age text is an age recipient, age1…, and its private key exported in Age is an age identity, the text the age command line reads with -i.

generate_secret_key(algorithm) gives a new symmetric key: 32 octets for a cipher and for HmacSha256, 64 for HmacSha512. SecretKey(bytes, algorithm) builds one from octets the program already holds. A secret key prints with no thumbprint, since it has no public half:

from crypto import *

[str(generate_secret_key(Aes256Gcm)), str(SecretKey(Bytes("Jefe"), HmacSha256))]
["SecretKey<Aes256Gcm>(redacted)", "SecretKey<HmacSha256>(redacted)"]

A cipher's key is exactly 32 octets, and a MAC's at least one. Another length is an evaluation error naming it:

from crypto import *

SecretKey(Bytes("short"), Aes256Gcm)
execute failure[invalid-argument]: a Aes256Gcm key is 32 octets, and this one is 5
 --> <expression>:3:1
  |
3 | SecretKey(Bytes("short"), Aes256Gcm)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Signatures

A signature is Bytes, detached from what it signs. sign(key) signs a String's UTF-8 octets or a Bytes value with a private key of a signing algorithm. Every scheme signs deterministically: one key and one message give one signature. The Ed25519 test key signs the empty message as RFC 8032 says:

from crypto import *
from encoding import *

key = read_private_key(file("examples/crypto/keys/rfc8032-test-1.pem"), Pem, Ed25519)
"" | sign(key) | encode(Hex)
"e5564300c360ac729086e2cc806e828a84877f1eb8e5d974d873e065224901555fb8821590a33bacc61e39701cf9b46bd25bf5f0595bbe24655141438e7a100b"
Algorithm Signature
Ed25519 64 octets, R and S
P256 64 octets, r and s, as JWS ES256
Rsa the modulus length, PKCS #1 v1.5 over SHA-256, as JWS RS256

verify(signature, key) checks a signature with the public key. A signature of another message, of another key, of the wrong length or malformed gives false, never an error:

from crypto import *

pair = generate_key_pair(Ed25519)
signature = "ship order 1042" | sign(pair.private)
["ship order 1042".verify(signature, pair.public),
 "ship order 1043".verify(signature, pair.public),
 "ship order 1042".verify(signature[0:63], pair.public)]
[true, false, false]

OpenSSL writes a P-256 signature as ASN.1 DER. p256_signature_from_der converts one to the 64 octets verify reads, and gives none for octets that are not one.

Signing a tree

sign_data(key) signs a Data tree over its canonical JSON, the RFC 8785 text chapter 30 fingerprints, and verify_data(signature, key) checks one. Two trees equal under HQL equality have one signature, whatever their key order:

from crypto import *

pair = generate_key_pair(Ed25519)
a = Data("{\"name\": \"hql\", \"version\": \"0.4.0\"}", Json)
b = Data("{\"version\": \"0.4.0\", \"name\": \"hql\"}", Json)
a.sign_data(pair.private) == b.sign_data(pair.private)
true

A verifier in another language checks the signature with any RFC 8785 implementation. The examples hold two releases, each a manifest and a base64 signature by the test key. The second manifest has its version changed, and its signature fails:

from crypto import *
from encoding import *

type Release {
    manifest  : Data
    signature : String
}

public = read_public_key(file("examples/crypto/keys/rfc8032-test-1.pub.pem"), Pem, Ed25519)
releases : Seq<Release> = source(file("examples/crypto/releases.jsonl"))
releases
| map(r => r.manifest.verify_data(r.signature.to_bytes(Base64) | unwrap_or(Bytes("")), public))
[true, false]

A String and a Data tree are signed by different names: sign signs a text's octets, and sign_data a tree's canonical JSON, in which a string is written with its quotes.

MACs

A MAC is a tag computed under a shared SecretKey of HmacSha256 or HmacSha512: 32 octets for the first, 64 for the second. mac(key) gives the tag. A webhook's secret is the length its sender chose, and a MAC key takes any length from one octet. RFC 4231's test case 2:

from crypto import *
from encoding import *

key = SecretKey(Bytes("Jefe"), HmacSha256)
"what do ya want for nothing?" | mac(key) | encode(Hex)
"5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843"

verify_mac(tag, key) checks a tag, comparing it in time independent of where it first differs. == over two Bytes is not constant-time, and a received tag is checked with verify_mac:

from crypto import *
from encoding import *

key = SecretKey(Bytes("Jefe"), HmacSha256)
tag = Bytes("5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843", Hex)
["what do ya want for nothing?".verify_mac(tag, key),
 "what do ya want for something?".verify_mac(tag, key)]
[true, false]

Password hashes

A password hash is an Argon2id PHC string: $argon2id$v=19$m=19456,t=2,p=1$, then the salt and the hash. hash_password draws a new 16-octet salt on every call, so it is effectful and two hashes of one password differ. verify_password(hash) gives whether a password is the one the hash was made from, under the parameters the hash names:

from crypto import *

stored = "correct horse battery staple" | hash_password
["correct horse battery staple".verify_password(stored),
 "Tr0ub4dor&3".verify_password(stored),
 stored.starts_with("$argon2id$v=19$m=19456,t=2,p=1$")]
[true, false, true]

memory_kib, iterations and parallelism raise the cost. A cost below OWASP's minimum is an evaluation error naming the parameter:

from crypto import *

"correct horse battery staple" | hash_password(memory_kib = 4096)
execute failure[invalid-argument]: `memory_kib` is 19456 to 1048576, and 4096 is outside: a cost below OWASP's minimum is refused
 --> <expression>:3:34
  |
3 | "correct horse battery staple" | hash_password(memory_kib = 4096)
  |                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

verify_password reads Argon2id, Argon2i and Argon2d hashes. A stored hash of another scheme, such as bcrypt's $2b$, is an evaluation error naming the scheme, not a wrong password.

Encryption

Design only

This section describes functionality that is not in the current release. It is written as if it existed: these programs do not run against the current binary, and names, types and outputs may change before they ship.

Shared-key encryption is authenticated: encrypt(key) under a SecretKey of Aes256Gcm or ChaCha20Poly1305 gives a new 12-octet nonce, the ciphertext and a 16-octet tag, as one Bytes. decrypt(key) gives the plaintext, or none for octets that are not an encryption of anything under that key:

from crypto import *

key = generate_secret_key(ChaCha20Poly1305)
sealed = "meet at the north gate" | encrypt(key)
[(sealed.decrypt(key) | and_then(to_text)) == some("meet at the north gate"),
 sealed[:-1].decrypt(key) | is_none]
[true, true]

Encryption to a recipient writes an age v1 message to one or more X25519 public keys, and any of their private keys decrypts it. The age command line reads and writes the same octets:

from crypto import *

pair = generate_key_pair(X25519)
sealed = "J45.909" | encrypt_to([pair.public])
sealed | decrypt_with(pair.private) | and_then(to_text)
"J45.909"

An encrypted endpoint encrypts what a format writes. encrypted_to wraps an endpoint that target writes, and decrypted_with one that source reads; orders.jsonl.age is JSON Lines, encrypted:

from crypto import *

pair = generate_key_pair(X25519)
orders = [{id: 1, total: 12.50}, {id: 2, total: 7.25}]
orders | target(encrypted_to(file("orders.jsonl.age"), [pair.public]))
source(decrypted_with(file("orders.jsonl.age"), pair.private)) | map(o => o.id)
[1, 2]

The key vault

Design only

This section describes functionality that is not in the current release. It is written as if it existed: these programs do not run against the current binary, and names, keys, types and outputs may change before they ship.

A key vault is a store of named keys that .hql/config.toml configures, as it configures an endpoint. Its secrets are named by *_env keys and never written:

[keyvault.team]
kind = "file"
path = "keys.age"
passphrase_env = "HQL_TEAM_VAULT_PASSPHRASE"

The module crypto.keyvault, imported with import crypto.keyvault and reached as keyvault, names a key by the vault's id and the key's name. keyvault.key gives a KeyRef, which holds the two names and no key material. A stored private key signs and decrypts inside the vault and is never a value of the program:

import crypto
import crypto.keyvault

signer = keyvault.key("team", "release-2026", Ed25519)
manifest = {name: "hql", version: "0.4.0"}
signature = keyvault.sign_data(manifest, signer)
manifest.verify_data(signature, keyvault.public_key(signer))
true

keyvault.keys lists a vault's keys by name, algorithm and what each one does:

import crypto.keyvault

keyvault.keys("team") | map(k => {name: k.name, algorithm: k.algorithm, decrypts: k.decrypts})
[{algorithm: "X25519", decrypts: true, name: "backup-2026"}, {algorithm: "Ed25519", decrypts: false, name: "release-2026"}]

A vault is a file store encrypted with age, the operating system's keychain, an ssh_agent, or aws_kms. In an ssh_agent or aws_kms vault the private key never enters the hql process. hql keyvault generates, imports, lists and deletes the keys of a store; a program does none of these.