gopgext/am
import "gopgext/am"
Package am writes index access methods in Go: new kinds of indexes, like pgvector’s IVFFlat and HNSW, created with CREATE ACCESS METHOD. The extension registers a Method under the access method’s name (in main):
am.Register("ivfflat", am.Method{
Options: []am.Option{{Name: "lists", Default: 100, Min: 1, Max: 32768}},
OrderBy: true, // ORDER BY col <-> query uses the index
Build: build, // CREATE INDEX: reads the table's rows, writes records
Insert: insert, // a new row
Vacuum: vacuum, // removes the records of deleted rows
Scan: scan, // the rows matching scan keys, or nearest first
})
and its sql/ files create the access method, with one handler function per access method (all with this package’s C symbol), and its operator classes:
CREATE FUNCTION ivfflat_handler(internal) RETURNS index_am_handler
AS 'MODULE_PATHNAME', 'gopgext_am_handler' LANGUAGE C;
CREATE ACCESS METHOD ivfflat TYPE INDEX HANDLER ivfflat_handler;
CREATE OPERATOR CLASS vector_l2_ops FOR TYPE vector USING ivfflat AS
OPERATOR 1 <-> (vector, vector) FOR ORDER BY float_ops;
Storage: an Index is a meta blob (Meta, SetMeta) and chains of records (NewChain, Append, Records, Get, Update, Delete): gopgext keeps them in the index’s pages, WAL-logged (so they survive crashes and replicate). Each of those operations is atomic. Modifications only happen in Build, Insert and Vacuum, which never run concurrently for one index (they are serialized by a lock), while scans run concurrently with them and see each operation either entirely or not at all. A record holds at most about 8,100 bytes, the meta blob about 8,000.
Go errors (and panics) become PostgreSQL errors, as in SQL functions.
Index#
func Register(name string, m Method)- type ChainID
- type Index
func (idx *Index) Append(chain ChainID, b []byte) RecordIDfunc (idx *Index) Columns() intfunc (idx *Index) Delete(id RecordID)func (idx *Index) Get(id RecordID) []bytefunc (idx *Index) ID() int64func (idx *Index) Meta() []bytefunc (idx *Index) Name() stringfunc (idx *Index) NewChain() ChainIDfunc (idx *Index) OpFamily(col int) stringfunc (idx *Index) Option(name string) intfunc (idx *Index) Records(chain ChainID) iter.Seq2[RecordID, []byte]func (idx *Index) SetMeta(b []byte)func (idx *Index) Update(id RecordID, b []byte)
- type Method
- type Option
- type RecordID
- type Result
- type Rows
- type ScanKey
- type Scanner
- type TID
- type Value
Functions#
func Register#
func Register(name string, m Method)
Register registers the access method named name (call it in main).
Types#
type ChainID#
type ChainID int64
ChainID identifies a chain of records.
type Index#
type Index struct {
// contains filtered or unexported fields
}
Index is an index of the access method.
func (*Index) Append#
func (idx *Index) Append(chain ChainID, b []byte) RecordID
Append adds a record at the end of a chain.
func (*Index) Columns#
func (idx *Index) Columns() int
Columns returns the number of key columns.
func (*Index) Delete#
func (idx *Index) Delete(id RecordID)
Delete deletes a record.
func (*Index) Get#
func (idx *Index) Get(id RecordID) []byte
Get returns a copy of a record.
func (*Index) ID#
func (idx *Index) ID() int64
ID identifies the index’s storage: it changes when the index is rebuilt (REINDEX, TRUNCATE), so it can key caches of its contents.
func (*Index) Meta#
func (idx *Index) Meta() []byte
Meta returns the meta blob (empty in a new index).
func (*Index) Name#
func (idx *Index) Name() string
Name returns the access method’s name.
func (*Index) NewChain#
func (idx *Index) NewChain() ChainID
NewChain creates an empty chain of records.
func (*Index) OpFamily#
func (idx *Index) OpFamily(col int) string
OpFamily returns the name of the operator family of column col (0-based): its operator class’s name, unless created with FAMILY.
func (*Index) Option#
func (idx *Index) Option(name string) int
Option returns the value of an option of the index.
func (*Index) Records#
func (idx *Index) Records(chain ChainID) iter.Seq2[RecordID, []byte]
Records returns the records of a chain, in order (copies). Records appended while ranging may or may not be seen.
func (*Index) SetMeta#
func (idx *Index) SetMeta(b []byte)
SetMeta replaces the meta blob.
func (*Index) Update#
func (idx *Index) Update(id RecordID, b []byte)
Update replaces a record with one of the same size.
type Method#
type Method struct {
Options []Option
// SupportFunctions is the number of support functions operator
// classes can declare (FUNCTION n).
SupportFunctions int
// OrderBy: scans can return rows ordered by an operator (ORDER BY col
// <-> value): operator classes declare it FOR ORDER BY.
OrderBy bool
// MultiColumn: indexes can have several columns.
MultiColumn bool
// Build fills a new, empty index with the rows of the table, and
// returns the number of index entries.
Build func(idx *Index, rows Rows) (int, error)
// Insert indexes a new row.
Insert func(idx *Index, tid TID, values []Value) error
// Vacuum removes the entries of the rows for which dead is true, and
// returns how many it removed and how many remain.
Vacuum func(idx *Index, dead func(TID) bool) (removed, remaining int, err error)
// Cost returns the fraction of the index a scan reads (for the
// planner); nil: all of it.
Cost func(idx *Index) float64
// Scan starts a scan: keys are conditions, orderBys orderings.
Scan func(idx *Index, keys, orderBys []ScanKey) (Scanner, error)
}
Method is an index access method.
type Option#
type Option struct {
Name string
Desc string
Default int
Min, Max int
}
Option is an integer option of the indexes (CREATE INDEX … WITH (lists = 100)).
type RecordID#
type RecordID int64
RecordID identifies a record. IDs of deleted records may be reused.
type Result#
type Result struct {
TID TID
// Distances are the values of the ORDER BY operators for the row (one
// per ordering key), in ascending order across results.
Distances []float64
// Recheck asks PostgreSQL to check the conditions on the row.
Recheck bool
// RecheckOrder means Distances are lower bounds: PostgreSQL computes
// the exact ones (with the operators) and reorders.
RecheckOrder bool
// NullDistance: the distances are NULL (a NULL value, or a NULL
// query): such rows come last.
NullDistance bool
}
Result is a row a scan returns.
type Rows#
type Rows = iter.Seq2[TID, []Value]
Rows are the rows of the table being indexed (Build): their TID and the values of the index’s columns (NULL ones included). Ranging over it scans the table; it can be scanned more than once.
type ScanKey#
type ScanKey struct {
Column int // 0-based
Strategy int
Arg Value
}
ScanKey is a condition (WHERE col op Arg) or an ordering (ORDER BY col op Arg) of a scan: Strategy is the operator’s number in the operator class.
type Scanner#
type Scanner interface {
Next() (r Result, ok bool, err error)
}
Scanner returns the results of a scan, one at a time, until ok is false.
func Results#
func Results(rs []Result) Scanner
Results returns a Scanner over rs.
type TID#
type TID struct {
Block uint32
Offset uint16
}
TID identifies a table row (a heap tuple).
func TIDFromBytes#
func TIDFromBytes(b []byte) TID
TIDFromBytes is the inverse of TID.Bytes.
func (TID) Bytes#
func (t TID) Bytes() []byte
Bytes returns t as 6 bytes, to store it in a record.
type Value#
type Value struct {
// contains filtered or unexported fields
}
Value is an indexed value or the argument of a scan key.
func (Value) Bytes#
func (v Value) Bytes() []byte
Bytes returns a copy of the contents of a variable-length value.
func (Value) Datum#
func (v Value) Datum() uint64
Datum returns the raw Datum.
func (Value) Float64#
func (v Value) Float64() float64
Float64 returns a double precision value.
func (Value) Int64#
func (v Value) Int64() int64
Int64 returns a bigint (or a smaller integer type) value.
func (Value) IsNull#
func (v Value) IsNull() bool
IsNull reports whether the value is NULL.
func (Value) View#
func (v Value) View() []byte
View returns the contents of a variable-length value (bytea, or a type stored like it), not copied: valid during the call (scan keys: until the scan ends).