gopgext/tsearch

import "gopgext/tsearch"

Package tsearch writes full text search dictionary templates and parsers in Go, and gives Go code PostgreSQL’s text search services: data files in $SHAREDIR/tsearch_data, lower-casing, and calling dictionaries.

A dictionary template is registered under the name of its init function (in main or a package initializer):

tsearch.RegisterDictionary("dintdict_init", func(opts []tsearch.Option) (tsearch.Dictionary, error) {
	d := &intDict{maxLen: 6}
	for _, o := range opts { ... } // CREATE/ALTER TEXT SEARCH DICTIONARY options
	return d, nil
})

func (d *intDict) Lexize(word string) ([]tsearch.Lexeme, error) { ... }

and the project’s sql/ files declare the functions with this package’s C symbols, and the template:

CREATE FUNCTION dintdict_init(internal) RETURNS internal
	AS 'MODULE_PATHNAME', 'gopgext_tsearch_init' LANGUAGE C STRICT;
CREATE FUNCTION dintdict_lexize(internal, internal, internal, internal) RETURNS internal
	AS 'MODULE_PATHNAME', 'gopgext_tsearch_lexize' LANGUAGE C STRICT;
CREATE TEXT SEARCH TEMPLATE intdict_template (LEXIZE = dintdict_lexize, INIT = dintdict_init);

The Dictionary init returns is the dictionary’s state: PostgreSQL caches it for the backend’s life, and tsearch drops it when PostgreSQL does (ALTER TEXT SEARCH DICTIONARY, the end of the statement that checked the options of CREATE TEXT SEARCH DICTIONARY…).

Parsers are registered under the names of their start and lextype functions (see RegisterParser).

Index#

Constants#

const (
	AddPos = 0x01 // TSL_ADDPOS: the lexeme takes a position of its own
	Prefix = 0x02 // TSL_PREFIX: prefix match (in tsqueries: lexeme:*)
	Filter = 0x04 // TSL_FILTER: a filtering dictionary: the next one gets this lexeme
)

Flags of a Lexeme (TSL_* in tsearch/ts_public.h).

Variables#

var StopWord = []Lexeme{}

StopWord is the result of Lexize for a stop word.

Functions#

func CharLen#

func CharLen(s string) int

CharLen returns the length in bytes of the first character of s in the server encoding (pg_mblen), at most len(s); 0 for “”.

func ConfigFile#

func ConfigFile(name, ext string) string

ConfigFile returns the path of text search data file name.ext in $SHAREDIR/tsearch_data (get_tsearch_config_filename: name can’t contain “/” or other unsafe characters). A name with a “.” can’t end in ext.

func FindDictionary#

func FindDictionary(name string) (pg.Oid, error)

FindDictionary returns the text search dictionary name in the schema of the SQL function being called, whatever the search_path (as unaccent(text) finds the unaccent dictionary of its extension).

func Live#

func Live() int

Live returns the number of dictionaries and parses alive in this backend (to check that none leak).

func Lower#

func Lower(s string) string

Lower lower-cases s with the database’s default collation, like text search does (str_tolower).

func ReadConfigFile#

func ReadConfigFile(path string) iter.Seq2[string, error]

ReadConfigFile reads a text search data file (a path from ConfigFile) line by line, like tsearch_readline: lines are converted from UTF-8 to the server encoding, and keep their line terminator. If the file can’t be opened, it yields an *OpenError and stops. A line that can’t be converted yields its error (a *pg.Error, e.g. code 22P05 untranslatable_character), and reading goes on with the next line if the loop continues.

While the loop runs, messages PostgreSQL reports (pg.Warning, errors of C helpers) get the context “line N of configuration file …”, as in C modules; errors Go code returns are raised after the file is closed, so they don’t.

func RegisterDictionary#

func RegisterDictionary(name string, init func(options []Option) (Dictionary, error))

RegisterDictionary registers the dictionary template whose init function (symbol gopgext_tsearch_init) is the SQL function name; its lexize function has symbol gopgext_tsearch_lexize.

func RegisterParser#

func RegisterParser(start, lextype string, p Parser)

RegisterParser registers parser p under the names of its SQL functions start (internal, int4) RETURNS internal, symbol gopgext_tsearch_start; and lextype (internal) RETURNS internal, symbol gopgext_tsearch_lextype. Its gettoken function (internal, internal, internal) RETURNS internal has symbol gopgext_tsearch_gettoken, its end function (internal) RETURNS void gopgext_tsearch_end. The headline function can be prsd_headline.

Types#

type Dictionary#

type Dictionary interface {
	// Lexize returns the lexemes of word: nil if the dictionary doesn't
	// know it (the next dictionary of the configuration gets it), an empty
	// non-nil slice (StopWord) for a word to drop.
	Lexize(word string) ([]Lexeme, error)
}

Dictionary is a text search dictionary’s state.

type Lexeme#

type Lexeme struct {
	Text    string
	Variant uint16 // nvariant
	Flags   uint16 // AddPos, Prefix, Filter
}

Lexeme is a normalized form of a word. Lexemes with the same Variant are one alternative (e.g. the parts of a compound word); different variants are alternatives.

func Lexize#

func Lexize(dict pg.Oid, word string) []Lexeme

Lexize runs text search dictionary dict on word, like ts_lexize(): nil if the dictionary doesn’t know it, an empty slice for a stop word. Errors (an unknown dictionary…) are raised.

type OpenError#

type OpenError struct {
	Path   string
	Reason string // strerror
}

OpenError is the error of a data file that can’t be opened: modules report it with their own message, e.g. “could not open synonym file”.

func (*OpenError) Error#

func (e *OpenError) Error() string

type Option#

type Option struct {
	Name     string // lower-cased, unless quoted
	Value    string // the value as text (defGetString), "" without one
	HasValue bool   // false for an option without "= value"

}

Option is an option of CREATE/ALTER TEXT SEARCH DICTIONARY (a DefElem).

func (Option) Bool#

func (o Option) Bool() (bool, error)

Bool returns the value as a boolean, like defGetBoolean: none, true, on, 1 are true; false, off, 0 are false.

func (Option) Text#

func (o Option) Text() (string, error)

Text returns the value, like defGetString: an error if there is none.

type Parser#

type Parser interface {
	// TokenTypes returns the token types Next returns.
	TokenTypes() []TokenType
	// Parse starts parsing text.
	Parse(text string) Tokenizer
}

Parser is a text search parser.

type TokenType#

type TokenType struct {
	ID          int // 1 to 255
	Alias       string
	Description string
}

TokenType is a token type of a parser (ts_token_type()).

type Tokenizer#

type Tokenizer interface {
	// Next returns the next token and its type; type 0 at the end.
	Next() (typ int, token string)
}

Tokenizer returns the tokens of a text, in order.