Skip to content

Release Notes: v0.1.8

Date: 2026-06-13

This release adds exact-match field filtering to SourceDocument and the index query layer. Callers can now tag documents with arbitrary key-value metadata at index time and scope searches to a matching subset without post-filtering the result set. A multi-term variant builds the allowed-doc set once and reuses it across all queries, making filtered multi-term searches efficient at scale.

Highlights

  • Bumped crate version from 0.1.7 to 0.1.8.
  • Added filters: BTreeMap<String, String> field to SourceDocument, DocRecord, and PersistedDocRecord.
  • Added IndexStore::query_filtered, single-term filtered search.
  • Added IndexStore::query_filtered_multi, multi-term filtered search; builds the allowed-doc set once regardless of query count.
  • Added MemoryIndex::query_with_filters and MemoryIndex::query_with_filters_and_lexical for filtered in-memory queries.
  • Added MemoryIndex::query_with_filters_multi, a multi-term variant that reuses the allowed-doc set across all queries.
  • Propagated filters through all serialization round-trips (PersistedDocRecord From conversions, assemble_doc_record, source_document_from_record).
  • Updated MarkdownAdapter, engine.rs, and the haystack benchmark binary to supply an empty filters map on construction.

SourceDocument.filters

Documents can now carry arbitrary key-value metadata:

let doc = SourceDocument {
    filters: BTreeMap::from([
        ("run_id".into(), "abc-123".into()),
        ("workspace_id".into(), "ws-456".into()),
    ]),
    ..
};

All filter entries must match (AND semantics) for a document to be included in results. An empty filters map means no restriction.

IndexStore::query_filtered

let results = store.query_filtered(query, top_k, &filters)?;

Runs BM25 first (via tantivy), then applies the filter to the candidate set before returning results.

IndexStore::query_filtered_multi

let per_term_results = store.query_filtered_multi(&queries, top_k, &filters)?;

Runs N BM25 searches, one per query term, because tantivy cannot batch these, but builds the allowed-doc set from filters only once. At scale this avoids scanning all documents once per query term.

Returns Vec<Vec<SearchResult>> in the same order as the input queries slice.

Performance Note

For workloads with many query terms against a large document corpus, prefer query_filtered_multi over calling query_filtered in a loop. The filter scan cost is O(n_docs) and is paid exactly once regardless of query count.