Vector search

ShrikeDB stores embeddings in named indexes and returns their entity identifiers through streaming, packed, batch, and Cypher query surfaces. Use the same entity ID for a vector, graph node, and key-value record to move from similarity search to connected context and application payloads.

How vector data is organized

Every vector belongs to an index name and an entity ID. An optional schema ID provides another namespace boundary. The vector dimension and distance metric must remain consistent for an index.

Value Purpose Example

Index name

Separates embeddings by model and use case.

incident_text_v3, product_image_v2

Entity ID

Joins a search result to graph and KV data.

The UID used for the corresponding graph node

Schema ID

Selects a schema-scoped vector partition when the deployment uses one.

Application or tenant schema identifier

Vector

Ordered floating-point embedding generated by the selected model.

A 384- or 768-dimensional array

k

Maximum nearest neighbors requested.

20

ef_search

HNSW search breadth. Larger values generally examine more candidates.

128, 200, 256

Name indexes after the embedding source and model version. Model upgrades usually change the vector space, so load the new embeddings into a new index instead of mixing them with the prior model.

Try vectors in ShrikeDB Cloud

Open ShrikeDB Cloud, sign in, and choose Vectors. Upload a JSONL file with one record per line:

{"entity_id":"10042","vector":[0.18,-0.04,0.61,0.22]}
{"entity_id":"10043","vector":[0.20,-0.03,0.58,0.19]}

Name the index, inspect its row count and dimension, and load it. Enter a query vector to see nearest entity IDs and scores. Open in Cypher carries the query into the editor, where you can add graph patterns. Query a graph node in the Query view, copy its UID, and use that decimal string as entity_id when you want a vector hit to bind to the node. The Cloud Query view requests UIDs as strings to preserve all 64 bits in the browser. Each named index keeps one fixed dimension; different indexes may use different dimensions.

Cloud also exposes authenticated POST /api/v1/vector/insert, POST /api/v1/vector/search, and GET /api/v1/vector/indexes. The public HTTP workflow shows requests with a tenant API key. The free workspace admits batches of at most 256 vectors and 2 MiB per request within its storage budget.

Insert vectors with gRPC

InsertVector writes one vector. BulkInsertVectors amortizes request overhead for loaders and returns the accepted count.

grpcurl -plaintext \
  -import-path ./proto \
  -proto shrikedb.v2.proto \
  -d '{
    "indexName": "incident_text_v3",
    "entityId": {"value": "10042"},
    "vector": [0.18, -0.04, 0.61, 0.22]
  }' \
  127.0.0.1:50051 \
  shrikedb.v2.ShrikeDbService/InsertVector

For a batch:

grpcurl -plaintext \
  -import-path ./proto \
  -proto shrikedb.v2.proto \
  -d '{
    "indexName": "incident_text_v3",
    "vectors": [
      {"entityId": {"value": "10042"}, "vector": [0.18, -0.04, 0.61, 0.22]},
      {"entityId": {"value": "10043"}, "vector": [0.20, -0.03, 0.58, 0.19]}
    ]
  }' \
  127.0.0.1:50051 \
  shrikedb.v2.ShrikeDbService/BulkInsertVectors

Production connections may require TLS, bearer metadata, and a database selector. Follow Connect To ShrikeDB for connection settings.

Search through the vector API

RPC Response Use

VectorSearch

A stream of VectorSearchResult messages

Consume results incrementally.

VectorSearchPacked

Parallel arrays of entity IDs and scores

Reduce protobuf framing and allocation overhead.

VectorSearchBatchPacked

One packed result for each query vector

Submit multiple independent searches in one request.

 
grpcurl -plaintext \
  -import-path ./proto \
  -proto shrikedb.v2.proto \
  -d '{
    "indexName": "incident_text_v3",
    "queryVector": [0.17, -0.02, 0.60, 0.24],
    "k": 20,
    "efSearch": 200
  }' \
  127.0.0.1:50051 \
  shrikedb.v2.ShrikeDbService/VectorSearchPacked

snapshot_version pins a search to a published database snapshot when the caller already has that version. VectorFilter.entity_ids restricts candidates to a supplied UID set. Use filters when the allowed population is already known; use a Cypher graph pattern when graph structure defines the population.

Search inside Cypher

vector.knn makes vector results part of a Cypher row pipeline:

CALL vector.knn(
  "incident_text_v3",
  [0.17, -0.02, 0.60, 0.24],
  40,
  {ef: 200}
)
YIELD node AS incident, score
RETURN incident, score
LIMIT 20

The index name and k are literals in this execution path. The query vector can be a literal or a request parameter. The procedure yields the matching node UID and its metric score.

Use k as the candidate budget and LIMIT as the final result budget. When graph or property predicates remove candidates after the vector stage, request enough candidates to leave a useful final set.

Join vector results to graph context

A vector result is a UID. Bind that UID to a graph node and continue with an ordinary pattern:

CALL vector.knn("incident_text_v3", $query_embedding, 100, {ef: 256})
YIELD node AS incident, score
MATCH (incident:Incident)-[:INVOLVES]->(asset:Asset)
RETURN incident, asset, score
LIMIT 25

This retrieves vector candidates first, then follows a directed relationship for each candidate. Use the same UID for the vector entry and graph node. See Multimodal and hybrid queries for a complete mixed query.

Hydrate final results from KV

Keep large documents or opaque application payloads in KV and fetch them after narrowing the result:

CALL vector.knn("incident_text_v3", $query_embedding, 50, {ef: 200})
YIELD node AS incident, score
MATCH (incident:Incident)-[:INVOLVES]->(asset:Asset)
CALL kv.get("default", incident) YIELD value AS incident_json
RETURN incident, asset, score, incident_json
LIMIT 20

The first kv.get argument names the public KV namespace. PutKeyValue writes to default; use the incident UID as its key. KV reads occur after the graph expansion and final row limit.

Choose search parameters

Begin with a representative evaluation set whose relevant neighbors are known. Record recall at the required k, median latency, p95 latency, index size, ingestion rate, and concurrency. Then change one parameter at a time.

  • Increase ef_search when recall is insufficient and the latency budget permits more exploration.

  • Increase k when downstream graph or metadata filters leave too few rows.

  • Keep k close to the application requirement for unfiltered nearest-neighbor requests.

  • Use packed and batch-packed RPCs when response framing dominates many small requests.

  • Separate model versions into named indexes and compare them with the same relevance set.

For partitioned HNSW searches, ef_search controls exploration within the partitions selected for a query. Increasing it cannot recover neighbors in partitions that were skipped. The standard ANN route searches all partitions unless an operator sets a partition budget or starts the server with SHRIKEDB_HNSW_EXPERIMENTAL_SEMANTIC_SHORTLIST=1. Calibrate any shorter route against an exhaustive sample from the same embedding model before using its throughput figure as a quality-matched comparison.

Search scores only have meaning in the metric and embedding space used by that index. Do not compare raw scores from unrelated models as if they shared a scale.

Durability and restart behavior

Acknowledged vectors are part of durable database state. Small partitions and newly acknowledged entries can remain in a pending search buffer while HNSW construction proceeds; shutdown persists that state and recovery restores search visibility. Keep the entire configured data directory on persistent storage and use the normal server shutdown path.

Named indexes are scoped independently. Reusing an entity UID in two index names does not merge their vectors. Schema IDs and embedding model metadata provide further partition identity. Use the vector namespace health report when diagnosing a missing or unexpectedly empty index.

Inspect and troubleshoot a search

Symptom Check

No results

Index name, schema/database scope, vector dimension, recovered namespace inventory, and any entity filter.

Unexpected neighbors

Embedding normalization, model version, metric, query preprocessing, and recall against an exhaustive sample.

Too few rows after a graph join

Increase vector candidate k, inspect the graph predicate, and verify shared entity IDs.

Latency grows with a new setting

Compare ef_search, candidate count, rescoring, packed response use, and concurrent request load.

Results differ after restart

Confirm the same data directory and database selection, then inspect vector recovery and namespace health.

Use PROFILE for a Cypher composition and server vector traces for direct vector RPCs. Record planning, vector search, graph expansion, KV hydration, and result delivery separately.

Related documentation