Semantic Search

Semantic search retrieves indexed document chunks ranked by embedding cosine similarity. Unlike Wire (headline + metadata), results include passage text you can cite, plus title, scores, entity names on the chunk, and source provenance.

When to use

Use semantic search when you need to:

• Pull evidence passages (numbers, quotes, guidance language)
• Research a theme hard to express as exact filters
• Cover a company or event from multiple angles (semantic_search_multi)
• Backtest what the corpus said in a time window
• Filter by publisher, chunk sentiment, utility, or coarse entity types

Do not use semantic search when you need:

• What just broke / novelty clusters → Wire
• Exact company-name monitoring → Wire entities or retrieve_entity
• Guaranteed entity match — semantic match is by meaning, not registry id

Use cases

Evidence for a specific claim — Pull quoteable passages that support a narrow factual question (guidance, numbers, language).

Code
{
  "query": "STMicroelectronics AI data center revenue guidance 2026 2027",
  "top_k": 8,
  "start_timestamp": "2026-07-01T00:00:00Z",
  "utility_score_ge": 0.7
}

Thematic macro / geopolitics — Find passages about a theme even when tickers and headlines don’t say it cleanly.

Code
{
  "query": "Red Sea Houthi tanker disruption Brent crude shipping",
  "top_k": 10,
  "start_timestamp": "2026-07-01T00:00:00Z",
  "cosine_similarity_ge": 0.7
}

Positive earnings reaction language — Screen for upbeat beat-and-raise style coverage.

Code
{
  "query": "company beat earnings estimates raised guidance",
  "top_k": 10,
  "start_timestamp": "2026-07-20T00:00:00Z",
  "sentiment_score_ge": 0.4
}

Negative tone screen — Find passages stressing cost, disruption, or margin pressure.

Code
{
  "query": "airline fuel cost Middle East disruption margin pressure",
  "top_k": 10,
  "start_timestamp": "2026-07-01T00:00:00Z",
  "sentiment_score_le": -0.3
}

Publisher-constrained research — Same research question, limited to publishers you trust.

Code
{
  "query": "Apple services revenue growth China",
  "top_k": 10,
  "source_metadata_filters": [
    { "key": "publisher", "include_any": ["seekingalpha.com", "www.cnbc.com"] }
  ],
  "start_timestamp": "2026-01-01T00:00:00Z"
}

Multi-angle company deep-dive — Cover several facets of one company in one call instead of repeating single searches.

Code
{
  "queries": [
    "Tesla Q2 automotive margins",
    "Tesla Robotaxi Cybercab rollout",
    "Tesla Optimus humanoid robot guidance"
  ],
  "total_top_k": 30,
  "strategy": "round_robin",
  "start_timestamp": "2026-07-01T00:00:00Z",
  "utility_score_ge": 0.5
}

Multi-query banking theme — Cover NIM, credit quality, and the Fed rate path in one call (default round_robin diversification across queries).

Code
{
  "queries": [
    "bank net interest margin outlook",
    "regional bank credit quality deposits",
    "Federal Reserve rate path bank earnings"
  ],
  "total_top_k": 20,
  "start_timestamp": "2026-07-01T00:00:00Z"
}

Backtest window — See what the corpus said during a fixed period, not the whole default history.

Code
{
  "query": "bank net interest margin outlook",
  "top_k": 20,
  "start_timestamp": "2026-07-20T00:00:00Z",
  "end_timestamp": "2026-07-22T00:00:00Z"
}

What you get back

Each hit is a chunk, not a full article.

Prefer specific queries + modest top_k (5–15). Skim by document_id.

Defaults that shape every call

Future end_timestamp is treated as realtime (no upper bound).

How to write queries

1. Specific topical phrases beat bare names. Tesla Q2 earnings margin guidance Robotaxi → cosine ~0.77–0.80. Query Tesla alone → ~0.69 on generic pieces.
2. Keyword-dense topic packs work well. Red Sea Houthi oil tanker Brent crude slightly outperformed a long natural-language question on the same theme.
3. Questions are fine when they encode entities + mechanism; fluff words rarely help.
4. Facet the problem for multi. Split “margins / Robotaxi / Optimus” into separate queries.
5. Not an entity resolver. A query mentioning easyJet can still return Ryanair fuel passages — embedding neighborhood, not registry match.

Filters

semantic_search_multi

Required: queries with min 2, max 50. One call, server-side dedupe, consolidation strategies.

After consolidation, the returned list is still ordered by cosine_similarity. Strategy changes which chunks make the cut, not the final display sort.

round_robin / rank_sum pull more facet diversity; cosine_similarity can keep consecutive chunks from one article near the top. Deduping is by chunk — several chunks from one document_id can still appear.

Workflow

1. Decide Wire vs semantic search (headlines/novelty vs passage evidence).
2. Write a specific query (or 2–5 facet queries for multi). Set start_timestamp explicitly.
3. Tighten with utility_score_ge, optional cosine floor ~0.70, publisher/entity_types only when needed.
4. Collapse by document_id; quote content with URL from source.source_metadata.
5. For live story / who printed first, switch to Wire with canonical names from chunk entities.

Limitations

• Passage retrieval, not novelty-aware news clustering.
• Same document can dominate top_k via multiple chunks — collapse by document_id.
• Not an exact entity filter; related peers/themes bleed in.
• canonical_entity_names can be incomplete vs the text.
• Publisher filter is a closed enum of domain-style strings — different from Wire publishers.
• Default start_timestamp and utility_score_ge silently shape every naive call.
• Aggressive cosine_similarity_ge easily returns [].
• Corpus mixes news, YouTube transcripts, and social — transcripts are noisy but sometimes uniquely informative.

FAQ

Q: Why are Fed-rate results from months ago when I wanted “now”?
A: Default start_timestamp is 2026-01-01; ranking is pure cosine, not recency. Set a recent start_timestamp.

Q: Why did cosine_similarity_ge: 0.75 return nothing?
A: Top hits may sit at ~0.72–0.73. Lower the floor or improve the query.

Q: I searched easyJet and got Ryanair — bug?
A: Semantic neighborhood for airline + fuel. For name-locked monitoring use Wire entities.

Q: Why so many near-identical chunks?
A: Chunk-level index. Collapse on document_id before presenting sources.

Q: Wire publisher: "bloomberg" vs semantic www.cnbc.com?
A: Different vocabularies. Semantic publisher filter only accepts its schema enum.

Q: When is multi worth it?
A: ≥2 distinct facets. Prefer multi over parallel single calls for dedupe.

Q: Does round_robin scramble the final order?
A: Selection is diversified; the response list is still sorted by cosine similarity.