Filtered vs Unfiltered Searches

Understanding when MarkLogic uses filtered versus unfiltered searches is essential for both accuracy and performance. The default behaviour varies by function, which surprises many developers. A filtered search uses indexes to find candidates, then opens each document to confirm the match. An unfiltered search relies solely on index resolution — faster, but able to return false positives.

cts:search() defaults to filtered; cts:uris() and other lexicon-based functions default to unfiltered. cts:search() returns full documents where accuracy matters; lexicon functions are optimised for speed. Path expressions in XQuery (like fn:doc(...)/parent/child) are a special case — when used as the root context in cts:search(), "filtered" and "unfiltered" produce materially different results. Unfiltered returns the first node in the fragment regardless of whether it genuinely matches; filtered validates each node and returns only genuine matches. The MarkLogic docs warn that unfiltered searches should only be performed at top-level nodes or fragment roots, and that path expressions below the fragment root can produce unexpected answers and stop after the first matching node per fragment.

Unfiltered is not a performance option — it is a decision to trust index resolution. That trust is reasonable for many workloads, but becomes a material risk with proximity or near queries, phrase queries (especially three or more words), wildcard queries, case- or diacritic-sensitive searches, geospatial precision requirements, reverse geospatial queries such as polygon containment (which can return the entire dataset as false positives when unfiltered), negation, repeated substructures within fragments, searches below fragment root, or index transition states during a reindex. For business-critical retrieval, audit, compliance, or migration workflows, treat filtered execution as the safer default unless the query shape, database configuration, and index coverage are known to support accurate unfiltered resolution.

One instructive edge case: cts:column-range-query() — a query constrained by a TDE column. cts:uris((), (), cts:column-range-query("schema", "view", "column", "=", "value")) returns the expected URIs. cts:search(fn:collection(), cts:column-range-query("schema", "view", "column", "=", "value")) with the same query may return zero results. Query tracing shows that cts:search() did correctly resolve the matching fragments — then discarded all of them in the filter pass. The likely cause is that the SPARQL resolution powering cts:column-range-query() interacts with filtering in a way that eliminates every candidate. The fix is to pass "unfiltered" to cts:search(). This is the kind of behaviour that costs real debugging time if you do not know it exists.

# Filtered Searches

# How Filtered Search Works

When a search runs in filtered mode, MarkLogic performs a two-phase operation:

1. Index resolution — the query is evaluated against the indexes to produce a set of candidate document fragments.
2. Filtering — each candidate is opened and examined against the query to confirm it genuinely matches.

This guarantees accuracy at the cost of document retrieval. In most transactional workloads this is the right trade-off, but in high-throughput read scenarios the per-document cost can add up.

# Sample Data

The examples in Sections 1 and 2 assume the llamaverse (v2.0+) is deployed. The llamaverse contains JSON documents for each llama resident, stored in the llamaverse collection. Raw llama documents follow the URI pattern /cleverllamas/llamaverse/raw/wild-llamas/llamas/{uuid}.json. We use the cleverllamas-llama user to read the data, as recommended.

The llamaverse sample data is freely available from github.com/cleverllamas/llamaverse — see the llamaverse article for full setup instructions.

The path expression examples in Section 3 use the llamaverse movement history document — a single JSON document at /cleverllamas/llamaverse/content/llama-movement/llama_location_history.json that records 3,360 GPS movement entries for multiple llama residents. An excerpt is shown inline in each example.

The filtered-versus-unfiltered path expression example uses Bradley's health record — an XML document introduced in llamaverse 2.0. The document structure is shown in full in the code group below.

# cts:search() — Filtered by Default

cts:search() is filtered unless you explicitly say otherwise. MarkLogic resolves index candidates and then opens each document to confirm it genuinely matches before returning it.

Here we use a cts:near-query() to search for llama profiles where the words "llama" and "cooking" appear within 1 word of each other. Every llama profile contains both words, but they are always separated by many words in the description text — so the filter step removes every candidate:

xquery version "1.0-ml";

(: FILTERED search (the default).                                         :)
(: MarkLogic uses the word index to find candidates, then opens each      :)
(: document to verify the match. Only documents where "llama" and         :)
(: "cooking" genuinely appear within 1 word of each other are returned.   :)
let $query := cts:near-query((
  cts:word-query("llama"),
  cts:word-query("cooking")
), 1)
for $doc in cts:search(fn:collection("llamaverse"), $query)
return fn:string($doc/name)

(: empty sequence :)

(: The filter re-checks each candidate document and confirms that         :)
(: "llama" and "cooking" are never within 1 word of each other.           :)
(: No false positives reach the caller.                                   :)

# Explicitly Requesting Unfiltered

Passing "unfiltered" skips the document-opening step entirely. MarkLogic returns whatever the word index resolves to. Using the same near-query, every document that contains both "llama" and "cooking" is returned — regardless of how far apart those words actually are:

xquery version "1.0-ml";

(: UNFILTERED search.                                                     :)
(: MarkLogic returns whatever the word index resolves to, without         :)
(: opening any document to verify proximity.                              :)
(: Every document containing both "llama" and "cooking" is a candidate — :)
(: and every one of those is a false positive.                            :)
let $query := cts:near-query((
  cts:word-query("llama"),
  cts:word-query("cooking")
), 1)
for $doc in cts:search(fn:collection("llamaverse"), $query, "unfiltered")[1 to 5]
return fn:string($doc/name)

"Aaron"
"Angela"
"Anthony"
"Bradley"
"Chloe"
(: ... 1,015 more llama names :)

(: All 1,020 documents containing both "llama" and "cooking" are returned.:)
(: The words exist in every document but are never adjacent — every       :)
(: result here is a false positive. The filtered version returns zero.    :)

# Unfiltered Searches

# How Unfiltered Search Works

Lexicon-based functions such as cts:uris(), cts:values(), and cts:collections() resolve purely from index data. No documents are opened at all. This makes them extremely fast, but the results reflect what the index contains, not necessarily what the documents currently contain.

# cts:uris() — Unfiltered by Default

Here we retrieve the URIs of all llamaverse documents that contain the word "Bradley" — a llama resident of the sanctuary known for his love of poetry and chess:

xquery version "1.0-ml";

(: cts:uris() is UNFILTERED by default.                                   :)
(: It resolves URIs entirely from the URI lexicon and word indexes —      :)
(: no documents are opened. Very fast, but accuracy depends entirely on  :)
(: the current state of the indexes.                                      :)
cts:uris((), (), cts:and-query((
  cts:collection-query("llamaverse"),
  cts:word-query("Bradley")
)))

/cleverllamas/llamaverse/raw/wild-llamas/llamas/223a3aaa-3d69-4323-bdd9-a84e00f61631.json

# Forcing Filtered Behaviour on cts:uris()

When accuracy matters more than speed, pass "filtered" as the fifth argument. MarkLogic will open and verify each candidate before including its URI in the results:

xquery version "1.0-ml";

(: FILTERED cts:uris() — more accurate, but each candidate document      :)
(: must be opened and checked, so it is slower than the default.         :)
cts:uris((), (), cts:and-query((
  cts:collection-query("llamaverse"),
  cts:word-query("Bradley")
)), (), "filtered")

/cleverllamas/llamaverse/raw/wild-llamas/llamas/223a3aaa-3d69-4323-bdd9-a84e00f61631.json

The difference becomes significant in large databases. Unfiltered results may include URIs for documents that matched at index time but no longer match — this commonly occurs with stemmed word forms or queries near fragment boundaries.

# Path Expressions

# Filtered and Unfiltered Produce Different Results Below the Fragment Root

When a path expression such as fn:doc(...)/parent/child is used as the root context in cts:search(), you are searching below the fragment root. In this situation, "filtered" and "unfiltered" do not behave the same way — and the difference is not a minor performance trade-off. Unfiltered actively returns the wrong answer.

Consider Bradley's health record — an XML document with two <checkup> nodes. Checkup 1 has vet=Dr. Aguilar, outcome=healthy. Checkup 2 has vet=Dr. Aguilar, outcome=treatment-required. We query for the checkup where outcome="treatment-required" — that is checkup 2. With "unfiltered", MarkLogic returns checkup 1: a false positive. It returns the first node in the fragment without verifying whether it genuinely satisfies the query. With "filtered", MarkLogic opens the document, validates each node, and correctly returns checkup 2.

<healthRecord llamaId="223a3aaa-3d69-4323-bdd9-a84e00f61631">
  <checkup sequence="1">
    <vet>Dr. Aguilar</vet>
    <outcome>healthy</outcome>
    <notes>Routine annual check. Weight within normal range. Fleece in excellent condition.</notes>
  </checkup>
  <checkup sequence="2">
    <vet>Dr. Aguilar</vet>
    <outcome>treatment-required</outcome>
    <notes>Follow-up visit. Mild respiratory symptoms noted. Course of antibiotics prescribed.</notes>
  </checkup>
</healthRecord>

xquery version "1.0-ml";

(: UNFILTERED search below the fragment root.                             :)
(: Bradley's health record contains two <checkup> nodes:                  :)
(:   checkup 1: vet=Dr. Aguilar, outcome=healthy                          :)
(:   checkup 2: vet=Dr. Aguilar, outcome=treatment-required               :)
(:                                                                        :)
(: We query for the checkup where outcome="treatment-required" — that     :)
(: is checkup 2. With "unfiltered", MarkLogic returns the FIRST node in   :)
(: the fragment regardless of whether it genuinely satisfies the query.   :)
let $uri   := "/cleverllamas/llamaverse/content/health-records/223a3aaa-3d69-4323-bdd9-a84e00f61631.xml"
let $query := cts:and-query((
  cts:element-value-query(xs:QName("vet"),     "Dr. Aguilar"),
  cts:element-value-query(xs:QName("outcome"), "treatment-required")
))
return
  cts:search(
    fn:doc($uri)/healthRecord/checkup,
    $query,
    "unfiltered"
  )

<checkup sequence="1">
  <vet>Dr. Aguilar</vet>
  <outcome>healthy</outcome>
  <notes>Routine annual check. Weight within normal range. Fleece in excellent condition.</notes>
</checkup>

(: FALSE POSITIVE — checkup 1 is returned, but outcome="healthy",        :)
(: not "treatment-required". MarkLogic returned the first node in the    :)
(: fragment without checking whether it genuinely satisfies the query.   :)

xquery version "1.0-ml";

(: The same query, using "filtered" (the default).                        :)
(: MarkLogic opens the document and validates each <checkup> node against :)
(: the query. Only the node that genuinely satisfies both conditions      :)
(: is returned.                                                           :)
let $uri   := "/cleverllamas/llamaverse/content/health-records/223a3aaa-3d69-4323-bdd9-a84e00f61631.xml"
let $query := cts:and-query((
  cts:element-value-query(xs:QName("vet"),     "Dr. Aguilar"),
  cts:element-value-query(xs:QName("outcome"), "treatment-required")
))
return
  cts:search(
    fn:doc($uri)/healthRecord/checkup,
    $query,
    "filtered"
  )

<checkup sequence="2">
  <vet>Dr. Aguilar</vet>
  <outcome>treatment-required</outcome>
  <notes>Follow-up visit. Mild respiratory symptoms noted. Course of antibiotics prescribed.</notes>
</checkup>

(: CORRECT — checkup 2 is the only node that genuinely satisfies         :)
(: both conditions.                                                       :)

# Only the First Match Per Document is Returned

There is a related behaviour specific to unfiltered path expression searches: MarkLogic returns only the first matching node per fragment, even when multiple nodes in the same document match the query.

To illustrate this, consider the llamaverse movement history document. It contains 3,360 GPS entries across all llama residents — Aaron alone has 168 entries, each identified by his llamaId. Searching for Aaron's ID using an unfiltered path expression into the array returns only the first matching entry:

{
  "envelope": {
    "headers": {
      "type": "llama-movement"
    },
    "instance": {
      "llama-movement": [
        {
          "llamaId": "0c8bdb0d-ac62-49b7-ac74-94dbba46efa5",
          "timestamp": "2025-05-18T12:00:00Z",
          "coordinates": [ -9.9219172, 53.5136376 ]
        },
        {
          "llamaId": "0c8bdb0d-ac62-49b7-ac74-94dbba46efa5",
          "timestamp": "2025-05-18T13:00:00Z",
          "coordinates": [ -9.9229262, 53.5143298 ]
        },
        {
          "llamaId": "0c8bdb0d-ac62-49b7-ac74-94dbba46efa5",
          "timestamp": "2025-05-18T14:00:00Z",
          "coordinates": [ -9.9221079, 53.5136388 ]
        }
      ]
    }
  }
}

xquery version "1.0-ml";

(: UNFILTERED cts:search() with a path expression root.                   :)
(: Even though Aaron has 168 movement entries in this document,           :)
(: only the FIRST matching node is returned. This is unfiltered           :)
(: behaviour below the fragment root — MarkLogic stops after the first   :)
(: matching node per fragment.                                            :)
let $uri := "/cleverllamas/llamaverse/content/llama-movement/llama_location_history.json"
let $query := cts:json-property-value-query(
  "llamaId", "0c8bdb0d-ac62-49b7-ac74-94dbba46efa5"
)
let $results :=
  cts:search(
    fn:doc($uri)/envelope/instance/array-node("llama-movement")/object-node(),
    $query,
    "unfiltered"
  )
return (
  fn:count($results),    (: Returns 1, not 168 :)
  $results               (: Returns only the first matching movement entry :)
)

1
{"llamaId": "0c8bdb0d-ac62-49b7-ac74-94dbba46efa5", "timestamp": "2025-05-18T12:00:00Z", "coordinates": [-9.9219172, 53.5136376]}

This pattern — using fn:doc() directly and iterating with cts:contains() — is the recommended approach when you need node-level granularity within a document. Using a path expression as the cts:search() root is fine for document discovery when you need it to run unfiltered, but for collecting all matching nodes within a document, or for any use case where accuracy matters, it is not the right tool.

# Configuration Settings That Affect Accuracy

Beyond the filtered and unfiltered options on individual queries, accuracy is also shaped by how the database is configured. The sections below summarise the configuration settings and query options where filtered versus unfiltered execution can materially affect result accuracy. The central pattern is consistent: filtered search validates candidate fragments; unfiltered search relies on index resolution, and the reliability of that resolution depends on the indexes that are available and how they are configured.

# Search Execution Options

# Fragment Structure

# Position Indexes

Position indexes allow MarkLogic to validate word order and proximity during index resolution. Without them, unfiltered results for near and phrase queries are more likely to contain false positives.

# Phrase Indexes and Boundary Settings

# Sensitivity Settings

# Wildcard Searches

# Tokenization

# Geospatial Searches

# Negation Queries

# Reindexing and Operational State

# Quick Reference

The tables below summarise the filtered/unfiltered default behaviour across MarkLogic's retrieval and query-constrained lexicon interfaces. Functions where filtered/unfiltered is not meaningful — such as direct URI reads via fn:doc, cts:doc, REST /v1/documents, and SPARQL endpoints — are excluded.

# Core CTS Search

# Search API

# REST and Client APIs

# REST Client API

# Java Client API

# Node.js Client API

# Server-Side JavaScript (JSearch)

# CTS Lexicons

# Value Lexicons

# Word Lexicons

# URI and Collection Lexicons

# Tuples, Co-occurrences and Ranges

# Triples

# Optic