Element Range Queries — Scalar Type Implications

Range indexes look simple in the Admin UI: pick an element, pick a type, save the configuration, and move on. In practice, that scalar type is one of the most consequential choices you make in a MarkLogic database. It determines what gets indexed, how values are compared, which queries match, which values silently disappear from the index, and how much memory and disk the index consumes.

In other words, it is a tiny dropdown with surprisingly expensive consequences.

That is why element range query bugs often feel mysterious. The query syntax is usually fine. The hidden issue is typically that the index type does not reflect the shape of the data. If you treat numbers as strings, comparisons become lexicographic. If you treat messy user input as integers, invalid values may be skipped or may cause the entire ingest to fail depending on database configuration — and decimals are truncated. If you treat local times as dateTime values without a timezone strategy, UTC normalisation can surprise you later.

The central lesson is simple: scalar types determine comparison semantics. The database is not just storing a value, it is storing a typed interpretation of that value. Once you understand that, range query behaviour stops feeling arbitrary and starts feeling predictable.

# Working Assumptions for the Examples

The examples below use the llamaverse sample data wherever possible. The examples assume the llamaverse (v2.1+) is deployed. The llamaverse sample data is freely available from github.com/cleverllamas/llamaverse — see the llamaverse article for full setup instructions.

Raw llama documents live under /cleverllamas/llamaverse/raw/wild-llamas/llamas/{uuid}.json and are in the wild-llamas collection. The examples assume queries are executed as the cleverllamas-llama user, which keeps them aligned with the rest of the site.

Some examples also show XML fragments because element range indexes are an XML-native concept and the contrast with JSON path range indexes is instructive. The underlying ideas are the same in both models — only the index definition and constructor name differ.

The llamaverse path range indexes used in this article are:

# Supported Scalar Types at a Glance

MarkLogic supports a broad set of scalar types for range indexes. Choosing between them is not just a storage question — it is also a question of semantics, query expressiveness, and operational safety.

A practical rule: choose the narrowest type that matches the real domain of the value, not the prettiest type in your sample data. If the value is conceptually numeric, use a numeric index. If it is conceptually a date, use a date-aware index. If it is conceptually a code, title, or label, use a string index and pay attention to collation.

# Scalar Type Determines Comparison Semantics

The scalar type controls how operators such as =, <, >, <=, and >= are interpreted. The exact same stored text can produce very different results depending on whether the index sees it as a number or a string.

# The Classic String Gotcha

Suppose an element or JSON property contains values that look numeric but are indexed as strings. The comparison is then lexicographic, not numeric. That means "9" is greater than "10" because the comparison starts with the first character.

xquery version "1.0-ml";

(: Illustrates lexicographic vs numeric ordering.                             :)
(: With a string range index, comparisons are lexicographic, not numeric.     :)
(: "9" is greater than "10" because "9" sorts after "1" as text.             :)
for $value in ("2", "9", "10", "42")
where $value gt "10"
return $value

"2"
"9"
"42"

That behaviour is not a MarkLogic quirk. It is the expected consequence of asking a string index to do string work. The bug is modelling numeric data as text and then expecting numeric ordering from it. No database can guess your intent here.

# Numeric Indexes Behave the Way Humans Expect

xquery version "1.0-ml";

(: With a numeric range index, comparisons are numeric.                       :)
for $value in (2, 9, 10, 42)
where $value gt 10
return $value

42

Exactly the same principle applies to temporal values. A dateTime index compares timestamps as timestamps. A string index compares their textual representation. ISO 8601 strings often sort in a useful way, but only if the formatting is consistent and you never need true temporal normalisation.

# Index Typing Is Not Schema Typing

There is one boundary that is worth making explicit, especially for XML teams that use schema-aware content. The scalar type on a range index is not the same thing as the runtime type of a value in an XQuery expression.

If an XML document has been schema-validated, element and attribute values can carry typed values rather than behaving like untyped text. That can change the behaviour of FLWOR clauses, where conditions, order by logic, and ordinary comparison expressions. In other words, the same lexical content can behave differently in query code depending on whether it is untyped, schema-typed, or explicitly cast.

That matters because developers often see one comparison behave correctly in a FLWOR expression and then assume a range query will behave the same way automatically. It may not. A schema can influence how XQuery evaluates values at runtime, but a range query still depends on the configured scalar type of the index that backs it.

For example, schema-typed XML may cause <age>10</age> to behave as an integer in a comparison, while a string range index over the same field still compares lexically. That is not a contradiction. It is two different typing systems operating at two different layers.

The practical rule is simple: if you care about predictable range-query behaviour, align the index scalar type with the real domain of the value even when schema-aware XML already gives you helpful runtime typing in FLWOR expressions.

# What Gets Indexed, Dropped, or Truncated

Range indexes are not magic mirrors of your source documents. At index time, MarkLogic coerces values to the configured scalar type. That has three common outcomes: the value is indexed successfully; the value is dropped because it cannot be coerced; or the value is coerced with loss of precision, such as truncating a decimal into an integer.

The silent-drop behaviour is the one that catches teams most often. If a document contains the element, but the value does not coerce to the configured type, the document effectively has no index entry for that value. A later range query cannot match what was never indexed.

The core llamaverse llama data is intentionally clean, so the first step is to verify that profile directly on the existing wild-llamas collection. If you want to reproduce coercion edge cases locally, modify one llama's heightCm value in your local dataset copy and rerun the check.

const wild = cts.collectionQuery('wild-llamas');
const heightRef = cts.pathReference('/heightCm');

let ints = 0;
let floats = 0;
let strings = 0;
let missing = 0;

const uris = cts.uris('', null, wild).toArray();
for (const uri of uris) {
  const doc = fn.doc(uri).toObject();
  const height = doc.heightCm;
  if (height === null || height === undefined) {
    missing += 1;
  } else if (typeof height === 'number') {
    if (Number.isInteger(height)) {
      ints += 1;
    } else {
      floats += 1;
    }
  } else if (typeof height === 'string') {
    strings += 1;
  }
}

const sampleUri = '/cleverllamas/llamaverse/raw/wild-llamas/llamas/00048384-cb13-4557-805e-a6b4e57f7eab.json';
const sampleDoc = fn.doc(sampleUri).toObject();
const sampleIndex = cts.values(heightRef, null, cts.documentQuery(sampleUri)).toArray();

xdmp.log('wild-llamas docs=' + uris.length);
xdmp.log('heightCm type profile: int=' + ints + ', float=' + floats + ', string=' + strings + ', missing=' + missing);
xdmp.log(sampleUri + ' => sourceValue=' + sampleDoc.heightCm + ' => index=' + (sampleIndex.length ? sampleIndex[0] : 'no heightCm index entry'));

// Optional local experiment for coercion edge cases:
// Update one wild-llama document's heightCm in a local dev copy, reindex, and rerun this script.
// For example set heightCm to 'unknown' and compare sourceValue vs index entry.

wild-llamas docs=3000
heightCm type profile: int=3000, float=0, string=0, missing=0
/cleverllamas/llamaverse/raw/wild-llamas/llamas/00048384-cb13-4557-805e-a6b4e57f7eab.json => sourceValue=145 => index=145

If preserving original numeric fidelity matters, do not use int just because the current data sample looks mostly whole-numbered. Use decimal or double instead.

# Practical Guidance on Invalid Values

MarkLogic gives you explicit control over what happens when a value cannot be coerced to the configured scalar type. Each range index definition includes an invalid-values property with two possible settings:

Neither setting is universally correct. The right choice depends on your data quality guarantees and how much you can tolerate silent index gaps versus hard failures at write time.

ignore (the default): The database accepts any document regardless of whether it can index a value. This is forgiving but dangerous: messy data lands silently, the document exists in the database, and the only symptom is that it never appears in range query results. In a high-volume pipeline this can mean thousands of documents are effectively invisible to range-based queries before anyone notices. Use ignore only when you have a separate data quality gate and are prepared to tolerate and monitor gaps.

reject: The write transaction fails if any indexed field contains a value that cannot be coerced. This surfaces data quality problems immediately at the point of ingest, before bad data enters your database. The trade-off is that a single malformed field in an otherwise valid document blocks the entire save. Use reject when your schema is stable, your source data is under your control, or when invisible index gaps are more dangerous than failed writes.

The invalid-values setting lives on each individual index definition, so you can use reject on critical fields and ignore on optional ones within the same database.

When a field is user-entered or integration-sourced, assume messy data will arrive eventually. If you choose ignore, validate before insert or normalise into a clean indexed field — do not rely on the index silently absorbing bad data without visibility into what was dropped.

# String Indexes and Collation

A string range index is not just a string range index — it is a string range index under a specific collation. That collation determines ordering, equality behaviour, case handling, and language-specific comparison rules.

If you configure a string range index with one collation and query it with another, MarkLogic will throw an error because the requested index does not exist for that collation. This is why string range query examples often include the collation option explicitly.

# dateTime Indexes Normalise to UTC

dateTime range indexes deserve special attention because the index stores instants, not presentation formats. When values include timezone offsets, MarkLogic normalises them to their UTC-equivalent instant during comparison. That is usually correct and desirable — it only feels surprising when developers compare displayed local times rather than the underlying instant.

xquery version "1.0-ml";

(: dateTime range indexes normalise to UTC. These two values represent the    :)
(: same instant: 10am Eastern Standard Time = 3pm UTC.                       :)
let $stored := xs:dateTime("2024-01-15T10:00:00-05:00")
let $query  := xs:dateTime("2024-01-15T15:00:00Z")
return $stored eq $query

true

The operational implication is straightforward: store explicit timezones whenever possible. If one system sends 2024-01-15T10:00:00-05:00 and another sends 2024-01-15T15:00:00Z, a dateTime index treats those as equivalent. If a producer sends a time without timezone information, you now have an ambiguity that no index can solve later.

# Query Implications

If you are doing "all events since 9am Dublin time" style logic, convert the boundary once into a real xs:dateTime with timezone before building the range query. Do not compare textual local-time fragments and hope the sort order remains meaningful across offsets.

# Multiple Indexes on the Same Field

Sometimes a single field needs more than one semantic interpretation. A common example is a value that is mostly numeric but also needs exact string-style matching, prefix grouping, or human-readable sorting. MarkLogic lets you define multiple range indexes on the same element or property with different scalar types where that makes sense operationally.

That does not mean you should index everything twice by default. It means you should model the access patterns honestly. In many systems, the cleaner design is two fields: index one as an integer for computation and one as a string for presentation. That keeps each index honest and avoids subtle coercion surprises.

# Performance Characteristics by Scalar Type

Performance is never just about raw speed — it is about storage density, cache behaviour, comparison cost, and index selectivity. Some broad patterns hold consistently enough to use as design guidance.

If all you need is a count or threshold, an integer index is hard to beat. If you need exact decimal comparisons, accept the slightly higher cost and use decimal. If you need text ordering, pay the string cost but configure collation intentionally rather than accidentally.

# Element Range Queries vs JSON Path Range Queries

The concepts are the same in XML and JSON. The constructor names differ because the indexed nodes differ. For XML elements, use cts:element-range-query(). For JSON path expressions, use cts:path-range-query(). Both require the corresponding range index to exist.

# What Happens Without the Index

This is one of the most important corrections to common MarkLogic folklore. Range queries do not gracefully degrade into a slow filter-only fallback when the required range index is missing. For cts:element-range-query() and cts:path-range-query(), MarkLogic throws an exception because the range index is mandatory.

xquery version "1.0-ml";

(: Attempting a range query on an element with no range index configured.    :)
cts:search(
  fn:collection(),
  cts:element-range-query(xs:QName("age"), ">=", 5)
)

XDMP-ELEMRIDXNOTFOUND: cts:element-range-query(xs:QName("age"), ">=", 5) --
Element range index for age not found

That distinction matters operationally. If a deployment pipeline forgets to promote an index, the application does not merely become slower — it fails. That is good in one sense because silent slowness is hard to diagnose, but it also means index management is part of application correctness.

# If You Truly Need a Fallback

Use explicit application logic. Catch the missing-index exception and run a slower manual pass, or provide a feature flag that disables range-query features until the index is present. Do not assume MarkLogic will improvise on your behalf for range semantics.

# Nulls, Missing Values, and Empty Structures

An element or property that is absent simply does not contribute an index entry. That sounds obvious, but it has practical implications for analytics and faceting logic. If half of your documents omit a field, a range query only sees the half that actually have an indexable value.

This is also why "find documents where a field is missing" is not a range-query problem. It is a presence-query problem, often expressed as a negated element/property existence or a structured query that combines the business scope with cts:not-query().

xquery version "1.0-ml";

(: Pattern for finding documents without a particular property.              :)
(: The missing value is not in the range index, so query for absence         :)
(: explicitly using cts:not-query and a property scope query.                :)
cts:search(
  fn:collection("wild-llamas"),
  cts:not-query(cts:json-property-scope-query("heightCm", cts:true-query()))
)

# Index Configuration via the Management API

Serious teams keep index definitions in source control rather than only in click-path memory. MarkLogic supports this through the Management REST API. The payload structure for path range indexes is:

{
  "range-path-index": [
    {
      "scalar-type": "int",
      "path-expression": "/heightCm",
      "collation": "",
      "range-value-positions": false,
      "invalid-values": "reject"
    }
  ]
}

Send this as a PUT to /manage/v2/databases/{database-name}/properties. Note that a PUT replaces the entire array, so always include all existing indexes in the payload when adding a new one. Adding indexes to a populated database triggers reindexing — plan this the same way you plan schema migrations.

The equivalent for XML element range indexes uses the range-element-index key with namespace-uri and localname fields in place of path-expression.

# Llamaverse Examples

The llamaverse llama documents have heightCm (int) and name (string) path range indexes configured. These examples run against the wild-llamas collection.

# Querying by height

xquery version "1.0-ml";

(: Query llamas taller than 180cm using a path range index on /heightCm. :)
(: The index must exist before this query will run. :)
for $doc in cts:search(
  fn:collection("wild-llamas"),
  cts:path-range-query("/heightCm", ">", xs:int(180))
)[1 to 5]
let $root := $doc/node()
order by xs:int(fn:string($root/heightCm)) descending
return fn:string($root/name) || " — " || fn:string($root/heightCm) || "cm"

Jade — 200cm
Sean — 198cm
Sean — 195cm
Debra — 188cm
David — 186cm

# Querying by name

xquery version "1.0-ml";

(: Query llamas whose name sorts at or after "S" using a codepoint string collation. :)
(: Requires a path range index on /name with collation http://marklogic.com/collation/ :)
for $doc in cts:search(
  fn:collection("wild-llamas"),
  cts:path-range-query(
    "/name",
    ">=",
    "S",
    ("collation=http://marklogic.com/collation/")
  )
)[1 to 5]
let $root := $doc/node()
order by fn:string($root/name)
return fn:string($root/name)

Sean
Sean
Susan
Todd
Xavier

# Quick Reference

# Common Design Patterns

# Pattern 1: Separate storage field from search field

If upstream systems send dirty values, preserve the raw string for auditing and index a cleaned numeric or temporal field for search. That gives you reliable range semantics without losing the original source payload.

# Pattern 2: Multiple indexes for one business concept

Sometimes the business concept is singular while the search semantics are plural. A code might need both lexical prefix searches and numeric ordering. Either dual-index the same field carefully or split it into explicit search-oriented fields.

# Pattern 3: Treat index changes as application changes

A missing or mis-typed range index changes correctness, not just speed. That means index configuration belongs in deployment automation, code review, and release planning.

# Troubleshooting Checklist

# Final Takeaways

Range queries are only as correct as the scalar types beneath them. An index is not just acceleration, it is interpretation. Use numeric types for numeric questions. Use temporal types for temporal questions. Use string types when lexical semantics are genuinely what you want, and configure collation explicitly. If the data is messy, clean it before it reaches the index or index a normalised companion field.

That discipline pays off twice: your results are more trustworthy, and your indexes are typically smaller, faster, and easier to reason about. It also means fewer late-night mysteries, which every human and every clever llama can appreciate.