Product

Columnar, object-backed, boring

Telemetry is append-only, time-ordered and overwhelmingly never read. Designing for that makes it cheap enough to keep for a year.

Design decisions

Object storage is the database

Segments live in object storage; compute is stateless and scales with the query, not with the data.

Vectorised scans

A full-text search over a month of logs is a scan, and scans are fast when the layout is columnar and the filters push down.

No rehydration

There is no cold tier that you have to restore before querying. Old data is slower by a factor of three, not a factor of a working day.

Cardinality is free

Labels are dictionary-encoded columns, not a key in a series index. Cardinality changes scan cost slightly and storage cost barely.

Exactly-once ingest

Idempotency keys on every batch, so an agent retry after a network blip does not double your bill.

Data stays put

EU or US region, chosen per account. Query compute runs in the same region; nothing crosses for convenience.

Sending data

bash

exporters:
  otlphttp:
    endpoint: https://ingest.eu.mvp-01.123ok.uk
    headers:
      authorization: Bearer ${LY_TOKEN}
    compression: zstd

bash

remote_write:
  - url: https://ingest.eu.mvp-01.123ok.uk/api/v1/write
    authorization:
      credentials: ${LY_TOKEN}
    queue_config:
      max_samples_per_send: 5000

bash

[sinks.lumberyard]
type = "http"
inputs = ["parse_nginx"]
uri = "https://ingest.eu.mvp-01.123ok.uk/api/v1/logs"
encoding.codec = "json"
compression = "zstd"

bash

# same syntax across signals
logs {service="checkout", level="error"}
  | json | status >= 500
  | rate(5m) by (endpoint)

traces {service="checkout", duration > 2s}
  | group by (db.statement) | p95(duration)