Skip to main content
releasev0.10.0 has been released! Check the release notes

Apache Pinot™

Realtime distributed OLAP datastore, designed to answer OLAP queries with low latencyPinotOverviewUSE-CASESUser-facingData ProductsBusinessIntelligenceAnomalyDetectionSOURCESEVENTSSmart IndexBlazing-FastPerformantAggregationPre-MaterializationSegment Optimizer

Pinot is proven at scale in LinkedIn powers 50+ user-facing apps and serving 100k+ queries

What is Apache Pinot?

Features#

Blazing Fast

Pinot is designed to answer OLAP queries with low latency on immutable data and mutable data(Upsert Support)

Pluggable indexing

Pluggable indexing technologies - Sorted Index, Bitmap Index, Inverted Index, StarTree Index, Bloom Filter, Range Index, Text Search Index(Lucence/FST), Json Index, Geospatial Index

Near Real time ingestion

Near Realtime ingestion with Apache Kafka, Apache Pulsar, Kinesissupports JSON, Avro, ProtoBuf, Thrift formats

Horizontally scalable

Horizontally scalable and fault tolerant

Joins using Trino or PrestoDB

Joins are currently not supported, but this problem can be overcome by using Trino or PrestoDB for querying

SQL-like Query Interface (PQL)

SQL like language that supports selection, aggregation, filtering, group by, order by, distinct queries on data

Hybrid tables

Consist of of both offline and realtime table. Use realtime table only to cover segments for which offline data may not be available yet

Anomaly Detection

Run ML Algorithms to detect Anomalies on the data stored in Pinot. Use ThirdEye with Pinot for Anomaly Detection and Root Cause Analysis

Smart Alerts in ThirdEye

Detect the right anomalies by customizing anomaly detect flow and notification flow

Ingest and Query Options#

Ingest with Kafka, Spark, HDFS or Cloud Storages
Query using PQL(Pinot Query Language ), SQL or Trino/Presto(supports Joins)
PinotOverviewQUERYSQLPQLJoins inTrino or PrestoSOURCESEVENTSSmart IndexBlazing-FastPerformantAggregationPre-MaterializationSegment Optimizer

User-Facing Analytics#

Building Latency Sensitive User Facing Analytics via Apache Pinot

Using Apache Kafka and Apache Pinot for User-Facing Analytics

Installs Everywhere#

Pinot can be installed using docker with Trino/Presto
Helm or K8s crds
On-Premise
Public Cloud
Locally

Install:

helm repo add pinot https://raw.githubusercontent.com/apache/pinot/master/kubernetes/helmkubectl create ns pinothelm install pinot pinot/pinot -n pinot --set cluster.name=pinot

Or choose your preferred method: