Apache Pinot™: Realtime distributed OLAP datastore

Features#

Blazing Fast

Pinot is designed to answer OLAP queries with low latency on immutable data and mutable data(Upsert Support)

Pluggable indexing

Pluggable indexing technologies - Sorted Index, Bitmap Index, Inverted Index, StarTree Index, Bloom Filter, Range Index, Text Search Index(Lucence/FST), Json Index, Geospatial Index

Near Real time ingestion

Near Realtime ingestion with Apache Kafka, Apache Pulsar, Kinesissupports JSON, Avro, ProtoBuf, Thrift formats

Horizontally scalable

Horizontally scalable and fault tolerant

Joins using Trino or PrestoDB

Joins are currently not supported, but this problem can be overcome by using Trino or PrestoDB for querying

SQL-like Query Interface (PQL)

SQL like language that supports selection, aggregation, filtering, group by, order by, distinct queries on data

Hybrid tables

Consist of of both offline and realtime table. Use realtime table only to cover segments for which offline data may not be available yet

Anomaly Detection

Run ML Algorithms to detect Anomalies on the data stored in Pinot. Use ThirdEye with Pinot for Anomaly Detection and Root Cause Analysis

Smart Alerts in ThirdEye

Detect the right anomalies by customizing anomaly detect flow and notification flow

Ingest and Query Options#

Ingest with Kafka, Spark, HDFS or Cloud Storages

Query using PQL(Pinot Query Language ), SQL or Trino/Presto(supports Joins)

User-Facing Analytics#

Building Latency Sensitive User Facing Analytics via Apache Pinot

Using Apache Kafka and Apache Pinot for User-Facing Analytics

Installs Everywhere#

Pinot can be installed using docker with Trino/Presto

Helm or K8s crds

On-Premise

Public Cloud

Locally

Install:

Using Helm
Using Binary
Build From Source

helm repo add pinot https://raw.githubusercontent.com/apache/pinot/master/kubernetes/helmkubectl create ns pinothelm install pinot pinot/pinot -n pinot --set cluster.name=pinot

VERSION=0.10.0wget https://downloads.apache.org/pinot/apache-pinot-$VERSION/apache-pinot-$VERSION-bin.tar.gztar vxf apache-pinot-*-bin.tar.gzcd apache-pinot-*-binbin/quick-start-batch.sh

# Clone a repogit clone https://github.com/apache/pinot.gitcd pinot
# Build Pinotmvn clean install -DskipTests -Pbin-dist
# Run the Quick Democd pinot-distribution/target/apache-pinot-*-SNAPSHOT-bin/apache-pinot-*-SNAPSHOT-binbin/quick-start-batch.sh

Or choose your preferred method: