Apache Pinot™
Realtime distributed OLAP datastore, designed to answer OLAP queries with low latency
Pinot is proven at scale in LinkedIn powers 50+ user-facing apps and serving 100k+ queries
What is Apache Pinot?
#
FeaturesBlazing Fast
Pinot is designed to answer OLAP queries with low latency on immutable data and mutable data(Upsert Support)
Pluggable indexing
Pluggable indexing technologies - Sorted Index, Bitmap Index, Inverted Index, StarTree Index, Bloom Filter, Range Index, Text Search Index(Lucence/FST), Json Index, Geospatial Index
Near Real time ingestion
Near Realtime ingestion with Apache Kafka, Apache Pulsar, Kinesissupports JSON, Avro, ProtoBuf, Thrift formats
Horizontally scalable
Horizontally scalable and fault tolerant
JOINS in Pinot
Apache Pinot supports dimension lookup joins currently. Full SQL join support in Apache Pinot is coming soon. In the interim, full SQL joins can be performed by querying Pinot via the Trino or PrestoDB connector.
SQL Query Interface
Apache Pinot provides SQL interface for querying interactively, as well as programmatically via rest-api.
Hybrid tables
Consist of of both offline and realtime table. Use realtime table only to cover segments for which offline data may not be available yet
Anomaly Detection
Run ML Algorithms to detect Anomalies on the data stored in Pinot. Use ThirdEye with Pinot for Anomaly Detection and Root Cause Analysis
Smart Alerts in ThirdEye
Detect the right anomalies by customizing anomaly detect flow and notification flow
#
Ingest and Query Options#
User-Facing AnalyticsBuilding Latency Sensitive User Facing Analytics via Apache Pinot
Using Apache Kafka and Apache Pinot for User-Facing Analytics
#
Installs EverywhereInstall:
- Using Helm
- Using Binary
- Build From Source
helm repo add pinot https://raw.githubusercontent.com/apache/pinot/master/kubernetes/helmkubectl create ns pinothelm install pinot pinot/pinot -n pinot --set cluster.name=pinot
VERSION=0.12.1wget https://downloads.apache.org/pinot/apache-pinot-$VERSION/apache-pinot-$VERSION-bin.tar.gztar vxf apache-pinot-*-bin.tar.gzcd apache-pinot-*-binbin/quick-start-batch.sh
# Clone a repogit clone https://github.com/apache/pinot.gitcd pinot
# Build Pinotmvn clean install -DskipTests -Pbin-dist
# Run the Quick Democd pinot-distribution/target/apache-pinot-*-SNAPSHOT-bin/apache-pinot-*-SNAPSHOT-binbin/quick-start-batch.sh