Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Continuous SQL with Apache Streaming
Timothy Spann
Developer Advocate
streamnative.io
StreamNative Solution
Application Messaging Data Pipelines Real-time Contextual Analytics
Tiered Storage
A...
streamnative.io
Speaker Bio
DZone Zone Leader and Big Data MVB;
@PaasDev
https://github.com/tspannhw https://www.datainmot...
streamnative.io
FLaNK and FLiP Stacks
● Apache Flink
● Apache NiFi
● Apache Kafka
● Apache Flink
● Apache Pulsar
● StreamN...
streamnative.io
I Can Haz
Data?
Today’s Data. REST and Websocket JSON
{"symbol":"CLDR",
"uuid":"10640832-f139-4b82-8780-e3...
streamnative.io
End to End Streaming Demo Pipeline
Enterprise
sources
Weather
Errors
Aggregates
Alerts
Stocks
ETL
Analytic...
streamnative.io
All Data - Anytime - Anywhere - Multi-Cloud - Multi-Protocol
Multi-
inges
t
Multi-
inges
t
Multi-ingest Me...
streamnative.io
Apache Pulsar is Cloud-Native Messaging and
Event-Streaming Platform
streamnative.io
Apache Pulsar Overview
Enable Geo-Replicated Messaging
● Pub-Sub
● Geo-Replication
● Pulsar Functions
● Ho...
streamnative.io
What are the Benefits of Pulsar?
Data Durability
Scalability Geo-Replication
Multi-Tenancy
Unified Messagin...
streamnative.io
A Unified Messaging Platform
Message Queuing
Data Streaming
streamnative.io
Upcoming - Flink + Pulsar (FLiP)
https://flink.apache.org/2019/05/03/pulsar-flink.html
https://github.com/...
streamnative.io
Apache Flink
Apache Flink is a distributed stream
processing system.
It is capable of providing high throu...
streamnative.io
Connect with the Community & Stay Up-To-Date
● Join the Pulsar Slack channel - Apache-Pulsar.slack.com
● F...
streamnative.io
● https://github.com/tspannhw/StreamingSQLExamples
● https://github.com/streamnative/pulsar-flink
● https:...
streamnative.io
Pulsar Summit Europe
October 6, 2021
Pulsar Summit Asia
November 20-21, 2021
Contact us at partners@pulsar...
streamnative.io
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

Continuous SQL with Apache Streaming (FLaNK and FLiP)

Download to read offline

18 aug2021
Continuous SQL with Apache Streaming (FLaNK and FLiP)
https://emamo.com/event/worldfestival-2021/s/pro-talk-continuous-sql-with-flink-WR115a
In this talk, I will walk through how someone can set up and run continuous SQL queries against Pulsar topics utilizing Apache Flink. We will walk through creating Pulsar topics, schemas and publishing data.

We will then cover consuming Pulsar data, joining Pulsar topics and inserting new events into Pulsar topics as they arrive. This basic overview will show hands-on techniques, tips and examples of how to do this using Pulsar tools.

https://github.com/tspannhw/FLiP-IoT
https://github.com/tspannhw/SpeakerProfile/tree/main/2021/talks

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Continuous SQL with Apache Streaming (FLaNK and FLiP)

  1. 1. Continuous SQL with Apache Streaming Timothy Spann Developer Advocate
  2. 2. streamnative.io StreamNative Solution Application Messaging Data Pipelines Real-time Contextual Analytics Tiered Storage APP Layer Computing Layer Storage Layer StreamNative Platform IaaS Layer Micro Service Notification Dashboard Risk Control Auditing Payment ETL
  3. 3. streamnative.io Speaker Bio DZone Zone Leader and Big Data MVB; @PaasDev https://github.com/tspannhw https://www.datainmotion.dev/ https://github.com/tspannhw/SpeakerProfile https://dev.to/tspannhw https://sessionize.com/tspann/ https://www.slideshare.net/bunkertor Developer Advocate
  4. 4. streamnative.io FLaNK and FLiP Stacks ● Apache Flink ● Apache NiFi ● Apache Kafka ● Apache Flink ● Apache Pulsar ● StreamNative's Flink Connector for Pulsar ● Apache +++ Apache projects are the way for all streaming use cases.
  5. 5. streamnative.io I Can Haz Data? Today’s Data. REST and Websocket JSON {"symbol":"CLDR", "uuid":"10640832-f139-4b82-8780-e3ad37b3d0 ce", "ts":1618529574078, "dt":1612098900000, "datetime":"2021/01/31 08:15:00", "open":"12.24500", "close":"12.25500", "high":"12.25500", "volume":"12353", "low":"12.24500"}
  6. 6. streamnative.io End to End Streaming Demo Pipeline Enterprise sources Weather Errors Aggregates Alerts Stocks ETL Analytics Streaming SQL Clickstream Market data Machine logs Social
  7. 7. streamnative.io All Data - Anytime - Anywhere - Multi-Cloud - Multi-Protocol Multi- inges t Multi- inges t Multi-ingest Merge Priority
  8. 8. streamnative.io Apache Pulsar is Cloud-Native Messaging and Event-Streaming Platform
  9. 9. streamnative.io Apache Pulsar Overview Enable Geo-Replicated Messaging ● Pub-Sub ● Geo-Replication ● Pulsar Functions ● Horizontal Scalability ● Multi-tenancy ● Tiered Persistent Storage ● Pulsar Connectors ● REST API ● CLI ● Many clients available ● Four Different Subscription Types ● Multi-Protocol Support ○ MQTT ○ AMQP ○ JMS ○ Kafka ○ ...
  10. 10. streamnative.io What are the Benefits of Pulsar? Data Durability Scalability Geo-Replication Multi-Tenancy Unified Messaging Model
  11. 11. streamnative.io A Unified Messaging Platform Message Queuing Data Streaming
  12. 12. streamnative.io Upcoming - Flink + Pulsar (FLiP) https://flink.apache.org/2019/05/03/pulsar-flink.html https://github.com/streamnative/pulsar-flink https://streamnative.io/en/blog/release/2021-04-20-flink-sql- on-streamnative-cloud
  13. 13. streamnative.io Apache Flink Apache Flink is a distributed stream processing system. It is capable of providing high throughput, near real-time processing of streams from Pulsar. It is ideal for ambitious Stream Processing compared to Pulsar’s model of lightweight Stream Processing. End-to-end exactly-once stream processing https://streamnative.io/en/blog/release/2021-06-14-exactly-once-semantics-with-transactions-in-pulsar
  14. 14. streamnative.io Connect with the Community & Stay Up-To-Date ● Join the Pulsar Slack channel - Apache-Pulsar.slack.com ● Follow @streamnativeio and @apache_pulsar on Twitter ● Subscribe to Monthly Pulsar Newsletter for major news, events, project updates, and resources in the Pulsar community
  15. 15. streamnative.io ● https://github.com/tspannhw/StreamingSQLExamples ● https://github.com/streamnative/pulsar-flink ● https://www.linkedin.com/pulse/2021-schedule-tim-spann/ ● https://github.com/tspannhw/SpeakerProfile/blob/main/2021/talks/20210729_HailHydrate!FromStream toLake_TimSpann.pdf ● https://streamnative.io/en/blog/release/2021-04-20-flink-sql-on-streamnative-cloud ● https://docs.streamnative.io/cloud/stable/compute/flink-sql Deeper Content @PaasDev https://www.pulsardeveloper.com/ timothyspann
  16. 16. streamnative.io Pulsar Summit Europe October 6, 2021 Pulsar Summit Asia November 20-21, 2021 Contact us at partners@pulsar-summit.org to become a sponsor or partner
  17. 17. streamnative.io
  • bunkertor

    Aug. 18, 2021

18 aug2021 Continuous SQL with Apache Streaming (FLaNK and FLiP) https://emamo.com/event/worldfestival-2021/s/pro-talk-continuous-sql-with-flink-WR115a In this talk, I will walk through how someone can set up and run continuous SQL queries against Pulsar topics utilizing Apache Flink. We will walk through creating Pulsar topics, schemas and publishing data. We will then cover consuming Pulsar data, joining Pulsar topics and inserting new events into Pulsar topics as they arrive. This basic overview will show hands-on techniques, tips and examples of how to do this using Pulsar tools. https://github.com/tspannhw/FLiP-IoT https://github.com/tspannhw/SpeakerProfile/tree/main/2021/talks

Views

Total views

163

On Slideshare

0

From embeds

0

Number of embeds

64

Actions

Downloads

0

Shares

0

Comments

0

Likes

1

×