SlideShare uma empresa Scribd logo
1 de 55
Baixar para ler offline
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Debanjan Saha
deban@amazon.com
GM, Amazon Aurora & Glue
SRV308
Deep Dive on Amazon Aurora
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Amazon Aurora?
D atabas e r eimagined for the c loud
 Speed and availability of high-end commercial databases
 Simplicity and cost-effectiveness of open source databases
 Drop-in compatibility with MySQL and PostgreSQL
 Simple pay as you go pricing
Delivered as a managed service
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Re-imagining the relational database
1
2
3
Scale-out, distributed architecture
Service-oriented architecture leveraging AWS services
Automate administrative tasks – fully managed service
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale-out, distributed architecture
 Purpose-built log-structured distributed
storage system designed for databases
 Storage volume is striped across
hundreds of storage nodes distributed
over 3 different availability zones
 Six copies of data, two copies in each
availability zone to protect against
AZ+1 failures
 Plan to apply same principles to other
layers of the stack
Master Replica
Availability
Zone 1
Shared storage volume
Availability
Zone 2
Availability
Zone 3
Storage nodes with SSDs
SQL
Transactions
Caching
SQL
Transactions
Caching
SQL
Transactions
Caching
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Leveraging AWS services
Lambda
S3
IAM
CloudWatch
Invoke Lambda events from stored procedures/triggers
Load data from S3, store snapshots and backups in S3
Use IAM roles to manage database access control
Upload systems metrics and audit logs to CloudWatch
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automate administrative tasks
Schema design
Query construction
Query optimization
Automatic fail-over
Backup & recovery
Isolation & security
Industry compliance
Push-button scaling
Automated patching
Advanced monitoring
Routine maintenance
Takes care of your time-consuming database management tasks,
freeing you to focus on your applications and business
You
AWS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora is used by
¾ of the top 100
AWS customers
Fastest growing
service in AWS history
Aurora customer adoption
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who is moving to Aurora and why?
Customers using
open source engines
 Higher performance – up to 5x
 Better availability and durability
 Reduces cost – up to 60%
 Easy migration; no application change
 One tenth of the cost; no licenses
 Integration with cloud ecosystem
 Comparable performance and availability
 Migration tooling and services
Customers using
Commercial engines
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AURORA PERFORMANCE
5x faster than MySQL; 3x faster than PostgreSQL
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora MySQL performance
WRITE PERFORMANCE READ PERFORMANCE
MySQL SysBench results; R4.16XL: 64cores / 488 GB RAM
Aurora read write throughput compared to MySQL 5.6
based on industry standard benchmarks.
Aurora MySQL 5.6
0
100000
200000
300000
400000
500000
600000
700000
0
50000
100000
150000
200000
250000
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora PostgreSQL performance
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
10 15 20 25 30 35 40 45 50 55 60
Throughput,tps
Minutes
pgbench throughput over time, 150 GiB, 1024 clients
PostgreSQL (Single AZ) Amazon Aurora (Three AZs)
W h i l e r u n n i n g p g b e n c h a t l o a d , t h r o u g h p u t i s
3 x m o r e c o n s i s t e n t t h a n P o s t g r e S Q L
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How did we achieve this?
Do fewer IOs
Minimize network packets
Cache prior results
Offload the database engine
DO LESS WORK
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
BE MORE EFFICIENT
DATABASES ARE ALL ABOUT I/O
NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND
HIGH-THROUGHPUT PROCESSING IS ALL ABOUT CONTEXT SWITCHES
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora I/O profile
BINLOG DATA DOUBLE-WRITELOG FRM FILES
TYPE OF WRITE
MYSQL WITH REPLICA
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBS
Amazon Elastic
Block Store (EBS)
Primary
Instance
Replica
Instance
1
2
3
4
5
AZ 1 AZ 3
Primary
Instance
Amazon S3
AZ 2
Replica
Instance
ASYNC
4/6 QUORUM
DISTRIBUT
ED WRITES
Replica
Instance
AMAZON AURORA
780K transactions
7,388K I/Os per million txns (excludes mirroring, standby)
Average 7.4 I/Os per transaction
MySQL I/O profile for 30 min Sysbench run
27,378K transactions 35X MORE
0.95 I/Os per transaction (6X amplification) 7.7X LESS
Aurora IO profile for 30 min Sysbench run
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
IO traffic in Aurora (storage node)
LOG RECORDS
Primary
Instance
INCOMING QUEUE
STORAGE NODE
S3 BACKUP
1
2
3
4
5
6
7
8
UPDATE
QUEUE
ACK
HOT
LOG
DATA
BLOCKS
POINT IN TIME
SNAPSHOT
GC
SCRUB
COALESCE
SORT
GROUP
PEER TO PEER GOSSIPPeer
Storage
Nodes
All steps are asynchronous
Only steps 1 and 2 are in foreground latency path
Input queue is 46X less than MySQL (unamplified, per node)
Favor latency-sensitive operations
Use disk space to buffer against spikes in activity
OBSERVATIONS
IO FLOW
① Receive record and add to in-memory queue
② Persist record and ACK
③ Organize records and identify gaps in log
④ Gossip with peers to fill in holes
⑤ Coalesce log records into new data block versions
⑥ Periodically stage log and new block versions to S3
⑦ Periodically garbage collect old versions
⑧ Periodically validate CRC codes on blocks
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora lock management
Scan
Delete
Scan
Delete
Insert
Scan Scan
Insert
Delete
Scan
Insert
Insert
MySQL lock manager Aurora lock manager
 Same locking semantics as MySQL
 Concurrent access to lock chains
Needed to support many concurrent sessions, high update throughput
 Multiple scanners in individual lock chains
 Lock-free deadlock detection
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Asynchronous key prefetch
Latency improvement factor vs. Batched Key Access (BKA) join
algorithm. Decision support benchmark, R3.8xlarge
14.57
-
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
Query-1
Query-2
Query-3
Query-4
Query-5
Query-6
Query-7
Query-8
Query-9
Query-10
Query-11
Query-12
Query-13
Query-14
Query-15
Query-16
Query-17
Query-18
Query-19
Query-20
Query-21
Query-22
Cold buffer
Works for queries using Batched Key
Access (BKA) join algorithm + Multi-
Range Read (MRR) Optimization
Performs a secondary to primary
index lookup during JOIN evaluation
Uses background thread to
asynchronously load pages into
memory
AKP used in queries 2, 5, 8, 9, 11, 13, 17, 18, 19, 20, 21
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Batched scans
Standard MySQL pprocesses rows one-at-
a-time resulting in high overhead due to:
 Repeated function calls
 Locking and latching
 cursor store and restore
 InnoDB to MySQL format conversion
Amazon Aurora scan tuples from the
InnoDB buffer pool in batches for
 Table full scans
 Index full scans
 Index range scans Latency improvement factor vs. Batched Key Access (BKA)
join algorithm Decision support benchmark, R3.8xlarge
1.78X
-
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
Query-1
Query-2
Query-3
Query-4
Query-5
Query-6
Query-7
Query-8
Query-9
Query-10
Query-11
Query-12
Query-13
Query-14
Query-15
Query-16
Query-17
Query-18
Query-19
Query-20
Query-21
Query-22
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MONITORING DATABASE PERFORMANCE
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Performance Insights
Dashboard showing load on DB
• Easy
• Powerful
Identifies source of bottlenecks
• Top SQL
Adjustable time frame
• Hour, day, week , month
• Up to 35 days of data
Max CPU
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Performance Insights
What if you could find out
what your databases were
doing, or had been doing?
Performance Insights is the
answer:
 Lets you see your overall
instance load
 Drill down by SQL
statement, by time, or by
calling host
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WHAT ABOUT AVAILABILITY?
“Performance only matters if your database is up”
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
6-way replicated storage
Sur vives c atas tr ophic failur es
Six copies across three availability zones
4 out 6 write quorum; 3 out of 6 read quorum
Peer-to-peer replication for repairs
Volume striped across hundreds of storage nodes
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
Read and write availabilityRead availability
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Up to 15 promotable read replicas
Master
Read
Replica
Read
Replica
Read
Replica
Shared distributed storage volume
Reader end-point
► Up to 15 promotable read replicas across multiple availability zones
► Re-do log based replication leads to low replica lag – typically < 10ms
► Reader end-point with load balancing and auto-scaling * NEW *
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instant crash recovery
TRADITIONAL DATABASE
Have to replay logs since the last
checkpoint
Typically 5 minutes between checkpoints
Single-threaded in MySQL; requires a
large number of disk accesses
AMAZON AURORA
Underlying storage replays redo records
on demand as part of a disk read
Parallel, distributed, asynchronous
No replay for startup
Checkpointed Data Redo Log
Crash at T0 requires
a re-application of the
SQL in the redo log since
last checkpoint
T0 T0
Crash at T0 will result in redo logs being
applied to each segment on demand, in
parallel, asynchronously
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Database fail-over time
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
0 - 5s – 30% of fail-overs
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
5 - 10s – 40% of fail-overs
0%
10%
20%
30%
40%
50%
60%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
10 - 20s – 25% of fail-overs
0%
5%
10%
15%
20%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
20 - 30s – 5% of fail-overs
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RECENT INNOVATIONS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Availability is about more than HW failures
Aur or a s olutions for availability dis r uptions
1. Patching your database software
 Zero Downtime Patching
2. Large-scale copying and reorganizations
 Fast Cloning
3. DBA errors requiring database restores
 Backtrack
4. Disasters
 Global replication
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Zero downtime patching
Networking
state
Application
state
Storage Service
App
state
Net
state
App
state
Net
state
BeforeZDP
New DB
Engine
Old DB
Engine
New DB
Engine
Old DB
Engine
WithZDP
User sessions terminate
during patching
User sessions remain
active through patching
Storage Service
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fast database cloning
Clone database without copying data
 Creation of a clone is nearly instantaneous
 Data copy happens only on write – when
original and cloned volume data differ
Example use cases
 Clone a production DB to run tests
 Reorganize a database
 Save a point in time snapshot for analysis
without impacting production system.
PRODUCTION DATABASE
CLONE CLONE
CLONE
DEV/TEST
APPLICATIONS
BENCHMARKS
PRODUCTION
APPLICATIONS
PRODUCTION
APPLICATIONS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Database backtrack
Backtrack brings the database to a point in time without requiring restore from backups
• Backtracking from an unintentional DML or DDL operation
• Backtrack is not destructive. You can backtrack multiple times to find the right point in time
t0 t1 t2
t0 t1
t2
t3 t4
t3
t4
Rewind to t1
Rewind to t3
Invisible Invisible
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How does backtrack work?
We keep periodic snapshot of each segment; we also preserve the redo logs
For backtrack, we identify the appropriate segment snapshots
Apply log streams to segment snapshots in parallel and asynchronously
SEGMENT
SNAPSHOT
LOG
RECORDS
RECOVERY
POINT
SEGMENT 1
SEGMENT 2
SEGMENT 3
TIME
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Online DDL: MySQL vs. Aurora
 Full Table copy in the background
 Rebuilds all indexes – can take hours or days
 DDL operation impacts DML throughput
 Table lock applied to apply DML changes
INDEX
LEAFLEAFLEAF LEAF
INDEX
ROOT
table name operation column-name time-stamp
Table 1
Table 2
Table 3
add-col
add-col
add-col
column-abc
column-qpr
column-xyz
t1
t2
t3
 Use schema versioning to decode the block.
 Modify-on-write primitive to upgrade to latest schema
 Currently support add NULLable column at end of table
 Add column anywhere and with default coming soon.
MySQL Aurora
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Online DDL performance
On r3.large
On r3.8xlarge
Aurora MySQL 5.6 MySQL 5.7
10GB table 0.27 sec 3,960 sec 1,600 sec
50GB table 0.25 sec 23,400 sec 5,040 sec
100GB table 0.26 sec 53,460 sec 9,720 sec
Aurora MySQL 5.6 MySQL 5.7
10GB table 0.06 sec 900 sec 1,080 sec
50GB table 0.08 sec 4,680 sec 5,040 sec
100GB table 0.15 sec 14,400 sec 9,720 sec
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Global replication - logical
Fas ter dis as ter r ec over y and enhanc ed data loc ality
Promote read-replica to a master
for faster recovery in the event of
disaster
Bring data close to your
customer’s applications in
different regions
Promote to a master for easy
migration
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Global replication – physical
AZ 1 AZ 3
Primary
Instance
AZ 2
Aurora
Replica
Aurora
Replica
ReplicationServer
ReplicationAgentREDO LOG FRM FILES
TYPE OF WRITE
Aurora
Replica
(optional)
Region 1: Primary Aurora Cluster Region 2: Read Replica
AZ 1
Consistently fast, low-lag, high-performance replication for global relational databases
• Global-scale replication in seconds or less
• Dedicated replication infrastructure ensures unconstrained performance
• Local reads, faster recovery, tighter DR objectives, and seamless cross-region migration
Async.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AURORA SERVERLESS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Serverless use cases
• Infrequently used applications (e.g. a
low-volume blog site)
• Applications with intermittent or
unpredictable load (e.g. a news site)
• Development or test databases not
needed on nights or weekends
• Consolidated fleets of multi-tenant
SaaS applications
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Serverless
• Starts up on demand,
shuts down when not in
use
• Scales up & down
automatically
• No application impact
when scaling
• Pay per second, 1 minute
minimum
WARM POOL
OF INSTANCES
APPLICATION
DATABASE STORAGE
SCALABLE DB CAPACITY
REQUEST ROUTER
DATABASE END-POINT
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instance provisioning and scaling
• First request triggers instance
provisioning. Usually 1-3 seconds
• Instance auto-scales up and down as
workload changes. Usually 1-3
seconds
• Instances hibernate after user-defined
period of inactivity
• Scaling operations are transparent to
the application – user sessions are
not terminated
• Database storage is persisted until
explicitly deleted by user
DATABASE STORAGE
WARM POOL
APPLICATION
REQUEST
ROUTER
CURRENT
INSTANCE
NEW
INSTANCE
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling to and from zero
• If desired, instances are removed after
user-defined period of inactivity
• While database is stopped, you only pay
for storage
• A database request triggers instance
provisioning (usually under 5 seconds)
• Scaling to zero is more expensive as
there is no running server from which to
copy the buffer cache
WARM POOL
INSTANCE
DATABASE STORAGE
REQUEST
ROUTER
APPLICATION
PROXY ENDPOINT
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AURORA MULTI-MASTER
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed Lock Manager
GLOBAL
RESOURCE
MANAGER
SQL
TRANSACTIONS
CACHING
LOGGING
SQL
TRANSACTIONS
CACHING
LOGGING
SHARED DISK CLUSTER
STORAGE
APPLICATION
LOCKING PROTOCOL MESSAGES
SHARED STORAGE
M1 M2 M3
M1 M1 M1M2 M3 M2
Cons
Heavyweight cache coherency traffic, on per-lock basis
Networking can be expensive
Negative scaling when hot blocks
Pros
All data available to all nodes
Easy to build applications
Similar cache coherency as in multi-processors
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consensus with two phase or Paxos commit
DATA
RANGE #1
DATA
RANGE #2
DATA
RANGE #4
DATA
RANGE #3
DATA
RANGE #5
L
L L
L
L
SHARED NOTHING
SQL
TRANSACTIONS
CACHING
LOGGING
SQL
TRANSACTIONS
CACHING
LOGGING
APPLICATION
STORAGE STORAGE
Cons
Heavyweight commit and membership change protocols
Range partitioning can result in hot partitions, not just hot
blocks. Re-partitioning expensive.
Cross partition operations expensive. Better at small
requests
Pros
Query broken up and sent to data node
Less coherence traffic – just for commits
Can scale to many nodes
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Conflict resolution using distributed ledgers
• There are many “oases” of
consistency in Aurora
• The database nodes know
transaction orders from that
node
• The storage nodes know
transactions orders applied
at that node
• Only have conflicts when
data changed at both
multiple database nodes
AND multiple storage nodes
• Much less coordination
required
2 3 4 5 61
BT1 [P1]
BT2 [P1]
BT3 [P1]
BT4 [P1]
BT1
BT2
BT3
BT4
Page1
Quorum
OT1
OT2
OT3
OT4
Page 1 Page 2
2 3 4 5 61
OT1[P2]
OT2[P2]
OT3[P2]
OT4[P2]
PAGE1 PAGE2
MASTER
MASTER
Page 2
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hierarchical conflict resolution
• Both masters are writing to
two pages P1 and P2
• BLUE master wins the
quorum at page P1;
ORANGE master wins
quorum at P2
• Both masters recognize the
conflict and have two
choices: (1) roll back the
transactions or (2) escalate
to regional resolver
• Regional arbitrator decides
who wins the tie breaker .
2 3 4 5 61
BT1 [P1]
OT1 [P1]
2 3 4 5 61
OT1[P2]
BT1[P2]
PAGE1 PAGE2
MASTER
MASTER
BT1 OT1
Regional
resolver
Page 1 Page 2 Page 1 Page 2
Quorum
X X
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Multi-region Multi-Master
Write accepted locally
Optimistic concurrency control – no distributed lock
manager, no chatty lock management protocol
REGION 1 REGION 2
HEAD NODES HEAD NODES
MULTI-AZ STORAGE VOLUME MULTI-AZ STORAGE VOLUME
LOCAL PARTITION LOCAL PARTITIONREMOTE PARTITION REMOTE PARTITION
Conflicts handled hierarchically – at head nodes,
at storage nodes, at AZ and region level arbitrators
Near-linear performance scaling when there is no
or low levels of conflicts
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AURORA PARALLEL QUERY
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Parallel query processing
Aurora storage has thousands of CPUs
 Presents opportunity to push down and parallelize
query processing using the storage fleet
 Moving processing close to data reduces network traffic
and latency
However, there are significant challenges
 Data stored in storage node is not range partitioned –
require full scans
 Data may be in-flight
 Read views may not allow viewing most recent data
 Not all functions can be pushed down to storage nodes
DATABASE NODE
STORAGE NODES
PUSH DOWN
PREDICATES
AGGREGATE
RESULTS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Processing at head node
STORAGE NODES
OPTIMIZER
EXECUTOR
INNODB
NETWORK STORAGE DRIVER
AGGREGATOR
APPLICATION
PARTIAL
RESULTS
STREAM
RESULTS
IN-FLIGHT
DATA
PQ CONTEXT
PQ PLAN
Query Optimizer produces PQ Plan and
creates PQ context based on leaf page
discovery
PQ request is sent to storage node along
with PQ context
Storage node produces
 Partial results streams with processed
stable rows
 Raw stream of unprocessed rows with
pending undos
Head node aggregates these data
streams to produce final results
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Processing at storage node
Each storage node runs up to 16 PQ
processes, each associated with a
parallel query
PQ process receives PQ context
 List of pages to scan
 Read view and projections
 Expression evaluation byte code
PQ process makes two passes through
the page list
 Pass 1: Filter evaluation on
InnoDB formatted raw data
 Pass 2: Expression evaluation
on MySQL formatted data
PQ PROCESS
PQ PROCESS
Up to 16
TO/FROM HEAD NODE
STORAGE
NODE PROCESS
PAGE LISTS
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Example: parallel hash join
Head node sends hash join context to
storage node consisting of:
 List of pages – Page ID and LSNs
 Join expression byte code
 Read view – what rows to include
 Projections – what columns to select
PQ process performs the following steps:
 Scans through the lists of pages
 Builds partial bloom filters; collect stats
 Sends two streams to head node
• Rows with matching hashes
• Rows with pending undo processing
Head node performs the Join function after
merging data streams from different storage
nodes
JOIN CONTEXT
PAGE SCAN
STABLE ROWS
ROWS WITH
PENDING UNDO
BLOOM
FILTER
FILTERED AND PROJECTED
ROW STREAM
UNPROCESSED
ROW STREAM
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Parallel query performance
1.71 1.31
100.16
142.4
41.39
11.58
117.06
1.77
290.66
1.15
5.61
48.87
29.95
4.29
2.76
83.47
14.35
2.27
1.27
1
2
4
8
16
32
64
128
256
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Enjoy trying new features?
AURORA SERVERLESS AURORA MULTI-MASTER
AURORA PARALLEL QUERY
PERFORMAN
CE INSIGHTS
Scales DB capacity
to match app needs
Scales out
writes across
multiple AZs
Improves
performance of
large queries
PERFORMANCE INSIGHTS
Assess DB
load & tune
performance
Sign up for previews : aws .amaz on.c om/rds/aurora/
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session
survey in the summit mobile app.

Mais conteúdo relacionado

Mais procurados

Aws glue를 통한 손쉬운 데이터 전처리 작업하기
Aws glue를 통한 손쉬운 데이터 전처리 작업하기Aws glue를 통한 손쉬운 데이터 전처리 작업하기
Aws glue를 통한 손쉬운 데이터 전처리 작업하기Amazon Web Services Korea
 
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...Amazon Web Services
 
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...Amazon Web Services Japan
 
Introduction to AWS Storage Services
Introduction to AWS Storage ServicesIntroduction to AWS Storage Services
Introduction to AWS Storage ServicesAmazon Web Services
 
Amazon Aurora Deep Dive (db tech showcase 2016)
Amazon Aurora Deep Dive (db tech showcase 2016)Amazon Aurora Deep Dive (db tech showcase 2016)
Amazon Aurora Deep Dive (db tech showcase 2016)Amazon Web Services Japan
 
Introduction to Amazon Relational Database Service (Amazon RDS)
Introduction to Amazon Relational Database Service (Amazon RDS)Introduction to Amazon Relational Database Service (Amazon RDS)
Introduction to Amazon Relational Database Service (Amazon RDS)Amazon Web Services
 
Amazon S3 & Amazon Glacier - Object Storage Overview
Amazon S3 & Amazon Glacier - Object Storage OverviewAmazon S3 & Amazon Glacier - Object Storage Overview
Amazon S3 & Amazon Glacier - Object Storage OverviewAmazon Web Services
 
Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016
Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016
Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016Amazon Web Services
 
ABCs of AWS: S3
ABCs of AWS: S3ABCs of AWS: S3
ABCs of AWS: S3Mark Cohen
 
AWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-RayAWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-RayAmazon Web Services Japan
 
Introduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless ApplicationsIntroduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless ApplicationsAmazon Web Services
 
AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법
AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법
AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법Amazon Web Services Korea
 

Mais procurados (20)

Aws glue를 통한 손쉬운 데이터 전처리 작업하기
Aws glue를 통한 손쉬운 데이터 전처리 작업하기Aws glue를 통한 손쉬운 데이터 전처리 작업하기
Aws glue를 통한 손쉬운 데이터 전처리 작업하기
 
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
Aurora Serverless: Scalable, Cost-Effective Application Deployment (DAT336) -...
 
Amazon S3 Masterclass
Amazon S3 MasterclassAmazon S3 Masterclass
Amazon S3 Masterclass
 
ElastiCache & Redis
ElastiCache & RedisElastiCache & Redis
ElastiCache & Redis
 
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...
 
Introduction to AWS Storage Services
Introduction to AWS Storage ServicesIntroduction to AWS Storage Services
Introduction to AWS Storage Services
 
Amazon Aurora Deep Dive (db tech showcase 2016)
Amazon Aurora Deep Dive (db tech showcase 2016)Amazon Aurora Deep Dive (db tech showcase 2016)
Amazon Aurora Deep Dive (db tech showcase 2016)
 
AWS for Backup and Recovery
AWS for Backup and RecoveryAWS for Backup and Recovery
AWS for Backup and Recovery
 
Deep Dive on Amazon S3
Deep Dive on Amazon S3Deep Dive on Amazon S3
Deep Dive on Amazon S3
 
Amazon Aurora: Under the Hood
Amazon Aurora: Under the HoodAmazon Aurora: Under the Hood
Amazon Aurora: Under the Hood
 
Introduction to Amazon Relational Database Service (Amazon RDS)
Introduction to Amazon Relational Database Service (Amazon RDS)Introduction to Amazon Relational Database Service (Amazon RDS)
Introduction to Amazon Relational Database Service (Amazon RDS)
 
Amazon S3 & Amazon Glacier - Object Storage Overview
Amazon S3 & Amazon Glacier - Object Storage OverviewAmazon S3 & Amazon Glacier - Object Storage Overview
Amazon S3 & Amazon Glacier - Object Storage Overview
 
Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016
Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016
Amazon RDS with Amazon Aurora | AWS Public Sector Summit 2016
 
ABCs of AWS: S3
ABCs of AWS: S3ABCs of AWS: S3
ABCs of AWS: S3
 
AWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-RayAWS Black Belt Online Seminar 2017 AWS X-Ray
AWS Black Belt Online Seminar 2017 AWS X-Ray
 
Deep Dive on Amazon Aurora
Deep Dive on Amazon AuroraDeep Dive on Amazon Aurora
Deep Dive on Amazon Aurora
 
AWS Black Belt Online Seminar Amazon Aurora
AWS Black Belt Online Seminar Amazon AuroraAWS Black Belt Online Seminar Amazon Aurora
AWS Black Belt Online Seminar Amazon Aurora
 
Introduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless ApplicationsIntroduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless Applications
 
BDA311 Introduction to AWS Glue
BDA311 Introduction to AWS GlueBDA311 Introduction to AWS Glue
BDA311 Introduction to AWS Glue
 
AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법
AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법
AWS Backup을 이용한 데이터베이스의 백업 자동화와 편리한 복구방법
 

Semelhante a SRV308 Deep Dive on Amazon Aurora

Amazon Aurora: Deep Dive - SRV308 - Chicago AWS Summit
Amazon Aurora: Deep Dive - SRV308 - Chicago AWS SummitAmazon Aurora: Deep Dive - SRV308 - Chicago AWS Summit
Amazon Aurora: Deep Dive - SRV308 - Chicago AWS SummitAmazon Web Services
 
Amazon Aurora - Rajeev Chakrabarti
Amazon Aurora - Rajeev ChakrabartiAmazon Aurora - Rajeev Chakrabarti
Amazon Aurora - Rajeev ChakrabartiAmazon Web Services
 
Amazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration ServiceAmazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration ServiceAmazon Web Services
 
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018Amazon Web Services
 
[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...
[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...
[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...Amazon Web Services
 
Deep Dive - Amazon Relational Database Services_AWSPSSummit_Singapore
Deep Dive - Amazon Relational Database Services_AWSPSSummit_SingaporeDeep Dive - Amazon Relational Database Services_AWSPSSummit_Singapore
Deep Dive - Amazon Relational Database Services_AWSPSSummit_SingaporeAmazon Web Services
 
Introducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech Talks
Introducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech TalksIntroducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech Talks
Introducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech TalksAmazon Web Services
 
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...Amazon Web Services
 
What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018
What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018
What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018Amazon Web Services
 
Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018
Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018
Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018Amazon Web Services
 
Amazon Aurora (MySQL, Postgres)
Amazon Aurora (MySQL, Postgres)Amazon Aurora (MySQL, Postgres)
Amazon Aurora (MySQL, Postgres)AWS Germany
 
What's new in Amazon Aurora - ADB203 - Chicago AWS Summit
What's new in Amazon Aurora - ADB203 - Chicago AWS SummitWhat's new in Amazon Aurora - ADB203 - Chicago AWS Summit
What's new in Amazon Aurora - ADB203 - Chicago AWS SummitAmazon Web Services
 
ElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech Talks
ElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech TalksElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech Talks
ElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech TalksAmazon Web Services
 

Semelhante a SRV308 Deep Dive on Amazon Aurora (20)

Amazon Aurora_Deep Dive
Amazon Aurora_Deep DiveAmazon Aurora_Deep Dive
Amazon Aurora_Deep Dive
 
Amazon Aurora
Amazon AuroraAmazon Aurora
Amazon Aurora
 
Amazon Aurora: Deep Dive - SRV308 - Chicago AWS Summit
Amazon Aurora: Deep Dive - SRV308 - Chicago AWS SummitAmazon Aurora: Deep Dive - SRV308 - Chicago AWS Summit
Amazon Aurora: Deep Dive - SRV308 - Chicago AWS Summit
 
Amazon Aurora
Amazon AuroraAmazon Aurora
Amazon Aurora
 
Amazon Aurora: Database Week SF
Amazon Aurora: Database Week SFAmazon Aurora: Database Week SF
Amazon Aurora: Database Week SF
 
Amazon Aurora - Rajeev Chakrabarti
Amazon Aurora - Rajeev ChakrabartiAmazon Aurora - Rajeev Chakrabarti
Amazon Aurora - Rajeev Chakrabarti
 
Amazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration ServiceAmazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration Service
 
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018
 
Aurora Deep Dive | AWS Floor28
Aurora Deep Dive | AWS Floor28Aurora Deep Dive | AWS Floor28
Aurora Deep Dive | AWS Floor28
 
[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...
[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...
[REPEAT 1] Deep Dive on Amazon Aurora with MySQL Compatibility (DAT304-R1) - ...
 
Amazon Aurora
Amazon AuroraAmazon Aurora
Amazon Aurora
 
Deep Dive - Amazon Relational Database Services_AWSPSSummit_Singapore
Deep Dive - Amazon Relational Database Services_AWSPSSummit_SingaporeDeep Dive - Amazon Relational Database Services_AWSPSSummit_Singapore
Deep Dive - Amazon Relational Database Services_AWSPSSummit_Singapore
 
Introducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech Talks
Introducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech TalksIntroducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech Talks
Introducing Amazon Aurora with PostgreSQL Compatibility - AWS Online Tech Talks
 
Managed Relational Databases
Managed Relational DatabasesManaged Relational Databases
Managed Relational Databases
 
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...
 
What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018
What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018
What's New in Amazon Aurora (DAT204-R1) - AWS re:Invent 2018
 
Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018
Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018
Deep Dive on Amazon Neptune (DAT403) - AWS re:Invent 2018
 
Amazon Aurora (MySQL, Postgres)
Amazon Aurora (MySQL, Postgres)Amazon Aurora (MySQL, Postgres)
Amazon Aurora (MySQL, Postgres)
 
What's new in Amazon Aurora - ADB203 - Chicago AWS Summit
What's new in Amazon Aurora - ADB203 - Chicago AWS SummitWhat's new in Amazon Aurora - ADB203 - Chicago AWS Summit
What's new in Amazon Aurora - ADB203 - Chicago AWS Summit
 
ElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech Talks
ElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech TalksElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech Talks
ElastiCache: Deep Dive Best Practices and Usage Patterns - AWS Online Tech Talks
 

Mais de Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

Mais de Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

SRV308 Deep Dive on Amazon Aurora

  • 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Debanjan Saha deban@amazon.com GM, Amazon Aurora & Glue SRV308 Deep Dive on Amazon Aurora
  • 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is Amazon Aurora? D atabas e r eimagined for the c loud  Speed and availability of high-end commercial databases  Simplicity and cost-effectiveness of open source databases  Drop-in compatibility with MySQL and PostgreSQL  Simple pay as you go pricing Delivered as a managed service
  • 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Re-imagining the relational database 1 2 3 Scale-out, distributed architecture Service-oriented architecture leveraging AWS services Automate administrative tasks – fully managed service
  • 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scale-out, distributed architecture  Purpose-built log-structured distributed storage system designed for databases  Storage volume is striped across hundreds of storage nodes distributed over 3 different availability zones  Six copies of data, two copies in each availability zone to protect against AZ+1 failures  Plan to apply same principles to other layers of the stack Master Replica Availability Zone 1 Shared storage volume Availability Zone 2 Availability Zone 3 Storage nodes with SSDs SQL Transactions Caching SQL Transactions Caching SQL Transactions Caching
  • 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Leveraging AWS services Lambda S3 IAM CloudWatch Invoke Lambda events from stored procedures/triggers Load data from S3, store snapshots and backups in S3 Use IAM roles to manage database access control Upload systems metrics and audit logs to CloudWatch
  • 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Automate administrative tasks Schema design Query construction Query optimization Automatic fail-over Backup & recovery Isolation & security Industry compliance Push-button scaling Automated patching Advanced monitoring Routine maintenance Takes care of your time-consuming database management tasks, freeing you to focus on your applications and business You AWS
  • 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora is used by ¾ of the top 100 AWS customers Fastest growing service in AWS history Aurora customer adoption
  • 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Who is moving to Aurora and why? Customers using open source engines  Higher performance – up to 5x  Better availability and durability  Reduces cost – up to 60%  Easy migration; no application change  One tenth of the cost; no licenses  Integration with cloud ecosystem  Comparable performance and availability  Migration tooling and services Customers using Commercial engines
  • 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AURORA PERFORMANCE 5x faster than MySQL; 3x faster than PostgreSQL
  • 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora MySQL performance WRITE PERFORMANCE READ PERFORMANCE MySQL SysBench results; R4.16XL: 64cores / 488 GB RAM Aurora read write throughput compared to MySQL 5.6 based on industry standard benchmarks. Aurora MySQL 5.6 0 100000 200000 300000 400000 500000 600000 700000 0 50000 100000 150000 200000 250000
  • 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora PostgreSQL performance 0 5000 10000 15000 20000 25000 30000 35000 40000 45000 10 15 20 25 30 35 40 45 50 55 60 Throughput,tps Minutes pgbench throughput over time, 150 GiB, 1024 clients PostgreSQL (Single AZ) Amazon Aurora (Three AZs) W h i l e r u n n i n g p g b e n c h a t l o a d , t h r o u g h p u t i s 3 x m o r e c o n s i s t e n t t h a n P o s t g r e S Q L
  • 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How did we achieve this? Do fewer IOs Minimize network packets Cache prior results Offload the database engine DO LESS WORK Process asynchronously Reduce latency path Use lock-free data structures Batch operations together BE MORE EFFICIENT DATABASES ARE ALL ABOUT I/O NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND HIGH-THROUGHPUT PROCESSING IS ALL ABOUT CONTEXT SWITCHES
  • 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora I/O profile BINLOG DATA DOUBLE-WRITELOG FRM FILES TYPE OF WRITE MYSQL WITH REPLICA EBS mirrorEBS mirror AZ 1 AZ 2 Amazon S3 EBS Amazon Elastic Block Store (EBS) Primary Instance Replica Instance 1 2 3 4 5 AZ 1 AZ 3 Primary Instance Amazon S3 AZ 2 Replica Instance ASYNC 4/6 QUORUM DISTRIBUT ED WRITES Replica Instance AMAZON AURORA 780K transactions 7,388K I/Os per million txns (excludes mirroring, standby) Average 7.4 I/Os per transaction MySQL I/O profile for 30 min Sysbench run 27,378K transactions 35X MORE 0.95 I/Os per transaction (6X amplification) 7.7X LESS Aurora IO profile for 30 min Sysbench run
  • 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. IO traffic in Aurora (storage node) LOG RECORDS Primary Instance INCOMING QUEUE STORAGE NODE S3 BACKUP 1 2 3 4 5 6 7 8 UPDATE QUEUE ACK HOT LOG DATA BLOCKS POINT IN TIME SNAPSHOT GC SCRUB COALESCE SORT GROUP PEER TO PEER GOSSIPPeer Storage Nodes All steps are asynchronous Only steps 1 and 2 are in foreground latency path Input queue is 46X less than MySQL (unamplified, per node) Favor latency-sensitive operations Use disk space to buffer against spikes in activity OBSERVATIONS IO FLOW ① Receive record and add to in-memory queue ② Persist record and ACK ③ Organize records and identify gaps in log ④ Gossip with peers to fill in holes ⑤ Coalesce log records into new data block versions ⑥ Periodically stage log and new block versions to S3 ⑦ Periodically garbage collect old versions ⑧ Periodically validate CRC codes on blocks
  • 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora lock management Scan Delete Scan Delete Insert Scan Scan Insert Delete Scan Insert Insert MySQL lock manager Aurora lock manager  Same locking semantics as MySQL  Concurrent access to lock chains Needed to support many concurrent sessions, high update throughput  Multiple scanners in individual lock chains  Lock-free deadlock detection
  • 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Asynchronous key prefetch Latency improvement factor vs. Batched Key Access (BKA) join algorithm. Decision support benchmark, R3.8xlarge 14.57 - 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 Query-1 Query-2 Query-3 Query-4 Query-5 Query-6 Query-7 Query-8 Query-9 Query-10 Query-11 Query-12 Query-13 Query-14 Query-15 Query-16 Query-17 Query-18 Query-19 Query-20 Query-21 Query-22 Cold buffer Works for queries using Batched Key Access (BKA) join algorithm + Multi- Range Read (MRR) Optimization Performs a secondary to primary index lookup during JOIN evaluation Uses background thread to asynchronously load pages into memory AKP used in queries 2, 5, 8, 9, 11, 13, 17, 18, 19, 20, 21
  • 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Batched scans Standard MySQL pprocesses rows one-at- a-time resulting in high overhead due to:  Repeated function calls  Locking and latching  cursor store and restore  InnoDB to MySQL format conversion Amazon Aurora scan tuples from the InnoDB buffer pool in batches for  Table full scans  Index full scans  Index range scans Latency improvement factor vs. Batched Key Access (BKA) join algorithm Decision support benchmark, R3.8xlarge 1.78X - 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 Query-1 Query-2 Query-3 Query-4 Query-5 Query-6 Query-7 Query-8 Query-9 Query-10 Query-11 Query-12 Query-13 Query-14 Query-15 Query-16 Query-17 Query-18 Query-19 Query-20 Query-21 Query-22
  • 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. MONITORING DATABASE PERFORMANCE
  • 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Performance Insights Dashboard showing load on DB • Easy • Powerful Identifies source of bottlenecks • Top SQL Adjustable time frame • Hour, day, week , month • Up to 35 days of data Max CPU
  • 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Performance Insights What if you could find out what your databases were doing, or had been doing? Performance Insights is the answer:  Lets you see your overall instance load  Drill down by SQL statement, by time, or by calling host
  • 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. WHAT ABOUT AVAILABILITY? “Performance only matters if your database is up”
  • 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 6-way replicated storage Sur vives c atas tr ophic failur es Six copies across three availability zones 4 out 6 write quorum; 3 out of 6 read quorum Peer-to-peer replication for repairs Volume striped across hundreds of storage nodes SQL Transaction AZ 1 AZ 2 AZ 3 Caching SQL Transaction AZ 1 AZ 2 AZ 3 Caching Read and write availabilityRead availability
  • 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Up to 15 promotable read replicas Master Read Replica Read Replica Read Replica Shared distributed storage volume Reader end-point ► Up to 15 promotable read replicas across multiple availability zones ► Re-do log based replication leads to low replica lag – typically < 10ms ► Reader end-point with load balancing and auto-scaling * NEW *
  • 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Instant crash recovery TRADITIONAL DATABASE Have to replay logs since the last checkpoint Typically 5 minutes between checkpoints Single-threaded in MySQL; requires a large number of disk accesses AMAZON AURORA Underlying storage replays redo records on demand as part of a disk read Parallel, distributed, asynchronous No replay for startup Checkpointed Data Redo Log Crash at T0 requires a re-application of the SQL in the redo log since last checkpoint T0 T0 Crash at T0 will result in redo logs being applied to each segment on demand, in parallel, asynchronously
  • 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database fail-over time 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 0 - 5s – 30% of fail-overs 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% 50.00% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 5 - 10s – 40% of fail-overs 0% 10% 20% 30% 40% 50% 60% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 10 - 20s – 25% of fail-overs 0% 5% 10% 15% 20% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 20 - 30s – 5% of fail-overs
  • 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. RECENT INNOVATIONS
  • 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Availability is about more than HW failures Aur or a s olutions for availability dis r uptions 1. Patching your database software  Zero Downtime Patching 2. Large-scale copying and reorganizations  Fast Cloning 3. DBA errors requiring database restores  Backtrack 4. Disasters  Global replication
  • 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Zero downtime patching Networking state Application state Storage Service App state Net state App state Net state BeforeZDP New DB Engine Old DB Engine New DB Engine Old DB Engine WithZDP User sessions terminate during patching User sessions remain active through patching Storage Service
  • 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fast database cloning Clone database without copying data  Creation of a clone is nearly instantaneous  Data copy happens only on write – when original and cloned volume data differ Example use cases  Clone a production DB to run tests  Reorganize a database  Save a point in time snapshot for analysis without impacting production system. PRODUCTION DATABASE CLONE CLONE CLONE DEV/TEST APPLICATIONS BENCHMARKS PRODUCTION APPLICATIONS PRODUCTION APPLICATIONS
  • 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database backtrack Backtrack brings the database to a point in time without requiring restore from backups • Backtracking from an unintentional DML or DDL operation • Backtrack is not destructive. You can backtrack multiple times to find the right point in time t0 t1 t2 t0 t1 t2 t3 t4 t3 t4 Rewind to t1 Rewind to t3 Invisible Invisible
  • 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How does backtrack work? We keep periodic snapshot of each segment; we also preserve the redo logs For backtrack, we identify the appropriate segment snapshots Apply log streams to segment snapshots in parallel and asynchronously SEGMENT SNAPSHOT LOG RECORDS RECOVERY POINT SEGMENT 1 SEGMENT 2 SEGMENT 3 TIME
  • 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Online DDL: MySQL vs. Aurora  Full Table copy in the background  Rebuilds all indexes – can take hours or days  DDL operation impacts DML throughput  Table lock applied to apply DML changes INDEX LEAFLEAFLEAF LEAF INDEX ROOT table name operation column-name time-stamp Table 1 Table 2 Table 3 add-col add-col add-col column-abc column-qpr column-xyz t1 t2 t3  Use schema versioning to decode the block.  Modify-on-write primitive to upgrade to latest schema  Currently support add NULLable column at end of table  Add column anywhere and with default coming soon. MySQL Aurora
  • 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Online DDL performance On r3.large On r3.8xlarge Aurora MySQL 5.6 MySQL 5.7 10GB table 0.27 sec 3,960 sec 1,600 sec 50GB table 0.25 sec 23,400 sec 5,040 sec 100GB table 0.26 sec 53,460 sec 9,720 sec Aurora MySQL 5.6 MySQL 5.7 10GB table 0.06 sec 900 sec 1,080 sec 50GB table 0.08 sec 4,680 sec 5,040 sec 100GB table 0.15 sec 14,400 sec 9,720 sec
  • 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Global replication - logical Fas ter dis as ter r ec over y and enhanc ed data loc ality Promote read-replica to a master for faster recovery in the event of disaster Bring data close to your customer’s applications in different regions Promote to a master for easy migration
  • 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Global replication – physical AZ 1 AZ 3 Primary Instance AZ 2 Aurora Replica Aurora Replica ReplicationServer ReplicationAgentREDO LOG FRM FILES TYPE OF WRITE Aurora Replica (optional) Region 1: Primary Aurora Cluster Region 2: Read Replica AZ 1 Consistently fast, low-lag, high-performance replication for global relational databases • Global-scale replication in seconds or less • Dedicated replication infrastructure ensures unconstrained performance • Local reads, faster recovery, tighter DR objectives, and seamless cross-region migration Async.
  • 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AURORA SERVERLESS
  • 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora Serverless use cases • Infrequently used applications (e.g. a low-volume blog site) • Applications with intermittent or unpredictable load (e.g. a news site) • Development or test databases not needed on nights or weekends • Consolidated fleets of multi-tenant SaaS applications
  • 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora Serverless • Starts up on demand, shuts down when not in use • Scales up & down automatically • No application impact when scaling • Pay per second, 1 minute minimum WARM POOL OF INSTANCES APPLICATION DATABASE STORAGE SCALABLE DB CAPACITY REQUEST ROUTER DATABASE END-POINT
  • 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Instance provisioning and scaling • First request triggers instance provisioning. Usually 1-3 seconds • Instance auto-scales up and down as workload changes. Usually 1-3 seconds • Instances hibernate after user-defined period of inactivity • Scaling operations are transparent to the application – user sessions are not terminated • Database storage is persisted until explicitly deleted by user DATABASE STORAGE WARM POOL APPLICATION REQUEST ROUTER CURRENT INSTANCE NEW INSTANCE
  • 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scaling to and from zero • If desired, instances are removed after user-defined period of inactivity • While database is stopped, you only pay for storage • A database request triggers instance provisioning (usually under 5 seconds) • Scaling to zero is more expensive as there is no running server from which to copy the buffer cache WARM POOL INSTANCE DATABASE STORAGE REQUEST ROUTER APPLICATION PROXY ENDPOINT
  • 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AURORA MULTI-MASTER
  • 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Distributed Lock Manager GLOBAL RESOURCE MANAGER SQL TRANSACTIONS CACHING LOGGING SQL TRANSACTIONS CACHING LOGGING SHARED DISK CLUSTER STORAGE APPLICATION LOCKING PROTOCOL MESSAGES SHARED STORAGE M1 M2 M3 M1 M1 M1M2 M3 M2 Cons Heavyweight cache coherency traffic, on per-lock basis Networking can be expensive Negative scaling when hot blocks Pros All data available to all nodes Easy to build applications Similar cache coherency as in multi-processors
  • 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Consensus with two phase or Paxos commit DATA RANGE #1 DATA RANGE #2 DATA RANGE #4 DATA RANGE #3 DATA RANGE #5 L L L L L SHARED NOTHING SQL TRANSACTIONS CACHING LOGGING SQL TRANSACTIONS CACHING LOGGING APPLICATION STORAGE STORAGE Cons Heavyweight commit and membership change protocols Range partitioning can result in hot partitions, not just hot blocks. Re-partitioning expensive. Cross partition operations expensive. Better at small requests Pros Query broken up and sent to data node Less coherence traffic – just for commits Can scale to many nodes
  • 44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Conflict resolution using distributed ledgers • There are many “oases” of consistency in Aurora • The database nodes know transaction orders from that node • The storage nodes know transactions orders applied at that node • Only have conflicts when data changed at both multiple database nodes AND multiple storage nodes • Much less coordination required 2 3 4 5 61 BT1 [P1] BT2 [P1] BT3 [P1] BT4 [P1] BT1 BT2 BT3 BT4 Page1 Quorum OT1 OT2 OT3 OT4 Page 1 Page 2 2 3 4 5 61 OT1[P2] OT2[P2] OT3[P2] OT4[P2] PAGE1 PAGE2 MASTER MASTER Page 2
  • 45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hierarchical conflict resolution • Both masters are writing to two pages P1 and P2 • BLUE master wins the quorum at page P1; ORANGE master wins quorum at P2 • Both masters recognize the conflict and have two choices: (1) roll back the transactions or (2) escalate to regional resolver • Regional arbitrator decides who wins the tie breaker . 2 3 4 5 61 BT1 [P1] OT1 [P1] 2 3 4 5 61 OT1[P2] BT1[P2] PAGE1 PAGE2 MASTER MASTER BT1 OT1 Regional resolver Page 1 Page 2 Page 1 Page 2 Quorum X X
  • 46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-region Multi-Master Write accepted locally Optimistic concurrency control – no distributed lock manager, no chatty lock management protocol REGION 1 REGION 2 HEAD NODES HEAD NODES MULTI-AZ STORAGE VOLUME MULTI-AZ STORAGE VOLUME LOCAL PARTITION LOCAL PARTITIONREMOTE PARTITION REMOTE PARTITION Conflicts handled hierarchically – at head nodes, at storage nodes, at AZ and region level arbitrators Near-linear performance scaling when there is no or low levels of conflicts
  • 47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AURORA PARALLEL QUERY
  • 48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Parallel query processing Aurora storage has thousands of CPUs  Presents opportunity to push down and parallelize query processing using the storage fleet  Moving processing close to data reduces network traffic and latency However, there are significant challenges  Data stored in storage node is not range partitioned – require full scans  Data may be in-flight  Read views may not allow viewing most recent data  Not all functions can be pushed down to storage nodes DATABASE NODE STORAGE NODES PUSH DOWN PREDICATES AGGREGATE RESULTS
  • 49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Processing at head node STORAGE NODES OPTIMIZER EXECUTOR INNODB NETWORK STORAGE DRIVER AGGREGATOR APPLICATION PARTIAL RESULTS STREAM RESULTS IN-FLIGHT DATA PQ CONTEXT PQ PLAN Query Optimizer produces PQ Plan and creates PQ context based on leaf page discovery PQ request is sent to storage node along with PQ context Storage node produces  Partial results streams with processed stable rows  Raw stream of unprocessed rows with pending undos Head node aggregates these data streams to produce final results
  • 50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Processing at storage node Each storage node runs up to 16 PQ processes, each associated with a parallel query PQ process receives PQ context  List of pages to scan  Read view and projections  Expression evaluation byte code PQ process makes two passes through the page list  Pass 1: Filter evaluation on InnoDB formatted raw data  Pass 2: Expression evaluation on MySQL formatted data PQ PROCESS PQ PROCESS Up to 16 TO/FROM HEAD NODE STORAGE NODE PROCESS PAGE LISTS
  • 51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Example: parallel hash join Head node sends hash join context to storage node consisting of:  List of pages – Page ID and LSNs  Join expression byte code  Read view – what rows to include  Projections – what columns to select PQ process performs the following steps:  Scans through the lists of pages  Builds partial bloom filters; collect stats  Sends two streams to head node • Rows with matching hashes • Rows with pending undo processing Head node performs the Join function after merging data streams from different storage nodes JOIN CONTEXT PAGE SCAN STABLE ROWS ROWS WITH PENDING UNDO BLOOM FILTER FILTERED AND PROJECTED ROW STREAM UNPROCESSED ROW STREAM
  • 52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Parallel query performance 1.71 1.31 100.16 142.4 41.39 11.58 117.06 1.77 290.66 1.15 5.61 48.87 29.95 4.29 2.76 83.47 14.35 2.27 1.27 1 2 4 8 16 32 64 128 256
  • 53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Enjoy trying new features? AURORA SERVERLESS AURORA MULTI-MASTER AURORA PARALLEL QUERY PERFORMAN CE INSIGHTS Scales DB capacity to match app needs Scales out writes across multiple AZs Improves performance of large queries PERFORMANCE INSIGHTS Assess DB load & tune performance Sign up for previews : aws .amaz on.c om/rds/aurora/
  • 54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you!
  • 55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Please complete the session survey in the summit mobile app.