An Introduction to Graph: Database, Analytics, and Cloud Services

Analytics and Data Summit 2018
An Introduction to Graph: Database, Analytics, and Cloud Services
Jean Ihm Hans Viehmann Jan Michels
Product Manager US Product Manager EMEA Principal Member of Technical Staff
Oracle Oracle Oracle
@JeanIhm @SpatialHannes

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Program Agenda
Product Introduction
Use Cases
Feature Overview
Demo
1
2
3
4
3

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 4
https://twitter.jeffprod.com
Following, no follow back
Follower, no follow back
Follow each other

https://www.macalester.edu/~abeverid/thrones.html
A social network generated from
A Storm of Swords.
The color of a vertex indicates its
community.
The size of a vertex corresponds
to its PageRank value, and the
size of its label corresponds to its
betweenness centrality.
An edge’s thickness represents
its weight.

Oracle’s Spatial and Graph Strategy
Enabling Spatial and Graph use cases on every platform
6
Oracle Big Data
Spatial and Graph
Oracle Database
Spatial and Graph
Spatial and Graph in
Oracle Cloud
Database Cloud Service
Exadata Cloud Service

Two Graph Data Models
RDF Data Model
• Data federation
• Knowledge representation
• Semantic Web
Social Network
Analysis
 Financial
 Retail, Marketing
 Social Media
 Smart Manufacturing
Linked Data
Semantic Web
Property Graph Model
• Path Analytics
• Social Network Analysis
• Entity analytics
 Life Sciences
 Health Care
 Publishing
 Finance
Use Case Graph Model Industry Domain
RDF Use Case Sessions: Linked open data initiatives from Ireland/Japan, tax fraud
detection in Florence, Italy – Wednesday 1:00-2:45

Graph Database Features:
• Scalability, Performance, Security
• Graph analytics
• Graph Visualization
• Graph Query Language
• Standard interfaces
• Integration with Machine Learning tools
8
Courtesy Tom Sawyer Perspectives
Courtesy Linkurious

Oracle Big Data Spatial and Graph
• Available for Big Data platform
– Hadoop, HBase, Oracle NoSQL
• Supported both on BDA and
commodity hardware
– CDH and Hortonworks
• Database connectivity through Big
Data Connectors or Big Data SQL
• Included in Big Data Cloud Service
Oracle Spatial and Graph (Database)
• Available with Oracle 12.2/18.1
• Using tables for graph persistence
• Graph views on relational data
• In-database graph analytics
– Sparsification, shortest path, page rank,
triangle counting, WCC, sub graphs
• SQL queries possible
• Included in Database Cloud Service
9
Graph Product Options

Use Cases
10

Graph Analysis for Business Insight
11
Identify
Influencers
Discover Graph Patterns
in Big Data
Generate
Recommendations

Some Use Case Scenarios
• Finance
– Customer 360, Fraud detection
• Public Sector
– Tax Evasion, Crime network analysis
• Retail
– Recommendation, sentiment analysis
• Manufacturing
– Analyzing complex bill of materials (BoM)

Financial Services
• Model customer relationship to products, services, people, places.
• Analyze money customer’s flow between non-bank to bank accounts
• Combine internal CRM data with enterprise and social media content
• Identify high-value customers across business divisions
• Enhance new product/service opportunities
• Provide Real-time recommendations
13
Applying Graph Analysis To Improve Customer Service
Use Case Session: Follow the Money: A Graph Model for
Monitoring Bank Transactions - Wednesday 10:05am

Customer 360 Analysis
14

Tax Fraud Analysis
Chinese Province Tax Office
Challenge:
– Modeling relationships between individuals and corporations
– Ingest documents, social media, web content, and publicly
available open data
– Create a ‘picture’ of the taxpayer network
• Taxpayer relationship with other taxpayers
• If a company structure, identify associated directors and shareholders in
that company
• Relationship between taxpayer’s and their associates’ financial affairs
• Identify relevant intermediaries acting on behalf of taxpayer
– Explore tax evasion and fraud, trigger a formal case
investigation
15

Public Security: Analyzing Criminal Networks
Chinese Police Department
Business Requirement
– Model relationships between known and suspected
criminals
– Ingest documents, social media, web content, chat
rooms, flight records, hotel stay registries, and
publically available open datasets.
How graph analysis solves the problem
• Search for known individuals in web of content
• Analyze relationship with other criminals, travel
history, addresses, employers
• Relationship between suspects and their financial
affairs
Courtesy Tom Sawyer Perspectives
Use Case Session: Construct a
Public Security Data Analysis
Engine Using Oracle Big Data
Solution (Justin Zou) Wednesday
10:35am

Analyzing Blockchain Ledger Transactions
• Distributed Ledgers being adopted in
Finance, Public Sector
• Load and manage massive transactions
from a distributed digital ledger
• Efficiently traverse a blockchain
transaction graph
• Query and visualize – search for
patterns of activity
17
Land Management, Banking, Public Services Tech Session: Analyzing Blockchain
and Bitcoin Transaction Data as
Graph (Z. Wu / X. Lopez) Thursday
8:45am

CyberNetwork Modeling & Analysis
• Model existing cloud infrastructure
entities as a connected graph
– servers, switches, storage, firewalls…
• Analysis of network behavior
– System impact of change to network topology
– Reachability analysis to understand which
routines, libraries, servers, routers, etc., are
affected by a modification
– Nework intrusion detection (next slide)
Monitoring Cloud Infrastructure Services

Automotive Manufacturing
Support high variance, short innovation cycles of complex autos
Voice of
Customers
Model Based
Design
Configuration & BOM
Simulation
3D CAD & CAE
Factory Resources
Financial
Requirements
• Unified graph representation of BoM,
Configuration, CAE, Simulation...
• Generate “graph view” of relational
data, or model instance data as graph
• Apply graph query and search across
BoM and configuration models
• Apply graph analytics
• Scale to trillions of nodes and edges
Graph View of Enterprise Data
Use Case Session: Graph
Modeling and Analysis for
Complex Automotive Data
Management (Mazda)
Wednesday 11:10 am

Feature Overview
20

Oracle Graph Analytics Architecture
Scalable and Persistent Storage
Graph Data Access Layer API
Graph Analytics
In-memory Analytic Engine
Blueprints & SolrCloud / Lucene
Property Graph Support on
Apache HBase, Oracle NoSQL or Oracle 12.2
RESTWebService
Python,Perl,PHP,Ruby,
Javascript,…
Java APIs
Java APIs/JDBC/SQL/PLSQL
21
CytoscapePlug-in
RIntegration(OAAgraph)
Sparkintegration
SQL*Plus, …

Interacting with the Graph
• Access through APIs
– Implementation of Apache Tinkerpop Blueprints APIs
– Based on Java, REST plus SolR Cloud/Lucene support for text search
• Scripting
– Groovy, Python, Javascript, ...
– Apache Zeppelin integration, Javascript (Node.js) language binding
• Graphical UIs
– Cytoscape, plug-in available
– Commercial Tools such as TomSawyer Perspectives
On-premise product geared towards data scientists and developers
22

Graph Analysis Algorithms can be very hard to code ...
• Example: Find the size of the 2-hop network of vertices (Gremlin+Python)
• Single API call instead
– Analysis in memory, in parallel
Oracle Big Data Spatial and Graph comes with 40+ pre-built algorithms
sum([v.query()
.direction(blueprints.Direction.OUT).count()
for v in OPGIterator(v0.query()
.direction(blueprints.Direction.OUT)
.vertices().iterator())])

Example: Betweenness Centrality in Big Data Graph
Code
24
D
A
C
E
B
F
I
J
H
K
G
analyst.vertexBetweennessCentrality(pg)
.getTopKValues(15)

Structure Evaluation
– Conductance
– countTriangles
– inDegreeDistribution
– outDegreeDistribution
– partitionConductance
– partitionModularity
– sparsify
– K-Core computes
Community Detection
– communitiesLabelPropagation
Ranking
– closenessCentralityUnitLength
– degreeCentrality
– eigenvectorCentrality
– Hyperlink-Induced Topic Search (HITS)
– inDegreeCentrality
– nodeBetweennessCentrality
– outDegreeCentrality
– Pagerank, weighted Pagerank
– approximatePagerank
– personalizedPagerank
– randomWalkWithRestart
25
Graph Analysis Algorithms (1)

Pathfinding
– fattestPath
– shortestPathBellmanFord
– shortestPathBellmanFordReverse
– shortestPathDijkstra
– shortestPathDijkstraBidirectional
– shortestPathFilteredDijkstra
– shortestPathFilteredDijkstraBidirectional
– shortestPathHopDist
– shortestPathHopDistReverse
Recommendation
– salsa
– personalizedSalsa
– whomToFollow
Classic - Connected Components
– sccKosaraju
– sccTarjan
– wcc
26
Graph Analysis Algorithms (2)

• Multi-purpose notebook for data
analysis and visualization
– Default for Oracle Cloud Services
• Enables to embed interactive
execution inside Browsers
• Renders execution results with
plots and tables within Browsers
• PGX provides a hook (interpreter)
for Zeppelin integration
27
Apache Zeppelin integration

Combining Graph Analytics and Machine Learning
Graph Analytics
• Compute graph metric(s)
• Explore graph or compute
new metrics using ML result
Machine Learning
• Build predictive model
using graph metric
• Build model(s) and
score or classify data
28
Add to
structured data
Add to graph

OAAgraph integration with R
• OAAgraph integrates in-memory engine into ORE and ORAAH
• Adds powerful graph analytics and querying capabilities to existing
analytical portfolio of ORE and ORAAH
• Built in algorithms of PGX available as R functions
• PGQL pattern matching
• Concept of “cursor” allows browsing of in-memory analytical results using
R data structures (R data frame), allows further client-side processing in R
• Exporting data back to Database / Spark allows persistence of results and
further processing using existing ORE and ORAAH analytical functions
Dedicated session by Mark Hornick and Sungpack Hong in Big Data track

SPARK
SPARK
SPARK
PGX
PGX
Graph Analytics on SPARK
• Use SPARK for conventional tabular
data processing (RDD, Dataframe, -set)
• Define graph view of the data
– View it as node table and edge table
• Load into PGX
• Execute graph algorithms in PGX
– Orders of magnitude faster than GraphX
– More scaleable
• Push analysis results back into SPARK
as additional tables
• Continue SPARK analysis
SPARK data structure and
communication mechanism not
optimized for graph analysis workloads

PGX Performance (Query) vs. Neo4j
Hardware: 88 x Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz - 256 RAM
Warmup: ignore the first two runs, measure the third run
Index usage: no special indexes (use defaults)
1
10
100
1000
10000
100000
1000000
10000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Time(ms)
Query
Code Graph Dataset: 5M nodes and 64M edges
Neo4j 3.0.7 PGX 2.3.0
76x
142x
71x
8x 4x
22x
57x
30x 15x
2x
12x
613x
307x
34X
> 800,000x
4x
20x
Recursive path queries
Numbers shown are “hot” numbers:
Neo4j is no longer accessing disk.

Distributed Graph Analysis Engine
• Oracle Big Data Spatial and Graph uses very compact graph representation
– Can fit graph with ~23bn edges into one BDA node
• Distributed implementation scales beyond this
– Processing even larger graphs with several machines in a cluster (scale-out)
– Interconnected through fast network (Ethernet or, ideally, Infiniband)
• Integrated with YARN for resource management
– Same client interface, but not all APIs implemented yet
• Again, much faster than other implementations
– Comprehensive performance comparison with GraphX, GraphLab
Handling extremely large graphs

• SQL-like syntax but with graph
pattern description and property
access
– Interactive (real-time) analysis
– Supporting aggregates, comparison,
such as max, min, order by, group by
• Finding a given pattern in graph
– Fraud detection
– Anomaly detection
– Subgraph extraction
– ...
• Proposed for standardization by
Oracle
– Specification available on-line
– Open-sourced front-end (i.e. parser)
Pattern matching using PGQL
https://github.com/oracle/pgql-lang
33

PGQL Example query – PGQL 1.1
• Find all instances of a given pattern/template in data graph
• Fast, scaleable query mechanism
SELECT v3.name, v3.age
FROM socialNetworkGraph
MATCH (v1:Person) –[:friendOf]-> (v2:Person) –[:knows]-> (v3:Person)
WHERE v1.name = ‘Amber’
query
Query: Find all people who are
known to friends of ‘Amber’.
34
socialNetwork
Graph
100
:Person
name = ‘Amber’
age = 25
200:Person
name = ‘Paul’
age = 30
300
:Person
name = ‘Heather’
age = 27
777
:Company
name = ‘Oracle’
location =
‘Redwood City’
:worksAt{1831}
startDate = ’09/01/2015’
:friendOf{1173}
:knows{2200}
:friendOf {2513}
since = ’08/01/2014’

Graph Query Language Standardization – Status Quo
• Why Property Graphs with SQL?
• Who is involved in the standardization of SQL extensions for Property
Graphs?
• What might the SQL extensions for Property Graphs look like?
The SQL Standard and Property Graphs

Why Property Graphs with SQL?
• Users are using both SQL data and Property Graph data
• High interest in PGs from:
– Users
– Vendors
• Building consensus among the players on:
– Syntax
– Semantics
• Application development is easier, better, quicker, faster if only one
interface

Who is involved?
• ISO:
– JTC 1/SC32/WG3
– USA, Germany, Japan, UK, Canada, China
• ANSI:
– INCITS / DM32 / DM32.2 / DM32.2 Ad Hoc Group on SQL Extensions for Property
Graphs
– Oracle, Neo4j, Microsoft, IBM, SAP/Sybase, Teradata, JCC Consulting, Actian
Aka the ISO SQL committee
Aka the US SQL committee

SQL extensions for Property Graphs (PGs)
• Goal: define extensions to query property graphs
– Agree on one (or possibly more) representation of PGs in SQL
• Most obvious, in tables
• Maybe later, some “native” storage format
– Agree on the way to query PGs in SQL
• Query PGs “natively” (use the power of pattern matching)
– Maybe later embed (full) SQL in PG queries
• Represent result as a table (unleash the power of SQL on the result)
• Maybe later DML operations on a property graph directly
• Targeted for SQL:2020
– No concrete proposal/agreement yet

Property Graph Definition (DDL) – Example
• Example:
CREATE PROPERTY GRAPH myGraph
VERTEX TABLES (Person, Message)
EDGE TABLES (
Created SOURCE Person DESTINATION Message,
Commented SOURCE Person DESTINATION Message )
• Existing tables (or views): Person, Message, Created, Commented
• Implementation can infer keys from (primary/foreign keys) of underlying tables
• All columns of each table are exposed as properties of the corresponding
vertex/edge (tables)
Create a PG w/two vertex tables
and two edge tables.

DDL – Example (cont.)
• Example for optional clauses
CREATE PROPERTY GRAPH myGraph
VERTEX TABLES (
People KEY ( id )
LABEL Person
PROPERTIES ( emailAddress AS email ),
Messages KEY ( id )
LABEL Message
PROPERTIES ( created as creationDate, content ) )
EDGE TABLES (
CreatedMessage KEY ( id )
SOURCE KEY ( creator ) REFERENCES People
DESTINATION KEY ( message ) REFERENES Messages
LABEL Created NO PROPERTIES,
CommentedOnMessage KEY ( id )
SOURCE KEY ( commenter ) REFERENCES People
DESTINATION KEY ( message ) REFERENCES Messages
LABEL Commented NO PROPERTIES )
Same PG as before –
but fine-grained control over
labels, properties, etc.

Querying PGs - Example
SELECT GT.creationDate, GT.content
FROM myGraph GRAPH_TABLE (
MATCH
(Creator IS Person WHERE Creator.email = :email1)
-[ IS Created ]->
(M IS Message)
<-[ IS Commented ]-
(Commenter IS Person WHERE Commenter.email = :email2)
WHERE ALL_DIFFERENT (Creator, Commenter)
ONE ROW PER MATCH
COLUMNS (
M.creationDate,
M.content )
) AS GT
Get the creationDate and content of the
messages created by one person
("email1") and commented on by
another person ("email2").

Using PGQL with the Oracle Database
• New feature in Oracle 18c
• Provides easy and natural way to express graph queries
– SQL translation is performed automatically behind the scenes
– Users don’t need to worry about writing complex SQL
• Querying data on disk
– Store graphs which do not fit into memory
– Dynamic data can be queried without the need to constantly push updates to in-
memory snapshots
– Joint analysis with other data in the database
42
PGQL-to-SQL conversion in Oracle Spatial and Graph

Graph Visualization – Cytoscape, Vis.js, ...
47

Graph Visualization – Commercial Tools
Dedicated session on TomSawyer Perspectives integration
48
TomSawyer Presentation: Analyze the Global
Knowledge Graph – Wednesday 9:00-9:50

• A new PGX client
– Based on JavaScript (Node.JS)
– Support both PGX shared memory and
distributed runtime
• Design highlights
– Leveraging latest ES6 language features
– ES6 classes and getters
– Arrow Functions
– Chaining of Promises
– Clean error handling
Node.js Client PGX shared
memoryNode.js
PGX
Client
https
RESTInterface
PGX
distributed
const p = pgx.connect("http://myhost:7007", options)
p.then(session => {
return session.readGraphWithProperties(json)
}).then(graph => {
let analyst = graph.session.analyst
return analyst.pagerank(graph, 0.001, 0.85, 100)
}).then(rank => {
rank.topK(5).foreach(elem => console.log(elem))
let q = "SELECT n, n.pagerank WHERE (n)"
return rank.graph.queryPgql(q)
}).catch(function(err) {
console.log("error: " + err);
})
Code Example:

Linkurious Ogma
• Server-based (Node.JS)
• Light-weight JavaScript
visualizer
• Powerful rendering
• Oracle integration
50
https://www.slideshare.net/Linkurious/how-to-visualize-oracle-big-data-spatial-and-graph-with-ogma
https://linkurio.us/visualize-oracle-graph-data-ogma-library/
https://linkurio.us

Demo
51

Autonomous Graph Cloud Service
• Simple: Fully-managed, pre-configured, automatically tuned
• Fully Managed: “One-click” deployment, no installation, zero configuration
• Comprehensive: Data management, analytics, domain workflows
• Developer Ready: Low code /zero code, APIs, and query language
• Data Scientist Friendly: Graph Modeler, Studio, and Notebooks
Session: Sneak Preview: Graph Cloud Services and Spatial Console
for Database Cloud (Jim Steiner, Jayant Sharma) Tuesday 11:15am

Summary
Graph capabilities in Oracle Big Data Spatial and Graph
• Graph databases are powerful tools, complementing relational databases
– Especially strong for analysis of graph topology and multi-hop relationships
• Graph analytics offer new insight
– Especially relationships, dependencies and behavioural patterns
• Oracle Property Graph technology offers
– Comprehensive analytics through various APIs, integration with relational database
– Scaleable, parallel in-memory processing
– Secure and scaleable graph storage using Oracle NoSQL, HBase or Oracle Database
• Available both on-premise or in the Cloud
53

Analytics and Data Summit – Graph Sessions
Tuesday
• An Introduction to Graph: Database, Analytics, and Cloud Services (Ihm, Viehmann, Michels) 10:10am
• Sneak Preview: Graph Cloud Services and Spatial Console for Database Cloud (Jim Steiner, Jayant
Sharma) 11:15am
Wednesday
• Analyze the Global Knowledge Graph with Visualization, Cloud, & Spatial Tech (Tom Sawyer
Software) 9:00am
• Follow the Money: A Graph Model for Monitoring Bank Transactions (Federico Calabria) 10:05am
• Graph Modeling and Analysis for Complex Automotive Data Management (Mazda) 11:10am
• Construct a Public Security Data Analysis Engine Using Oracle Big Data Solution (Justin Zou)
11:35am
54
*All sessions in room 103 unless otherwise noted

Wednesday (continued)
• A Graph-Enabled Data Discovery Platform Supporting GDPR Compliance (W. Wcislo) 2:45pm
• Anomaly Detection in Medicare Provider Data Set using OAAgraph (M. Hornick, S. Hong)
4:30pm
Thursday
• Analyzing Blockchain and Bitcoin Transaction Data as Graph (Zhe Wu, Xavier Lopez) 8:45am
• Ingesting streaming data into Graph database (Trivadis) 9:50am
• Fake News, Trolls, Bots and Money Laundering – Find the truth with Graphs (Jim Steiner,
Sungpack Hong) 10:55am
• Measuring Published Citation using Graph Modeling and Analysis (The Institute of Statistical
Mathematics, Tokyo, Japan) 1:55pm
55

• Hands-on Labs *Held in 500 building room 104
– Property Graph 101 on Oracle Database 12.2 for the completely clueless (Vlamis Software) Tuesday,
3:35pm
– Using Property Graph and Graph Analytics on NoSQL DB to analyze data on Meetup.com (Karin Patenge)
Wednesday, 2:10pm
• RDF Graph sessions on Wednesday
– Serving Ireland's Authoritative Geospatial Data as Linked Data (Trinity College Dublin) 1:00pm
– Publication of Statistical Linked Open Data in Japan with Oracle RDF graph (National Statistics Center of
Japan) 1:25pm
– Discovering Tax Fraud in Florence, Italy with a Semantic Graph Database (Evodevo) 2:20pm
56

Engage with the Spatial and Graph SIG
Promotes interaction and
communication to drive
the market for spatial
technology and data
Members connect and
exchange knowledge via
online communities and
at conferences and
events
• Talk with us at the Summit!
•
• Join us online
• tinyurl.com/oraclespatialcommunity
• Search for “Oracle Spatial and Graph Community”
• Contact us:
oraclespatialsig@gmail.com @oraspatialsig
Morning Arrivals
Tues & Wed
7:45-8:30 a.m.
Registration Area
Birds of a Feather
Lunch
Wednesday
12-1pm
Auditorium
Receptions
Tues & Wed
evenings
Spatial Table,
lobby

An Introduction to Graph: Database, Analytics, and Cloud Services

An Introduction to Graph: Database, Analytics, and Cloud Services

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a An Introduction to Graph: Database, Analytics, and Cloud Services

Semelhante a An Introduction to Graph: Database, Analytics, and Cloud Services (20)

Mais de Jean Ihm

Mais de Jean Ihm (9)

Último

Último (20)

An Introduction to Graph: Database, Analytics, and Cloud Services