As developers using a functional programming language, we embrace immutable collections and pure functions in our code. However, we often have separate rules for how we work with our persistent collections in databases. In this talk we'll look at why & how to take a more functional approach to persistence with databases, from managing the connections, using immutable table rows, and considering event-source persistence

1. Functional Database
Strategies
Scala Bay • March 20, 2017

2. Hello
Scala Bay
Hello
Scala Bay

3. YesQL

4. by Jason Swartz
@swartzrock

6. Functional Database
Strategies

7. Step One
Buy A Functional Database

8. Silly! Databases Are
mutable, not
functional!

9. Well?
Aren’tthey?

10. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

11. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

12. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

13. Remember That Your
Database
is mutable.

14. Avoid Sharing
Your State

15. ● Avoid shared mutable SESSIONS
● Avoid shared mutable CURSORS
● Mutating state? Cool! But MODEL IT FIRST
● Execute state changes at THE EDGE
Safe Database Interactions

16. Doobie
A Typelevel Project

17. Are these steps? Or a Monad?

18. val xa: Transactor[Task] =
DriverManagerTransactor[Task](
"org.postgresql.Driver",
"jdbc:postgresql://localhost/issues",
"u", "p"
)

19. case class Issue(id: Int, issueId: Int, title: String,
assignee: Int, status: String)
val issue: Issue = sql"""
select id, issue_id, title, assignee, status
from issues where issue_id = ${req.issue_id}
order by id desc limit 1
"""
.query[Issue]
.unique
.transact(xa)
.unsafePerformSync

20. ● ResultSet to Case Class Conversion
● Async IO modeling with Task (scalaz version)
● Think in SQL, not an ORM
● Plain SQL with sql"" interpolation
● SQL functions supported
Why Doobie?

21. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

22. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

23. Let’s Talk About
Immutable Tables

26. Create-Read-
Update-Delete

27. GET /issues
GET /issues/{id}
POST /issues
PUT /issues/{id}
Issue Endpoints

28. How Do These Events
Affect The Database?

29. 1. POST /issues title=’Config ELB’
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+

30. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+

31. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+

32. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+
Not Bad.

33. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+
Do You Know How We Got Here?

34. +------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+
Do You Know How We Got Here?

35. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+
Do You Know How We Got Here?

36. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+
Why is ‘assignee’ NULL?

37. Mutable Table Rows
Lose History

38. Immutable Table Rows
Keep Their History

39. Let’s Try To
Lock Down
Our State

40. 1. POST /issues title=’Config ELB’
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+

41. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+

42. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:19 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+

43. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:19 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+

44. 1. GET /issues/1
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:19 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+

45. Tables Are Mutable,
But Table Rows Should Be
Immutable

46. In Other Words, Tables
Should Be
Append-Only

47. How Do You Make An
Append-Only
Table?

48. One: Don’t Let Your DB User
Make
Changes

49. Grant select, insert on
issues to my-db-user;
-- tested on Postgresql

50. Thank You!
Goodbye!

51. Two: Pick The Right
Columns

52. 1. GET /issues/1
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:19 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | NULL | true |
+------+-------------+------------+----------+-------+

53. create table issues (
id serial,
created timestamp default now(),
issue_id int default nextval(‘iseq’),
title text,
assignee int,
done boolean default false
)

54. 1. GET /issues/1
+------+-------------+------------+------------+----------+-------+
| id | created | issue_id | title | assignee | done |
+------+-------------+------------+------------+----------+-------+
| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |
+------+-------------+------------+------------+----------+-------+
| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
+------+-------------+------------+------------+----------+-------+
| 4 | 09-18 18:24 | 1 | Config ELB | NULL | true |
+------+-------------+------------+------------+----------+-------+

55. select * from issues
where issue_id = :issue_id
order by id desc limit 1;

56. What Defines An
Issue,
The Version or the Entity?

57. ISSUES
+------+-------------+------------+------------+----------+-------+
| id | created | issue_id | title | assignee | done |
+------+-------------+------------+------------+----------+-------+
| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |
+------+-------------+------------+------------+----------+-------+
| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+

58. create table issues (
id serial,
created timestamp default now(),
issue_id int default nextval(‘iseq’),
title text,
assignee int,
done boolean default false )
create table notifications (
id serial,
created timestamp default now(),
issue_row_id int references issues )

59. ISSUES
+------+-------------+------------+------------+----------+-------+
| id | created | issue_id | title | assignee | done |
+------+-------------+------------+------------+----------+-------+
| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |
+------+-------------+------------+------------+----------+-------+
| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
NOTIFICATIONS
+------+-------------+--------------+
| id | created | issue_row_id |
+------+-------------+--------------+
| 1 | 09-18 19:13 | 2 |
+------+-------------+--------------+

60. ISSUES
+------+-------------+------------+------------+----------+-------+
| id | created | issue_id | title | assignee | done |
+------+-------------+------------+------------+----------+-------+
| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |
+------+-------------+------------+------------+----------+-------+
| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |
+------+-------------+------------+------------+----------+-------+
NOTIFICATIONS
+------+-------------+--------------+
| id | created | issue_row_id |
+------+-------------+--------------+
| 1 | 09-18 19:13 | 2 |
+------+-------------+--------------+

61. Issue Notifications Are
Immutable
In The Database

62. That’s The Basics Of
Immutability
In Table Rows

63. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

64. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

65. SQL:2003 expands
Groups into Windows

66. Works Great In
Postgresql

67. ● Aggregation functions, eg sum(), rank(), avg()
● Window definitions with over()
● Grouping with partition by, order
Window Functions

68. +------+------------+------------+----------+-------+
| id | issue_id | title | assignee | done |
+------+------------+------------+----------+-------+
| 1 | 1 | Config ELB | NULL | false |
+------+------------+------------+----------+-------+
| 2 | 1 | Config ELB | 10 | false |
+------+------------+------------+----------+-------+
| 3 | 2 | Bug to fix | 11 | false |
+------+------------+------------+----------+-------+
| 4 | 2 | Bug to fix | 11 | true |
+------+------------+------------+----------+-------+

69. +------+------------+------------+----------+-------+
| id | issue_id | title | assignee | done |
+------+------------+------------+----------+-------+
| 1 | 1 | Config ELB | NULL | false |
+------+------------+------------+----------+-------+
| 2 | 1 | Config ELB | 10 | false |
+------+------------+------------+----------+-------+
| 3 | 2 | Bug to fix | 11 | false |
+------+------------+------------+----------+-------+
| 4 | 2 | Bug to fix | 11 | true |
+------+------------+------------+----------+-------+

70. with latest_issues as (
select *, row_number() over (
partition by issue_id
order by id
desc
)
from issues where created > now() - interval '7 day'
)
select * from latest_issues where row_number = 1

71. with latest_issues as (
select *, row_number() over (
partition by issue_id
order by id
desc
)
from issues where created > now() - interval '7 day'
)
select * from latest_issues where row_number = 1

72. with latest_issues as (
select *, row_number() over (
partition by issue_id
order by id
desc
)
from issues where created > now() - interval '7 day'
)
select * from latest_issues where row_number = 1

73. with latest_issues as (
select *, row_number() over (
partition by issue_id
order by id
desc
)
from issues where created > now() - interval '7 day'
)
select * from latest_issues where row_number = 1

74. +------+------------+------------+----------+-------+------------+
| id | issue_id | title | assignee | done | row_number |
+------+------------+------------+----------+-------+------------+
| 1 | 1 | Config ELB | NULL | false | 2 |
+------+------------+------------+----------+-------+------------+
| 2 | 1 | Config ELB | 10 | false | 1 |
+------+------------+------------+----------+-------+------------+
| 3 | 2 | Bug to fix | 11 | false | 2 |
+------+------------+------------+----------+-------+------------+
| 4 | 2 | Bug to fix | 11 | true | 1 |
+------+------------+------------+----------+-------+------------+

75. +------+------------+------------+----------+-------+------------+
| id | issue_id | title | assignee | done | row_number |
+------+------------+------------+----------+-------+------------+
| 1 | 1 | Config ELB | NULL | false | 2 |
+------+------------+------------+----------+-------+------------+
| 2 | 1 | Config ELB | 10 | false | 1 |
+------+------------+------------+----------+-------+------------+
| 3 | 2 | Bug to fix | 11 | false | 2 |
+------+------------+------------+----------+-------+------------+
| 4 | 2 | Bug to fix | 11 | true | 1 |
+------+------------+------------+----------+-------+------------+

76. +------+------------+------------+----------+-------+------------+
| id | issue_id | title | assignee | done | row_number |
+------+------------+------------+----------+-------+------------+
| 1 | 1 | Config ELB | NULL | false | 2 |
+------+------------+------------+----------+-------+------------+
| 2 | 1 | Config ELB | 10 | false | 1 |
+------+------------+------------+----------+-------+------------+
| 3 | 2 | Bug to fix | 11 | false | 2 |
+------+------------+------------+----------+-------+------------+
| 4 | 2 | Bug to fix | 11 | true | 1 |
+------+------------+------------+----------+-------+------------+

77. That Was
Window
Functions

78. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

79. 1. Functional DB’s In Scala
a. Doobie
2. Functional DB’s In The Database
a. Immutable Rows
b. Window Functions
c. Events, Not State
Agenda

80. You Know How To Maintain
State

81. Do We Still Need State?

82. Let’s Talk About
Event-Sourcing

83. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | 10 | true |
+------+-------------+------------+----------+-------+

84. 1. POST /issues title=’Config ELB’
2. PUT /issues/1 assignee=10
3. PUT /issues/1 done=true
+------+-------------+------------+----------+-------+
| id | updated | title | assignee | done |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:13 | Config ELB | NULL | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:16 | Config ELB | 10 | false |
+------+-------------+------------+----------+-------+
| 1 | 09-18 18:24 | Config ELB | 10 | true |
+------+-------------+------------+----------+-------+
Events
States

89. Now We’re Storing
Events,
Not States

90. create table issue_events (
id serial,
created timestamp default now(),
issue_id int default nextval(‘iseq’),
originator text,
payload text
)

91. 1. POST /issue/1/event ‘Originator: 4a48239-8a..’
payload=’<Update val=”done=true”>’
+----+-------------+----------+------------+---------+
| id | created | issue_id | originator | payload |
+----+-------------+----------+------------+---------+
| 14 | 09-18 18:50 | 1 | 4a482... | <...> |
+----+-------------+----------+------------+---------+

92. Create Events And
Simulate The State

93. 1. Create-Issue
Issue(“Config ELB”, null, false);
Real Events
Virtual States

94. 1. Create-Issue
2. Assign-Issue
Issue(“Config ELB”, 10, false);
Real Events
Virtual States

95. 1. Create-Issue
2. Assign-Issue
3. Complete-Issue
Issue(“Config ELB”, 10, true);
Real Events
Virtual States

96. So Why Use
Event-Sourcing?

97. 1. High Write Performance
2. Potential for Command/Query Separation
3. Auditable
4. Replayable
5. Undo-able
6. Monitorable
Reasons For Event-Sourcing

98. It’s Like Having Control
Over The Versions Of
Your State Changes

99. It’s Like Having Control
Over The Versions Of
Your Data

100. It’s Like Git
For Your Data

101. 1. Frankly, It’s Weird
2. Requires Events. No Events, No Event-Sourcing.
3. As Of March 2017, It’s Still Non-Standard
Reasons Against Event-Sourcing

102. Wait! We’re
Scala
developers!

103. Who Cares About Being
Non-Standard?

104. That About Sums Up Event
Sourcing

105. That About Sums Up
Database
Interactions

106. Okay, Actually That’s The
Entire Talk
Unless There’s More Time

107. Functional Database
Strategies
Scala Bay • March 20, 2017

108. by Jason Swartz
@swartzrock

109. Thank You
For Attending

110. Fin

111. THIS SPACE
LEFT BLANK