Jan W. Veldsink, Master in the Art of AI at Nyenrode Business University, Rabobank and Grio talks about the ethics of Machine Learning and how to assess this with BigML. *Machine Learning School for Business Schools 2021: Virtual Conference.

1. Jan W
. Veldsink MSc
THE ART OF
ARTIFICIAL INTELLIGENCE
TRUSTWORTHY AI
Jan W Veldsink MSc

2. Jan W
. Veldsink MSc

3. Modular MBA
2021
AI and Digital Security
AISEC
drs. Jan W Veldsink

4. Jan W
. Veldsink MSc
“Everybody has a plan
until they get punched in the mouth.”
Resilience: testing

5. Jan W
. Veldsink MSc
&

6. Jan W
. Veldsink MSc
Rabo - Compliance Analytics
• Fraud
• AML / Sanctions /TF / HT /
MAC / Integrity
• Data-science / Big-Data /
Business Rules / Machine
learning / Semantics /
Network reasoning / Logic /
BigML / Riskshield
Expertise
• Monitoring
• Visualisation of results in the
employee facing systems
• Adaptation Business Oriented
Machine Learning / Decision
Engineering
Challenges
Tooling
DataScience
Visualization
BusinessIntelligence
Business engineering
Type of projects
• Fraud detection
• AML
• CDDD / KYC
• Name Screening
• Data quality
• TRIM -Transaction Risk
Monitoring
• Misuse of systems
• Classification of documents
Complince Data
Science & Analytics

7. Jan W
. Veldsink MSc
Projects in BigML / Machine Learning Platform
DataDrift detection Agri Default Prediction
Fraud detection
CDD - Anomalies
Data driven Audit
False Positive Reduction
Classification Documents
KYC - Anomalies

8. Jan W
. Veldsink MSc
Organizing ML
Role Fte’s
Lead 1
Team 2
Data-support 1
Virtual team 4 - 8
IT -support 0,25
Some small team
A place to experiment
and work
ML/AI expert
Domain Data
expert
Business Domain
expert
Data
Labels
Model
A place to work on
business projects
AI(BigML)-Desk

9. Jan W
. Veldsink MSc
Artificial Intelligence - Watson

10. Jan W
. Veldsink MSc
Artificial Intelligence & Ethics
Wordcloud taken from the Human and Tech document: “Moreel kompas werkversie 200820”

11. Jan W
. Veldsink MSc
Trustworthy AI? > 50 pages..
EthicsGuidelinesforTrustworthyAI EU

12. Jan W Veldsink / Grio
CONFIDENTIAL
© AXVECO 2020. All rights reserved
EU 7 Criteria for Trustworthy AI
1. Human agency and oversight
Fundamental rights, human agency and human oversight
2. Technical Robustness and safety
Resilience to attack and security, fall back plan and general
safety, accuracy, reliability and reproducibility
3. Privacy and data governance
Respect for privacy, quality and integrity of data, and access to
data
4. Transparency
Traceability, explainability and communication
5. Diversity, non discrimination and fairness
The avoidance of unfair bias, accessibility and universal design,
and stakeholder participation
6. Societal and environmental wellbeing
Sustainability and environmental friendliness, social impact,
society and democracy
7. Accountability
Auditability, minimisation and reporting of negative impact,
trade-offs and redress

13. Jan W
. Veldsink MSc

14. Jan W
. Veldsink MSc
https://raceandtechnology.wordpress.com/2014/12/10/genetics-a-double-edged-sword-2/

15. Jan W
. Veldsink MSc
Fairness of Assessment

16. Jan W
. Veldsink MSc
The need for an AI/ML platform?
Auditability
Repeatability

17. Jan W
. Veldsink MSc
Situational Ethics
Situational ethics or situation ethics takes into account the particular context of
an act when evaluating it ethically, rather than judging it according to absolute
moral standards. With the intent to have a fair basis for judgments or action, one
looks to personal ideals of what is appropriate to guide them, rather than an
unchanging universal code of conduct, such as Biblical law under divine command
theory or the Kantian categorical imperative.[1] Proponents of situational
approaches to ethics include existentialist philosophers Sartre, de Beauvoir,
Jaspers, and Heidegger.[2
]
https://en.wikipedia.org/wiki/Situational_ethics

18. Jan W
. Veldsink MSc
Where to Start?
Step
1
Finish
“Let’s predict
milk production!”
“Here are the cows that
produce the best milk
And WHY”
Step
2
- - - - - - - -
???

19. Jan W
. Veldsink MSc
Where to Start ETHICS?
Step
1
“Let’s predict
milk production!”
Evaluation
Modeling
Data
Understanding
Data
Preparation
Business
Understanding
Deployment

20. Jan W
. Veldsink MSc
The Mindmap: AI Ethics Window
AI Ethics Window
Initiation The business need for AI/ML
Business task within the vision/
strategy of the organisation
The question
Is the question specific,
transparent and focussed on AI/
ML?
Is the question is Business?
From the business perspective
there is a justification for the
question:
The question also includes the
description of the data that will be
used to answer the question!
In the “Model creation” the data is
exactly defined
Can the question be answered
with or without the consent of the
user / employee / customer?
Is a person / organisation the
subject of the question
Can people get hurt / die?
Model Creation
There is no reuse of previous
models
The model is made for the
question and only answers the
question that has been
formulated
Every decision is explained by
the model
Only the data formulated in the
question was used for modeling
Tested against “Ethics Frame”
Will it be designed for dynamic /
calibration / learning?
Model execution / Use
The model is applied for the
purpose of answering the
question!
No questions are answered with
the model other than the ones
asked.
Are personal values or autonomy
at stake?
Permission from user / employee /
customer is required
Human in the loop
Is there a human making the last
decision?
Can the end-user influence the
actions
Feedback
Monitor
Datadrift Model competence
Model performance
Ethics performance
Decision performance
Destruction of the model
The AI model is completely
removed
Parts of the AI are left for reuse
Recycling of parts / of the model
Is the model intended for reuse,
for example a language model?
Ethics frame / non discriminatory
labels
Bias
Demographic blindness:
decisions are made using a
limited set of features that are
highly uncorrelated with protected
classes, that is, groups of people
protected by laws or policies.
Demographic parity: outcomes
are proportionally equal for all
protected classes.
Equal opportunity: true-positive
rates are equal for each protected
class.
Equal odds: true-positive and
false-positive rates are equal for
each protected class.
Race
Gender
M/F/U
Religion
Etnics
Name
Place of birth
Nationality
Societal class
Zip code
Earnings / salary / funds
Data assessment
Is the data a representative
sample?
Does the data contain any of the
“discriminating” data points
Technology assessment
Viable
Are we using the right AI
paradigm to solve the problem
Valuable
Does AI/ML add value
Vital
How critical is the use of AI
4-Way test
Is it the truth?
Is it fair to all concerned?
Will it build goodwill and better
relations?
Will it be beneficial to all
concerned?

21. Jan W
. Veldsink MSc
AI - A business driven process!
Destruction Reuse
Business Initiation The Question Model Creation Model Use

22. Jan W Veldsink / Grio
CONFIDENTIAL
Business Initiation
Business Initiation
•Describe the kind of business Vision / Mission
•Ethical framework for your business
•The business need for AI/ML
•Business tasks within the mission/vision/strategy of the
organization

23. Jan W Veldsink / Grio
CONFIDENTIAL
The Question
•Is the question specific, transparent and focussed on
AI/ML?
•Is the question put in Business terms?
• From the business perspective there is a justification for the question?
• Does the question violate the ethic frame?
•The question also includes the description of the data that will be used to
answer the question!
• Including the proposed data fields/sources and samples
•Can the question be answered with or without the consent of the user /
employee / customer?
• Is a person / organization the subject of the question
• Can people get hurt / die?
The Question

24. Jan W Veldsink / Grio
CONFIDENTIAL
Model Creation
•Is there reuse of models?
•Is designed for dynamic /calibration / learning?
•The model is made for the question and only answers the
question that has been formulated!
•Every decision is explained by the model!
•Only the data formulated in the question was used for
modelling!
•Tested against “Ethics Frame”, ad the outcome aligns with
the business ethics?
Model Creation

25. Jan W Veldsink / Grio
CONFIDENTIAL
Model Execution / Use
•The model is applied for the purpose of answering the
question!
•Are personal values or autonomy at stake?
•Permission from user / employee / customer is required
•Human in the loop /Is there a human making the last decision?
•Can the end-user influence the actions
•Safeguards:
• Feedback/ Monitor / Datadrift / Model competence /Model
performance / Ethics performance /Decision performance
Model Use

26. Jan W Veldsink / Grio
CONFIDENTIAL
Destruction / Reuse
•Destruction of the model
• The AI model is completely removed
• Parts of the AI are left for reuse
•Reuse of parts / of the model
• Is the model intended for reuse, for example a language
model?
Destruction
Reuse

27. Jan W
. Veldsink MSc
Artificial Intelligence - Alexa

28. Jan W
. Veldsink MSc
Destruction / Reuse

29. Jan W
. Veldsink MSc
Ethical Frameworks

30. Jan W Veldsink / Grio
CONFIDENTIAL
Bias Assessment
•Demographic blindness: decisions are made using a
limited set of features that are highly uncorrelated with
protected classes, that is, groups of people protected by
laws or policies.
•Demographic parity: outcomes are proportionally equal for
all protected classes.
•Equal opportunity: true-positive rates are equal for each
protected class.
•Equal odds: true-positive and false-positive rates are equal
for each protected class.

31. Jan W Veldsink / Grio
CONFIDENTIAL
Data Assessment
•Is the data for training the model a representative sample?
•Does the data contain any of the “discriminating” data
points
• Race
• Gender
• Religion
• Ethnics
• Nationality
• Societal class
Data

32. Jan W
. Veldsink MSc
What
is (algorithmic) accountability?
• Responsibility (bad predictions) vs.
accountability (accurate, but “unethical”
predictions)
• “With great power comes great responsibility”
• Accountability – being answerable for
managing quality, risks, results, and for
compliance with policies wrt assigned
responsibilities

33. Jan W
. Veldsink MSc
Discrimination Prevention: Why?
ML algorithms are considered to be inherently
objective, or to have no bad intent, but:
• models are as good as the underlying (biased)
data they learn from,
• algorithms may reinforce human prejudices,
• while human decision makers may
discriminate occasionally, algorithms would
discriminate systematically.

34. Jan W
. Veldsink MSc
Why Can Predictive Models Discriminate?
• Labels are “wrong”<=historically biased
decisions
– stereotypes wrt race, ethnicity, gender, age
– economic incentives
• Data is incomplete => omitted variable bias
– leaving out important causal factor(s)
– the model compensates for the missing factor by over-
or underestimating the effect of other factor(s).
• Sampling bias
Note:
• we assume there is no intent to discriminate

35. Jan W
. Veldsink MSc
Discrimination Prevention: How?
Two main questions:
• How to define a good measure for
discrimination?
• How to incorporate constraints into ML
algorithms?
Social-technical, legal-technical and technical
issues

36. Jan W
. Veldsink MSc
Predicting with SensitiveAttributes
Dataset with
Two attributes:
- Anomaly Score
- Gender Attribute
Split 80%
Train
Split 20%
Test
Build Tree model
Target = GENDER
Evaluate model
Test
If Phi > 0.2 then
BIAS!!
Trained
Model(tree)
Data without Bias
attribute GENDER
Anomaly
Detector
Data with Anomaly
Score
Batch
Anomaly Score

37. Jan W
. Veldsink MSc
Discrimination-aware Solutions
• Task: Learn a predictor that
:
• Maximizes accurac
y
• Minimizes discriminatio
n
• Remove sensitive attributes?

38. Jan W Veldsink / Grio
CONFIDENTIAL
AI = Innovation = Systems thinking = Reflection
•Interestingly, Toyota, the most successful automobile company in the world, has no
standardised cost-control system used for centralised control. Toyota measures many
things. They fully comply with regulations in the many countries in which they operate.
But they do not use their cost measurements in ways that most other large corporations
do. They measure for learning rather than controlling, for helping local people see
how they are doing, and for continuous improvement, not for centralised control.

39. Jan W
. Veldsink MSc

40. Jan W
. Veldsink MSc
CONFIDENTIAL
Drs. Jan Veldsink

41. Jan W Veldsink / Grio
CONFIDENTIAL