13. Visual Intelligence Made Easy
Easily customize your own state-of-the-art computer
vision models that fit to your unique use case. Just bring
a few examples of labeled images and let Custom Vision
do the hard work.
Custom Vision
Azure Cognitive Services
19. Role of Cognitive Services
making it easier to infuseAI
frameworks
TensorFlow KerasPytorch Onnx
ML
Extract intent Detect empty shelvesExtract identity
effort
pioneer
expert
developer
everyone
Pre-built,
customizable
services
face OCR text vision speech translation
QnA
......
use case featuresF1 F2 F3 F4
how deep in the stack?
generic
specialized
technology
21. Video Indexer
Azure Cognitive Services
Capability Add-on
Language identification
Detect spoken language &
support multi language content
Linguistic Transcript
Convert speech to text in 10
languages and allow extensions
Captioning
Create captions in three formats:
vtt, ttml, srt
Two channels processing
Auto detect, balancing, separate
transcript and merge to single
timeline
Noise reduction
Clear up telephony audio or
noisy recordings (based on
Skype filters)
Transcript customization
Fit to Industry, market and
domain specific terms
Speaker enumeration
Map and understand who spoke
when
Speaker statistics
Statistics for speakers speech
ratios in the audio
Visual text recognition
Extract and group text that
appears in video as overlay,
slides or background
Keyword extraction
Find out the keywords discussed
in each segment
Sentiment analysis
Compare levels of positive vs
negative spoken or written
moments over the timeline
Visual content moderation
Detect explicit visuals such
nudity and racy content
Labels identification
Tag objects such as cat, table,
car, ball etc. when they appear
Brand detection
Track brand mentions in speech
or on screen overheads with
option to customize
Celebrity identification
Identify celebrities and see their
biography
Keyframe extraction
Auto detection of stable
keyframes in a movie
Shot detection
Detect when a shot starts/ends
based on visual analysis
Black frame detection
Identification of black frames in
a movie
Audio effects
Identify audio effects such as
clapping, silence, speech
Thumbnail extraction
Automatically extract best face
selection image
Artifacts
Rich next level of details via
artifact files
Inline editing
Make manual fixes for errors
detected
Sub-clipping
Source video is stored once for
multiple playlists of video
segments
Search
Understand the context of
search results
Widgets
Easily embed delightful widgets
of the insights and player
Recommendations
Find more videos with similar
people discussing similar topics
Custom face identification
Customize face identification
model
Face detection
Detect and group faces in the
video
Rest API
Easily integrate with your
application with REST API
Text content moderation
Detect explicit text in audio
Translation
immediate translate of source to
54 languages
Topic inferencing
Identify main topics of the
video
Emotions identification
Detect emotions expressed in
speech vocal signals
23. Speech
Azure Cognitive Services
For optimal result in Speech to Text, customize:
• acoustic models for your use environments, such as vehicles
• field-specific vocabulary and grammar, such as medical or IT
• pronunciation of abbreviations and acronyms, such as "IOU" for "I owe you."
• Speaker Recognition
• Speech to Text
• Text to Speech
24. Language Understanding (LUIS) aims to be the most comprehensive cloud-based service for
conversational understanding, and the easiest to use for developers with no AI expertise:
• extracts intent/action and entities from user utterances
• includes dictionaries, which can be extended
with customer-specific terms
Highlights:
• now easier to integrate with
Azure Bot, Speech and QnA Maker services
Language Understanding
Azure Cognitive Services
27. Extract insights from customer feedback and social network postings.
I had a wonderful trip to Seattle and
enjoyed seeing the Space Needle!
• submit up to 100 calls per minute, each with up to 1,000 documents of up to 5,000 characters each
Links backed by Wikipedia
in select languages
Text Analytics
Azure Cognitive Services
29. translate dynamic content in your mobile, desktop and web apps
automatically detect languages
transliterate into different alphabets
Text Translator –
Translation
Azure Cognitive Services
60+ supported languages: https://docs.microsoft.com/en-us/azure/cognitive-services/Translator/language-support
31. Resources:
• Blog
• Case Study
• Documentation
• Samples
Content Moderator helps businesses manage risks associated with user generated content
(UGC) by using machine-assisted content moderation APIs and a human review tool.
Features include:
Detection of potential adult, racy, and offensive, illegal, and unwanted image & video content
Identification of possible profanity and undesirable text
Built-in human review tool for improving the results
Workflows that allow you to add other API’s and extract more content insights
Customers using Content Moderator technologies include:
Online marketplaces/e-Commerce sites for moderating catalogs and chatbots
Social media/messaging/gaming platforms for moderating user content and digital assets
Enterprises/K-12 solution providers moderating user content and information chatbots
Global content moderation service providers using AI to augment human moderation teams
Content Moderator
Azure Cognitive Services
33. Personalization
Azure Cognitive Services
Personalization enables you to prioritize relevant content and user
experiences, so they adapt to your audience’s preferences in a live,
ongoing learning loop.
Personalization uses the power of Reinforcement Learning to make the whole learning cycle at digital
speed: it builds its own models, determines what experiments are worth trying, and learns from the real
world with a simple reward feedback score that reflects your business goals.
Personal Content
Personal Interactions
Personal Priorities
👍
36. DevOps MLOps
Code testing
Code reproducibility
App deployment
Model retraining
Model validation
Model reproducibility
Model deployment
37. App developer
using Azure DevOps
MLOps with Azure Machine Learning
Build appCollaborate Test app Release app Monitor app
Model reproducibility Model retrainingModel deploymentModel validation
Data scientist using
Azure Machine Learning
38. MLOps with Azure Machine Learning
Code, dataset, and
environment versioning
Model reproducibility Model retrainingModel deploymentModel validation
Build appCollaborate Test app Release app Monitor app
App developer
using Azure DevOps
Data scientist using
Azure Machine Learning
39. MLOps with Azure Machine Learning
Model validation
& profiling
Model reproducibility Model retrainingModel deploymentModel validation
Train model Validate
model
Build appCollaborate Test app Release app Monitor app
App developer
using Azure DevOps
Data scientist using
Azure Machine Learning
40. MLOps with Azure Machine Learning
Model packaging
Simple deployment
across cloud and edge
Model reproducibility Model retrainingModel deploymentModel validation
Train model Validate
model
Deploy
model
Build appCollaborate Test app Release app Monitor app
App developer
using Azure DevOps
Data scientist using
Azure Machine Learning
41. MLOps with Azure Machine Learning
Model
management
& monitoring
Model performance
analysis
Model reproducibility Model retrainingModel deploymentModel validation
Train model Validate
model
Deploy
model
Monitor
model
Retrain model
Build appCollaborate Test app Release app Monitor app
App developer
using Azure DevOps
Data scientist using
Azure Machine Learning
42. MLOps with Azure Machine Learning
Train model Validate
model
Deploy
model
Monitor
model
Retrain model
Model reproducibility Model retrainingModel deploymentModel validation
Build appCollaborate Test app Release app Monitor app
Azure Machine Learning extension for Azure DevOps
App developer
using Azure DevOps
Data scientist using
Azure Machine Learning
43. MLOps with Azure Machine Learning
Model reproducibility Model retrainingModel deploymentModel validation
Train model Validate
model
Deploy
model
Monitor
model
Retrain model
Build appCollaborate Test app Release app Monitor app
Audit trail management and model interpretability
App developer
using Azure DevOps
Data scientist using
Azure Machine Learning
44. Steps to Build a Machine Learning Solution
1
Problem
Framing
2
Get/Prepar
e Data
3
Develop
Model
4
Deploy
Model
5
Evaluate /
Track
Performanc
e
3.1
Analysis/
Metric
definition
3.2
Feature
Engineering
3.3
Model
Training
3.4
Parameter
Tuning
3.5
Evaluation
45. ● Create a model
○ Step 1: Get data
○ Step 2: Prepare the data
○ Step 3: Define features
● Train the model
○ Step 4: Choose and apply a learning algorithm
● Score and test the model
○ Step 5: Predict