Enviar pesquisa
Carregar
Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
•
2 gostaram
•
707 visualizações
Sugree Phatanapherom
Seguir
Presentation of academic research with 140-char limit
Leia menos
Leia mais
Tecnologia
Negócios
Denunciar
Compartilhar
Denunciar
Compartilhar
1 de 83
Baixar agora
Baixar para ler offline
Recomendados
Event Scheduling
Event Scheduling
Ayesha Kanwal
Example Solutions for Scheduling and Work Planning
Example Solutions for Scheduling and Work Planning
SIS Group International
Why Average Response Time is not a right measure of your web application's pe...
Why Average Response Time is not a right measure of your web application's pe...
vodQA
Performance testing basics
Performance testing basics
Charu Anand
Mining model for hotel recommendations (Kaggle Challenge)
Mining model for hotel recommendations (Kaggle Challenge)
Arjun Varma
Time advance mehcanism
Time advance mehcanism
Nikhil Sharma
Predictive control 1 introduction
Predictive control 1 introduction
jamestpp
UC4 - One Automation
UC4 - One Automation
k1k2sdad
Mais conteúdo relacionado
Semelhante a Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
Switched Reluctance Motor ( Srm ) Essay
Switched Reluctance Motor ( Srm ) Essay
Sarah Robinson
Data Mining and Analytics
Data Mining and Analytics
Nathaniel Palmer
Scheduling Of A Scheduling Strategy
Scheduling Of A Scheduling Strategy
Christina Ramirez
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
CrimsonPublishersRDMS
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Editor IJCATR
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
genetic paper
genetic paper
Swathi Rampur
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
CSCJournals
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CUO VEERANAN VEERANAN
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CUO VEERANAN VEERANAN
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
ijceronline
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
SIMUL8 Corporation
Parallel Algorithm mid.pdf
Parallel Algorithm mid.pdf
MdShafiqulIslam945669
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
Carasik BPM ECM
Carasik BPM ECM
Bob Carasik
Analytics for Process Excellence
Analytics for Process Excellence
Denis Gagné
Human Opinion Dynamics Based Job Scheduling
Human Opinion Dynamics Based Job Scheduling
Annerys Sanchez
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET Journal
performance
performance
manogallery
Semelhante a Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
(20)
Switched Reluctance Motor ( Srm ) Essay
Switched Reluctance Motor ( Srm ) Essay
Data Mining and Analytics
Data Mining and Analytics
Scheduling Of A Scheduling Strategy
Scheduling Of A Scheduling Strategy
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Workforce Management & BPM Integration
genetic paper
genetic paper
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
Parallel Algorithm mid.pdf
Parallel Algorithm mid.pdf
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Carasik BPM ECM
Carasik BPM ECM
Analytics for Process Excellence
Analytics for Process Excellence
Human Opinion Dynamics Based Job Scheduling
Human Opinion Dynamics Based Job Scheduling
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
performance
performance
Mais de Sugree Phatanapherom
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Sugree Phatanapherom
@sugree and Twitter
@sugree and Twitter
Sugree Phatanapherom
Behind the madness
Behind the madness
Sugree Phatanapherom
drupal.in.th
drupal.in.th
Sugree Phatanapherom
Twitter API and Startup Ideas
Twitter API and Startup Ideas
Sugree Phatanapherom
Readme Read Sugree
Readme Read Sugree
Sugree Phatanapherom
SCMSWeb and Condor-G Demonstration
SCMSWeb and Condor-G Demonstration
Sugree Phatanapherom
Hand-on Resources II: Extending SCMSWeb
Hand-on Resources II: Extending SCMSWeb
Sugree Phatanapherom
Drupal: blog and beyond
Drupal: blog and beyond
Sugree Phatanapherom
The Spirit of Open Source
The Spirit of Open Source
Sugree Phatanapherom
mbpurple - the replacement twitter im
mbpurple - the replacement twitter im
Sugree Phatanapherom
jibjib - ultimate twitter client for your phone
jibjib - ultimate twitter client for your phone
Sugree Phatanapherom
Next Web Application - Brainstorm
Next Web Application - Brainstorm
Sugree Phatanapherom
Optimizing Drupal for Mobile Devices
Optimizing Drupal for Mobile Devices
Sugree Phatanapherom
Call for Students: Google Summer of Code 2008
Call for Students: Google Summer of Code 2008
Sugree Phatanapherom
Twitter Rules
Twitter Rules
Sugree Phatanapherom
Mais de Sugree Phatanapherom
(16)
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
@sugree and Twitter
@sugree and Twitter
Behind the madness
Behind the madness
drupal.in.th
drupal.in.th
Twitter API and Startup Ideas
Twitter API and Startup Ideas
Readme Read Sugree
Readme Read Sugree
SCMSWeb and Condor-G Demonstration
SCMSWeb and Condor-G Demonstration
Hand-on Resources II: Extending SCMSWeb
Hand-on Resources II: Extending SCMSWeb
Drupal: blog and beyond
Drupal: blog and beyond
The Spirit of Open Source
The Spirit of Open Source
mbpurple - the replacement twitter im
mbpurple - the replacement twitter im
jibjib - ultimate twitter client for your phone
jibjib - ultimate twitter client for your phone
Next Web Application - Brainstorm
Next Web Application - Brainstorm
Optimizing Drupal for Mobile Devices
Optimizing Drupal for Mobile Devices
Call for Students: Google Summer of Code 2008
Call for Students: Google Summer of Code 2008
Twitter Rules
Twitter Rules
Último
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
GDSC PJATK
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
Seth Reyes
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
YounusS2
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
Adam Moalla
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
Matsuo Lab
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
Matt Ray
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
Md Hossain Ali
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
dgelyza
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
DianaGray10
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
Mahmoud Rabie
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UbiTrack UK
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
Brian Pichman
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
Adtran
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
SkyPlanner
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
Bachir Benyammi
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
Liveplex
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
Jamie (Taka) Wang
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
infogdgmi
Designing A Time bound resource download URL
Designing A Time bound resource download URL
Runcy Oommen
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
DianaGray10
Último
(20)
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
Designing A Time bound resource download URL
Designing A Time bound resource download URL
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
1.
Automatic Self-Tuning Architecture
for Batch Scheduler on Large Scale Computing System
2.
I am Sugree
Phatanapherom from Kasetsart University.
3.
This research is
a co-work with Asst. Prof. Putchong Uthayopas.
4.
Ready, steady, go.
5.
What is batch
scheduler?
6.
Batch scheduler is
responsible to schedule jobs to execute on resources at the right time.
7.
Why do we
need batch scheduler?
8.
To utilize resources
efficiently.
9.
To finish all
jobs as fast as possible.
10.
To minimize power
consumption.
11.
In general, it
is so called "resource scheduling problem".
12.
Jobs, Resources and
Time time resources
13.
In this research,
main criteria is to minimize cost to run the resources.
14.
Back to the
past, most works focused on improving algorithms.
15.
To simplify the
problem, this research limits scope job characteristics to independent sequential jobs.
16.
In short, a
job contains the one and only one task.
17.
In other words,
job = task.
18.
Scheduling Algorithms Scheduling
On-line Batch RR OLB MET MCT MinMin MaxMin Sufferage XSufferage CMinMin CMaxMin CSufferage
19.
There are on-line
and batch scheduling.
20.
The most simple
algorithm is "Round Robin".
21.
"Opportunistic Load Balancing"
assigns job to the next available machine.
22.
"Minimum Execution Time"
assigns job to the fastest machine.
23.
"Minimum Completion Time"
assigns job to the machine with minimum completion time for that job.
24.
Next are batch
scheduling algorithms.
25.
"MinMin" assigns shortest
job to the fastest machine.
26.
"MaxMin" assign longest
job to the fastest machine.
27.
"Sufferage" is reassignable
MaxMin.
28.
"XSufferage" is Sufferage
with data locality.
29.
CMinMin, CMaxMin and
CSufferage are derivative with costing.
30.
How to verify?
How to evaluate?
31.
The answer is
simulation. Why?
32.
Closed. Controllable. Reproducible.
33.
Simulation is assumption
and modeling.
34.
Grid is a
meta-scheduler and underlying cluster schedulers managing hosts.
35.
Grid Grid Scheduler
Cluster Scheduler Host Cluster Scheduler Cluster Scheduler jobs Host
36.
Interconnection between scheduler
and processors are dedicated.
37.
Network Scheduler Processor
Storage Processor Processor Processor
38.
Job consists of
inputs, outputs and executable.
39.
Job Executable Input
Output Machine
40.
Operations are 2
steps; mapping and scheduling.
41.
Mapping "job" to
"machine".
42.
Schedule "job" to
the exact time.
43.
In short, the
result is generic priority index.
44.
45.
Time ready time
execution time deadline period before deadline time
46.
Cost cumulative cost
cost cost
47.
Experimented based on
GAMESS job log in ThaiGrid to assume a small and a big system and named them, KUGrid and ThaiGrid, respectively.
48.
Makespan and cost
are observed.
49.
Makespan is the
period of time from when the first job submitted to the last job finished.
50.
Price-Performance
51.
Cost
52.
Makespan
53.
Looks great! Any
problems? Yes!
54.
Priority index contains
5 factors. What are the right values?
55.
What are the
factors of those factors?
56.
There are so
many dependencies. Job characteristics. Resource characteristics. User characteristics.
57.
This problem is
so called "Multi-variate Optimization".
58.
Plus, a bit
more complex with evaluation in simulator.
59.
How to solve?
60.
Optimization Architecture Optimizer
Simulator Simulator Simulator Simulator Batch Scheduler Monitoring System Accounting System
61.
Optimization Algorithm?
62.
Particle Swarm Optimization
is selected as the first one to try.
63.
The position of
each particle in n-dimension plane represents solution.
64.
PSO is social
influence in various scopes.
65.
Local, neighbor and
global.
66.
Usually, one trust
oneself, friends and the world, respectively. The level of trust.
67.
PSO
68.
How to fully
automate self-tuning process?
69.
Historical data are
the key.
70.
The quality of
solution depends on optimizer.
71.
Running optimizer longer
may return better solution.
72.
Precision of using
historical data depends on data period and amount of data.
73.
How to use
historical data? Log replay or estimation.
74.
How to maximize
solution quality to near optimal?
75.
Just run more
simulations using the whole grid system to optimize itself at night!
76.
Results? Please accept
my apologize. They are not published yet.
77.
Conclusion.
78.
Flexible algorithms introduce
more adjustable factors.
79.
The factors are
vary from time to time.
80.
In other view,
these algorithms are improved by external optimization periodically.
81.
Particle swarm optimization
is selected to solve multi-variate optimization.
82.
Improve scheduler by
scheduler itself.
83.
Any questions?
Baixar agora