IoT and BigData
•
4 gostaram•1,686 visualizações
An introduction to BigData and how it can be used in combination with the internet of things.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (20)
Semelhante a IoT and BigData
Semelhante a IoT and BigData (20)
Mais de Daan Gerits
Último
Último (20)
IoT and BigData
- 2. WhoAmI Entrepreneurial geek Belgian Community Micro-clusters for BigData enthousiasts BigData Services
- 3. Agenda Part 1 : BigData Part 2 : IoT and BigData Part 3 : Discussions
- 5. The Buzz Started by Google ● ● GFS Paper (2003) MapReduce Paper (2004) Storing and processing “Big” Data ~ multiple petabytes
- 6. The Need Large amount of data IOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOI IOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOI IOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOI OIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOI OIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOII IOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOI OOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOIIOIOIOIIIOIOO OOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOO OIOIIIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIO IOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOI OIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOII OIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOII OIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOI OIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOI OIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOIIOIOIOIII OIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOI OOOOIOIIIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOI OIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOI OIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOI OIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOI OIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOI OIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOII OIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOII OIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOI OIOIIIOIOOOOIOIIOIOIIOIIOIOIOIIIOIOOOOI
- 7. The Need OIII IOII IOII OO OIII OIII High Speed at which IOII data becomes available IOII OO OIII OIII IOII IOII OO OIII
- 8. The Need IOII IOII The Different ways data is structured IOII IOII IOII IOII IOIIIOIIIOII IOII IOII IOII IOII IOII IOII IOII IOII IOII IOII IOIIIOII IOII IOII IOII IOII IOII IOII IOII IOII
- 9. Technologies
- 11. Hadoop Platform for Batch Processing Yahoo! Doug Cutting Mike Cafarella Fetching and processing webpages
- 12. Storage: HDFS
- 13. Processing: YARN
- 15. Apache Storm Platform for Realtime Processing Looks like Complex Event Processing (CEP) Backtype Nathan Marz Acquired by Twitter Like “ever-running” MapReduce jobs
- 16. Apache Storm
- 17. Apache Storm
- 18. Apache Storm
- 19. Apache Storm
- 20. Apache Storm
- 21. Myths “BigData is all about storing data” “BigData is only for big companies having lots of data” “There are no BigData use cases in Belgium” “BigData is only being used for analytics”
- 22. Dangers People are hard to find Technologies are “different” Too much technologies You need hardware Incorrect information
- 23. IoT & BigData Part 2
- 24. Iot as datasource Use data from “Things” As raw as possible No matter which format Store everything
- 26. Iot as datasource Adapt the processing flow based on the data flowing through it: - Disable faulty sensors - Take processing shortcuts
- 27. Iot as datasource Control other “Things” based on the data flowing through: - Rain predicted => disable sprinklers - Check-out at work, trip home will take 60 min => start heating in 30 min Would be great to link with a rule-engine (Drools e.g.)
- 28. Iot as datasource Store data for more complex processing: - Pattern or behaviour recognition - Forecasting - Prevention
- 29. Iot as datasource Storage: Every event is a record Every event is immutable Every event has: a source a timestamp a type a value
- 31. Iot as Platform - Idea Quantity > Quality One platform to Store Process Nodes can be Anything with a controller Anything that can store data
- 32. Iot as Platform - Risks Complex Resource Management Exploit Capabilities Complex Data Management Fragmented Data Complex Security Model Obfuscate Data
- 33. Iot as Platform - Gains Keep your friends close, but your data closer. Delegate processing Use resources in the best way possible
- 34. What do you think? Part 3