Altan Çakır
[introductory/intermediate] Introduction to Big Data with Apache Spark
Summary
Apache Spark, open-source cluster-computing framework providing a fast and general engine for large-scale processing, has been one of the exciting technologies in recent years for the big data development. The main idea behind this technology is to provide a memory abstraction, which allows us to efficiently share data across the different stages of a map-reduce job or provide in-memory data sharing. Our lecture starts with a brief introduction to Spark and its Hadoop related ecosystem, and then shows some common techniques – classification, collaborative filtering, and anomaly detection, among others, ETL pipeline with distributed deep learning. If you have an entry-level understanding of machine learning and statistics, and program in Python or Scala, you will find these subjects useful for working on your own big data challenges.
Syllabus
- Introduction to Data Analysis with Apache Spark
- Spark Programming Model with RDD objects and DataFrames
- Running Spark Applications on Hadoop / Cloud-based Cluster Systems
- Spark SQL
- Spark Streaming
- Machine Learning with Spark MLlib/ML
- Deep Learning with Spark
- Advanced Analytics Applications with Spark
- Anaysis of real world applications
References
https://spark.apache.org, Unified Analytics Engine for Big Data.
Advanced Analytics with Spark: Patterns For Learning From Data At Scale, A. Teller, M. Pumperla, M. Malohlava.
Mastering Machine Learning with Apache Spark 2.x, S. Amirgodshi, M. Rajendran, B. Hall, S. Mei.
Pre-requisites
Python, Machine Learning, Distributed Computing.
Short bio
Prof. Altan Çakır has a Ph.D. degree in physics from Karlsruhe Institute of Technology (KIT), Germany. During his Ph.D. he was responsible for a scientific research based on new physics searches in the CMS detector at the Large Hadron Collider (LHC) at European Nuclear Research Laboratory (CERN). Thereafter, he was granted as a post-doctoral research fellow at Deutsches Elektronen-Synchotron (DESY), a national nuclear research center in Hamburg, Germany, where he spent more than 5 years, and then got his present full professor position at Istanbul Technical University (ITU), Istanbul in 2016. Currently, Altan Çakır is Dean of Research at ITU, a group leader of ITU-CMS group at CERN and leads a research project and group at the CMS detector. Furthermore, he was a visiting faculty at Fermi National Accelerator Laboratory (Fermilab), Illinois, USA in 2017. His group’s expertise is focused around machine learning techniques in large scale data analysis. However, their research is very much interdisciplinary, with expertise in the group ranging from science and big data synthesis to economy, industrial applications and operations research. Today, he is consulting various companies worldwide, and sharing his expertise in big data application areas, strategies, skills and competencies based on real-world scenarios.
Prof. Çakır was involved in a large number of high-profile research projects at CERN, DESY, Fermilab and ITU during the last seventeen years. He enjoys being able to integrate his research and teaching key concepts of science and big data technologies. It’s rewarding to be part of the development of the next generation of scientists, engineers and help students move on to careers all over the world, in academia, industry and government. Prof. Çakır is founder of Parton Big Data Analytics and Consulting and co-founder of AI-based advertising intelligence company Adin.AI.
Lectures on big data are periodically given by Prof. Altan Çakır in Big Data and Business Analytics Program (http://bigdatamaster.itu.edu.tr) at Istanbul Technical University: Big Data Technologies and Applications, Machine Learning with Big Data. All in all, Altan Çakır is co-president of AITR – Turkish Artificial Intelligence Platform (aitr.org.tr), an executive member of ITU AI Center (ai.itu.edu.tr) and Turkish Informatics Foundation, and one of the lecturers of Cambridge BigDat 2019, University of Cambridge, United Kingdom.
