1. komponenta

Lecture type	Total
Lectures	15
Exercises	30

* Load is given in academic hour (1 academic hour = 45 minutes)

COURSE AIMS AND OBJECTIVES: To enable students to:
- work with tools for developing complex data pipelines
- develop aggregation frameworks with respect to the level of abstraction towards the system user
- understand the theory of data flows and implement systems for processing data flows in real time
- configure relational and non-relational databases for scalable data storage and reading
- implement object-oriented advanced components for searching relational and non-relational databases
- design and implement data solutions in the cloud.

COURSE DESCRIPTION AND SYLLABUS:
1. Development of data pipelines and aggregation frameworks. Transformations. Types of data pipeline systems. State machines in data pipelines. Workflow Monitoring. Levels of use of processing systems. Contents of auxiliary packages. Levels of parallelization in data processing.
2. Data stream processing. Motivation and basics of data streams. Message systems. Window operations. Triggers and watermarks. Data stream processing systems. Kappa, Lambda and hybrid architectures.
3. Storage of analytical results. Types of aggregates and analysis results. Optimizations when storing large amounts of data. Data search systems. Interfaces for generating images and reports.
4. Cloud data engineering. Fundamentals of cloud engineering. Cloud infrastructure. Cloud security. Implementation of cloud data processing systems.

1. Streaming systems: the what, where, when, and how of large-scale data processing, Akidau, T., Chernyak, S., & Lax, R., O'Reilly Media, Inc., 2018.
2. Designing data-intensive applications: The big ideas behind reliable, scalable, and maintainable systems, Kleppmann, M., O'Reilly Media, Inc., 2017.
3. Big Data, Cloud Computing, and Data Science Engineering (Vol. 844), Lee, R. (Ed.), Berlin/Heidelberg, Germany: Springer, 2019.
4. Data Stream Processing and Analytics, https://vasia.github.io/dspa21/index.html, Boston University, CS 591 K1, 2021.
5. Big Data Processing, https://burcuku.github.io/cse2520-bigdata/, TU Delft, CSE2520, 2022.

Enrollment :
Passed : Advanced database systems

Ostali izborni predmeti - Regular study - Computer Science and Mathematics

Ostali izborni predmeti - Regular study - Computer Science and Mathematics

Ostali izborni predmeti - Regular study - Computer Science and Mathematics

Ostali izborni predmeti - Regular study - Computer Science and Mathematics