Fixed multiple TODOs + Added Glossary + PDF styling (#14)

* added code size and image scale

* added exemplary .pdfs

* bookdown and latex settings

* preamle latex styles

* uncomment cover for epub

* txt with necesarry packaes to build in Rstudio

* modified airflow prerequisites

* added airflow config description

* renamed python to Python

* added Jupyterhub to ml workflow tools

* added workspace image

* restructured chapter 1 and 2

* renames infrastructure chapters

* UI to user interface

* mlops to ml platform

* added glossary initial

* run black .

* added initial glossary and acronyms

* added acronyms example

* added seldon in different sections

* added Glossary to end of book

* renamed 9.5 section

* modified gitignore

* removed acronyms and !expr acronymsdown::add_filter()

* updated glossary

* renamed to sbl

* acknowledgement as unnumbered

* comments to preamble

.gitignore

@@ -7,4 +7,23 @@
 ./manuscript/docs/*
 manuscript/docs
 ./manuscript/svg-inkscape/*
-manuscript/svg-inkscape
+manuscript/svg-inkscape
+manuscript/.Rbuildignore
+manuscript/_main.log
+manuscript/_main.log
+manuscript/_main.Rmd
+manuscript/_main.log
+manuscript/_main.aux
+manuscript/_main.fdb_latexmk
+manuscript/_main.fls
+manuscript/_main.idx
+manuscript/_main.log
+manuscript/_main.md
+manuscript/_main.pdf
+manuscript/_main.rds
+manuscript/_main.synctex.gz
+manuscript/_main.tex
+manuscript/_main.toc
+manuscript/renderd79f54ebd616.rds
+manuscript/_main.log
+manuscript/_main.rds

install_packages.txt

@@ -0,0 +1,10 @@
+brew install harfbuzz
+brew install fribidi
+brew install libgit2
+---
+install.packages("usethis", verbose=TRUE)
+---
+brew install libtiff
+brew install libxt
+
+install.packages("formatR")

manuscript/00-Preamble.Rmd

@@ -6,8 +6,15 @@ This project started out of an interest in multiple domains. At first, I was wor
 
 The overall aim is to build and create a MLOps architecture based on Airflow running on AWS EKS. Ideally, this architecture is create using terraform. Model tracking might be done using MLFlow, Data tracking using DVC. Further mentioned might be best practices in software development, CI/CD, Docker, and pipelines. I might also include a small Data Science use case utilizing the Airflow Cluster we built.
 
+The book contains two sections with distinct focuses. The first section comprises Chapters 3 to 6, which consist of tutorials on the specific tools aforementioned. These chapters also serve as prerequisites for the subsequent sections. Among these tutorials, the chapters dedicated to *Airflow* and *MLflow* are oriented towards Data Scientists, providing insights into their usage. The chapters centered around *Kubernetes* and *Terraform* target Data- and MLOps Engineers, offering detailed guidance on deploying and managing these tools.
+
+The second section, comprising Chapters 7 to 9, delves into an exemplary machine learning Platform. This section demands a strong background in engineering due to its complexity. While these chapters cover the essential tools introduced in the previous section, they may not explore certain intricate aspects used like OAuth authentication and networking details in great depth. Moreover, it is crucial to note that the ML Platform example presented is not intended for production deployment, as there should be significant security concerns considered. Instead, its main purpose is to serve as an informative illustration of ML platforms and MLOps engineering principles.
+
+Chapter 1 and 2 serve as an introduction to the domain of MLOps
+
 ### A work in progress {.unlisted .unnumbered}
 
 This project / book / tutorial / whatever this is or will be, startet by explaining the concept of Kubernetes. The plan is to continuously update it by further sections. Since there is no deadline, there is no timeline, and I am also not sure whether there will exist something final to be honest.
 
 This document is written during my journey in the realm of MLOps. It is therefore in a state of continuous development. 
+

manuscript/01-Introduction.Rmd

@@ -1,4 +1,5 @@
-# Introduction
+
+# Introduction {#introduction}
 
 Imagine being able to effortlessly deploy, manage, and monitor your machine learning models with ease. No more headaches from version control issues, data drift, and model performance degradation. That's the power of MLOps. *"MLOps Engineering: Building, Deploying, and Managing Machine Learning Workflows with Airflow and MLflow on Kubernetes"* takes you on a journey through the principles, practices, and platforms of MLOps. You'll learn how to create an end-to-end pipeline for machine learning projects, using cutting-edge tools and techniques like Kubernetes, Terraform, and GitOps, and working with tools to ease your machine learning workflow such as Apache Airflow and MLflow Tracking. Before we begin, let's have a more closer look on what MLOps actually is, what principles it incorporates, and how it distinguished from traditional DevOps.
 

manuscript/01.1-Introduction-Machine_Learning_Workflow.Rmd

@@ -1,27 +1,24 @@
+
 ## Machine Learning Workflow
 
 A machine learning workflow typically involves several stages. These stages are closely related and sometimes overlap as some stages may involve multiple iterations. In the following, the machine learning workflow is broken down to five different stages to make things easier, and give an overview.
 
-![](images/01-Introduction/ml-lifecycle.svg)
+![ML lifecycle](images/01-Introduction/ml-lifecycle.svg){ width=100% }
 
 **1. Data Preparation**
 In the first stage, data used to train a machine learning model is collected, cleaned, and preprocessed. Preprocessing includes tasks to remove missing or duplicate data, normalize data, and split data into a training and testing set.
-
 **2. Model Building** 
 In the second stage, a machine learning model is selected and trained using the prepared data. This includes tasks such as selecting an appropriate algorithm as a machine learning model, training the model, and tuning the model's parameters to improve its performance.
-
 **3. Model Evaluation**
 Afterward, the performance of the trained model is evaluated using the test data set. This includes tasks such as measuring the accuracy and other performance metrics, comparing the performance of different models, and identifying potential issues with the model.
-
 **4. Model Deployment** 
 Finally, the selected and optimized model is deployed to a production environment where it can be used to make predictions on new data. This stage includes tasks like scaling the model to handle large amounts of data, and deploying the model to different environments to be used in different contexts
-
 **5. Model Monitoring and Maintenance**
 It is important to monitor the model performance and update the model as needed, once the model is deployed. This includes tasks such as collecting feedback from the model, monitoring the model's performance metrics, and updating the model as necessary.
 
 Each stage is often handled by the same tool or platform which makes a clear differentiation across stages and tools fairly difficult. Further, some machine learning workflows will not have all the steps, or they might have some variations. A machine learning workflow is thereby not a walk in the park and the actual model code is just a small piece of the work. 
 
-![Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small black box in the middle. The required surrounding infrastructure is vast and complex. (D. Sculley, et al., 2015)](images/01-Introduction/ml-sculley.svg)
+![Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small black box in the middle. The required surrounding infrastructure is vast and complex. (D. Sculley, et al., 2015)](images/01-Introduction/ml-sculley.svg){ width=100% }
 
 Working with and developing machine learning models, monitoring their performance, and continuously retraining it on new data with possible alternative models can be challenging and involves the right tools.
 
@@ -55,8 +52,13 @@ Model Tracking tools are often used at the development and testing stages of the
 
 Model Serving refers to the process of deploying a machine learning model in a production environment, so it can be used to make predictions on new data. This includes tasks such as scaling the model to handle large amounts of data, deploying the model to different environments, and monitoring the performance of the deployed model. Model serving tools are specifically used at the deployment stage of the machine learning workflow and can handle the necessary tasks mentioned beforehand.
 
-There are multiple tools that integrate the funtionality of serving models, each different in its specific use cases, for example *TensorFlow*, *Kubernetes*, *DataRobot*, or also the already mentioned tools *MLflow* and *Airflow*.
+There are multiple tools that integrate the funtionality of serving models, each different in its specific use cases, for example *KF Serve*, *BentoML*, *Seldon*, or also the already mentioned tools like *MLflow*.
+
+#### Integrated development environment
+
+An Integrated Development Environment (IDE) is a software application that provides a comprehensive and unified workspace for software developers and programmers to write, edit, debug, and manage code. It is not solely used for the development of machine learning models but rather for all software, or code related work. It typically combines a text editor with features such as code completion, syntax highlighting, version control integration, debugging tools, and project management functionalities. IDEs streamline the software development process by offering a cohesive environment where developers can efficiently write and test code, navigate through projects, access documentation, and collaborate with team members. 
 
+It is thereby necessary to include an IDE in our list as ML workflow tools as it is a necessary component when developing machine learning models in a collaborative and scalable way. A well known and established tool is *JupyterHub* which allows organizations to provide a shared and collaborative environment for data scientists, researchers, and developers. It enables the deployment of interactive Jupyter notebooks to create, edit, and execute Jupyter notebooks within a centralized infrastructure to develop data driven code. 
 
 ### Developing Machine Learning Models
 

manuscript/01.2-Introduction-MLOps.Rmd

@@ -25,7 +25,7 @@ The goal of DevOps is to automate and streamline the process of building, testin
 
 DevOps focuses on the deployment and management of software in general (or *traditional* sofware), while MLOps focuses specifically on the deployment and management of machine learning models in a production environment. The goal is basically the same as in DevOps, yet deploying a machine learning model. While this is achieved by the same tools and best practices used in DevOps, deploying machine learning models (compared to software) adds a lot of complexity to the process.
 
-![Traditional vs ML software](images/01-Introduction/mlops-vs-devops.svg)
+![Traditional vs ML software](images/01-Introduction/mlops-vs-devops.svg){ width=100% }
 
 Machine learning models are not just lines of code, but also require large amounts of data, and specialized hardware, to function properly. Further, machine learning models and their complex algorithms might need to change when there is a shift in new data. This process of ensuring that machine learning models are accurate and reliable with new data leads to additional challenges. 
 Another key difference is that MLOps places a great emphasis on model governance, which ensures that machine learning models are compliant with relevant regulations and standards. The above list of tools within DevOps can be extended to the following for MLOps.
@@ -42,7 +42,7 @@ It's important to note that the specific tools used in MLOps and DevOps may vary
 
 Incorporating the tools introduced by DevOps and MLOps can extend the machine learning workflow outlined in the previous section, resulting in a complete MLOps lifecycle that covers each stage of the machine learning process while integrating automation practices.
 
-![](images/01-Introduction/mlops-lifecycle.svg)
+![](images/01-Introduction/mlops-lifecycle.svg){ width=100% }
 
 Integrating MLOps into machine learning projects introduces additional complexity into the workflow. Although the development stage can be carried out on a local machine, subsequent stages are typically executed within a cloud platform. Additionally, the transition from one stage to another is automated using tools like CI/CD, which automate testing and deployment.
 

manuscript/01.3-Introduction-Roles_and_Tasks.Rmd

@@ -2,7 +2,7 @@
 
 The MLOps lifecycle typically involves several key roles and responsibilities, each with their own specific tasks and objectives.
 
-![Roles and their operating areas within the MLOps lifecycle](images/01-Introduction/mlops-roles-and-tasks.svg)
+![Roles and their operating areas within the MLOps lifecycle](images/01-Introduction/mlops-roles-and-tasks.svg){ width=100% }
 
 ### Data Engineer