GitHub - xuhuajie990/devops-exercises: Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions

ℹ️ This repo contains questions and exercises on various technical topics, sometimes related to DevOps and SRE

📊 There are currently 2624 exercises and questions

⚠️ You can use these for preparing for an interview but most of the questions and exercises don't represent an actual interview. Please read FAQ page for more details

🛑 If you are interested in pursuing a career as DevOps engineer, learning some of the concepts mentioned here would be useful, but you should know it's not about learning all the topics and technologies mentioned in this repository

📝 You can add more exercises by submitting pull requests :) Read about contribution guidelines here

DevOps	Git	Network	Hardware	Kubernetes
Software Development	Python	Go	Perl	Regex
Cloud	AWS	Azure	Google Cloud Platform	OpenStack
Operating System	Linux	Virtualization	DNS	Shell Scripting
Databases	SQL	Mongo	Testing	Big Data
CI/CD	Certificates	Containers	OpenShift	Storage
Terraform	Puppet	Distributed	Questions you can ask	Ansible
Observability	Prometheus	Circle CI		Grafana
Argo	Soft Skills	Security	System Design
Chaos Engineering	Misc	Elastic	Kafka

Network

In general, what do you need in order to communicate?

A common language (for the two ends to understand)

A way to address who you want to communicate with

A Connection (so the content of the communication can reach the recipients)

What is TCP/IP?

A set of protocols that define how two or more devices can communicate with each other.

To learn more about TCP/IP, read here

What is Ethernet?

Ethernet simply refers to the most common type of Local Area Network (LAN) used today. A LAN—in contrast to a WAN (Wide Area Network), which spans a larger geographical area—is a connected network of computers in a small area, like your office, college campus, or even home.

What is a MAC address? What is it used for?

A MAC address is a unique identification number or code used to identify individual devices on the network.

Packets that are sent on the ethernet are always coming from a MAC address and sent to a MAC address. If a network adapter is receiving a packet, it is comparing the packet’s destination MAC address to the adapter’s own MAC address.

When is this MAC address used?: ff:ff:ff:ff:ff:ff

When a device sends a packet to the broadcast MAC address (FF:FF:FF:FF:FF:FF), it is delivered to all stations on the local network. Ethernet broadcasts are used to resolve IP addresses to MAC addresses (by ARP) at the data link layer.

What is an IP address?

An Internet Protocol address (IP address) is a numerical label assigned to each device connected to a computer network that uses the Internet Protocol for communication.An IP address serves two main functions: host or network interface identification and location addressing.

Explain the subnet mask and give an example

A Subnet mask is a 32-bit number that masks an IP address and divides the IP addresses into network addresses and host addresses. Subnet Mask is made by setting network bits to all "1"s and setting host bits to all "0"s. Within a given network, out of the total usable host addresses, two are always reserved for specific purposes and cannot be allocated to any host. These are the first address, which is reserved as a network address (a.k.a network ID), and the last address used for network broadcast.

Example

What is a private IP address? In which scenarios/system designs, one should use it?

Private IP addresses are assigned to the hosts in the same network to communicate with one another. As the name "private" suggests, the devices having the private IP addresses assigned can't be reached by the devices from any external network. For example, if I am living in a hostel and I want my hostel mates to join the game server I have hosted, I will ask them to join via my server's private IP address, since the network is local to the hostel.

What is a public IP address? In which scenarios/system designs, one should use it?

A public IP address is a public-facing IP address. In the event that you were hosting a game server that you want your friends to join, you will give your friends your public IP address to allow their computers to identify and locate your network and server in order for the connection to take place. One time that you would not need to use a public-facing IP address is in the event that you were playing with friends who were connected to the same network as you, in that case, you would use a private IP address. In order for someone to be able to connect to your server that is located internally, you will have to set up a port forward to tell your router to allow traffic from the public domain into your network and vice versa.

Explain the OSI model. What layers there are? What each layer is responsible for?

Application: user end (HTTP is here)

Presentation: establishes context between application-layer entities (Encryption is here)

Session: establishes, manages, and terminates the connections

Transport: transfers variable-length data sequences from a source to a destination host (TCP & UDP are here)

Network: transfers datagrams from one network to another (IP is here)

Data link: provides a link between two directly connected nodes (MAC is here)

Physical: the electrical and physical spec of the data connection (Bits are here)

You can read more about the OSI model in penguintutor.com

For each of the following determines to which OSI layer it belongs:

Error correction
Packets routing
Cables and electrical signals
MAC address
IP address
Terminate connections
3 way handshake

Error correction - Data link

Packets routing - Network

Cables and electrical signals - Physical

MAC address - Data link

IP address - Network

Terminate connections - Session

3-way handshake - Transport

What delivery schemes are you familiar with?

Unicast: One-to-one communication where there is one sender and one receiver.

Broadcast: Sending a message to everyone in the network. The address ff:ff:ff:ff:ff:ff is used for broadcasting. Two common protocols which use broadcast are ARP and DHCP.

Multicast: Sending a message to a group of subscribers. It can be one-to-many or many-to-many.

What is CSMA/CD? Is it used in modern ethernet networks?

CSMA/CD stands for Carrier Sense Multiple Access / Collision Detection. Its primary focus is to manage access to a shared medium/bus where only one host can transmit at a given point in time.

CSMA/CD algorithm:

Before sending a frame, it checks whether another host is already transmitting a frame.

If no one is transmitting, it starts transmitting the frame.

If two hosts transmit at the same time, we have a collision.

Both hosts stop sending the frame and they send everyone a 'jam signal' notifying everyone that a collision occurred

They are waiting for a random time before sending it again

Once each host waited for a random time, they try to send the frame again and so the cycle starts again

Describe the following network devices and the difference between them:

router
switch
hub

A router, switch, and hub are all network devices used to connect devices in a local area network (LAN). However, each device operates differently and has its specific use cases. Here is a brief description of each device and the differences between them:

Router: a network device that connects multiple network segments together. It operates at the network layer (Layer 3) of the OSI model and uses routing protocols to direct data between networks. Routers use IP addresses to identify devices and route data packets to the correct destination.

Switch: a network device that connects multiple devices on a LAN. It operates at the data link layer (Layer 2) of the OSI model and uses MAC addresses to identify devices and direct data packets to the correct destination. Switches allow devices on the same network to communicate with each other more efficiently and can prevent data collisions that can occur when multiple devices send data simultaneously.

Hub: a network device that connects multiple devices through a single cable and is used to connect multiple devices without segmenting a network. However, unlike a switch, it operates at the physical layer (Layer 1) of the OSI model and simply broadcasts data packets to all devices connected to it, regardless of whether the device is the intended recipient or not. This means that data collisions can occur, and the network's efficiency can suffer as a result. Hubs are generally not used in modern network setups, as switches are more efficient and provide better network performance.

What is a "Collision Domain"?

A collision domain is a network segment in which devices can potentially interfere with each other by attempting to transmit data at the same time. When two devices transmit data at the same time, it can cause a collision, resulting in lost or corrupted data. In a collision domain, all devices share the same bandwidth, and any device can potentially interfere with the transmission of data by other devices.

What is a "Broadcast Domain"?

A broadcast domain is a network segment in which all devices can communicate with each other by sending broadcast messages. A broadcast message is a message that is sent to all devices in a network rather than a specific device. In a broadcast domain, all devices can receive and process broadcast messages, regardless of whether the message was intended for them or not.

three computers connected to a switch. How many collision domains are there? How many broadcast domains?

Three collision domains and one broadcast domain

How does a router work?

A router is a physical or virtual appliance that passes information between two or more packet-switched computer networks. A router inspects a given data packet's destination Internet Protocol address (IP address), calculates the best way for it to reach its destination, and then forwards it accordingly.

What is NAT?

Network Address Translation (NAT) is a process in which one or more local IP addresses are translated into one or more Global IP address and vice versa in order to provide Internet access to the local hosts.

What is a proxy? How does it work? What do we need it for?

A proxy server acts as a gateway between you and the internet. It’s an intermediary server separating end users from the websites they browse.

If you’re using a proxy server, internet traffic flows through the proxy server on its way to the address you requested. The request then comes back through that same proxy server (there are exceptions to this rule), and then the proxy server forwards the data received from the website to you.

Proxy servers provide varying levels of functionality, security, and privacy depending on your use case, needs, or company policy.

What is TCP? How does it work? What is the 3-way handshake?

TCP 3-way handshake or three-way handshake is a process that is used in a TCP/IP network to make a connection between server and client.

A three-way handshake is primarily used to create a TCP socket connection. It works when:

A client node sends an SYN data packet over an IP network to a server on the same or an external network. The objective of this packet is to ask/infer if the server is open for new connections.

The target server must have open ports that can accept and initiate new connections. When the server receives the SYN packet from the client node, it responds and returns a confirmation receipt – the ACK packet or SYN/ACK packet.

The client node receives the SYN/ACK from the server and responds with an ACK packet.

What is round-trip delay or round-trip time?

From wikipedia: "the length of time it takes for a signal to be sent plus the length of time it takes for an acknowledgment of that signal to be received"

Bonus question: what is the RTT of LAN?

How does an SSL handshake work?

SSL handshake is a process that establishes a secure connection between a client and a server.

The client sends a Client Hello message to the server, which includes the client's version of the SSL/TLS protocol, a list of the cryptographic algorithms supported by the client, and a random value.

The server responds with a Server Hello message, which includes the server's version of the SSL/TLS protocol, a random value, and a session ID.

The server sends a Certificate message, which contains the server's certificate.

The server sends a Server Hello Done message, which indicates that the server is done sending messages for the Server Hello phase.

The client sends a Client Key Exchange message, which contains the client's public key.

The client sends a Change Cipher Spec message, which notifies the server that the client is about to send a message encrypted with the new cipher spec.

The client sends an Encrypted Handshake Message, which contains the pre-master secret encrypted with the server's public key.

The server sends a Change Cipher Spec message, which notifies the client that the server is about to send a message encrypted with the new cipher spec.

The server sends an Encrypted Handshake Message, which contains the pre-master secret encrypted with the client's public key.

The client and server can now exchange application data.

What is the difference between TCP and UDP?

TCP establishes a connection between the client and the server to guarantee the order of the packages, on the other hand, UDP does not establish a connection between the client and server and doesn't handle package orders. This makes UDP more lightweight than TCP and a perfect candidate for services like streaming.

Penguintutor.com provides a good explanation.

What TCP/IP protocols are you familiar with?

Explain the "default gateway"

A default gateway serves as an access point or IP router that a networked computer uses to send information to a computer in another network or the internet.

What is ARP? How does it work?

ARP stands for Address Resolution Protocol. When you try to ping an IP address on your local network, say 192.168.1.1, your system has to turn the IP address 192.168.1.1 into a MAC address. This involves using ARP to resolve the address, hence its name.

Systems keep an ARP look-up table where they store information about what IP addresses are associated with what MAC addresses. When trying to send a packet to an IP address, the system will first consult this table to see if it already knows the MAC address. If there is a value cached, ARP is not used.

What is TTL? What does it help to prevent?

TTL (Time to Live) is a value in an IP (Internet Protocol) packet that determines how many hops or routers a packet can travel before it is discarded. Each time a packet is forwarded by a router, the TTL value is decreased by one. When the TTL value reaches zero, the packet is dropped, and an ICMP (Internet Control Message Protocol) message is sent back to the sender indicating that the packet has expired.

TTL is used to prevent packets from circulating indefinitely in the network, which can cause congestion and degrade network performance.

It also helps to prevent packets from being trapped in routing loops, where packets continuously travel between the same set of routers without ever reaching their destination.

In addition, TTL can be used to help detect and prevent IP spoofing attacks, where an attacker attempts to impersonate another device on the network by using a false or fake IP address. By limiting the number of hops that a packet can travel, TTL can help prevent packets from being routed to destinations that are not legitimate.

What is DHCP? How does it work?

It stands for Dynamic Host Configuration Protocol and allocates IP addresses, subnet masks, and gateways to hosts. This is how it works:

A host upon entering a network broadcasts a message in search of a DHCP server (DHCP DISCOVER)

An offer message is sent back by the DHCP server as a packet containing lease time, subnet mask, IP addresses, etc (DHCP OFFER)

Depending on which offer is accepted, the client sends back a reply broadcast letting all DHCP servers know (DHCP REQUEST)

The server sends an acknowledgment (DHCP ACK)

Read more here

Can you have two DHCP servers on the same network? How does it work?

It is possible to have two DHCP servers on the same network, however, it is not recommended, and it is important to configure them carefully to prevent conflicts and configuration problems.

When two DHCP servers are configured on the same network, there is a risk that both servers will assign IP addresses and other network configuration settings to the same device, which can cause conflicts and connectivity issues. Additionally, if the DHCP servers are configured with different network settings or options, devices on the network may receive conflicting or inconsistent configuration settings.

However, in some cases, it may be necessary to have two DHCP servers on the same network, such as in large networks where one DHCP server may not be able to handle all the requests. In such cases, DHCP servers can be configured to serve different IP address ranges or different subnets, so they do not interfere with each other.

What is SSL tunneling? How does it work?

SSL (Secure Sockets Layer) tunneling is a technique used to establish a secure, encrypted connection between two endpoints over an insecure network, such as the Internet. The SSL tunnel is created by encapsulating the traffic within an SSL connection, which provides confidentiality, integrity, and authentication.

Here's how SSL tunneling works:

A client initiates an SSL connection to a server, which involves a handshake process to establish the SSL session.

Once the SSL session is established, the client and server negotiate encryption parameters, such as the encryption algorithm and key length, then exchange digital certificates to authenticate each other.

The client then sends traffic through the SSL tunnel to the server, which decrypts the traffic and forwards it to its destination.

The server sends traffic back through the SSL tunnel to the client, which decrypts the traffic and forwards it to the application.

What is a socket? Where can you see the list of sockets in your system?

A socket is a software endpoint that enables two-way communication between processes over a network. Sockets provide a standardized interface for network communication, allowing applications to send and receive data across a network. To view the list of open sockets on a Linux system: netstat -an

This command displays a list of all open sockets, along with their protocol, local address, foreign address, and state.

What is IPv6? Why should we consider using it if we have IPv4?

IPv6 (Internet Protocol version 6) is the latest version of the Internet Protocol (IP), which is used to identify and communicate with devices on a network. IPv6 addresses are 128-bit addresses and are expressed in hexadecimal notation, such as 2001:0db8:85a3:0000:0000:8a2e:0370:7334.

There are several reasons why we should consider using IPv6 over IPv4:

Address space: IPv4 has a limited address space, which has been exhausted in many parts of the world. IPv6 provides a much larger address space, allowing for trillions of unique IP addresses.

Security: IPv6 includes built-in support for IPsec, which provides end-to-end encryption and authentication for network traffic.

Performance: IPv6 includes features that can help to improve network performance, such as multicast routing, which allows a single packet to be sent to multiple destinations simultaneously.

Simplified network configuration: IPv6 includes features that can simplify network configuration, such as stateless autoconfiguration, which allows devices to automatically configure their own IPv6 addresses without the need for a DHCP server.

Better mobility support: IPv6 includes features that can improve mobility support, such as Mobile IPv6, which allows devices to maintain their IPv6 addresses as they move between different networks.

What is VLAN?

A VLAN (Virtual Local Area Network) is a logical network that groups together a set of devices on a physical network, regardless of their physical location. VLANs are created by configuring network switches to assign a specific VLAN ID to frames sent by devices connected to a specific port or group of ports on the switch.

What is MTU?

MTU stands for Maximum Transmission Unit. It's the size of the largest PDU (protocol Data Unit) that can be sent in a single transaction.

What happens if you send a packet that is bigger than the MTU?

With the IPv4 protocol, the router can fragment the PDU and then send all the fragmented PDU through the transaction.

With IPv6 protocol, it issues an error to the user's computer.

True or False? Ping is using UDP because it doesn't care about reliable connection

False. Ping is actually using ICMP (Internet Control Message Protocol) which is a network protocol used to send diagnostic messages and control messages related to network communication.

What is SDN?

SDN stands for Software-Defined Networking. It is an approach to network management that emphasizes the centralization of network control, enabling administrators to manage network behavior through a software abstraction.

In a traditional network, network devices such as routers, switches, and firewalls are configured and managed individually, using specialized software or command-line interfaces. In contrast, SDN separates the network control plane from the data plane, allowing administrators to manage network behavior through a centralized software controller.

What is ICMP? What is it used for?

ICMP stands for Internet Control Message Protocol. It is a protocol used for diagnostic and control purposes in IP networks. It is a part of the Internet Protocol suite, operating at the network layer.

ICMP messages are used for a variety of purposes, including:

Error reporting: ICMP messages are used to report errors that occur in the network, such as a packet that could not be delivered to its destination.

Ping: ICMP is used to send ping messages, which are used to test whether a host or network is reachable and to measure the round-trip time for packets.

Path MTU discovery: ICMP is used to discover the Maximum Transmission Unit (MTU) of a path, which is the largest packet size that can be transmitted without fragmentation.

Traceroute: ICMP is used by the traceroute utility to trace the path that packets take through the network.

Router discovery: ICMP is used to discover the routers in a network.

What is NAT? How does it work?

NAT stands for Network Address Translation. It’s a way to map multiple local private addresses to a public one before transferring the information. Organizations that want multiple devices to employ a single IP address use NAT, as do most home routers. For example, your computer's private IP could be 192.168.1.100, but your router maps the traffic to its public IP (e.g. 1.1.1.1). Any device on the internet would see the traffic coming from your public IP (1.1.1.1) instead of your private IP (192.168.1.100).

Which port number is used in each of the following protocols?:

SSH
SMTP
HTTP
DNS
HTTPS
FTP
SFTP

SSH - 22

SMTP - 25

HTTP - 80

DNS - 53

HTTPS - 443

FTP - 21

SFTP - 22

Which factors affect network performance?

Several factors can affect network performance, including:

Bandwidth: The available bandwidth of a network connection can significantly impact its performance. Networks with limited bandwidth can experience slow data transfer rates, high latency, and poor responsiveness.

Latency: Latency refers to the delay that occurs when data is transmitted from one point in a network to another. High latency can result in slow network performance, especially for real-time applications like video conferencing and online gaming.

Network congestion: When too many devices are using a network at the same time, network congestion can occur, leading to slow data transfer rates and poor network performance.

Packet loss: Packet loss occurs when packets of data are dropped during transmission. This can result in slower network speeds and lower overall network performance.

Network topology: The physical layout of a network, including the placement of switches, routers, and other network devices, can impact network performance.

Network protocol: Different network protocols have different performance characteristics, which can impact network performance. For example, TCP is a reliable protocol that can guarantee the delivery of data, but it can also result in slower performance due to the overhead required for error checking and retransmission.

Network security: Security measures such as firewalls and encryption can impact network performance, especially if they require significant processing power or introduce additional latency.

Distance: The physical distance between devices on a network can impact network performance, especially for wireless networks where signal strength and interference can affect connectivity and data transfer rates.

What is APIPA?

APIPA is a set of IP addresses that devices are allocated when the main DHCP server is not reachable

What IP range does APIPA use?

APIPA uses the IP range: 169.254.0.1 - 169.254.255.254.

Control Plane and Data Plane

What does "control plane" refer to?

The control plane is a part of the network that decides how to route and forward packets to a different location.

What does "data plane" refer to?

The data plane is a part of the network that actually forwards the data/packets.

What does "management plane" refer to?

It refers to monitoring and management functions.

To which plane (data, control, ...) does creating routing tables belong to?

Control Plane.

Explain Spanning Tree Protocol (STP).

What is link aggregation? Why is it used?

What is Asymmetric Routing? How to deal with it?

What overlay (tunnel) protocols are you familiar with?

What is GRE? How does it work?

What is VXLAN? How does it work?

What is SNAT?

Explain OSPF.

OSPF (Open Shortest Path First) is a routing protocol that can be implemented on various types of routers. In general, OSPF is supported on most modern routers, including those from vendors such as Cisco, Juniper, and Huawei. The protocol is designed to work with IP-based networks, including both IPv4 and IPv6. Also, it uses a hierarchical network design, where routers are grouped into areas, with each area having its own topology map and routing table. This design helps to reduce the amount of routing information that needs to be exchanged between routers and improve network scalability.

The OSPF 4 Types of routers are:

Internal Router

Area Border Routers

Autonomous Systems Boundary Routers

Backbone Routers

Learn more about OSPF router types: https://www.educba.com/ospf-router-types/

What is latency?

Latency is the time taken for information to reach its destination from the source.

What is bandwidth?

Bandwidth is the capacity of a communication channel to measure how much data the latter can handle over a specific time period. More bandwidth would imply more traffic handling and thus more data transfer.

What is throughput?

Throughput refers to the measurement of the real amount of data transferred over a certain period of time across any transmission channel.

When performing a search query, what is more important, latency or throughput? And how to ensure that we manage global infrastructure?

Latency. To have good latency, a search query should be forwarded to the closest data center.

When uploading a video, what is more important, latency or throughput? And how to assure that?

Throughput. To have good throughput, the upload stream should be routed to an underutilized link.

What other considerations (except latency and throughput) are there when forwarding requests?

Keep caches updated (which means the request could be forwarded not to the closest data center)

Explain Spine & Leaf

What is Network Congestion? What can cause it?

Network congestion occurs when there is too much data to transmit on a network and it doesn't have enough capacity to handle the demand.
This can lead to increased latency and packet loss. The causes can be multiple, such as high network usage, large file transfers, malware, hardware issues, or network design problems.
To prevent network congestion, it's important to monitor your network usage and implement strategies to limit or manage the demand.

What can you tell me about the UDP packet format? What about the TCP packet format? How is it different?

What is the exponential backoff algorithm? Where is it used?

Using Hamming code, what would be the code word for the following data word 100111010001101?

00110011110100011101

Give examples of protocols found in the application layer

Hypertext Transfer Protocol (HTTP) - used for the webpages on the internet

Simple Mail Transfer Protocol (SMTP) - email transmission

Telecommunications Network - (TELNET) - terminal emulation to allow a client access to a telnet server

File Transfer Protocol (FTP) - facilitates the transfer of files between any two machines

Domain Name System (DNS) - domain name translation

Dynamic Host Configuration Protocol (DHCP) - allocates IP addresses, subnet masks, and gateways to hosts

Simple Network Management Protocol (SNMP) - gathers data on devices on the network

Give examples of protocols found in the Network Layer

Internet Protocol (IP) - assists in routing packets from one machine to another

Internet Control Message Protocol (ICMP) - lets one know what is going such as error messages and debugging information

What is HSTS?

HTTP Strict Transport Security is a web server directive that informs user agents and web browsers how to handle its connection through a response header sent at the very beginning and back to the browser. This forces connections over HTTPS encryption, disregarding any script's call to load any resource in that domain over HTTP.

Read more [here](https://www.globalsign.com/en/blog/what-is-hsts-and-how-do-i-use-it#:~:text=HTTP%20Strict%20Transport%20Security%20(HSTS,and%20back%20to%20the%20browser.)

Network - Misc

What is the Internet? Is it the same as the World Wide Web?

The internet refers to a network of networks, transferring huge amounts of data around the globe.
The World Wide Web is an application running on millions of servers, on top of the internet, accessed through what is known as the web browser

What is the ISP?

ISP (Internet Service Provider) is the local internet company provider.

Operating System

Operating System Exercises

Name	Topic	Objective & Instructions	Solution	Comments
Fork 101	Fork	Link	Link
Fork 102	Fork	Link	Link

Operating System - Self Assessment

What is an operating system?

From the book "Operating Systems: Three Easy Pieces":

"responsible for making it easy to run programs (even allowing you to seemingly run many at the same time), allowing programs to share memory, enabling programs to interact with devices, and other fun stuff like that".

Operating System - Process

Can you explain what is a process?

A process is a running program. A program is one or more instructions and the program (or process) is executed by the operating system.

If you had to design an API for processes in an operating system, what would this API look like?

It would support the following:

Create - allow to create new processes

Delete - allow to remove/destroy processes

State - allow to check the state of the process, whether it's running, stopped, waiting, etc.

Stop - allow to stop a running process

How a process is created?

The OS is reading program's code and any additional relevant data

Program's code is loaded into the memory or more specifically, into the address space of the process.

Memory is allocated for program's stack (aka run-time stack). The stack also initialized by the OS with data like argv, argc and parameters to main()

Memory is allocated for program's heap which is required for dynamically allocated data like the data structures linked lists and hash tables

I/O initialization tasks are performed, like in Unix/Linux based systems where each process has 3 file descriptors (input, output and error)

OS is running the program, starting from main()

True or False? The loading of the program into the memory is done eagerly (all at once)

False. It was true in the past but today's operating systems perform lazy loading which means only the relevant pieces required for the process to run are loaded first.

What are different states of a process?

Running - it's executing instructions

Ready - it's ready to run but for different reasons it's on hold

Blocked - it's waiting for some operation to complete. For example I/O disk request

What are some reasons for a process to become blocked?

I/O operations (e.g. Reading from a disk)

Waiting for a packet from a network

What is Inter Process Communication (IPC)?

What is "time sharing"?

Even when using a system with one physical CPU, it's possible to allow multiple users to work on it and run programs. This is possible with time sharing where computing resources are shared in a way it seems to the user the system has multiple CPUs but in fact it's simply one CPU shared by applying multiprogramming and multi-tasking.

What is "space sharing"?

Somewhat the opposite of time sharing. While in time sharing a resource is used for a while by one entity and then the same resource can be used by another resource, in space sharing the space is shared by multiple entities but in a way where it's not being transferred between them.
It's used by one entity until this entity decides to get rid of it. Take for example storage. In storage, a file is yours until you decide to delete it.

What component determines which process runs at a given moment in time?

CPU scheduler

Operating System - Memory

What is "virtual memory" and what purpose it serves?

Virtual memory combines your computer's RAM with temporary space on your hard disk. When RAM runs low, virtual memory helps to move data from RAM to a space called a paging file. Moving data to paging file can free up the RAM so your computer can complete its work. In general, the more RAM your computer has, the faster the programs run. https://www.minitool.com/lib/virtual-memory.html

What is demand paging?

What is copy-on-write?

Copy-on-write (COW) is a resource management concept, with the goal to reduce unnecessary copying of information. It is a concept which is implemented for instance within the POSIX fork syscall, which creates a duplicate process of the calling process.
The idea:

If resources are shared between 2 or more entities (for example shared memory segments between 2 processes) the resources don't need to be copied for every entity, but rather every entity has a READ operation access permission on the shared resource. (the shared segments are marked as read-only) (Think of every entity having a pointer to the location of the shared resource which can be dereferenced to read its value)

If one entity would perform a WRITE operation on a shared resource a problem would arise since the resource also would be permanently changed for ALL other entities sharing it. (Think of a process modifying some variables on the stack, or allocatingy some data dynamically on the heap, these changes to the shared resource would also apply for ALL other processes, this is definitely an undesirable behaviour)

As a solution only if a WRITE operation is about to be performed on a shared resource, this resource gets COPIED first and then the changes are applied.

What is a kernel, and what does it do?

The kernel is part of the operating system and is responsible for tasks like:

Allocating memory

Schedule processes

Control CPU

True or False? Some pieces of the code in the kernel are loaded into protected areas of the memory so applications can't overwritten them

True

What is POSIX?

Explain what is Semaphore and what its role in operating systems

What is cache? What is buffer?

Buffer: Reserved place in RAM which is used to hold data for temporary purposes Cache: Cache is usually used when processes reading and writing to the disk to make the process faster by making similar data used by different programs easily accessible.

Virtualization

What is Virtualization?

Virtualization uses software to create an abstraction layer over computer hardware that allows the hardware elements of a single computer—processors, memory, storage and more - to be divided into multiple virtual computers, commonly called virtual machines (VMs).

What is a hypervisor?

Red Hat: "A hypervisor is software that creates and runs virtual machines (VMs). A hypervisor, sometimes called a virtual machine monitor (VMM), isolates the hypervisor operating system and resources from the virtual machines and enables the creation and management of those VMs."

Read more here

What types of hypervisors are there?

Hosted hypervisors and bare-metal hypervisors.

What are the advantages and disadvantges of bare-metal hypervisor over a hosted hypervisor?

Due to having its own drivers and a direct access to hardware components, a baremetal hypervisor will often have better performances along with stability and scalability.

On the other hand, there will probably be some limitation regarding loading (any) drivers so a hosted hypervisor will usually benefit from having a better hardware compatibility.

What types of virtualization are there?

Operating system virtualization Network functions virtualization Desktop virtualization

Is containerization is a type of Virtualization?

Yes, it's a operating-system-level virtualization, where the kernel is shared and allows to use multiple isolated user-spaces instances.

How the introduction of virtual machines changed the industry and the way applications were deployed?

The introduction of virtual machines allowed companies to deploy multiple business applications on the same hardware while each application is separated from each other in secured way, where each is running on its own separate operating system.

Virtual Machines

Do we need virtual machines in the age of containers? Are they still relevant?

Yes, virtual machines are still relevant even in the age of containers. While containers provide a lightweight and portable alternative to virtual machines, they do have certain limitations. Virtual machines still matter because they offer isolation and security, can run different operating systems, and are good for legacy apps. Containers limitations for example are sharing the host kernel.

Prometheus

What is Prometheus? What are some of Prometheus's main features?

Prometheus is a popular open-source systems monitoring and alerting toolkit, originally developed at SoundCloud. It is designed to collect and store time-series data, and to allow for querying and analysis of that data using a powerful query language called PromQL. Prometheus is frequently used to monitor cloud-native applications, microservices, and other modern infrastructure.

Some of the main features of Prometheus include:

1. Data model: Prometheus uses a flexible data model that allows users to organize and label their time-series data in a way that makes sense for their particular use case. Labels are used to identify different dimensions of the data, such as the source of the data or the environment in which it was collected. 2. Pull-based architecture: Prometheus uses a pull-based model to collect data from targets, meaning that the Prometheus server actively queries its targets for metrics data at regular intervals. This architecture is more scalable and reliable than a push-based model, which would require every target to push data to the server. 3. Time-series database: Prometheus stores all of its data in a time-series database, which allows users to perform queries over time ranges and to aggregate and analyze their data in various ways. The database is optimized for write-heavy workloads, and can handle a high volume of data with low latency. 4. Alerting: Prometheus includes a powerful alerting system that allows users to define rules based on their metrics data and to send alerts when certain conditions are met. Alerts can be sent via email, chat, or other channels, and can be customized to include specific details about the problem. 5. Visualization: Prometheus has a built-in graphing and visualization tool, called PromDash, which allows users to create custom dashboards to monitor their systems and applications. PromDash supports a variety of graph types and visualization options, and can be customized using CSS and JavaScript.

Overall, Prometheus is a powerful and flexible tool for monitoring and analyzing systems and applications, and is widely used in the industry for cloud-native monitoring and observability.

In what scenarios it might be better to NOT use Prometheus?

From Prometheus documentation: "if you need 100% accuracy, such as for per-request billing".

Describe Prometheus architecture and components

The Prometheus architecture consists of four major components:

1. Prometheus Server: The Prometheus server is responsible for collecting and storing metrics data. It has a simple built-in storage layer that allows it to store time-series data in a time-ordered database. 2. Client Libraries: Prometheus provides a range of client libraries that enable applications to expose their metrics data in a format that can be ingested by the Prometheus server. These libraries are available for a range of programming languages, including Java, Python, and Go. 3. Exporters: Exporters are software components that expose existing metrics from third-party systems and make them available for ingestion by the Prometheus server. Prometheus provides exporters for a range of popular technologies, including MySQL, PostgreSQL, and Apache. 4. Alertmanager: The Alertmanager component is responsible for processing alerts generated by the Prometheus server. It can handle alerts from multiple sources and provides a range of features for deduplicating, grouping, and routing alerts to appropriate channels.

Overall, the Prometheus architecture is designed to be highly scalable and resilient. The server and client libraries can be deployed in a distributed fashion to support monitoring across large-scale, highly dynamic environments

Can you compare Prometheus to other solutions like InfluxDB for example?

Compared to other monitoring solutions, such as InfluxDB, Prometheus is known for its high performance and scalability. It can handle large volumes of data and can easily be integrated with other tools in the monitoring ecosystem. InfluxDB, on the other hand, is known for its ease of use and simplicity. It has a user-friendly interface and provides easy-to-use APIs for collecting and querying data.

Another popular solution, Nagios, is a more traditional monitoring system that relies on a push-based model for collecting data. Nagios has been around for a long time and is known for its stability and reliability. However, compared to Prometheus, Nagios lacks some of the more advanced features, such as multi-dimensional data model and powerful query language.

Overall, the choice of a monitoring solution depends on the specific needs and requirements of the organization. While Prometheus is a great choice for large-scale monitoring and alerting, InfluxDB may be a better fit for smaller environments that require ease of use and simplicity. Nagios remains a solid choice for organizations that prioritize stability and reliability over advanced features.

What is an Alert?

In Prometheus, an alert is a notification triggered when a specific condition or threshold is met. Alerts can be configured to trigger when certain metrics cross a certain threshold or when specific events occur. Once an alert is triggered, it can be routed to various channels, such as email, pager, or chat, to notify relevant teams or individuals to take appropriate action. Alerts are a critical component of any monitoring system, as they allow teams to proactively detect and respond to issues before they impact users or cause system downtime.

What is an Instance? What is a Job?

In Prometheus, an instance refers to a single target that is being monitored. For example, a single server or service. A job is a set of instances that perform the same function, such as a set of web servers serving the same application. Jobs allow you to define and manage a group of targets together.

In essence, an instance is an individual target that Prometheus collects metrics from, while a job is a collection of similar instances that can be managed as a group.

What core metrics types Prometheus supports?

Prometheus supports several types of metrics, including:
1. Counter: A monotonically increasing value used for tracking counts of events or samples. Examples include the number of requests processed or the total number of errors encountered. 2. Gauge: A value that can go up or down, such as CPU usage or memory usage. Unlike counters, gauge values can be arbitrary, meaning they can go up and down based on changes in the system being monitored. 3. Histogram: A set of observations or events that are divided into buckets based on their value. Histograms help in analyzing the distribution of a metric, such as request latencies or response sizes. 4. Summary: A summary is similar to a histogram, but instead of buckets, it provides a set of quantiles for the observed values. Summaries are useful for monitoring the distribution of request latencies or response sizes over time.

Prometheus also supports various functions and operators for aggregating and manipulating metrics, such as sum, max, min, and rate. These features make it a powerful tool for monitoring and alerting on system metrics.

What is an exporter? What is it used for?

The exporter serves as a bridge between the third-party system or application and Prometheus, making it possible for Prometheus to monitor and collect data from that system or application.
The exporter acts as a server, listening on a specific network port for requests from Prometheus to scrape metrics. It collects metrics from the third-party system or application and transforms them into a format that can be understood by Prometheus. The exporter then exposes these metrics to Prometheus via an HTTP endpoint, making them available for collection and analysis.

Exporters are commonly used to monitor various types of infrastructure components such as databases, web servers, and storage systems. For example, there are exporters available for monitoring popular databases such as MySQL and PostgreSQL, as well as web servers like Apache and Nginx.

Overall, exporters are a critical component of the Prometheus ecosystem, allowing for the monitoring of a wide range of systems and applications, and providing a high degree of flexibility and extensibility to the platform.

Which Prometheus best practices?

Here are three of them:
1. Label carefully: Careful and consistent labeling of metrics is crucial for effective querying and alerting. Labels should be clear, concise, and include all relevant information about the metric. 2. Keep metrics simple: The metrics exposed by exporters should be simple and focus on a single aspect of the system being monitored. This helps avoid confusion and ensures that the metrics are easily understandable by all members of the team. 3. Use alerting sparingly: While alerting is a powerful feature of Prometheus, it should be used sparingly and only for the most critical issues. Setting up too many alerts can lead to alert fatigue and result in important alerts being ignored. It is recommended to set up only the most important alerts and adjust the thresholds over time based on the actual frequency of alerts.

How to get total requests in a given period of time?

To get the total requests in a given period of time using Prometheus, you can use the *sum* function along with the *rate* function. Here is an example query that will give you the total number of requests in the last hour:
sum(rate(http_requests_total[1h]))

In this query, http_requests_total is the name of the metric that tracks the total number of HTTP requests, and the rate function calculates the per-second rate of requests over the last hour. The sum function then adds up all of the requests to give you the total number of requests in the last hour.

You can adjust the time range by changing the duration in the rate function. For example, if you wanted to get the total number of requests in the last day, you could change the function to rate(http_requests_total[1d]).

What HA in Prometheus means?

HA stands for High Availability. This means that the system is designed to be highly reliable and always available, even in the face of failures or other issues. In practice, this typically involves setting up multiple instances of Prometheus and ensuring that they are all synchronized and able to work together seamlessly. This can be achieved through a variety of techniques, such as load balancing, replication, and failover mechanisms. By implementing HA in Prometheus, users can ensure that their monitoring data is always available and up-to-date, even in the face of hardware or software failures, network issues, or other problems that might otherwise cause downtime or data loss.

How do you join two metrics?

In Prometheus, joining two metrics can be achieved using the *join()* function. The *join()* function combines two or more time series based on their label values. It takes two mandatory arguments: *on* and *table*. The on argument specifies the labels to join *on* and the *table* argument specifies the time series to join.
Here's an example of how to join two metrics using the join() function:

sum_series( join( on(service, instance) request_count_total, on(service, instance) error_count_total, ) )

In this example, the join() function combines the request_count_total and error_count_total time series based on their service and instance label values. The sum_series() function then calculates the sum of the resulting time series

How to write a query that returns the value of a label?

To write a query that returns the value of a label in Prometheus, you can use the *label_values* function. The *label_values* function takes two arguments: the name of the label and the name of the metric.
For example, if you have a metric called http_requests_total with a label called method, and you want to return all the values of the method label, you can use the following query:

label_values(http_requests_total, method)

This will return a list of all the values for the method label in the http_requests_total metric. You can then use this list in further queries or to filter your data.

How do you convert cpu_user_seconds to cpu usage in percentage?

To convert *cpu_user_seconds* to CPU usage in percentage, you need to divide it by the total elapsed time and the number of CPU cores, and then multiply by 100. The formula is as follows:
100 * sum(rate(process_cpu_user_seconds_total{job="<job-name>"}[<time-period>])) by (instance) / (<time-period> * <num-cpu-cores>)

Here, is the name of the job you want to query, is the time range you want to query (e.g. 5m, 1h), and is the number of CPU cores on the machine you are querying.

For example, to get the CPU usage in percentage for the last 5 minutes for a job named my-job running on a machine with 4 CPU cores, you can use the following query:

100 * sum(rate(process_cpu_user_seconds_total{job="my-job"}[5m])) by (instance) / (5m * 4)

Go

What are some characteristics of the Go programming language?

Strong and static typing - the type of the variables can't be changed over time and they have to be defined at compile time

Simplicity

Fast compile times

Built-in concurrency

Garbage collected

Platform independent

Compile to standalone binary - anything you need to run your app will be compiled into one binary. Very useful for version management in run-time.

Go also has good community.

What is the difference between var x int = 2 and x := 2?

The result is the same, a variable with the value 2.

With var x int = 2 we are setting the variable type to integer while with x := 2 we are letting Go figure out by itself the type.

True or False? In Go we can redeclare variables and once declared we must use it.

False. We can't redeclare variables but yes, we must used declared variables.

What libraries of Go have you used?

This should be answered based on your usage but some examples are:

fmt - formatted I/O

What is the problem with the following block of code? How to fix it?

func main() {
    var x float32 = 13.5
    var y int
    y = x
}

The following block of code tries to convert the integer 101 to a string but instead we get "e". Why is that? How to fix it?

package main

import "fmt"

func main() {
    var x int = 101
    var y string
    y = string(x)
    fmt.Println(y)
}

It looks what unicode value is set at 101 and uses it for converting the integer to a string. If you want to get "101" you should use the package "strconv" and replace y = string(x) with y = strconv.Itoa(x)

What is wrong with the following code?:

package main

func main() {
    var x = 2
    var y = 3
    const someConst = x + y
}

Constants in Go can only be declared using constant expressions. But x, y and their sum is variable.
const initializer x + y is not a constant