The IN and OUT instructions are used to access ports. These ports control many external devices, including those connected to the PCI bus.
Add a new user program pci to the members list in the root Cargo.toml file, and create
the outline with:
cargo new pciIn pci edit the Cargo.toml file, and add
[dependencies]
euralios_std = { path = "../euralios_std" }and copy build.rs from hello/ to pci/. The pci/src/main.rs file can contain a basic main function for now:
#![no_std]
#![no_main]
use euralios_std::debug_println;
#[no_mangle]
fn main() {
debug_println!("Hello world!");
}To build the executable when we execute “make run”, in the root
makefile we need to add the pci program:
.PHONY: run
run : user/hello user/pci
cargo run --release --bin kernelFor now in kernel/src/main.rs in the kernel_thread_main() function
replace “../../user/hello” with “../../user/pci”. Running
should now print “Hello world!”.
This program first writes to the PCI address port 0xCF8, and
then reads from the data port 0xCFC.
#[no_mangle]
fn main() {
debug_println!("Hello world!");
let device_info: u32;
let device_addr: u32 = 0x8000_0000;
const CONFIG_ADDRESS: u16 = 0x0CF8;
const CONFIG_DATA: u16 = 0x0CFC;
unsafe {
asm!("out dx, eax",
in("dx") CONFIG_ADDRESS,
in("eax") device_addr);
asm!("in eax, dx",
in("dx") CONFIG_DATA,
lateout("eax") device_info);
}
debug_println!("Device: {}", device_info);
}Note that the in and out instructions are unusual in only
using the eax register (or subsets of it) for the data,
and dx for the port number.
When this is run it produces a General Protection Fault (GPF):
The in and out instructions are privileged, with controls
on which processes can use them.
To allow a user program to write to ports we need to change the I/O Privilege Level (IOPL) to 3 (the ring our program is running in). This is set in bits 12 and 13 of RFLAGS, mask 0x3000.
In process.rs, new_user_thread modify the rflags:
context.rflags = 0x0200 + 0x3000;And now our program should run, printing a value for device_info!
It’s probably not a good idea for every program to be able to read and
write to ports because a program could do anything to attached
devices. We need to be able to decide if a process should have IO
privileges by passing an argument to new_user_thread. We could just
add a bool argument, but at the calling site a true or false
isn’t very informative. Rust doesn’t (yet?) have keyword arguments,
but the builder pattern seems to be a common replacement. That seems
too much for just a couple of parameters, so for now we’ll just do the
simple thing and define a struct:
pub struct Params {
pub handles: Vec<Arc<RwLock<Rendezvous>>>,
pub io_privileges: bool
}
pub fn new_user_thread(
bin: &[u8],
params: Params
) -> Result<u64, &'static str> {
...
}(somewhere before drain‘ing handles we need to define let mut
handles = params.handles;). The flags can now be set with
context.rflags = if params.io_privileges {
0x200 + 0x3000 // Interrupt enable + IOPL 3
} else {
0x200 // Interrupt enable
};At the calling site kernel_thread_main() in kernel/src/main.rs:
process::new_user_thread(
include_bytes!("../../user/pci"),
process::Params{
handles: Vec::from([
interrupts::keyboard_rendezvous(),
vga_rz
]),
io_privileges: true
});The user program should now print a device number if io_privileges
is true and cause a General Protection Fault if it’s false.
Device 8086:1237
According to the PCI ID repository this is vendor Intel Corporation, and 8086:1237 is 440FX - 82441FX PMC, the “Natoma” chipset.
There is a lot of information on the PCI bus on the OSDev wiki including tables of the bit offsets, and device classes.
First we can wrap up the configuration code reader with a struct representing a PCI bus location (bus, slot and function):
#[derive(Clone, Copy)]
struct PciLocation {
bus: u16,
slot: u16,
function: u16
}It’s a simple type so it derives Clone and Copy traits so it can be easily copied rather than moved.
We can then put the configuration reading code into a method:
const CONFIG_ADDRESS: u16 = 0xCF8;
const CONFIG_DATA: u16 = 0xCFC;
impl PciLocation {
/// Return PCI bus address
fn address(&self) -> u32 {
0x8000_0000
| ((self.bus as u32) << 16)
| ((self.slot as u32) << 11)
| ((self.function as u32) << 8)
}
fn read_register(&self, register: u8) -> u32 {
let addr = self.address()
| ((register as u32) << 2);
let value: u32;
unsafe {
asm!("out dx, eax",
in("dx") CONFIG_ADDRESS,
in("eax") addr,
options(nomem, nostack));
asm!("in eax, dx",
in("dx") CONFIG_DATA,
lateout("eax") value,
options(nomem, nostack));
}
value
}
}We then need a struct to represent a device which may be attached to a PCI location:
struct Device {
location: PciLocation,
vendor_id: u16, // Identifies the manufacturer of the device
device_id: u16, // Identifies the particular device. Valid IDs are allocated by the vendor
class: u8, // The type of function the device performs
subclass: u8, // The specific function the device performs
prog_if: u8, // register-level programming interface, if any
revision_id: u8 // revision identifier. Valid IDs are allocated by the vendor
}To create a Device struct we can query a PciLocation:
impl PciLocation {
fn get_device(&self) -> Option<Device> {
let reg_0 = self.read_register(0);
if reg_0 == 0xFFFF_FFFF {
return None // No device
}
let vendor_id = (reg_0 & 0xFFFF) as u16;
let device_id = (reg_0 >> 16) as u16;
let reg_2 = self.read_register(2);
let revision_id = (reg_2 & 0xFF) as u8;
let prog_if = ((reg_2 >> 8) & 0xFF) as u8;
let subclass = ((reg_2 >> 16) & 0xFF) as u8;
let class = ((reg_2 >> 24) & 0xFF) as u8;
Some(Device {
location: self.clone(),
vendor_id,
device_id,
class,
subclass,
prog_if,
revision_id
})
}
}We need some way to print these structs. PciLocation is straightforward:
use core::fmt;
impl fmt::Display for PciLocation {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "PCI {:04X}:{:02X}:{:02X}",
self.bus, self.slot, self.function)
}
}To print Device structs we can use:
impl fmt::Display for Device {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{} [{:04X}:{:04X}] {}:{}",
self.location, self.vendor_id, self.device_id,
self.class, self.subclass)
}
}but it would be nice to have a human-readable description of the
device class. There is a table of class codes on OSDev wiki, but we don’t
need the whole table for QEMU. A subset which will do for now is in this
class_str() method:
impl Device {
fn class_str(&self) -> &'static str {
match self.class {
0x0 => match self.subclass {
0 => "Non-VGA-Compatible Unclassified Device",
1 => "VGA-Compatible Unclassified Device",
_ => "Unknown",
},
0x1 => match self.subclass {
0x0 => "SCSI Bus Controller",
0x1 => "IDE Controller",
0x2 => "Floppy Disk Controller",
0x3 => "IPI Bus Controller",
0x4 => "RAID Controller",
0x5 => "ATA Controller",
0x6 => "Serial ATA Controller",
0x7 => "Serial Attached SCSI Controller",
0x8 => "Non-Volatile Memory Controller",
_ => "Mass Storage Controller"
}
0x2 => match self.subclass {
0x0 => "Ethernet Controller",
0x1 => "Token Ring Controller",
0x2 => "FDDI Controller",
0x3 => "ATM Controller",
0x4 => "ISDN Controller",
0x5 => "WorldFip Controller",
0x6 => "PICMG 2.14 Multi Computing Controller",
0x7 => "Infiniband Controller",
0x8 => "Fabric Controller",
_ => "Network Controller"
}
0x3 => match self.subclass {
0x0 => "VGA Compatible Controller",
0x1 => "XGA Controller",
0x2 => "3D Controller (Not VGA-Compatible)",
_ => "Display Controller"
}
0x4 => match self.subclass {
0x0 => "Multimedia Video Controller",
0x1 => "Multimedia Audio Controller",
0x2 => "Computer Telephony Device",
0x3 => "Audio Device",
_ => "Multimedia Controller"
}
0x5 => match self.subclass {
0x0 => "RAM Controller",
0x1 => "Flash Controller",
_ => "Memory Controller"
}
0x6 => match self.subclass {
0x0 => "Host Bridge",
0x1 => "ISA Bridge",
0x2 => "EISA Bridge",
0x3 => "MCA Bridge",
0x4 => "PCI-to-PCI Bridge",
0x5 => "PCMCIA Bridge",
0x6 => "NuBus Bridge",
0x7 => "CardBus Bridge",
0x8 => "RACEway Bridge",
0x9 => "PCI-to-PCI Bridge",
0xA => "InfiniBand-to-PCI Host Bridge",
_ => "Bridge"
}
_ => "Unknown"
}
}
}That allows Device to be formatted as:
impl fmt::Display for Device {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{} [{:04X}:{:04X}] {}",
self.location, self.vendor_id, self.device_id, self.class_str())
}
}Finally we can run a brute force check of all PCI locations, printing the devices:
#[no_mangle]
fn main() {
// Brute force check of all PCI slots
for bus in 0..256 {
for slot in 0..32 {
if let Some(device) = (
PciLocation{bus,
slot,
function:0}).get_device() {
debug_println!("Device {}", device);
}
}
}
}
In addition to the host and ISA bridges, and the VGA-compatible controller (1234:1111, the QEMU Virtual Video Controller), there is a network controller with vendor ID 8086 (Intel) and device ID 100E. This is the ID of the QEMU e1000 network card which is described here on OSDev.
Now that we can scan the PCI bus and find devices, it’s time to make that information available to other processes.
When scanning for devices we need to save the information somewhere we
can use. Fortunately user programs have a heap allocator so we can use
a Vec in pci/src/main.rs:
extern crate alloc;
use alloc::vec::Vec;
#[no_mangle]
fn main() {
let mut devices = Vec::new();
// Brute force check of all PCI slots
for bus in 0..256 {
...
debug_println!("Device {}", device);
devices.push(device);
...
}To make this information available to other processes we can
use an event loop at the end of main():
loop {
match syscalls::receive(0) {
Ok(message) => {
debug_println!("Received message");
},
Err(code) => {
debug_println!("Receive error {}", code);
}
}
}Running this you now should see the list of PCI devices as before, but now pressing a key (which sends a message to handle 0) prints “Received message”.
The PCI program will probably grow to have multiple functions, so we
need a convention to determine how to handle the message.
In the section on messaging we decided that rdi would always be a value
so we can use that to store the message type.
So far we only have one kind of message, a character sent from the
keyboard or sent to the VGA device. We can call that message type 0
and add it to kernel/src/rendezvous.rs and
euralios_std/src/syscalls.rs:
// Standard message types
pub const MESSAGE_TYPE_CHAR: u64 = 0;Then update the messages sent by keyboard_handler_inner() in
kernel/src/interrupts.rs so that the first value (in rdi) is the
message type, and character in the second value (rsi):
Message::Short(MESSAGE_TYPE_CHAR,
character as u64, 0));This message is received in hello/src/main.rs so needs to
be updated to get the character from the second value:
let value = match msg {
Message::Short(_, value, _) => value,
_ => 0
};The message is then received in kernel/src/vga_buffer.rs
listener() function which becomes:
fn listener() {
loop {
let err: u64;
let value: u64;
unsafe {
asm!("mov rax, 3", // sys_receive
"mov rdi, 0", // handle
"syscall",
lateout("rax") err,
lateout("rsi") value,
out("rdi") _,
out("rdx") _);
}
let ch = char::from_u32(value as u32).unwrap();
println!("VGA: {} , {} => {}", err, value, ch);
}
}Eventually we’ll probably have a set of system-wide standard message types, so we can leave low numbers for those, and use numbers above 256 (for example) for types specific to particular programs.
When pci receives a message we can now match the type of the message.
We can add a FIND_DEVICE message type to the standard library in euralios_std/src/message.rs
pub mod pci {
pub const FIND_DEVICE: u64 = 256;
}which other programs can send to find a device:
use euralios_std::message::pci;
fn main() {
...
debug_println!("Received message");
match message {
syscalls::Message::Short(
syscalls::MESSAGE_TYPE_CHAR, ch, _) => {
// A character e.g. from keyboard
debug_println!("Character: {}", ch);
}
syscalls::Message::Short(
pci::FIND_DEVICE, vendor, device) => {
// Find a device with given vendor and device ID
let vendor_id = (vendor & 0xFFFF) as u16;
let device_id = (device & 0xFFFF) as u16;
debug_println!("Finding device [{:04X}:{:04X}]",
vendor_id, device_id);
}
_ => {}
}When the pci program receives a FIND_DEVICE message, it
should look for the device, and return a message back to the
sender. The easiest way to do that is to send a message back to the
same rendezvous (handle 0 in this case). We need two message types: A
message containing a PCI location address (bus, slot, function), and
one for “device not found”:
pub mod pci {
pub const FIND_DEVICE: u64 = 256;
pub const ADDRESS: u64 = 384; // New
pub const NOTFOUND: u64 = 385; // New
}We can find the device by iterating over the devices vector with
a predicate which checks that the vendor and device IDs match those
requested:
if let Some(device) = devices.iter().find(
|&d| d.vendor_id == vendor_id &&
d.device_id == device_id) {
// Found
} else {
// Not found
}If the device is found then we can send the PCI location address back:
syscalls::send(0,
syscalls::Message::Short(
pci::ADDRESS,
device.location.address() as u64,
0));while if it’s not found then we send a different message:
syscalls::send(0,
syscalls::Message::Short(
pci::NOTFOUND,
0xFFFF_FFFF_FFFF_FFFF, 0));To use this interface, another task sends a FIND_DEVICE message to
the rendezvous corresponding to the pci program’s input, and then
waits for a ADDRESS or NOTFOUND message back i.e something like:
syscalls::send(handle,
syscalls::Message::Short(
pci::FIND_DEVICE,
0x8086_100E as u64, 0));
match syscalls::receive(handle) {
syscalls::Message::Short(
pci::ADDRESS, address, _) => {
// Do something with address
},
syscalls::Message::Short(
pci::NOTFOUND, _, _) => {
// Device not found
},
_ => {}
}There are a couple of issues with this interface which we will try to address in the next section:
- It doesn’t ensure that the return message goes to the process (A)
that sent the request: In between process A calling
sendand thepciprocess sending a reply, another process (B) may be scheduled which also sends a message to thepciprocess. We need asend_receivesyscall which only allows the process which received a message to send the next message, preventing another process from jumping in. - The process communicating with the
pciprocess needs a way to get the rendezvous handle.