Does FemtoRV32 include mtime and mtimecmp?

[jimmylu890303]

I am currently learning how to port FreeRTOS on FemtoRV, and I am referencing the FreeRTOS port on the srv32 repository. I have successfully run FreeRTOS on FemtoRV, but encountered some issues. Specifically, the vTaskStartScheduler method in FreeRTOS isn't behaving as expected. I suspect this is due to incorrect configuration of configMTIME_BASE_ADDRESS and configMTIMECMP_BASE_ADDRESS. In order to resolve this, I am trying to determine the correct address of mtime on FemtoRV. Unfortunately, I have been unable to find this information on my own, so I am seeking assistance. Thank you.

The code above corresponds to vPortSetupTimerInterrupt in FreeRTOS. I believe that the issue might be related to the vPortSetupTimerInterrupt function not working correctly. Consequently, vTaskStartScheduler is also affected and not functioning as intended.

 void vPortSetupTimerInterrupt( void )
    {
    uint32_t ulCurrentTimeHigh, ulCurrentTimeLow;
    volatile uint32_t * const pulTimeHigh = ( volatile uint32_t * const ) ( ( configMTIME_BASE_ADDRESS ) + 4UL ); /* 8-byte type so high 32-bit word is 4 bytes up. */
    volatile uint32_t * const pulTimeLow = ( volatile uint32_t * const ) ( configMTIME_BASE_ADDRESS );
    volatile uint32_t ulHartId;

        __asm volatile( "csrr %0, mhartid" : "=r"( ulHartId ) );
        pullMachineTimerCompareRegister  = ( volatile uint64_t * ) ( ullMachineTimerCompareRegisterBase + ( ulHartId * sizeof( uint64_t ) ) );
        uint32_t cycles32_;
        uint32_t cycles32h_;
        do
        {
            ulCurrentTimeHigh = *pulTimeHigh;
            ulCurrentTimeLow = *pulTimeLow;
        } while( ulCurrentTimeHigh != *pulTimeHigh );

        ullNextTime = ( uint64_t ) ulCurrentTimeHigh;
        ullNextTime <<= 32ULL; /* High 4-byte word is 32-bits up. */
        ullNextTime |= ( uint64_t ) ulCurrentTimeLow;
        ullNextTime += ( uint64_t ) uxTimerIncrementsForOneTick;
        *pullMachineTimerCompareRegister = ullNextTime;

        /* Prepare the time to use after the next tick interrupt. */
        ullNextTime += ( uint64_t ) uxTimerIncrementsForOneTick;
    }

[Mecrisp]

Great to see a FreeRTOS port for the FemtoRV! The issue you are encountering is that the mtime and mtimecmp CSRs are not implemented in FemtoRV, for timekeeping, only cycles and cyclesh are available:
https://github.com/BrunoLevy/learn-fpga/blob/master/FemtoRV/RTL/PROCESSOR/femtorv32_gracilis.v#L301-L306
But it should not be difficult to add more CSRs. Which of the FemtoRV cores are you using for your porting effort?

[jimmylu890303]

I am using femtorv32_gracilis for my porting.
As far as I know, mtime and mtimecmp are not CSR (Control and Status Register) registers; they are memory-mapped registers, similar to those found in any other memory-mapped device. However, I don't have any idea how to start the implementation.
Do you have any ideas or advice on implementing mtime and mtimecmp?

[jimmylu890303]

I need to develop a module similar to CLINT. Inside that module, I should implement the mtime and mtimecmp registers. Alternatively, I can implement the mtime and mtimecmp registers similar to other CSR registers in femtorv32_gracilis.v.

image ref

[Mecrisp]

I had to read a bit on mtime and mtimecmp myself; sorry for confusing you. These are not CSR registers, but memory mapped peripheral IO registers. As long as this is your only interrupt source, I think you can go just with a custom peripheral that is doing timekeeping. mtime is a 64 bit IO register incrementing every clock cycle (perhaps divided down to microseconds or another useful calibrated unit), and interrupt fires when mtime is greater or equal than mtimecmp. As far as I understand, there are no fixed IO address ranges defined, you can place these registers into the memory map whereever it suits you.

This is a different type of timer used together with the Femtorv32-Gracilis, which automatically generates a stream of interrupts in a fixed interval determined by a reload register, but the idea should be similar:

https://github.com/badgeteam/mch2022-firmware-ice40/blob/master/projects/RISCV-Playground/rtl/riscv_playground.v#L145-L157

For multiple interrupt sources, you have to go for your own interrupt controller. If you do, keep in mind that the current interrupt implementation is pulse sensitive, and will doubly trigger if you do something intended for level sensitive interrupts, but you can change this behaviour as described in here:

#104

[Mecrisp]

I think FreeRTOS support is a very solid enhancement for the tutorial; time to reach out to @BrunoLevy if he is interested in officially adding a FreeRTOS chapter in the learn-fpga coursework along with the necessary logic!

[BrunoLevy]

Hello @jimmylu890303 and @Mecrisp, yes it will be a fantastic addition to the tutorial series !
I think that gracilis (extended with the memory-mapped register plus the interrupt source) has everything needed.

• The first thing to do is of course to get the thing running. How to add a mapped register is explained here. Then wou'll need to wire the interrupt source.
• Then we'll need to write a clear explanation of how the Risc-V priviledged instruction set works. This will require some writing, because I think that the official specification here is very difficult to read:
  • it lists all possible CSRs, whereas we only need to explain a couple of them
  • clarify what are in-processor CSRs and memory-mapped ones (it is not super clear to me !)
  • explain what happens when an interrupt is fired and what happens when one returns from an interrupt
• We may also need to explain the RISC-V interrupt controller specification PLIC . It is unclear to me what is CLINT, what is PLIC etc..., need to read more.

For the tutorial, I'd like to continue with the "step by step incremental modification" approach of episode I, so the "scenario" could be something like (first draft):

• start from the 'quark' obtained at the end of episode I
• add interrupt_request wire, and mstatus, mtvec CSRs. Wire interrupt_request to a physical button. Write a simple example program that does something interesting. For instance, we could have an ascii animation of a bouncing ball, running in an infinite loop, and the interrupt adds a random force to the ball. With two buttons, we could write something like a 'pong' or 'breakout' game.
• add timer interrupt source. Write an example with minimalistic multitasking, demonstrating context swapping (@Mecrisp has it already). For instance, we could have two or three balls bouncing on the screen, each ball has its own thread.
• now an example with both timer interrupt source and buttons: multithreaded pong game, one thread for the ball, one thread for the paddle, one thread for game logic
• run FreeRTOS (maybe a couple of intermediary steps needed, in particular about simulation / verilator etc...)

[jimmylu890303]

Sorry for my late reply. Based on my current understanding, I need to implement a module similar to PLIC or CLINT. This module will interact with the gracilis core.

Therefore, my next steps involve studying the FemtoRV tutorials to gain a clear understanding of the structure.

Once I have a solid understanding, I will proceed to implement the PLIC for my FreeRTOS porting. This involves reading and learning more about PLIC and then incorporating the necessary changes for integration with FreeRTOS.

Thank you for your guidance, and I appreciate any further assistance.

If I come up with any ideas, I'll stay in touch with you,@Mecrisp and @BrunoLevy.

[BrunoLevy]

Yes we'll stay in touch ! Having a clean and easy to follow tutorial on interrupts and the priviledged ISA will be a fantastic addition to the tutorial series. There are different things to learn and to understand, I do not have a super clear idea on how everything works, but writing a tutorial about something is a good way of learning it ! I have created a placeholder for the upcoming tutorial, here, do not hesitate to update it (PR) when you underestand something (or to continue this thread if it is easier).

[jimmylu890303]

As far as I know, the flow of handling interrupts in RISC-V is as follows.

When a trap occurs, the flow of handling interrupts in RISC-V

1. The CPU raises mip.MEIP
   • When mip.MEIP transitions to 1, mie evaluates whether to set mie.MEIE to 1 (determining whether to initiate an interrupt).
2. The CPU records the current value of mepc and update mcause according to mcause trap table.
   • When an exception occurs : the mepc records the instruction address pointed to by the Program Counter, which is the address of the instruction where the exception occurred.
   • When an interrupt occurs : the mepc records the address of the next instruction pointed to by the Program Counter.
3. The CPU fetches mtvec and enters a trap.
4. mstatus.MIE is stored into mstatus.MPIE, and mstatus.MIE is set to 0.
5. The CPU claims the Interrupt ID and enters the Interrupt Service Routine (ISR).
6. The CPU handles the interrupt.
7. The CPU sends a completion signal with the interrupt ID.
8. The CPU executes mret, storing mstatus.MPIE back into mstatus.MIE, and returns to the mepc position.

In the case of gracilis, it only supports mepc, mtvec, mstatus, and mcause.Moreover, the implementation of mstatus and mcause is too simple. Both of them only have one bit, but in the spec, there are 32 bits, and each bit represents a different meaning.

So, the first thing I think we need to do is to implement complete Control and Status Registers (CSRs)

[jimmylu890303]

The second thing I think we need to do is to implement a CLINT module. I have referenced the srv32 project, and CLINT is responsible for pending interrupts to the processor. It has mtime and mtimecmp registers. The provided code is part of the CLINT module in the srv32 project.

output reg              timer_irq,
output reg              sw_irq,
output reg              ex_irq

else if (mtime_nxt >= mtimecmp)
    timer_irq           <= 1'b1;

// MSIP is used to trigger an interrupt. The external interrupt is at D[16],
// defined by srv32 core. This is used for software self-test, it connects
// ex_irq to the interrupt pin at top level of RTL code.
always @(posedge clk or negedge resetb)
begin
    if (!resetb) begin
        sw_irq          <= 1'b0;
        ex_irq          <= 1'b0;
    end else if (wready && waddr == MSIP_BASE) begin
        sw_irq          <= wdata[0];        // software interrupt
        ex_irq          <= wdata[16];       // external interrupt
    end
end

[jimmylu890303]

Third thing I think we need to do is to implement a complete interrupt mechanism. When an interrupt request is pending, the core should perform different operations based on the type of interrupt.

The provided code is part of the RISC-V core in the srv32 project. In the core design, it is evident that different actions are taken based on the type of interrupt. For instance, if a software interrupt (sw_irq) occurs, the mcause register will be updated to INT_MSI. Handling this type of interrupt involves performing specific operations with CSR (Control and Status Register) registers. The code demonstrates the logic and behavior associated with interrupt handling in the RISC-V core within the srv32 project.

localparam  [31: 0] TRAP_INST_ALIGN = 32'h0,        // Instruction address misaligned
                    TRAP_INST_FAIL  = 32'h1,        // Instruction access fault
                    TRAP_INST_ILL   = 32'h2,        // Illegal instruction
                    TRAP_BREAK      = 32'h3,        // Breakpoint
                    TRAP_LD_ALIGN   = 32'h4,        // Load address misaligned
                    TRAP_LD_FAIL    = 32'h5,        // Load access fault
                    TRAP_ST_ALIGN   = 32'h6,        // Store/AMO address misaligned
                    TRAP_ST_FAIL    = 32'h7,        // Store/AMO access fault
                    TRAP_ECALL      = 32'hb,        // Environment call from M-mode
                    INT_USI         = 1<<31|32'h0,  // User software interrupt
                    INT_SSI         = 1<<31|32'h1,  // Supervisor software interrupt
                    INT_MSI         = 1<<31|32'h3,  // Machine software interrupt
                    INT_UTIME       = 1<<31|32'h4,  // User timer interrupt
                    INT_STIME       = 1<<31|32'h5,  // Supervisor timer interrupt
                    INT_MTIME       = 1<<31|32'h7,  // Machine timer interrupt
                    INT_UEI         = 1<<31|32'h8,  // User external interrupt
                    INT_SEI         = 1<<31|32'h9,  // Supervisor external interrupt
                    INT_MEI         = 1<<31|32'hb;  // Machine external interrupt


assign ex_sw_irq  = sw_irq && csr_mstatus[MIE] && csr_mie[MSIE] &&
                    !ex_system_op && !ex_flush;

always @* begin    
    ex_mcause     = 32'h0;
    case(1'b1)
        ex_timer_irq       : ex_mcause = INT_MTIME;
        ex_sw_irq          : ex_mcause = INT_MSI;
        ex_interrupt       : ex_mcause = INT_MEI;
        ex_systemcall      : begin
            case (ex_imm[1:0])
                2'b00: ex_mcause   = TRAP_ECALL;
                2'b01: ex_mcause   = TRAP_BREAK;
                2'b10: ex_mcause   = csr_mcause; // uret, sret, mret
                default: begin
                    `ifndef SYNTHESIS
                    $display("Illegal system call at PC 0x%08x\n", ex_pc);
                    `endif
                    ex_mcause = TRAP_INST_ILL;
                end
            endcase
        end
    endcase
end

// update CSRs
ex_sw_irq : begin
    csr_mcause          <= INT_MSI;
    csr_mepc            <= {ex_ret_pc[31: 1], 1'b0};
    csr_mtval           <= 32'd0; // FIXME
    csr_mstatus[MPIE]   <= csr_mstatus[MIE];
    csr_mstatus[MIE]    <= 1'b0;
    csr_mip[MSIP]       <= 1'b1;

[jimmylu890303]

And then, I believe that porting FreeRTOS on FemtoRV could work correctly.

[Mecrisp]

You are embarking on the route to full support of everything, and who am I to stop you! But before you set sails, I want to give you a hint. You came to FemtoRV in the first place probably because Bruno and me extracted what is necessary to understand, and presented it in a -hopefully- readable and concise way. There are other cores that are much better in full support of everything! First, think of what FreeRTOS absolutely needs. And then how to minimally implement it. Start with that.

As far as I understand, FreeRTOS wants a 64 bit register that counts every clock cycle, and another 64 bit register that compares to that. Interrupt triggers when counter is greater or equal to the compare value.

Now on to how FemtoRV interrupts work. They trigger an interrupt response when they see the interrupt line being high for one clock cycle, even if the CPU already is in an interrupt handler at that moment.

With a cycle counter incrementing every clock cycle, iff the timer is the only interrupt source, you can go with this: If counter and compare are equal, pulse interrupt line. You can directly wire a test for equality into the interrupt wire. If compare register is written and the new value is greater or equal to counter register, pulse interrupt line also. OR both together (and route it through a flipflop because of critical path). This way you get the full "greater or equal" behaviour which FreeRTOS expects with the one-cycle-pulsed interrupt of the FemtoRV.

There you go. Two registers in IO space, and wiring it to the interrupt line. Done. FreeRTOS should work.

For writing the 64 bit compare register atomically, I recommend that you have a look which part is written first, store that in a temporary 32 bit shadow register, and upon writing the other half, the complete 64 bit compare register is updated at once. This way you circumvent triggering interrupts that might occur when writing high and low 32 bits separately.

Similar is possible for read access -- when reading the first half of the counter, freeze the other half in a shadow register in logic and deliver this later.

[jimmylu890303]

Thank you very much. You inspire me a lot. I will continue learning how to implement that with your advice!