Multi-threaded support #83
Comments
osvunikernel
pushed a commit
to osvunikernel/lwext4
that referenced
this issue
Jul 9, 2024
This patch modifies this fork of lwext4 to make is safe to interact with by multiple threads running in OSv. The key assumption is, that OSv VFS layer provides necessary locking around all interactions with lwext4 to guard ext filesystem metadata (i-node table, directory entries, etc) modifications confined to specific vnode. Beyond that, we add necessary locking around 3 key common data structures: - i-node bitmaps in ext4_ialloc.c - data block bitmaps in ext4_balloc.c - metadata blockcache in ext4_bcache.c and related files More specifically following functions are protected with inode_alloc_lock()/unlock() to make sure no two files/directories get assigned same inode number: - ext4_ialloc_alloc_inode() - ext4_ialloc_free_inode() Next, following functions are protected with block_alloc_lock()/unlock() to make sure no two files/directories use same data block: - ext4_balloc_alloc_block() - ext4_balloc_free_block() - ext4_balloc_free_blocks() Finally, these functions in ext4_bcache.c and related source files are protected with bcache_lock()/unlock() to make sure the global metadata block cache access is synchronized: - ext4_bcache_invalidate_lba() in __ext4_balloc_free_block() and __ext4_balloc_free_blocks() - ext4_bcache_find_get(), ext4_block_flush_buf() and ext4_bcache_free() in ext4_block_flush_lba() - ext4_block_get_noread(), ext4_bcache_test_flag() ext4_bcache_free() in ext4_block_get() - ext4_bcache_free() in ext4_block_set() - ext4_block_get_noread() in ext4_trans_block_get_noread() Ref gkostka#83 Ref cloudius-systems/osv#1179 Signed-off-by: Waldemar Kozaczuk <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
First of all, I wanted to congratulate @gkostka and other contributors on this excellent library.
Now, I am considering to integrate
lwext4into the OSv unikernel to addext4support. The only read-write filesystem OSv supports right now is ZFS which on the one hand has many excellent features but it is also quite "heavy" and has other downsides (not very good support on many Linux distributions).In my case, I would be integrating
lwext4with the VFS layer in a similar way this RTOS fork oflwextdoes - https://github.com/RT-Thread-packages/lwext4/blob/master/ports/rtthread/dfs_ext.c#L226-L243.However, I anticipate my integration will not be very performant in the OSv context when multiple application threads interact with the
lwext4layer. As I understand most high-level API functions in theext4.cuse exclusive global lock (seeEXT4_MP_LOCK(mp)andEXT4_MP_UNLOCK(mp)) which in effect enforces sequential processing. For example, no two files can be read, or written, or accessed simultaneously, unless I read the code incorrectly.The OSv's VFS layer locks access at each
vnodelevel so I could try to disableEXT4_MP_LOCKby never callingext4_mount_setup_locks()but I am afraid this will lead to the filesystem corruption. For example, we would need to make sure that thebcache, the inode tables, and the file metadata stay consistent and that no single block is used by two different files when multiple threads try to call variousext4_...functions inext4.c.Could you please advise where in the code should I add relevant thread synchronization primitives to enforce the data integrity of the ext4 filesystem? At least one part that seems obvious is
ext4_bcache, but I would imagine there is logic that allocates new blocks when appending to files or other places where we update inodes or block groups which would have to be guarded as well.Any help would be greatly appreciated.
The text was updated successfully, but these errors were encountered: