vcache #1153

mohrcore · 2021-06-11T19:28:27Z

We need basic vcache functionality to support any external filesystem.

…de_t to v_data

pj1031999

Please resolve conflicts.

(manually resolved conflicts introduced 4ce58f2 (cahirwpz#1104))

mohrcore · 2022-01-04T16:15:05Z

No idea on why the vmem test fails. It strikes me as very bizarre. Is the pool allocator broken, or am I just doing something wrong?

pj1031999 · 2022-01-08T19:24:56Z

@mohrcore value of VF_CACHED in your code is 0. The only place where you are using this flag calls pool allocator (and is always called). When you change VF_CACHED to 1 then your code will be working better.

Splitted the vnode cleanup into two VOPs (VOP_INACTIVE, VOP_RECLAIM). VOP_RECLAIM should be used only for vcached filesystems. Renamed some vcache procedures. vfs_vcache_return (earlier vfs_vcache_put) distinguishes now between detached an attached vnodes.

mohrcore · 2022-01-11T20:03:42Z

@pj1031999 thank you for you comment. That was certainly causing some issues. I've fixed that now.

mohrcore · 2022-01-11T20:17:46Z

Allow me to explain my reasoning behind adding a new VOP:

A vnode use count as I get it represents the number of entities that need direct access to data held by a vnode. A use count greater than zero, therefore implies that the vnode is not recycleable at the moment (ie. can't be on vcache freelist). When the use count drops to zero, the vnode should be moved to a vcache freelist and thus become recycleable. This will be done through a vfs_vcache_return procedure called when a vnode with VF_CACHED flag set gets its use counter dropped to zero. Right before that happens it's be crucial for a filesytem code to take action if a vnode is no longer valid, ie. it's corresponding inode is to be removed from a filesystem (eg. user has deleted a file). The filesystem code should detach any of its data from a vnode to prevent vcache from returning an outdated vnode on request that uses a new inode with the same number as the old one. This action can be done using vfs_vcache_detach procedure, which will mark the node "detached" and will prevent it from being put into any bucket within vcache in vfs_vcache_return. Such action could've been performed inside VOP_RECLAIM that used to be called when vnode usecount dropped to zero.

The problem is that it was the last point at which a filesystem code could do anything in regards to a vnode, before it gets recycled. That meant that any inodes that were allocated by a filesystem needed to be freed in that moment, which in regards to filesystems present on block devices makes the vcache impractical. (I guess alternatively they could be cached internally by a filesystem) A filesystem would need to flush an inode to a block device every time the use count drops to zero and load it back the moment the vnode goes back into use. Thus VOP_RECLAIM is now called only when the vcache system needs to recycle the vnode for use with another filesystem, or when the filesystem chooses to detach its data from a vnode. A vnode is expected to have its usecnt equal to 0 at this point. In VOP_RECLAIM a filesystem is required to ensure that all the resources it has associated to the vnode will be in correct state once the vnode is gone, so they could be attached later to another vnode if needed. For tmpfs it just means a little cleanup work (removing an inode if a file is expected to be removed). For something like ext2 it could mean flushing the inode to its block device (and given that block buffering will most likely require a vnode in order to identify a block, this could also mean flushing and de-buffering the entire data associated with that vnode).

Because it's still necessary to take action the moment vnode use count drops to zero, as described eariler (detaching filesystem data from invalid vnode), the old VOP_RECLAIMbecame VOP_INACTIVE.

According to NetBSD manual:

"VOP_INACTIVE() is called when the kernel is no longer using the vnode."
"VOP_RECLAIM() is called when a vnode is being reused for a different file system."

@cahirwpz what are your thoughts on this?

… two separate procedures to for better control over locks

mohrcore · 2022-01-24T21:32:57Z

vmem test still fails, but I'm unable to reproduce this failure locally. It seems to be some rare heizenbug. I'm unfamiliar with vmem at this point, so I have no clue what is a potential cause (other than a race condidtion I might be overlooking).

mohrcore · 2022-01-25T18:53:41Z

Changing the pool_alloc to kmalloc seems to solve the issue. I re-ran the entire testing pipeline three times with no failures after that small change.

My take is that the pool allocator is broken, or there's some limitation to its use that I'm not aware of. Or the vmem test is broken (I don't get what it does tho). Feel free to call it bs, but I don't see another explanation atm.

pj1031999 · 2022-05-01T19:09:46Z

include/sys/vnode.h

 typedef int vnode_ioctl_t(vnode_t *v, u_long cmd, void *data, file_t *fp);
+typedef int vnode_inactive(vnode_t *v);
 typedef int vnode_reclaim_t(vnode_t *v);


Inconsistent naming schema.

pj1031999 · 2022-05-01T19:14:52Z

sys/kern/tmpfs.c

+                                    .v_inactive = tmpfs_vop_inactive,
                                    .v_reclaim = tmpfs_vop_reclaim,
                                    .v_readlink = tmpfs_vop_readlink,
                                    .v_symlink = tmpfs_vop_symlink,
                                    .v_link = tmpfs_vop_link};


Please add , after last struct member - then shitty clang-format will be happy and it will produce human readable output :)

pj1031999 · 2022-05-01T19:17:47Z

sys/kern/tmpfs.c

-  mtx_lock(&v->tfn_timelock);
+  mtx_lock(&node->tfn_timelock);
  if (atime->tv_sec != VNOVAL)
-    v->tfn_atime = *atime;
+    node->tfn_atime = *atime;
  if (mtime->tv_sec != VNOVAL)
-    v->tfn_mtime = *mtime;
-  mtx_unlock(&v->tfn_timelock);
+    node->tfn_mtime = *mtime;
+  mtx_unlock(&node->tfn_timelock);


WITH_MTX_LOCK

pj1031999 · 2022-05-01T19:20:08Z

sys/kern/vfs_vcache.c

+static vcache_t vcache_hash(mount_t *mp, ino_t ino) {
+  return (((vcache_t)mp->mnt_vnodecovered >> 3) + ino) % VCACHE_BUCKET_CNT;
+}


Try to convince me that it's a good hash function

pj1031999 · 2022-05-01T19:26:15Z

sys/kern/vfs_vcache.c

+vnode_t *vfs_vcache_reborrow(mount_t *mp, ino_t ino) {
+  SCOPED_MTX_LOCK(&vcache_giant_lock);
+
+  return vcache_hashget(mp, ino);
+}


It's the only place that uses vcache_hashget in code. Why they are separated functions?

pj1031999 · 2022-05-01T19:28:56Z

sys/kern/vfs_vcache.c

+  error = vfs_vcache_detach(vn);
+  assert(error == 0);


What is a purpose of this assertion? Why you don't propagate error?

First steps towards vcache

5e3b5a9

mohrcore added the WiP not ready for code review label Jun 11, 2021

mohrcore added 5 commits June 12, 2021 00:30

Some tweaks and fixes in vfs_vcache

e6b0172

Trying to migrate tmpfs use of vnodes to vcache

716b189

Fixed hashing nodes with no mountpoint and tmpfs not binding tmpfs_no…

85f324f

…de_t to v_data

Bugfixes. There's still some work to be done

e129126

Fixed vnode dropping with vcache

6a748e7

mohrcore added review please review this PR and removed WiP not ready for code review labels Aug 8, 2021

pj1031999 requested changes Sep 12, 2021

View reviewed changes

mohrcore and others added 3 commits October 5, 2021 23:03

Merge branch 'master' of https://github.com/cahirwpz/mimiker into vcache

6a07b66

(manually resolved conflicts introduced 4ce58f2 (cahirwpz#1104))

Fixed formatting

d168671

Merge branch 'master' into vcache

b36b8a1

mohrcore added WiP not ready for code review and removed review please review this PR labels Jan 4, 2022

mohrcore added 2 commits January 11, 2022 01:07

Set VF_CACHED value to 1

fd5721d

Merge branch 'master' of https://github.com/cahirwpz/mimiker into vcache

58742a1

mohrcore force-pushed the vcache branch from 9d10f46 to 58742a1 Compare January 11, 2022 00:10

mohrcore added 2 commits January 11, 2022 19:44

Fixed formattign

111e0f9

mohrcore added 3 commits January 17, 2022 17:07

Cleanup and cosmetic changes

488297b

Splitted detaching fs data and removing vnode from vcache bucket into…

6663116

… two separate procedures to for better control over locks

Merge branch 'master' of https://github.com/cahirwpz/mimiker into vcache

f82a87c

mohrcore force-pushed the vcache branch from 11ed5c5 to d17921a Compare January 24, 2022 20:59

Fixed recursive lock in vcache

a6b7923

mohrcore force-pushed the vcache branch from d17921a to a6b7923 Compare January 24, 2022 21:00

Replaced pool allocator with kmalloc

7a745d5

Removed pool.h include

f05feb4

mohrcore added review please review this PR and removed WiP not ready for code review labels Jan 26, 2022

mohrcore mentioned this pull request Jan 26, 2022

Usage of pool allocator can cause vmem test to fail #1208

Closed

mohrcore requested a review from pj1031999 January 26, 2022 18:07

pj1031999 reviewed May 1, 2022

View reviewed changes

cahirwpz removed the review please review this PR label Jun 20, 2022

cahirwpz added the orphaned needs new owner label Oct 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vcache #1153

vcache #1153

mohrcore commented Jun 11, 2021

pj1031999 left a comment

mohrcore commented Jan 4, 2022

pj1031999 commented Jan 8, 2022

mohrcore commented Jan 11, 2022

mohrcore commented Jan 11, 2022

mohrcore commented Jan 24, 2022

mohrcore commented Jan 25, 2022 •

edited

Loading

pj1031999 May 1, 2022

pj1031999 May 1, 2022

pj1031999 May 1, 2022

pj1031999 May 1, 2022

pj1031999 May 1, 2022

pj1031999 May 1, 2022

vcache #1153

Are you sure you want to change the base?

vcache #1153

Conversation

mohrcore commented Jun 11, 2021

pj1031999 left a comment

Choose a reason for hiding this comment

mohrcore commented Jan 4, 2022

pj1031999 commented Jan 8, 2022

mohrcore commented Jan 11, 2022

mohrcore commented Jan 11, 2022

mohrcore commented Jan 24, 2022

mohrcore commented Jan 25, 2022 • edited Loading

pj1031999 May 1, 2022

Choose a reason for hiding this comment

pj1031999 May 1, 2022

Choose a reason for hiding this comment

pj1031999 May 1, 2022

Choose a reason for hiding this comment

pj1031999 May 1, 2022

Choose a reason for hiding this comment

pj1031999 May 1, 2022

Choose a reason for hiding this comment

pj1031999 May 1, 2022

Choose a reason for hiding this comment

mohrcore commented Jan 25, 2022 •

edited

Loading