Skip to content

Comments

DAOS ROMIO ADIO driver - P1#1

Open
mchaarawi wants to merge 40 commits intomasterfrom
daos_adio
Open

DAOS ROMIO ADIO driver - P1#1
mchaarawi wants to merge 40 commits intomasterfrom
daos_adio

Conversation

@mchaarawi
Copy link
Owner

No description provided.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- add fsync

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- delete_on_close mode
- bug fixes

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- make fsync for read also to refresh the epoch on the FH

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- fix async bugs and nbytes reporting in status

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- fix bug in non contiguous memory buf processing.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
…ash)

- expose DAOS array object hints to user, although DAOS only takes into account the stripe size for now.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
…les per DAOS containers

- MPI_File_delete will now not work since we don't have punch yet
- remove hints that are not needed anymore

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Use generic implementation of preallocate to DAOS driver instead of no-op.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
	src/mpi/romio/adio/ad_pvfs2/ad_pvfs2_write_list_classic.c
	src/mpi/romio/adio/common/ad_end.c
	src/mpi/romio/adio/common/ad_fstype.c
	src/mpi/romio/adio/common/ad_init.c
	src/mpi/romio/adio/common/ad_write_str_naive.c
	src/mpi/romio/adio/include/adio.h
	src/mpi/romio/adio/include/adioi.h
	src/mpi/romio/adio/include/adioi_fs_proto.h
	src/mpi/romio/mpi-io/read.c
	src/mpi/romio/mpi-io/write.c
	src/mpi/romio/test/hindexed.c
	src/mpi/romio/test/types_with_zeros.c
- use daos murmur hash for filename -> oid.lo
- fix bugs related to merge from mpich master

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- fix error handling issue in file_open
- rebase with latest DAOS

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
add script for DAOS supported tests

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
- fix some build warnings
- rebase with DAOS master

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
…ch (from the same rank).

  This allows overwrites from the same rank.

- On a collective IO, sync the epochs from all ranks and set to the max epoch consumed.

- Check for erros in async IO callback

- Delete now punches the object associated with the file, not delete the container.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
mchaarawi added 10 commits June 4, 2018 19:53
This is still not supported though till we store the correponding
oid for the file in a directory object (root) in the flat container.
Implementation for that is still to be done.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
  the implementation from here.
- update the cart check to d_hash_murmur64 which is what is used.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
tested with mpi4py io tests.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
tested with mpi4py io tests.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
…aos_adio

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	src/mpi/romio/adio/ad_daos/ad_daos.c
- switch to DFS for flat namespace instead of hashing file name to oid
- fix some bugs

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	src/mpi/romio/adio/common/flatten.c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants