Rename submission.dac to submission.rems

[csc-jm]

Description

Separated this task from the other tasks given in #588 for less chaotic reviewing. So this just refactors the naming scheme of the dac field in the submission object to rems.

Related issues

Partially fixes #588

Type of change

• Breaking change (probably for front-end, considering the endpoint name is altered)

Changes Made

• change the submission schema
• change the PUT endpoint name to /v1/submission/rems
• modify existing integration tests
• edit openapi specs

Testing

• Integration Tests

[blankdots]

looks good to me, but i think we should keep terminology consistent with the source, REMS has workflow and approvers/Handlers make up a DAC

[csc-felipe]

Looks good to me. Well done, finding all the occurrences to be renamed.

After Stefan's comments are resolved, please dismiss the review and request it again. ^^

[csc-jm]

While converting all these DAC namings, I also looked at this but wasn't sure if that dataset key needs to change or stay like that.

[blankdots]

While converting all these DAC namings, I also looked at this but wasn't sure if that dataset key needs to change or stay like that.

🤔 what is the dataset key in the lines you highlighted, could we get more details ?

[csc-felipe]

That's adding the REMS details to a dataset. For each dataset we create a resource, and a catalogue item in REMS. We don't use that information anywhere at the moment, but still, it should be preserved. What line 422 does is that it adds the REMS link that can be used to apply for access to the dataset to the dataset description that will be visible in Etsin.

My answer to @csc-jm would be that dac could be renamed to rems also in that case. I expect that later the frontend will display some information derived from these values.

[csc-felipe]

I do think that writing REMS DAC makes sense, as what we're doing is translating REMS data into DAC. But I don't mind if you prefer to have different wording. The most important imo is that it's consistent, and at least somewhat accurate.

[csc-felipe]

On the other hand, writing some documentation how the publishing process works would be good. I just wonder if now is a good time, or after we make the changes we've been planning.

[blankdots]

That's adding the REMS details to a dataset. For each dataset we create a resource, and a catalogue item in REMS. We don't use that information anywhere at the moment, but still, it should be preserved. What line 422 does is that it adds the REMS link that can be used to apply for access to the dataset to the dataset description that will be visible in Etsin.

hmm i see, so it is added to the: https://github.com/CSCfi/metadata-submitter/blob/develop/metadata_backend/helpers/schemas/ena_dataset.json or https://github.com/CSCfi/metadata-submitter/blob/develop/metadata_backend/helpers/schemas/ena_bpdataset.json this info - but we also have this information in https://github.com/CSCfi/metadata-submitter/blob/refactor/rename-submission-dac/metadata_backend/helpers/schemas/submission.json#L1165 is that correct ?

My question would be what do we do with a submission that has multiple datasets each with their own worflow in REMS because a submission can have multiple datasets (which correspond to a REMS resource) and for each one in rems you need to create a separate catalogue item, however https://github.com/CSCfi/metadata-submitter/blob/refactor/rename-submission-dac/metadata_backend/helpers/schemas/submission.json#L1165 allows for only one object not an array ?

[blankdots]

On the other hand, writing some documentation how the publishing process works would be good. I just wonder if now is a good time, or after we make the changes we've been planning.

yeah, i think this needs to come with some documentation on what represents a REMS DAC and how we map concepts, otherwise this will not go well with further development

[csc-felipe]

but we also have this information in

@blankdots, not really. Most of the information should be the same, but there are values which are specific to the dataset: resource_id and catalogue_item_id. Also the URL which is build from those values.

[csc-felipe]

As for the second question, in a meeting with other people, we decided that a submission has the same workflow / policy / dac for all datasets. That's why it's an object in the submission. Mostly so that the frontend and backend "remembers" what was the chosen one, and it doesn't have to be specified for each dataset.

But then we do need to create new records in REMS for each dataset, so that data ends up in the metadata_object.

[blankdots]

but we also have this information in

@blankdots, not really. Most of the information should be the same, but there are values which are specific to the dataset: resource_id and catalogue_item_id. Also the URL which is build from those values.

then we need to remove these lines:

• https://github.com/CSCfi/metadata-submitter/blob/refactor/rename-submission-dac/metadata_backend/helpers/schemas/submission.json#L1190
• https://github.com/CSCfi/metadata-submitter/blob/refactor/rename-submission-dac/metadata_backend/helpers/schemas/submission.json#L1194

[csc-felipe]

Indeed, there was a bug. The issue was created :) #649

[csc-jm]

Moved this momentarily to WIP because I need to push and test some things. Sorry if it still generated bunch unnecessary emails. I renamed the dac field of a dataset. I almost also fixed the issue #649 opened by @csc-felipe in this PR but wanted to clarify something first so didn't. Before this PR gets more confusing to review, we can maybe merge and follow up with more issues

[blankdots]

i am ok with changes so far.