Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

abstract CWL test: recusively validate #93

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mr-c
Copy link
Contributor

@mr-c mr-c commented Nov 4, 2023

Fixes: #51

Copy link

codecov bot commented Nov 4, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 40.41%. Comparing base (785cd40) to head (b355f82).

❗ Current head b355f82 differs from pull request most recent head 33c7572. Consider uploading reports for the commit 33c7572 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #93   +/-   ##
=======================================
  Coverage   40.41%   40.41%           
=======================================
  Files          48       48           
  Lines        1915     1915           
  Branches      433      433           
=======================================
  Hits          774      774           
  Misses       1025     1025           
  Partials      116      116           
Flag Coverage Δ
unittests 40.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mr-c mr-c marked this pull request as draft November 4, 2023 11:17
@mvdbeek
Copy link
Member

mvdbeek commented Nov 6, 2023

I guess it's "working":

https://github.com/galaxyproject/gxformat2/actions/runs/6754508062/job/18394521652?pr=93#step:5:123:

=================================== FAILURES ===================================
  _____________________________ test_abstract_export _____________________________
  
      def test_abstract_export():
          for name, example in EXAMPLES.items():
              format2, native = _both_formats(example)
  >           _run_example(format2, _examples_path_for(f"{name}_from_format2.cwl"))
  
  tests/test_export_abstract.py:59: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  tests/test_export_abstract.py:143: in _run_example
      loadingContext, uri = recursive_resolve_and_validate_document(
  .tox/py310-unit/lib/python3.10/site-packages/cwltool/load_tool.py:6[55](https://github.com/galaxyproject/gxformat2/actions/runs/6754508062/job/18394521652?pr=93#step:5:57): in recursive_resolve_and_validate_document
      tool = make_tool(uri, loadingContext)
  .tox/py310-unit/lib/python3.10/site-packages/cwltool/load_tool.py:[59](https://github.com/galaxyproject/gxformat2/actions/runs/6754508062/job/18394521652?pr=93#step:5:61)7: in make_tool
      tool = loadingContext.construct_tool_object(processobj, loadingContext)
  .tox/py310-unit/lib/python3.10/site-packages/cwltool/workflow.py:55: in default_make_tool
      return Workflow(toolpath_object, loadingContext)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <cwltool.workflow.Workflow object at 0x7fcd5223d9f0>
  toolpath_object = {'class': 'Workflow', 'cwlVersion': 'v1.2', 'doc': 'Simple workflow that no-op cats a file and then selects 10 random ...tests/examples/abstractcwl/BASIC_WORKFLOW_from_format2.cwl', 'http://commonwl.org/cwltool#original_cwlVersion': 'v1.2'}
  loadingContext = <cwltool.context.LoadingContext object at 0x7fcd51dae2f0>
  
      def __init__(
          self,
          toolpath_object: CommentedMap,
          loadingContext: LoadingContext,
      ) -> None:
          """Initialize this Workflow."""
          super().__init__(toolpath_object, loadingContext)
          self.provenance_object: Optional[ProvenanceProfile] = None
          if loadingContext.research_obj is not None:
              run_uuid: Optional[UUID] = None
              is_main = not loadingContext.prov_obj  # Not yet set
              if is_main:
                  run_uuid = loadingContext.research_obj.ro_uuid
      
              self.provenance_object = ProvenanceProfile(
                  loadingContext.research_obj,
                  full_name=loadingContext.cwl_full_name,
                  host_provenance=loadingContext.host_provenance,
                  user_provenance=loadingContext.user_provenance,
                  orcid=loadingContext.orcid,
                  run_uuid=run_uuid,
                  fsaccess=loadingContext.research_obj.fsaccess,
              )  # inherit RO UUID for main wf run
              # TODO: Is Workflow(..) only called when we are the main workflow?
              self.parent_wf = self.provenance_object
      
          # FIXME: Won't this overwrite prov_obj for nested workflows?
          loadingContext.prov_obj = self.provenance_object
          loadingContext = loadingContext.copy()
          loadingContext.requirements = self.requirements
          loadingContext.hints = self.hints
      
          self.steps: List[WorkflowStep] = []
          validation_errors = []
          for index, step in enumerate(self.tool.get("steps", [])):
              try:
                  self.steps.append(
                      self.make_workflow_step(step, index, loadingContext, loadingContext.prov_obj)
                  )
              except ValidationException as vexc:
                  if _logger.isEnabledFor(logging.DEBUG):
                      _logger.exception("Validation failed at")
                  validation_errors.append(vexc)
      
          if validation_errors:
  >           raise ValidationException("\n".join(str(v) for v in validation_errors))
  E           schema_salad.exceptions.ValidationException: tests/examples/abstractcwl/BASIC_WORKFLOW_from_format2.cwl:22:7: Workflow step output 'out_file1'
  E                                                                            does not correspond to
  E           tests/examples/abstractcwl/BASIC_WORKFLOW_from_format2.cwl:27:7:   tool output (expected '')

@mr-c
Copy link
Contributor Author

mr-c commented Nov 6, 2023

I guess it's "working":

Yeah 😄 but probably the abstract export should be fixed :-P

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CWL export validation not as strict as cwltool's
2 participants