Make the gem multijob

[safafa]

Description

This PR introduces new changes to make the gem multi-job:

FailingSpecFormatter

• It now generates failures_log_#{ENV.fetch('TEST_ENV_NUMBER', nil)}.yml and "exceptions_log_#{ENV.fetch('TEST_ENV_NUMBER', nil)}.yml" files. meaning that each job will have two YAML files storing failures and exceptions that we must remember to upload as artefacts when setting up the ci.yml.

failing_specs_detector:combine_log rake task

• fetches the generated YAML files and combines all the failures grouped by exceptions in a TXT file failing_specs_detectr_log.txt the task should be run after the spec jobs in a separate job and should download the uploaded artefacts.

The PR also adds .github/workflows ci configuration. the second jobs is an example on how the result will look like in the console after running the rake task

Test

To test this on a project you can follow these steps:

Add the Gem to the Gemfile

 gem "failing_spec_detector", github: "nebulab/failing_spec_detector", branch: "make-the-gem-multijob"

Add this to your Rakefile:

unless Rails.env.staging? || Rails.env.production?
  begin
    spec = Gem::Specification.find_by_name "failing_spec_detector"
    load "#{spec.gem_dir}/lib/tasks/failing_spec_detector/combine_log.rake"
  rescue
  end
end

Configure Github workflows, here's an example:

name: CI
on: push
jobs:
  specs:
    runs-on: ubuntu-latest

    env:
      MATRIX_FILENAME: rspec_${{ matrix.ci_node_index }}.yml

    strategy:
      fail-fast: false
      matrix:
        ci_node_total: [8]
        ci_node_index: [0, 1, 2, 3, 4, 5, 6, 7]

    steps:
     ........... (steps) ......
    - name: Run Specs
      env:
        RAILS_ENV: test
        CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
        CI_NODE_INDEX: ${{ matrix.ci_node_index }}
      run: bundle exec rake ci:specs
# Upload failure and exceptions log files after running the specs
    - name: Upload Failures
      if: ${{ always() }}
      uses: actions/upload-artifact@v3
      with:
        name: Failures_${{ matrix.ci_node_index }}
        path: failures_log_*.yml

    - name: Upload Exceptions
      if: ${{ always() }}
      uses: actions/upload-artifact@v3
      with:
        name: Exceptions_${{ matrix.ci_node_index }}
        path: exceptions_log_*.yml
  # Add the failing spec detector spec
  failing_spec_detector:
    name: Test print logs task
    if: ${{ failure() }}
    needs: [specs]
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2.0
          bundler-cache: true
      # Download Failures and Exceptions artifacts, add as many steps as the ci_node_total of the specs job
      - name: Download Failure_0
        uses: actions/download-artifact@v3
        with:
          name: Failures_0
      - name: Download Failure_1
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_1
      - name: Download Failure_2
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_2
      - name: Download Failure_3
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_3
      - name: Download Failure_4
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_4
      - name: Download Failure_5
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_5
      - name: Download Failure_6
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_6
      - name: Download Failure_7
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Failures_7
      - name: Download Exception_0
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_0
      - name: Download Exception_1
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_1
      - name: Download Exception_2
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_2
      - name: Download Exception_3
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_3
      - name: Download Exception_4
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_4
      - name: Download Exception_5
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_5
      - name: Download Exception_6
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_6
      - name: Download Exception_7
        if: ${{ always() }}
        uses: actions/download-artifact@v3
        with:
          name: Exceptions_7
      - name: Run combine_log task
        run: bundle exec rake failing_specs_detector:combine_log
      - name: Upload log file
        uses: actions/upload-artifact@v3
        with:
           name: failing_specs_log
           path: "failing_specs_detector_log.txt"

Download the log file from https://github.com/**/**/actions/runs/*** check the latest workflow run artifacts the failing_specs_log will be listed there you can download it and check it locally

Possible issues:
When using deprecations_detector alongside this gem, I ran into an issue when downloading the deprecations artifacts the action was also downloading the failures and exceptions logs which caused the deprecations job to fail. Please make sure to use the name attribute when downloading the deprecations. Example:

  - uses: actions/download-artifact@v3
        with:
          name: Deprecations
          path: spec/deprecations