Enable and develop acceptance testing

.gitignore

@@ -5,4 +5,4 @@ helm/**/*.lock
 *.tgz
 .vscode
 .env
-html-reports
+reports

package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffc-grants-addvalue-web",
-  "version": "2.38.9",
+  "version": "2.38.10",
   "description": "Web frontend for FTF adding value scheme",
   "homepage": "https://github.com/DEFRA/ffc-grants-addvalue-web",
   "main": "app/index.js",

test/acceptance/.dockerignore

@@ -0,0 +1,7 @@
+node_modules
+reports
+test-output
+.dockerignore
+docker-compose*
+Dockerfile
+README.md

test/acceptance/Dockerfile

@@ -1,7 +1,6 @@
-ARG PARENT_VERSION=1.2.1-node12.18.3
+ARG PARENT_VERSION=2.3.0-node22.4.0
 ARG PORT_DEBUG=9229
 
-# Development
 FROM defradigital/node-development:${PARENT_VERSION}
 ARG PARENT_VERSION
 LABEL uk.gov.defra.ffc.parent-image=defradigital/node-development:${PARENT_VERSION}
@@ -10,4 +9,5 @@ EXPOSE ${PORT_DEBUG}
 COPY --chown=node:node package*.json ./
 RUN npm install
 COPY --chown=node:node . .
-CMD [ "npm", "run", "test" ]
+#CMD [ "npm", "run", "test" ]
+CMD [ "ls" ]

test/acceptance/README.md

@@ -1,204 +1,84 @@
 # Acceptance Tests
+This folder contains the acceptance test project for the Adding Value Grant web app. 
 
-> Future Farming and Countryside Programme - Grants Frontend Acceptance Tests
-
-This folder contains the acceptance tests for the FFC Grants service components.
-
-The framework is (Cucumber)[https://cucumber.io/] and (webdriver.io)[https://webdriver.io/] based, containerised, expandable and based on the actively maintained webdriver.io Cucumber boilerplate project.
-
+The framework used is WebdriverIO with Cucumber and the tests are containerised by default, running against a single browser (Chrome)
+.
 ## Requirements
+- Docker
+- Node
+- npm
 
-- Docker Desktop 2.2.0.3 (42716) or higher
-- Node version 12 (not compatible with v13)
-  - if you have multiple versions of node, run `nvm use 12` to use version 12 for this alone
-
-# Quick start
-
-Docker is used to create a container for each of selenium-hub, chrome-browser and webdriver-cuke.
-* Selenium Hub allows concurrent execution of test cases
-* Chrome Browser is the browser specified in the configuration file `wdio.conf.js` by default
-* Webdriver.io along with Cucumber is this framework that defines the tests.
-
-## How to run the tests
-
-1. Set the root URL for the environment in the environment variable `TEST_ENVIRONMENT_ROOT_URL`
-
-2. If running against localhost, then no need to set `TEST_ENVIRONMENT_ROOT_URL` as it will default to `docker.host.internal:3000`.  Instead make sure the application container is running with `docker-compose up --build` in the root folder of this repository
-
-3. From the directory containing the dockerfile run `docker-compose run --rm wdio-cucumber`. This will run an acceptance test against the FFC-Demo web service.
-
-4. The test reports will be output to `./html-reports`. Note that WSL users need to run `mkdir -m 777 html-reports`. Read more about report configuration in the [rpii/wdio-hmtl-reporter docs](https://github.com/rpii/wdio-html-reporter)
-
-5. Now you are ready to maintain, extend or write your own features in the `./acceptance/features` directory
-
-## How to write a test
-
-Tests are written in [Gherkin syntax](https://cucumber.io/docs/reference)
-that means that you write down what's supposed to happen in a real language. All test files are located in
-`./acceptance/features/*` and have the file ending `.feature`. You will already find some test files in that
-directory. They should demonstrate, how tests could look like. Just create a new file and write your first
-test.
-
-__myFirstTest.feature__
-```gherkin
-Feature:
-    In order to keep my product stable
-    As a developer or product manager
-    I want to make sure that everything works as expected
+## Running tests inside a container (default)
+Docker is used to create containers for both the tests themselves (`wdio-cucumber`) and the Selenium instance of Chrome (`chrome-browser`).
 
-Scenario: Check title of website after search
-    Given I open the url "http://google.com"
-    When I set "WebdriverIO" to the inputfield "#lst-ib"
-    And I press "Enter"
-    Then I expect that the title is "WebdriverIO - Google Search"
-
-Scenario: Another test
-    Given ...
+1. Provide the following environment variables in a `.env` file in this directory:
 
 ```
-
-This test opens the browser and navigates them to google.com to check if the title contains the search
-query after doing a search. As you can see, it is pretty simple and understandable for everyone.
-
-# Using tags
-
-If you want to run only specific tests you can mark your features with tags. These tags will be placed before each feature like so:
-
-```gherkin
-@Tag
-Feature: ...
+TEST_ENVIRONMENT_ROOT_URL
+
+LOGIN_USERNAME
+LOGIN_PASSWORD
+
+SHAREPOINT_TENANT_ID
+SHAREPOINT_CLIENT_ID
+SHAREPOINT_CLIENT_SECRET
+SHAREPOINT_HOSTNAME
+SHAREPOINT_SITE_PATH
+SHAREPOINT_DOCUMENT_LIBRARY
+SHAREPOINT_UPLOAD_FOLDER
+SHAREPOINT_WORKSHEET
 ```
 
-To run only the tests with specific tag(s) use the `--cucumberOpts.tagExpression=` parameter like so:
+2. For ARM architectures, change the image used for Chrome in `docker-compose.yaml`:
 
-```sh
-$ npx wdio wdio.conf.js --cucumberOpts.tagExpression='@Tag or @AnotherTag'
 ```
+  selenium:
+    image: selenium/standalone-chrome
 
-For more tag options please see the [Cucumber.js documentation](https://docs.cucumber.io/tag-expressions/)
-
-# Pending test
+CHANGES TO..
 
-If you have failing or unimplemented tests you can mark them as "Pending" so they will get skipped.
+  selenium:
+    image: seleniarm/standalone-chromium
+```   
 
-```gherkin
-// skip whole feature file
-@Pending
-Feature: ...
+3. If running against `localhost` ensure the application container is running with `docker-compose up --build` from the root folder of this repository.
 
-// only skip a single scenario
-@Pending
-Scenario: ...
-```
+4. From the `/test/acceptance` directory run `docker-compose run --build --rm wdio-cucumber`. This will run all acceptance tests.
 
-# Adding new steps and snippets
+5. HTML reports will be output to `./reports`.
 
-The predefined snippets allow you to do a lot of common things but you might need extra snippets which
-are better aligned with your aims. To do so you will find all step definitions in `./acceptance/steps`. They
-are separated in `given`, `when` and `then`.
+## Running tests outside a container
+To run the tests outside a container:
 
-You define your snippet using regular expressions. This is pretty powerful as it allows you to create complex
-sentences with multiple options. Everything that's within `"([^"]*)?"` gets captured and appended to the
-callback. The last argument is always a callback function that you need to call when your step is done.
-You can access the browser and your WebdriverIO instance with `browser`.
-
-To assert values this boilerplate project comes with a [Chai](http://chaijs.com/) integration.
-
-# Comments
-
-You can add additional descriptive comments in your feature files.
+1. Comment out `hostname` and `port` in `wdio.conf.js`, i.e.:
+```js
+exports.config = {
+    //hostname: 'selenium',
+    //port: 4444,```
+```
+2. Run `npm run test`, this will execute the following script defined in `package.json`:
+```pwsh
+npx wdio run ./wdio.conf.js
+```
 
-```gherkin
-###
-  This is a
-  block comment
-###
-Feature: As a bystander
-    I can watch bottles falling from a wall
-    So that I can be mildly amused
+3. Run a specific test or tests with a tag:
+```pwsh
+npx wdio run ./wdio.conf.js --cucumberOpts.tags=@tag
+```
 
-# This is a single line comment
-Scenario: check if username is present
-    Given I login as "roboter" with password "test123"
-    Then the username "roboter" should be present in the header
+## Running tests in parallel
+Tests can be run in parallel at feature file level by increasing the number of instances available to WebdriverIO in `wdio.conf.js', e.g.:
+```js
+maxInstances: 3,
 ```
 
-# List of predefined steps
-
-Check out all predefined snippets. You can see examples of how these can be used in [the webdriver.io cucumber-boilerplate `sampleSnippets.feature`](https://github.com/webdriverio/cucumber-boilerplate/blob/master/src/features/sampleSnippets.feature).
-
-## Given steps
-
-- `I open the (url|site) "([^"]*)?"` <br>Open a site in the current browser window/tab
-- `the element "([^"]*)?" is( not)* visible` <br>Check the (in)visibility of an element
-- `the element "([^"]*)?" is( not)* enabled` <br>Check if an element is (not) enabled
-- `the element "([^"]*)?" is( not)* selected` <br>Check if an element is (not) selected
-- `the checkbox "([^"]*)?" is( not)* checked` <br>Check if a checkbox is (not) checked
-- `there is (an|no) element "([^"]*)?" on the page` <br>Check if an element (does not) exist
-- `the title is( not)* "([^"]*)?"` <br>Check the title of the current browser window/tab
-- `the element "([^"]*)?" contains( not)* the same text as element "([^"]*)?"` <br>Compare the text of two elements
-- `the (button|element) "([^"]*)?"( not)* contains the text "([^"]*)?"` <br>Check if an element contains the given text
-- `the (button|element) "([^"]*)?"( not)* contains any text` <br>Check if an element does not contain any text
-- `the (button|element) "([^"]*)?" is( not)* empty` <br>Check if an element is empty
-- `the page url is( not)* "([^"]*)?"` <br>Check the url of the current browser window/tab
-- `the( css)* attribute "([^"]*)?" from element "([^"]*)?" is( not)* "([^"]*)?"` <br>Check the value of an element's (css) attribute
-- `the cookie "([^"]*)?" contains( not)* the value "([^"]*)?"` <br>Check the value of a cookie
-- `the cookie "([^"]*)?" does( not)* exist` <br>Check the existence of a cookie
-- `the element "([^"]*)?" is( not)* ([\d]+)px (broad|tall)` <br>Check the width/height of an element
-- `the element "([^"]*)?" is( not)* positioned at ([\d]+)px on the (x|y) axis` <br>Check the position of an element
-- `I have a screen that is ([\d]+) by ([\d]+) pixels` <br>Set the browser size to a given size
-- `I have closed all but the first (window|tab)` <br>Close all but the first browser window/tab
-- `a (alertbox|confirmbox|prompt) is( not)* opened` <br>Check if a modal is opened
-
-## Then steps
-
-- `I expect that the title is( not)* "([^"]*)?"` <br>Check the title of the current browser window/tab
-- `I expect that element "([^"]*)?" does( not)* appear exactly "([^"]*)?" times` <br>Checks that the element is on the page a specific number of times
-- `I expect that element "([^"]*)?" is( not)* visible` <br>Check if a certain element is visible
-- `I expect that element "([^"]*)?" becomes( not)* visible` <br>Check if a certain element becomes visible
-- `I expect that element "([^"]*)?" is( not)* within the viewport` <br>Check if a certain element is within the current viewport
-- `I expect that element "([^"]*)?" does( not)* exist` <br>Check if a certain element exists
-- `I expect that element "([^"]*)?"( not)* contains the same text as element "([^"]*)?"` <br>Compare the text of two elements
-- `I expect that (button|element) "([^"]*)?"( not)* contains the text "([^"]*)?"` <br>Check if an element or input field contains the given text
-- `I expect that (button|element) "([^"]*)?"( not)* contains any text` <br>Check if an element or input field contains any text
-- `I expect that (button|elementelement) "([^"]*)?" is( not)* empty` <br>Check if an element or input field is empty
-- `I expect that the url ( not)* contains "([^"]*)?"` <br>Check if the the URL of the current browser window/tab contains a certain string
-- `I expect that the url is( not)* "([^"]*)?"` <br>Check if the the URL of the current browser window/tab is a certain string
-- `I expect that the path is( not)* "([^"]*)?"` <br>Check if the path of the URL of the current browser window/tab is a certain string
-- `I expect the url to( not)* contain "([^"]*)?"` <br>Check if the URL of the current browser window/tab contains a certain string
-- `I expect that the( css)* attribute "([^"]*)?" from element "([^"]*)?" is( not)* "([^"]*)?"` <br>Check the value of an element's (css) attribute
-- `I expect that checkbox "([^"]*)?" is( not)* checked` <br>Check if a check-box is (not) checked
-- `I expect that element "([^"]*)?" is( not)* selected` <br>Check if an element is (not) selected
-- `I expect that element "([^"]*)?" is( not)* enabled` <br>Check if an element is (not) enabled
-- `I expect that cookie "([^"]*)?"( not)* contains "([^"]*)?"` <br>Check if a cookie with a certain name contains a certain value
-- `I expect that cookie "([^"]*)?"( not)* exists` <br>Check if a cookie with a certain name exist
-- `I expect that element "([^"]*)?" is( not)* ([\d]+)px (broad|tall)` <br>Check the width/height of an element
-- `I expect that element "([^"]*)?" is( not)* positioned at ([\d]+)px on the (x|y) axis` <br>Check the position of an element
-- `I expect that element "([^"]*)?" (has|does not have) the class "([^"]*)?"` <br>Check if an element has a certain class
-- `I expect a new (window|tab) has( not)* been opened` <br>Check if a new window/tab has been opened
-- `I expect the url "([^"]*)?" is opened in a new (tab|window)` <br>Check if a URL is opened in a new browser window/tab
-- `I expect that element "([^"]*)?" is( not)* focused` <br>Check if an element has the focus
-- `I wait on element "([^"]*)?"( for (\d+)ms)*( to( not)* (be checked|be enabled|be selected|be visible|contain a text|contain a value|exist))*` <br>Wait for an element to be checked, enabled, selected, visible, contain a certain value or text or to exist
-- `I expect that a (alertbox|confirmbox|prompt) is( not)* opened` <br>Check if a modal is opened
-- `I expect that a (alertbox|confirmbox|prompt)( not)* contains the text "$text"` <br>Check the text of a modal
-
-## When steps
-
-- `I (click|doubleclick) on the (link|button|element) "([^"]*)?"` <br>(Double)click a link, button or element
-- `I (add|set) "([^"]*)?" to the inputfield "([^"]*)?"` <br>Add or set the content of an input field
-- `I clear the inputfield "([^"]*)?"` <br>Clear an input field
-- `I drag element "([^"]*)?" to element "([^"]*)?"` <br>Drag an element to another element
-- `I submit the form "([^"]*)?"` <br>Submit a form
-- `I pause for (\d+)ms` <br>Pause for a certain number of milliseconds
-- `I set a cookie "([^"]*)?" with the content "([^"]*)?"` <br>Set the content of a cookie with the given name to  the given string
-- `I delete the cookie "([^"]*)?"` <br>Delete the cookie with the given name
-- `I press "([^"]*)?"` <br>Press a given key. You’ll find all supported characters [here](https://w3c.github.io/webdriver/webdriver-spec.html#keyboard-actions). To do that, the value has to correspond to a key from the table.
-- `I (accept|dismiss) the (alertbox|confirmbox|prompt)` <br>Accept or dismiss a modal window
-- `I enter "([^"]*)?" into the prompt` <br>Enter a given text into a modal prompt
-- `I scroll to element "([^"]*)?"` <br>Scroll to a given element
-- `I close the last opened (window|tab)` <br>Close the last opened browser window/tab
-- `I focus the last opened (window|tab)` <br>Focus the last opened browser window/tab
-- `I log in to site with username "([^"]*)?" and password "([^"]*)?"` <br>Login to a site with the given username and password
-- `I select the (\d+)(st|nd|rd|th) option for element "([^"]*)?"` <br>Select an option based on it's index
-- `I select the option with the (name|value|text) "([^"]*)?" for element "([^"]*)?"` <br>Select an option based on its name, value or visible text
-- `I move to element "([^"]*)?"( with an offset of (\d+),(\d+))` <br>Move the mouse by an (optional) offset of the specified element
+## Running tests against multiple browsers
+Tests can be run against multiple browsers, currently only when running outside a container, by specifying additional capabilities together with more instances in `wdio.conf.js`:
+```js
+maxInstances: 10,
+capabilities: [
+    { acceptInsecureCerts: true, browserName: 'chrome' },
+    { acceptInsecureCerts: true, browserName: 'firefox' },
+    { acceptInsecureCerts: true, browserName: 'edge' }
+],
+```

test/acceptance/docker-compose.yaml

@@ -1,20 +1,18 @@
-version: "3.7"
 services:
   wdio-cucumber:
     build: .
     image: wdio-cucumber
-    container_name: webdriver-cuke
     depends_on:
       - selenium
-      - hub
-    environment:
-      TEST_ENVIRONMENT_ROOT_URL: ${TEST_ENVIRONMENT_ROOT_URL}
-      CHROME_ARGS: "--headless --ignore-certificate-errors"
+    volumes:
+      - ./reports/:/home/node/reports/
+    ports:
+      - "9228:9229"
 
   selenium:
-    image: selenium/standalone-chrome-debug
+    image: selenium/standalone-chrome
+    # change to following image for ARM architectures
+    #image: seleniarm/standalone-chromium
     container_name: chrome-browser
-
-  hub:
-    image: selenium/hub
-    container_name: selenium-hub
+    volumes:
+      - /dev/shm:/dev/shm