Lighthouse performance metrics setup #9304

ruchidh · 2025-01-30T00:23:27Z

Description

This CR is testing github workflow for performance testing, currently for job lighthouse.
This is not for merge purpose as of now. Please ignore license checks or other issues for now.

Issues Resolved

Screenshot

Testing the changes

Changelog

skip

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Ruchi Sharma <[email protected]>

angle943 · 2025-01-30T22:51:45Z

.lighthouserc.js

+      target: 'temporary-public-storage', // Required for GitHub integration
+    },
+  },
+};


which OS instance does this connect to?

for our BE, 9200. This temporary-public-storage , is a public instance of google report .

angle943 · 2025-01-30T22:54:05Z

.lighthouserc.js

+module.exports = {
+  ci: {
+    collect: {
+      url: [`${baseUrl}`, `${baseUrl}/app/data-explorer/discover`, `${baseUrl}/app/dashboards`], // Add more URLs as needed


Something i'm wondering about...

Without some specific actions, I don't think any results would be loaded in discover and no dashboards will be set up on dashboards page, so i'm not sure if we are getting the full benefit of having this.

Is it possible to start with some examples that'll load? perhaps the sample data (but even with that i think we need to modify the URL so we are hitting the correct page. For example /app/dashboards might just go to the list of dashboards you need to select from)

+1, you can actually hit the api or bulk insert saved objects and data

yes, have plan to do so, updating mock data

angle943 · 2025-01-30T22:56:25Z

.lighthouserc.js

+    assert: {
+      preset: 'lighthouse:recommended',
+      assertions: {
+        performance: ['error', { minScore: 0.9 }],


oof i think we need make the minscore much lower. and it might differ between the different URLs.

For example on dashboards page with the example data loaded, i am getting a score of 34 (yikes)

This is only for testing purpose, would change in final review.

angle943 · 2025-01-30T22:57:39Z

.github/workflows/performance_testing.yml

@@ -0,0 +1,51 @@
+name: Performance Testing


do we need:

# Copyright OpenSearch Contributors # SPDX-License-Identifier: Apache-2.0

angle943 · 2025-01-30T22:58:09Z

.lighthouserc.js

@@ -0,0 +1,28 @@
+const baseUrl = 'http://localhost:5601';


do we need a license here? (like .eslintrc.js)

kavilla · 2025-01-31T00:16:19Z

package.json

@@ -464,6 +465,7 @@
    "postcss": "^8.4.5",
    "prettier": "^2.1.1",
    "prop-types": "^15.7.2",
+    "puppeteer": "^24.1.1",


i would be hesitant to add this. i know some opensearch plugin uses puppeteer and they have to bundle a chrome binary so that it can not fail on some integration testing.

i think other plugin teams have used puppeteer-core? not sure haven't looked too deep into it

okay, let's discuss this

Signed-off-by: Ruchi Sharma <[email protected]>

kavilla · 2025-01-31T00:19:42Z

.github/workflows/performance_testing.yml

+
+      - name: Run OpenSearch
+        run: |
+          /bin/bash -c "./opensearch-2.17.0/opensearch-tar-install.sh &"


we should fix this if we have this somewhere else but if a different version then than 2.17 then this will not be right

kavilla · 2025-01-31T00:21:31Z

.github/workflows/performance_testing.yml

+        run: yarn global add @lhci/cli
+
+      - name: Build OpenSearch Dashboards
+        run: yarn build


build will build a platform distribution. are we sure we dont want to yarn osd bootstrap?

kavilla · 2025-01-31T00:25:19Z

.github/workflows/performance_testing.yml

+          sleep 30
+        shell: bash
+
+      - name: Install Lighthouse CI


is there not a github action that we can pull from the marketplace and run this?

kavilla · 2025-01-31T00:28:15Z

.lighthouserc.js

@@ -0,0 +1,28 @@
+const baseUrl = 'http://localhost:5601';


baseUrl technically doesn't have to be 5601. It can be configurable to be something like 443

kavilla · 2025-01-31T00:29:42Z

config/opensearch_dashboards.yml

nit: no changes observed besides white spacing

kavilla · 2025-01-31T00:30:47Z

package.json

@@ -271,6 +271,7 @@
    "@elastic/filesaver": "1.1.2",
    "@elastic/makelogs": "^6.1.0",
    "@faker-js/faker": "^7.6.0",
+    "@lhci/cli": "^0.14.0",


if this is a dev dependency then bootstrapping should be suffice enough correct? do we still need to add it the step to install it globally in the github workflow?

Signed-off-by: Ruchi Sharma <[email protected]>

angle943 · 2025-01-31T00:55:43Z

We talked about this in private, but just for some permanence, I will write TLDR version here. Some challenges with this approach are:

installing mock data to load the data
Telling Lighthouse the correct URL to see the loaded data (challenge is that for dashboard, the created dashboards have randomly generated IDs in the URL. For Discover, we will need to modify the URL to have all the correct data and timestamps to load the proper data)

One way we can solve this is leveraging Cypress: https://medium.com/@mudit56p/enhancing-ui-performance-testing-with-lighthouse-and-cypress-1ad69db0a150

Signed-off-by: Ruchi Sharma <[email protected]>

Ubuntu added 6 commits January 28, 2025 23:15

lighthouse ci added

8913f15

lighthouse setup

bb50042

lighthouse setup

49713ef

lighthouse setup

fb42ef5

lught house steup

3c0799e

ci metrics setup

3d5dddc

ruchidh added 2 commits January 30, 2025 21:32

fix missing lhci install from flow

20dbacf

Signed-off-by: Ruchi Sharma <[email protected]>

add yarn bootstrap to run dependencies

6636bf9

LDrago27 removed the first-time-contributor label Jan 30, 2025

angle943 reviewed Jan 30, 2025

View reviewed changes

.lighthouserc.js

@@ -0,0 +1,28 @@

const baseUrl = 'http://localhost:5601';

Copy link

Contributor

angle943 Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need a license here? (like .eslintrc.js)

ruchidh added 2 commits January 30, 2025 23:57

download opensearch

b937c42

add wait for for be server

3f15a5f

kavilla reviewed Jan 31, 2025

View reviewed changes

add node options

4aa842b

Signed-off-by: Ruchi Sharma <[email protected]>

kavilla reviewed Jan 31, 2025

View reviewed changes

config/opensearch_dashboards.yml

# data.savedQueriesNewUI.enabled: true

Copy link

Member

kavilla Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no changes observed besides white spacing

kavilla reviewed Jan 31, 2025

View reviewed changes

ruchidh added 4 commits January 31, 2025 00:41

remove yarn build and add boottsrap

33b9660

Signed-off-by: Ruchi Sharma <[email protected]>

update version

7f59b70

Signed-off-by: Ruchi Sharma <[email protected]>

change lhci from global to dev

4b63789

update version

20957e7

Signed-off-by: Ruchi Sharma <[email protected]>

ruchidh added 7 commits January 31, 2025 21:08

updare version

1bcb100

add mock data

664d4de

Signed-off-by: Ruchi Sharma <[email protected]>

add wait

f6c03a7

Signed-off-by: Ruchi Sharma <[email protected]>

add wait on

846bccf

Signed-off-by: Ruchi Sharma <[email protected]>

reevrt waiton opensearch

5f785ae

reevrt waiton opensearch

c566adf

Signed-off-by: Ruchi Sharma <[email protected]>

commetn wait on

1f70a7b

Signed-off-by: Ruchi Sharma <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lighthouse performance metrics setup #9304

Lighthouse performance metrics setup #9304

ruchidh commented Jan 30, 2025 •

edited

Loading

angle943 Jan 30, 2025

ruchidh Jan 31, 2025 •

edited

Loading

angle943 Jan 30, 2025 •

edited

Loading

kavilla Jan 31, 2025

ruchidh Jan 31, 2025

angle943 Jan 30, 2025

ruchidh Jan 30, 2025

angle943 Jan 30, 2025

angle943 Jan 30, 2025

kavilla Jan 31, 2025

kavilla Jan 31, 2025

ruchidh Jan 31, 2025

kavilla Jan 31, 2025

kavilla Jan 31, 2025

kavilla Jan 31, 2025

kavilla Jan 31, 2025

kavilla Jan 31, 2025

kavilla Jan 31, 2025

angle943 commented Jan 31, 2025

Lighthouse performance metrics setup #9304

Are you sure you want to change the base?

Lighthouse performance metrics setup #9304

Conversation

ruchidh commented Jan 30, 2025 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

Choose a reason for hiding this comment

ruchidh Jan 31, 2025 • edited Loading

Choose a reason for hiding this comment

angle943 Jan 30, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

angle943 commented Jan 31, 2025

ruchidh commented Jan 30, 2025 •

edited

Loading

ruchidh Jan 31, 2025 •

edited

Loading

angle943 Jan 30, 2025 •

edited

Loading