Skip to content

Commit

Permalink
Add benchmarks (#1217)
Browse files Browse the repository at this point in the history
* feat: add benchmark

* feat: add benchmark

* feat: add benchmark

* feat: add benchmark, drop node 16

* feat: add initial benchmark suite

* perf: tiny perf for isPlainObject

* perf: tiny perf for isPlainObject

* perf: tiny perf for isPlainObject

* perf: tiny perf for isPlainObject
  • Loading branch information
belgattitude authored May 17, 2024
1 parent e0592f7 commit da8474c
Show file tree
Hide file tree
Showing 26 changed files with 574 additions and 61 deletions.
2 changes: 1 addition & 1 deletion .changeset/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@
"access": "public",
"baseBranch": "main",
"updateInternalDependencies": "patch",
"ignore": []
"ignore": ["@bench/*"]
}
5 changes: 5 additions & 0 deletions .changeset/heavy-pumpkins-develop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@httpx/assert": minor
---

@httpx/assert - drop node 16
5 changes: 5 additions & 0 deletions .changeset/kind-tools-buy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@httpx/assert": minor
---

Small perf for isPlainObject and add benchmark
4 changes: 2 additions & 2 deletions .github/workflows/ci-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,11 @@ jobs:
- name: 🕵️ Typecheck
if: matrix.node-version == '20.x'
run: yarn nx run-many --target=typecheck --exclude='examples/**' --exclude='docs'
run: yarn nx run-many --target=typecheck --exclude='examples/**' --exclude='bench/**' --exclude='docs'

- name: 🔬 ESLint checks
if: matrix.node-version == '20.x'
run: yarn nx run-many --target=lint --exclude='examples/**' --exclude='docs'
run: yarn nx run-many --target=lint --exclude='examples/**' --exclude='bench/**' --exclude='docs'

- name: 🧪 Unit tests (no-coverage - node !== 20.x)
if: matrix.node-version != '20.x'
Expand Down
33 changes: 17 additions & 16 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,24 @@ If applicable:

## Local scripts

| Name | Description |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `yarn g:changeset` | Add a changeset to declare a new version |
| `yarn g:typecheck` | Run typechecks in all workspaces |
| `yarn g:lint` | Display linter issues in all workspaces |
| `yarn g:lint --fix` | Attempt to run linter auto-fix in all workspaces |
| `yarn g:test-unit` | Run unit tests in all workspaces |
| `yarn g:build` | Run build in all workspaces |
| `yarn g:clean` | Clean builds in all workspaces |
| `yarn g:check-dist` | Ensure build dist files passes es2017 (run `g:build` first). |
| `yarn g:check-size` | Ensure build files are within size limit (run `g:build` first). |
| `yarn g:docgen` | Build documentation (generally api doc) |
| `yarn clean:global-cache` | Clean tooling caches (eslint, jest...) |
| Name | Description |
|------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|
| `yarn g:changeset` | Add a changeset to declare a new version |
| `yarn g:typecheck` | Run typechecks in all workspaces |
| `yarn g:lint` | Display linter issues in all workspaces |
| `yarn g:lint --fix` | Attempt to run linter auto-fix in all workspaces |
| `yarn g:test-unit` | Run unit tests in all workspaces |
| `yarn g:build` | Run build in all workspaces |
| `yarn g:clean` | Clean builds in all workspaces |
| `yarn g:check-dist` | Ensure build dist files passes es2017 (run `g:build` first). |
| `yarn g:check-size` | Ensure build files are within size limit (run `g:build` first). |
| `yarn g:docgen` | Build documentation (generally api doc) |
| `yarn g:bench-dist` | Run comparative benchmarks of latest |
| `yarn clean:global-cache` | Clean tooling caches (eslint, jest...) |
| `yarn deps:check --dep dev` | Will print what packages can be upgraded globally (see also [.ncurc.yml](https://github.com/belgattitude/httpx/blob/main/.ncurc.yml)) |
| `yarn deps:update --dep dev` | Apply possible updates (run `yarn install && yarn dedupe` after) |
| `yarn check:install` | Verify if there's no peer-deps missing in packages |
| `yarn dedupe` | Built-in yarn deduplication of the lock file |
| `yarn deps:update --dep dev` | Apply possible updates (run `yarn install && yarn dedupe` after) |
| `yarn check:install` | Verify if there's no peer-deps missing in packages |
| `yarn dedupe` | Built-in yarn deduplication of the lock file |

## Git message format

Expand Down
6 changes: 6 additions & 0 deletions bench/assert/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
dist
out
build
coverage
_release

47 changes: 47 additions & 0 deletions bench/assert/.eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/**
* Specific eslint rules for this workspace, learn how to compose
* @link https://github.com/belgattitude/perso/tree/main/packages/eslint-config-bases
*/

// Workaround for https://github.com/eslint/eslint/issues/3458
require('@belgattitude/eslint-config-bases/patch/modern-module-resolution');

const {
getDefaultIgnorePatterns,
} = require('@belgattitude/eslint-config-bases/helpers');

module.exports = {
extends: [
'@belgattitude/eslint-config-bases/typescript',
'@belgattitude/eslint-config-bases/simple-import-sort',
'@belgattitude/eslint-config-bases/sonar',
'@belgattitude/eslint-config-bases/regexp',
'@belgattitude/eslint-config-bases/jest',
'@belgattitude/eslint-config-bases/performance',

// Apply prettier and disable incompatible rules
'@belgattitude/eslint-config-bases/prettier-plugin',
],
ignorePatterns: [
...getDefaultIgnorePatterns(),
'**/build',
'**/.cache',
'**/dist',
'**/_release',
'.cache',
'**/docs',
],
overrides: [
{
files: ['src/**/*.ts'],
rules: {},
},
],
parser: '@typescript-eslint/parser',
parserOptions: {
project: 'tsconfig.json',
tsconfigRootDir: __dirname,
},
root: true,
rules: {},
};
37 changes: 37 additions & 0 deletions bench/assert/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# dependencies
node_modules
/.pnp
.pnp.js

# production output
/dist
/build
/out

# tests
/coverage
/.nyc_output

# editor
/.idea
.project
.classpath
.c9/
*.launch
.settings/
*.sublime-workspace
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json

# os
.DS_Store

# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*

106 changes: 106 additions & 0 deletions bench/assert/bench/benchmark-results.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
{
"files": [
{
"filepath": "/home/sebastien/github/httpx/bench/assert/src/main.bench.ts",
"groups": [
{
"fullName": "src/main.bench.ts > isPlainObject",
"benchmarks": [
{
"id": "1895279271_0_0",
"sampleCount": 351597,
"name": "@httpx/assert: `isPlainObject(v)`",
"rank": 1,
"rme": 0.041483146163102426,
"totalTime": 500.0013865120709,
"min": 0.00137300044298172,
"max": 0.044540997594594955,
"hz": 703192.050031469,
"period": 0.0014220866119792571,
"mean": 0.0014220866119792571,
"variance": 3.185139714518867e-8,
"sd": 0.00017846959725731628,
"sem": 3.0098278970064616e-7,
"df": 351596,
"critical": 1.96,
"moe": 5.899262678132665e-7,
"p75": 0.0014100000262260437,
"p99": 0.002376001328229904,
"p995": 0.002542998641729355,
"p999": 0.0026690028607845306
},
{
"id": "1895279271_0_1",
"sampleCount": 253654,
"name": "@sindresorhus/is: `is.plainObject(v)`",
"rank": 3,
"rme": 0.059455874056027265,
"totalTime": 500.0017051026225,
"min": 0.0018820017576217651,
"max": 0.13113800063729286,
"hz": 507306.26998149726,
"period": 0.0019711958222721603,
"mean": 0.0019711958222721603,
"variance": 9.069401924380675e-8,
"sd": 0.00030115447737632384,
"sem": 5.979549517794933e-7,
"df": 253653,
"critical": 1.96,
"moe": 0.0000011719917054878067,
"p75": 0.0019720010459423065,
"p99": 0.0021730028092861176,
"p995": 0.0027900002896785736,
"p999": 0.0036289989948272705
},
{
"id": "1895279271_0_2",
"sampleCount": 341331,
"name": "is-plain-obj: `isPlainObj(v)`",
"rank": 2,
"rme": 0.02483356376002929,
"totalTime": 500.0007661022246,
"min": 0.0014320015907287598,
"max": 0.03036399930715561,
"hz": 682660.954023849,
"period": 0.001464856008104229,
"mean": 0.001464856008104229,
"variance": 1.1757948653429083e-8,
"sd": 0.00010843407514904658,
"sem": 1.855999748801957e-7,
"df": 341330,
"critical": 1.96,
"moe": 3.6377595076518354e-7,
"p75": 0.0014679990708827972,
"p99": 0.0014929994940757751,
"p995": 0.0015009976923465729,
"p999": 0.0023039989173412323
},
{
"id": "1895279271_0_3",
"sampleCount": 6027,
"name": "lodash-es: `_.isPlainObject(v)`",
"rank": 4,
"rme": 0.2664562411447772,
"totalTime": 500.0052360482514,
"min": 0.0788080021739006,
"max": 0.33079900220036507,
"hz": 12053.873770670642,
"period": 0.08296088203886699,
"mean": 0.08296088203886699,
"variance": 0.00007666330968425547,
"sd": 0.008755758658406219,
"sem": 0.0001127828815823049,
"df": 6026,
"critical": 1.96,
"moe": 0.00022105444790131759,
"p75": 0.08217500150203705,
"p99": 0.11034799739718437,
"p995": 0.12764700129628181,
"p999": 0.2455960027873516
}
]
}
]
}
]
}
55 changes: 55 additions & 0 deletions bench/assert/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"name": "@bench/assert",
"private": true,
"description": "benchmarks for @httpx/assert",
"version": "0.0.0-local",
"license": "MIT",
"author": {
"name": "Vanvelthem Sébastien",
"url": "https://github.com/belgattitude"
},
"homepage": "https://belgattitude.github.io/httpx/assert",
"repository": {
"type": "git",
"url": "https://github.com/belgattitude/httpx.git",
"directory": "bench/assert"
},
"sideEffects": false,
"type": "module",
"exports": {
"./package.json": "./package.json"
},
"scripts": {
"clean": "rimraf ./dist ./build ./coverage ./_release",
"fix-staged": "lint-staged --allow-empty",
"bench-dist": "vitest bench --run",
"tinybench": "tsx ./src/benchmarks.ts",
"lint-disabled": "eslint . --ext .ts,.tsx,.js,.jsx,.mjs,.cjs,.mts,.cts",
"typecheck-disabled": "tsc --project tsconfig.json --noEmit"
},
"dependencies": {
"@httpx/assert": "workspace:^"
},
"devDependencies": {
"@belgattitude/eslint-config-bases": "5.8.0",
"@sindresorhus/is": "6.3.1",
"@types/lodash-es": "4.17.12",
"cross-env": "7.0.3",
"esbuild": "0.21.3",
"eslint": "8.57.0",
"is-plain-obj": "4.1.0",
"lodash-es": "4.17.21",
"moderndash": "3.11.1",
"rimraf": "5.0.7",
"tinybench": "2.8.0",
"tsx": "4.10.4",
"type-fest": "4.18.2",
"typescript": "5.4.5",
"vite": "5.2.11",
"vite-tsconfig-paths": "4.3.2",
"vitest": "1.6.0"
},
"engines": {
"node": ">=18"
}
}
35 changes: 35 additions & 0 deletions bench/assert/src/benchmarks.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// import { isPlainObject } from '../dist/index.mjs';
import { isPlainObject } from '@httpx/assert';
import is from '@sindresorhus/is';
import { Bench } from 'tinybench';

const bench = new Bench({ time: 1000 });

const realLifeScenarios = [
...Array.from({ length: 70 }).map((_) => ({ a: Math.random() })),
...Array.from({ length: 10 }).fill(new Map()),
...Array.from({ length: 5 }).fill(null),
// eslint-disable-next-line unicorn/no-useless-undefined
...Array.from({ length: 5 }).fill(undefined),
...Array.from({ length: 10 }).fill('str'),
];
const { plainObject } = is;
bench
.add('@httpx/assert (isPlainObject)', () => {
realLifeScenarios.forEach((value) => isPlainObject(value));
})
.add('@sindresorhus/is (is.plainObject(v))', async () => {
realLifeScenarios.forEach((value) => is.plainObject(value));
})
.add(
'@sindresorhus/is (const { plainObject } = is; plainObject(v))',
async () => {
realLifeScenarios.forEach((value) => plainObject(value));
}
)
.todo('unimplemented bench');

await bench.warmup(); // make results more reliable, ref: https://github.com/tinylibs/tinybench/pull/50
await bench.run();

console.table(bench.table());
Loading

0 comments on commit da8474c

Please sign in to comment.