feat(editor): Add workflow evaluation run views (no-changelog) #12258
+3,458
−535
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add API endpoints for test metrics CRUD operations - Implement metrics input component with add/delete functionality - Simplify tags input component to use single tag selection
Add a new slot 'node-controls' to Canvas components hierarchy (Canvas, WorkflowCanvas, CanvasNode) to allow custom node controls injection. The slot provides node data and class toggle functionality to customize node appearance.
- Add tooltips to test definition steps explaining their purpose - Limit tag selection to single tag in test definitions - Add multiple-limit prop to N8nSelect component - Update TagsDropdown to support multiple-limit
- Extract route lists into dedicated arrays for better organization - Replace conditional tab selection with array-based lookup - Remove mainHeaderTab from router meta in favor of route-based logic
This commit adds several improvements to the test runner functionality: - Add test run detail view showing execution metrics and test case results - Track test run IDs in evaluation workflow executions for better traceability - Add dark mode support for metrics charts - Fix tooltip behavior in evaluation steps - Improve table cell handling for external links - Update API types and response handling - Clean up unused code and comments
This commit adds the ability to pin specific nodes during test runs. Key changes include: - Add new NodesPinning.vue component for managing pinned nodes in test definitions - Extend TestDefinitionRecord interface to support mockedNodes - Add node toolbar slot to Canvas component for custom node actions - Update CanvasNode to support dynamic class updates via event bus - Add UI for pinning/unpinning nodes with visual feedback - Remove debug console.logs from test definition store This feature allows users to specify which nodes' execution data should be preserved during test runs by pinning them in the workflow canvas.
- Refactor NodesPinning component to fix TS errors and improve node selection - Update test runs table layout with better column ordering and resizable columns - Add empty state for test runs with action to start new test - Show metrics chart only when metrics are available - Update UI text to use "test runs" instead of "test cases" - Improve test definition edit view with collapsible node selection - Fix workflow loading sequence in NodesPinning component - Add border and make columns resizable in TestDefinitionTable
…he-test-definition-screen
…tion support - Introduced a new method to update node classes for pinning/unpinning actions. - Refactored the disableAllNodes function to utilize the new updateNodeClasses method. - Added localization for the pin button tooltip in the - Updated CanvasNodeEventBusEvents type to include payload structure for node class updates.
…he-test-definition-screen
n8n-assistant
bot
added
core
- Updated import path for useMetricsChart to improve structure. - Enhanced chart options to include dynamic x-axis title based on locale. - Adjusted styling for metric selection to use max-width for better responsiveness.
OlegIvaniv
changed the title
Codecov ReportAttention: Patch coverage is 📢 Thoughts on this report? Let us know! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This pull request extends the test definition feature by adding new views and routes for managing and inspecting test runs, metrics, node pinning, and related execution data. Changes are behind a feature flag and mainly front-end.
Routes & Views
/evaluation/:testId/runs(TEST_DEFINITION_RUNS):Lists all runs for a given test definition. Users can navigate from here to individual run details.
/evaluation/:testId/runs/:runId(TEST_DEFINITION_RUNS_DETAIL):Displays details for a specific test run, including metrics and linked executions.
These integrate into the existing
/evaluationentry point:/evaluation(TEST_DEFINITION):Lists test definitions, now with actions to start runs and navigate to the test runs page.
/evaluation/:testId(TEST_DEFINITION_EDIT):Updated to show node-pinning functionality and allow configuration of metrics and tags. A modal is available for selecting nodes to pin.
CleanShot.2024-12-17.at.12.58.43.mp4
Key UI Components & Changes
Node Pinning (
NodesPinning.vue):Accessed from the test definition edit view, this modal lets users pin nodes. Pinned nodes keep consistent execution data across test runs.
Metrics Management (
MetricsInput.vue,MetricsChart.vue):Users can define metrics(
MetricsInput.vue) to track test performance. Metrics are displayed in charts over multiple runs (MetricsChart.vue) and handled with CRUD operations for metrics inuseTestDefinitionForm.ts->testDefinition.store.ee.ts->@/api/testDefinition.ee.N8nSelect & TagsDropdown:
The
N8nSelectandTagsDropdowncomponents have been updated to include amultipleLimitprop and to handle tag creation and selection more gracefully.Header Tabs (MainHeader.vue):
The top navigation now relies on arrays of route names to determine the active tab. This approach supports new test definition routes alongside existing workflow and execution routes.
State & API Integrations
State Store (testDefinition.store.ee.ts):
The store now manages:
It integrates with new API client endpoints defined in
@/api/testDefinition.ee.TestRunnerService (
test-runner.service.ee.ts):Temporarily includes logic to pass
testRunIdas metadata to evaluation workflow executions. This allows us to fetch executions related to a run.Note: This is a short-term solution; future backend improvements should supersede this.
Auto-Save & Polling
Related Linear tickets, Github issues, and Community forum posts
Review / Merge checklist
release/backport(if the PR is an urgent fix that needs to be backported)