Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add External Source (and External Event) Attribute Validation #1560

Open
wants to merge 46 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
1c0627d
Prototype of creating external source/event types thru Gateway
Nov 8, 2024
dc8e578
Prototype WIP
Nov 8, 2024
d409002
WIP Prototype of visualizing attributes
Nov 15, 2024
745a458
update to play better with gateway and source validation
Nov 15, 2024
1da36e5
Prototype cleanup, re-adding permissions, removing code that is now i…
Nov 15, 2024
8209f76
Lint/format cleanup
Nov 15, 2024
295f4a2
fix Uploading... bug
Nov 15, 2024
8d2abe3
additional error handling
Nov 18, 2024
e56d585
remove allowed event types from source type creation
Nov 18, 2024
5bf33fc
finish error handling
Nov 18, 2024
987ab78
complete error handling
Nov 18, 2024
14fc62a
Rework JsonSchema to translate to ValueSchema for UI
Nov 18, 2024
1fcbf5c
Cleanup
Nov 18, 2024
6fa0478
Use <Parameters> in Management modal
Nov 18, 2024
e420de5
Remove extra handling for errors
Nov 18, 2024
3f94f7b
Close creation modal after successful creation
Nov 18, 2024
92fc6ee
Add attributes to expected test output, fix formatting
Nov 19, 2024
8cbd634
Error handling fixes & e2e test updates
Nov 19, 2024
37d2d60
Improve handling for non-existent types
Nov 20, 2024
28be295
Alter failure toast messages
Nov 20, 2024
5745f17
Lint fix
Nov 20, 2024
a0ec888
Refactor createExternalSource to send parsed JSON as form data
Nov 21, 2024
b692fa1
update effects for new source type uploading system
Nov 22, 2024
165a8b8
WIP - Update modal for creating types from a single schema
Nov 22, 2024
546ed67
Rework Create & Manage modals into one, fix bug with ExternalEventForm
Nov 25, 2024
dcefe3e
Move external type management modal to sub-page of External Sources
Nov 26, 2024
893f84f
Update e2e tests for new 'type' page
Nov 26, 2024
9e90903
Updating derivation examples for e2e tests
Nov 26, 2024
8fcbd48
--amend
Nov 26, 2024
38e0803
Add support for arrays as series
Nov 27, 2024
29d7b70
Fix issue with auto-selecting source after upload
Nov 27, 2024
2fba232
Formatting
Nov 27, 2024
ff2495b
Always show all 3 tables in type manager
Dec 3, 2024
b22d55e
Linting & test fixes
Dec 3, 2024
d208686
Formatting on ExternalTypeManager, use Panels like Dictionaries page …
Dec 4, 2024
8afe276
Update formatting for Arrays in attribute schemas, add associated sou…
Dec 4, 2024
7835a15
Formatting
Dec 4, 2024
0631a4d
Properly translate nested JsonSchema-typed items to ValueSchema
Dec 4, 2024
cdda004
Add missing deselectSource
Dec 5, 2024
4051b42
E2E test fixes
Dec 12, 2024
eb32de9
Remove out-of-date comment
Dec 13, 2024
39614a9
Fix typo; names => name
Dec 13, 2024
074e5ef
update to allow automatic creation of types and allow either/or event…
Dec 30, 2024
efc7a1d
show required attributes
Jan 3, 2025
e04e89c
Fix bug with not displaying objects/structs nested within arrays/series
Jan 3, 2025
7273047
update tests
Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions e2e-tests/data/Schema_Example_Derivation.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
{
"event_types": {
"DerivationA": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
},
"DerivationB": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
},
"DerivationC": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
},
"DerivationD": {
"type": "object",
"required": [
"rules",
"notes",
"should_present"
],
"properties": {
"rules": {
"type": "string"
},
"notes": {
"type": "string"
},
"should_present": {
"type": "boolean"
}
}
}
},
"source_types": {
"DerivationTest": {
"type": "object",
"required": [
"exampleAttribute"
],
"properties": {
"exampleAttribute": {
"type": "string"
}
}
}
}
}
46 changes: 46 additions & 0 deletions e2e-tests/data/Schema_Example_Source.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"event_types": {
"ExampleEvent": {
"title": "ExampleEvent",
"description": "Schema for the attributes of the ExampleEvent External Event Type.",
"type": "object",
"properties": {
"example1": {
"type": "string"
},
"example2": {
"type": "number"
},
"optional": {
"type": "string"
}
},
"required": [
"example1",
"example2"
]
}
},
"source_types": {
"Example External Source": {
"title": "Example External Source",
"description": "Schema for the attributes of the Example External Source External Source Type.",
"type": "object",
"properties": {
"optional": {
"type": "string"
},
"version": {
"type": "number"
},
"spacecraft": {
"type": "string"
}
},
"required": [
"version",
"spacecraft"
]
}
}
}
11 changes: 4 additions & 7 deletions e2e-tests/data/example-external-source.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,9 @@
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-002T00:00:00Z"
},
"metadata": {
"attributes": {
"version": 1,
"spacecraft": [
"sc1",
"sc2"
]
"spacecraft": "sc1"
}
},
"events": [
Expand All @@ -21,10 +18,10 @@
"event_type": "ExampleEvent",
"start_time": "2022-001T12:00:00Z",
"duration": "00:00:00",
"properties": {
"attributes": {
"example1": "value",
"example2": 1
}
}
]
}
}
21 changes: 21 additions & 0 deletions e2e-tests/data/example-external-source_no-attr.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"source": {
"key": "ExampleExternalSource:example-external-source_no-attr.json",
"source_type": "Empty External Source",
"valid_at": "2024-001T00:00:00Z",
"period": {
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-002T00:00:00Z"
},
"attributes": {}
},
"events": [
{
"key": "EmptyEvent:1/sc/sc1:1:X/1",
"event_type": "EmptyEvent",
"start_time": "2022-001T12:00:00Z",
"duration": "00:00:00",
"attributes": {}
}
]
}
37 changes: 16 additions & 21 deletions e2e-tests/data/external-event-derivation-1.json
Original file line number Diff line number Diff line change
@@ -1,53 +1,48 @@
{
"source": {
"key": "external-event-derivation-1.json",
"source_type": "Derivation Test",
"source_type": "DerivationTest",
"valid_at": "2022-018T00:00:00Z",
"period": {
"start_time": "2022-005T00:00:00Z",
"end_time": "2022-011T00:00:00Z"
},
"metadata": {}
"attributes": {
"exampleAttribute": "TestString"
}
},
"events": [
{
"key": "2",
"event_type": "DerivationD",
"start_time": "2022-005T23:00:00Z",
"duration": "01:10:00",
"properties": {
"rules": [
3,
4
],
"notes": "subsumed by test 01, even though end lies outside of 01, also replaced by test 01 by key",
"should_present": false
"attributes": {
"rules": "3, 4",
"notes": "subsumed by test 01, even though end lies outside of 01, also replaced by test 01 by key",
"should_present": false
}
},
{
"key": "7",
"event_type": "DerivationC",
"start_time": "2022-009T23:00:00Z",
"duration": "02:00:00",
"properties": {
"rules": [
3
],
"notes": "subsumed by test 02, even though end lies outside of 01, because start time during 01",
"should_present": false
"attributes": {
"rules": "3",
"notes": "subsumed by test 02, even though end lies outside of 01, because start time during 01",
"should_present": false
}
},
{
"key": "8",
"event_type": "DerivationB",
"start_time": "2022-010T11:00:00Z",
"duration": "01:05:00",
"properties": {
"rules": [
1
],
"notes": "after everything, subsumed by nothing despite being from oldest file",
"should_present": true
"attributes": {
"rules": "1",
"notes": "after everything, subsumed by nothing despite being from oldest file",
"should_present": true
}
}
]
Expand Down
46 changes: 20 additions & 26 deletions e2e-tests/data/external-event-derivation-2.json
Original file line number Diff line number Diff line change
@@ -1,65 +1,59 @@
{
"source": {
"key": "external-event-derivation-2.json",
"source_type": "Derivation Test",
"source_type": "DerivationTest",
"valid_at": "2022-019T00:00:00Z",
"period": {
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-007T00:00:00Z"
},
"metadata": {}
"attributes": {
"exampleAttribute": "TestString"
}
},
"events": [
{
"key": "1",
"event_type": "DerivationA",
"start_time": "2022-001T00:00:00Z",
"duration": "02:10:00",
"properties": {
"rules": [
1
],
"notes": "before everything, subsumed by nothing",
"should_present": true
"attributes": {
"rules": "1",
"notes": "before everything, subsumed by nothing",
"should_present": true
}
},
{
"key": "2",
"event_type": "DerivationA",
"start_time": "2022-001T12:00:00Z",
"duration": "02:10:00",
"properties": {
"rules": [
4
],
"notes": "overwritten by key in later file, even with type change",
"should_present": false
"attributes": {
"rules": "4",
"notes": "overwritten by key in later file, even with type change",
"should_present": false
}
},
{
"key": "3",
"event_type": "DerivationB",
"start_time": "2022-002T23:00:00Z",
"duration": "03:00:00",
"properties": {
"rules": [
2
],
"notes": "starts before next file though occurs during next file, still included",
"should_present": true
"attributes": {
"rules": "2",
"notes": "starts before next file though occurs during next file, still included",
"should_present": true
}
},
{
"key": "4",
"event_type": "DerivationB",
"start_time": "2022-005T21:00:00Z",
"duration": "03:00:00",
"properties": {
"rules": [
3
],
"notes": "start subsumed by 02, not included in final result",
"should_present": false
"attributes": {
"rules": "3",
"notes": "start subsumed by 02, not included in final result",
"should_present": false
}
}
]
Expand Down
Loading
Loading