Commit 5e78640
Add WFSS L3 compatibility and search to plugin select dropdowns (#3729)
* Temporary test for new WFSS L3 files
* Starting work on WFSS L3 loader
* Added warning message for masked spectra
* Prepping for vue things
* Revert to using dropdown for simplicity
* This may change to the table in the future once I've got a handle on using the dropdown.
* This is (closely) following the prescription in the image importer
* Added SelectSpectraComponent
* Specifically for the SpectrumList importer.
* WIP testing with known spectrum list file
* WIP working on preloading spectra for user choice
* WIP working on getting spectra to load correctly with mask
* WIP added mask checking/application logic
* WIP fixed bugs, fixed some logic
* also working on tests to confirm this operates as expected
* WIP fixing up and adding to SelectSpectraComponent
* WIP prototype working version
* Uses exposure number + source ID to create the unique label/reference
* Will likely move checking for issues with creating a spectrum in the specutils parser rather than applying masks everywhere
* WIP spectrum_list works as expected
* Need to tidy things up and fix some display things.
* Likely need to mess with labels.
* Modified checks and generalized how the spectra show in the loader (index based if WFSS source_id can't be found).
* Also fixed up some things in the tests but they have to be formalized of course.
* WIP include search bar
* It's a bit rudimentary but it'll work for now. May attempt to clean it up later.
* Added v-if to check for search bool
* WIP added search, need to work on data labels
* Also remove list concatenation for now to speed up testing
* WIP still working on tests
* Added TODO
* Made search a bit more graceful
* Removed default option
* Still trying to make the search function more seamless
* Added padding below dropdown
* Updated data label in viewer
* Fix TODOs
* Vectorize SpectrumListConcatenated
* Some options to handle SpectrumListConcatenated
* Cleaned up some code
* Added disabling for when search is not necessary
* Mostly for SpectrumListConcatenated
* Reduced padding a bit
* Refactor logic for consistent labeling
* Remove most logic from output and put it into init where the action is happening anyway
* Add in suffix to options dict to keep everything together
* Modify SpectrumListConcatenated to avoid error due to no spectra selection since we're combining them all
* Use selected_obj_dict instead of selected_obj
* This allows us to keep everything together easier for labeling and whatnot
* Updated changelog
* Added TODO
* Remove search function
* Too awkward as is, will be implemented in plugin_select later
* Added warning for duplicated data labels
* Only show warning once...
to avoid inundating the user on back-to-back imports
* Initial commit of tests
* Made some fixtures and an initial test for is_valid
* Debugging getting some tests in
* Working on proper setup before moving forward
* Removed some unused code and slightly modified masking
* Fixed up tests to use custom test class
* Finally figured out the commands to instantiate the class directly...
* This allows some convenience when changing input and also avoids dealing with the rest of the loader structure.
* Commented out tests were generated by copilot and will be reviewed
* Reverted for now
* Fixed style issues
* Fixed style issues
* Fixed style issues
* Fixed style issues
* Remove old commented out code
* Adjusted explanation for spectral_mask
* Added a TODO and a comment about masks
* Updated tests to work correctly with masking
* Removed fully masked entries, Spectum objects can't be initialized fully masked, but they can have the masks updated afterward.
* Added a TODO and a comment about masks
* Remove SelectSpectraComponent to use SelectFileExtensionComponent
* They're functionally the same except I also added the ability to retrieve the original dict (and included a suffix key for data label purposes).
* Changes to account for viewer issues
* Included parent logic which helps keep spectra from the same file together however there is still the issue that multiple WFSS files load into the same viewer.
* When parenting+add to viewer happened at the same time in the loop, there would be an error when importing non WFSS + WFSS which stated that the dataset could not be found in `self.dataset.choices`. I could not find the source of this issue but managed to mitigate that by adding to the viewer after all the data had been added to the collection. I don't think this should happen, so I left a TODO in.
* Building out more tests
* Updated TODOs after discussion
* Added more tests, still WIP
* Fixed some bugs found through testing
* Adjusted naming for non-WFSS files to clarify file indices vs internal indices.
* Add split check for index based naming for existing data labels.
* Moved previous_data_label_messages to init to avoid overwrite on new call.
* Move previous_data_label_messages inside if to avoid duplicates.
* Use `default_viewer_reference` instead of hardcoded viewer for adding to viewer.
* Added a bunch of tests
* Some TODOs left over due to known bugs or odd behavior.
* Changed spectra meta to avoid duplicates and the headaches that that causes when forcing those to be ingested.
* Use custom fake class to override default_data_label_resolver to avoid labeling duplication and issues when checking viewer (although I still couldn't get a second viewer to pop out, at least naming is easier).
* Added mock patch for snackbar catching.
* re-arranged the code a bit.
* Added some basic concatenated tests... I may revisit these before merging the PR.
* Small adjustments to code for consistency/bugs
* Added check and default behavior for no spectra selected...
for compatability with config.load which doesn't specify a selection but needs something to be selected in order to avoid an error in get_selected_obj.
* Added observer to disable import button when nothing is selected (and to avoid the case where the user accidentally imports everything due to the selection being none -> all).
* Added a warning for the user if this does happen during the load process to save them the stress of wondering what's happening when it's attempting to load all the data.
* Added TODO in regards to the warning
* Added some tests, parametrized others, added some checks
* Notably checking initialization
* Also added tests for newly added functions in spectrum_list
* Adjust for changes in spectrum list importer
* Made several helper methods private
* Adjusted test for new functionality and added new WFSS test
* Fake Importers are added to the registry for the new spectrum_list tests so I had to exclude those in some of the old tests. I have added a fixture to conftest to help with this in case it comes up in the future.
* The WFSS file test is basic but should be OK because the hardcore testing happens in test_spectrum_list.py
* Style fixes
* Style fixes
* Adapt existing fixtures for spectrum list importer tests and vice versa
* Style fixes
* Added a docstring-ish comment to explain new fixture
* Fix tests to avoid warnings on CI
* Style fixes
* Found a couple more warning spots to fix
* Style fix
* Made observing more dynamic
* Since we determine which loader to use by checking them all, I was accidentally disabling import for the other possible loaders when attempting to use the spectrum_list loader without anything selected. This has now been fixed with two separate observers, one for format and one for spectra_selected.
* Updated tests
* Small adjustments
* Fixed and added tests
* Added tests for new observe functionality
* Fixed some small things in other tests
* Get coverage of snackbar message for fully masked spectra
* Style fixes
* Style fixe
* Change label name for quality of life
* This does not affect the label that is output although it may just a hair more confusing for being generic vs the descriptive data label in the viewer.
* Implemented search in plugin_select.vue
* Search now shows up in dropdown and actively filters (similar to as in app.vue).
* Added some padding below the dropdown box.
* Added exposures dropdown
* Used for WFSS fits files and potentially others in the future
* Added traitlets in spectrum_list.py
* Added methods and logic for filtering the source id dropdown by exposures.
* Update API hints
* Added tests for exposures stuff
* Removed no-longer applicable test
* Remove TODO
* Updated variable names to follow convention
* Updated CHANGES.rst
* Updated CHANGES.rst
* Update CHANGES.rst
* Update CHANGES.rst
* Removed references to parenting...
in preparation for a PR to address the viewer issue.
* Add staticmethod decorators
* Renamed variable to avoid shadowing
* Add TODO for skipped test
* Add `load_all` kwarg option for spectrum lists
* Update tests with load_all functionality
* Add load_selected and load_all functionality
* Condense tests and TODO
* Cover new load_selected attribute
* Update error message for none selected
* Remove load_all, allow wildcard filtering in load_selected
* Add dropdown for selection to SpectrumListConcatenated
* Update CHANGES.rst
Co-authored-by: Kyle Conroy <[email protected]>
* Revert to using `disable_dropdown` except for concatenated 2d spectra
* add coverage
* Add unit test for `_load_selected_helper`
* Fixed a bad assert in test
* Remove fully_masked_spectra list
* It seems the GUI can handle it after all!
* Style fixes
* Style fixes
* Update CHANGES.rst
* Moved source_id filtering to backend
* Use observer on exposure selection change to propagate this
* Added and modified tests to accommodate this
* Moved source_id filtering to backend
* Use observer on exposure selection change to propagate this
* Added and modified tests to accommodate this
* Add `suffix` to the list of keys in `SelectFileExtensionComponent` items and modified `spectrum_list` therein. Removed `get_selected_obj_dict`.
* Modified load_selected logic...
to use existing '*' wildcard attribute setting in `user_api` to select all.
* Remove `load_selected` and related functionality
* Adjust tests to use `user_api.spectra` instead
* Small adjustments and update tests
* Expose `exposures`. Don't set `exposures.selected` to an empty list, just check for `None`.
* Move check for 2D spectra into `SpectrumListConcatenatedImporter` init
* Use `spectra.select_all()` instead of '*'.
* Fix API hints
* Removed custom data_label check logic
* Style fixes
* Adjustment to label
* Change `spectra` to `sources`
* And other small bugs
* Updated vue file with spectra->sources
* Restored exposures.selected to initialize with empty list
* The dropdown didn't show otherwise and this is easier to remove in the future when the multiselect bug is fixed than other logic to guarantee it being there.
* Revert tests for exposures.selected initialization
* Added fully_masked test back and more complicated tests for ...selection_change
* Tentative commit for more extensive logic for format/selection change
* Minor fixes and comments
* Style fixes
* 2d spectrum as 1d should always convert to flux units
* clarify flux vs spectral flux density
* test coverage
* implement toggle to control whether to convert to flux density
* Fixes to SpectrumListConcatenated tests
* Accidentally included code from Kyle's PR
* codestyle
* No need to check is_valid in init anymore
* Updated test to remove old check
* Style fix
* Fixed tab issue from merge
* Tab issue
* Style fix
* Style fix
* Comment out SB conversion toggle for now
* Removed exposure dropdown and references therein
* Remove unused imports
* Force the flux to be read in from the specified flux column...
to avoid issues with different source types (point vs extended resulted in flux vs surface brightness which caused a big headache for everyone)
* Removed references to unit conversion, (properly) force flux column
* Restoring unit conversion code
---------
Co-authored-by: Kyle Conroy <[email protected]>1 parent 6bbb435 commit 5e78640
File tree
13 files changed
+1153
-126
lines changed- jdaviz
- components
- configs/specviz
- plugins/unit_conversion
- tests
- core
- loaders
- importers/spectrum_list
- parsers
13 files changed
+1153
-126
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
10 | 13 | | |
11 | 14 | | |
12 | 15 | | |
| |||
19 | 22 | | |
20 | 23 | | |
21 | 24 | | |
| 25 | + | |
| 26 | + | |
22 | 27 | | |
23 | 28 | | |
24 | 29 | | |
| |||
61 | 66 | | |
62 | 67 | | |
63 | 68 | | |
| 69 | + | |
| 70 | + | |
64 | 71 | | |
65 | 72 | | |
66 | 73 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
23 | 58 | | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 59 | + | |
| 60 | + | |
30 | 61 | | |
31 | 62 | | |
32 | | - | |
33 | | - | |
34 | | - | |
| 63 | + | |
35 | 64 | | |
36 | | - | |
37 | | - | |
38 | | - | |
| 65 | + | |
39 | 66 | | |
40 | 67 | | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
| 68 | + | |
61 | 69 | | |
62 | | - | |
| 70 | + | |
63 | 71 | | |
64 | 72 | | |
65 | 73 | | |
| |||
70 | 78 | | |
71 | 79 | | |
72 | 80 | | |
73 | | - | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
74 | 128 | | |
75 | 129 | | |
76 | 130 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
93 | 97 | | |
94 | 98 | | |
95 | 99 | | |
| |||
Lines changed: 5 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | 46 | | |
57 | 47 | | |
58 | 48 | | |
| |||
395 | 385 | | |
396 | 386 | | |
397 | 387 | | |
398 | | - | |
399 | | - | |
| 388 | + | |
| 389 | + | |
400 | 390 | | |
401 | 391 | | |
402 | 392 | | |
403 | 393 | | |
404 | 394 | | |
405 | 395 | | |
406 | 396 | | |
407 | | - | |
408 | | - | |
| 397 | + | |
| 398 | + | |
409 | 399 | | |
410 | 400 | | |
411 | 401 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
55 | 56 | | |
56 | 57 | | |
57 | 58 | | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
70 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
71 | 78 | | |
72 | 79 | | |
73 | 80 | | |
74 | 81 | | |
75 | | - | |
| 82 | + | |
76 | 83 | | |
77 | 84 | | |
78 | 85 | | |
| |||
417 | 424 | | |
418 | 425 | | |
419 | 426 | | |
420 | | - | |
| 427 | + | |
| 428 | + | |
421 | 429 | | |
422 | 430 | | |
423 | | - | |
| 431 | + | |
424 | 432 | | |
425 | 433 | | |
426 | 434 | | |
427 | 435 | | |
428 | 436 | | |
429 | 437 | | |
430 | | - | |
| 438 | + | |
431 | 439 | | |
432 | | - | |
433 | | - | |
| 440 | + | |
| 441 | + | |
434 | 442 | | |
435 | 443 | | |
436 | 444 | | |
| |||
0 commit comments