Update the testing pipeline#257
Merged
zezhishao merged 16 commits intoGestaltCogTeam:masterfrom Aug 22, 2025
Merged
Conversation
…hen testing on large-scale datasets.
…hen testing on large-scale datasets.
…hen testing on large-scale datasets.
…hen testing on large-scale datasets.
…hen testing on large-scale datasets.
zezhishao
reviewed
Aug 15, 2025
basicts/runners/base_tsf_runner.py
Outdated
| # For saving test results | ||
| self._inputs_memmap = None | ||
| self._pred_memmap = None | ||
| self._tgt_memmap = None |
Collaborator
There was a problem hiding this comment.
使用_target_memmap和_prediction_memap吧，保持和其他变量的一致。
basicts/runners/base_epoch_runner.py
Outdated
| raise NotImplementedError() | ||
|
|
||
| def test(self, train_epoch: Optional[int] = None, save_metrics: bool = False, save_results: bool = False) -> None: | ||
| def test(self, train_epoch: Optional[int] = None, save_metrics: bool = False, save_results: bool = False,) -> None: |
basicts/runners/base_epoch_runner.py
Outdated
| self.logger.info(f'Test metrics saved to {os.path.join(self.ckpt_save_dir, "test_metrics.json")}.') | ||
|
|
||
| self.on_test_end() | ||
| self.on_test_end(save_metrics) |
Collaborator
There was a problem hiding this comment.
可以将on_test_end中的代码移动到test函数中或者放到本函数中；
数据操作（例如计算、存储results和metrics）和操作之后的log尽量不要离得太远。
| history_data = self.select_input_features(history_data) | ||
| future_data_4_dec = self.select_input_features(future_data) | ||
|
|
||
|
|
||
| # Forward pass through the model | ||
| model_return = self.model(history_data=history_data, future_data=future_data_4_dec, | ||
| model_return = self.model(history_data=history_data, future_data=future_data_4_dec, |
basicts/runners/base_epoch_runner.py
Outdated
|
|
||
| @master_only | ||
| def on_test_end(self) -> None: | ||
| def on_test_end(self, save_metrics: bool = False) -> None: |
Collaborator
There was a problem hiding this comment.
把on_test_end内的代码移动到外面之后，这里的参数可以删掉了
| prediction, target, inputs = [], [], [] | ||
|
|
||
| for data in tqdm(self.test_data_loader): | ||
| for batch_idx, data in tqdm(enumerate(self.test_data_loader)): |
Collaborator
There was a problem hiding this comment.
enumerate要放在外面，否则会导致进度条不显示
… for missing values
… for missing values
… for missing values
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update the testing pipeline to fix potential memory overflow issues when testing on large-scale datasets.
The issue may happen when concatenating all test samples of a large dataset. The issue has been fixed, making the process more memory-efficient.