hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200

[bh-cai]

when I run the flow command, I got the issue: hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200

main_instance_segmentation.py general.experiment_name=test1_scannet_val_query_150_topk_500_dbscan_0.95 general.project_name=scannet_eval general.checkpoint='checkpoints/scannet/scannet_val.ckpt' general.train_mode=false general.eval_on_segments=true general.train_on_segments=true model.num_queries=150 general.topk_per_image=500 general.use_dbscan=true general.dbscan_eps=0.95

/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/utilities/seed.py:55: UserWarning: No seed found, seed set to 2801433411
rank_zero_warn(f"No seed found, seed set to {seed}")
Global seed set to 2801433411
EXPERIMENT ALREADY EXIST
{'target': 'pytorch_lightning.loggers.WandbLogger', 'project': '${general.project_name}', 'name': '${general.experiment_name}', 'save_dir': '${general.save_dir}', 'entity': 'manjusaka_labs', 'resume': 'allow', 'id': '${general.experiment_name}'}
wandb: Currently logged in as: bh_c (manjusaka_labs). Use wandb login --relogin to force relogin
wandb: wandb version 0.15.4 is available! To upgrade, please run:
wandb: $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.15.0
wandb: Run data is saved locally in saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95/wandb/run-20230609_012749-test1_scannet_val_query_150_topk_500_dbscan_0.95
wandb: Run wandb offline to turn off syncing.
wandb: Resuming run test1_scannet_val_query_150_topk_500_dbscan_0.95
wandb: ⭐️ View project at https://wandb.ai/manjusaka_labs/scannet_eval
wandb: 🚀 View run at https://wandb.ai/manjusaka_labs/scannet_eval/runs/test1_scannet_val_query_150_topk_500_dbscan_0.95
2023-06-09 01:27:54.018 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:91 - Key not found, it will be initialized randomly: model.scene_min
2023-06-09 01:27:54.019 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:91 - Key not found, it will be initialized randomly: model.scene_max
2023-06-09 01:27:54.145 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:100 - criterion.empty_weight not in loaded checkpoint
2023-06-09 01:27:54.149 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:115 - excessive key: model.scene_min
2023-06-09 01:27:54.149 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:115 - excessive key: model.scene_max
[2023-06-09 01:27:54,238][main][INFO] - {'general_train_mode': False, 'general_task': 'instance_segmentation', 'general_seed': None, 'general_checkpoint': 'checkpoints/scannet/scannet_val.ckpt', 'general_backbone_checkpoint': None, 'general_freeze_backbone': False, 'general_linear_probing_backbone': False, 'general_train_on_segments': True, 'general_eval_on_segments': True, 'general_filter_out_instances': False, 'general_save_visualizations': False, 'general_visualization_point_size': 20, 'general_decoder_id': -1, 'general_export': False, 'general_use_dbscan': True, 'general_ignore_class_threshold': 100, 'general_project_name': 'scannet_eval', 'general_workspace': 'jonasschult', 'general_experiment_name': 'test1_scannet_val_query_150_topk_500_dbscan_0.95', 'general_num_targets': 19, 'general_add_instance': True, 'general_dbscan_eps': 0.95, 'general_dbscan_min_points': 1, 'general_export_threshold': 0.0001, 'general_reps_per_epoch': 1, 'general_on_crops': False, 'general_scores_threshold': 0.0, 'general_iou_threshold': 1.0, 'general_area': 5, 'general_eval_inner_core': -1, 'general_topk_per_image': 500, 'general_ignore_mask_idx': [], 'general_max_batch_size': 99999999, 'general_save_dir': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'general_gpus': 1, 'data_train_mode': 'train', 'data_validation_mode': 'validation', 'data_test_mode': 'validation', 'data_ignore_label': 255, 'data_add_raw_coordinates': True, 'data_add_colors': True, 'data_add_normals': False, 'data_in_channels': 3, 'data_num_labels': 20, 'data_add_instance': True, 'data_task': 'instance_segmentation', 'data_pin_memory': False, 'data_num_workers': 4, 'data_batch_size': 5, 'data_test_batch_size': 1, 'data_cache_data': False, 'data_voxel_size': 0.02, 'data_reps_per_epoch': 1, 'data_cropping': False, 'data_cropping_args_min_points': 30000, 'data_cropping_args_aspect': 0.8, 'data_cropping_args_min_crop': 0.5, 'data_cropping_args_max_crop': 1.0, 'data_crop_min_size': 20000, 'data_crop_length': 6.0, 'data_cropping_v1': True, 'data_train_dataloader__target_': 'torch.utils.data.DataLoader', 'data_train_dataloader_shuffle': True, 'data_train_dataloader_pin_memory': False, 'data_train_dataloader_num_workers': 4, 'data_train_dataloader_batch_size': 5, 'data_validation_dataloader__target_': 'torch.utils.data.DataLoader', 'data_validation_dataloader_shuffle': False, 'data_validation_dataloader_pin_memory': False, 'data_validation_dataloader_num_workers': 4, 'data_validation_dataloader_batch_size': 1, 'data_test_dataloader__target_': 'torch.utils.data.DataLoader', 'data_test_dataloader_shuffle': False, 'data_test_dataloader_pin_memory': False, 'data_test_dataloader_num_workers': 4, 'data_test_dataloader_batch_size': 1, 'data_train_dataset__target_': 'datasets.semseg.SemanticSegmentationDataset', 'data_train_dataset_dataset_name': 'scannet', 'data_train_dataset_data_dir': 'data/processed/scannet', 'data_train_dataset_image_augmentations_path': 'conf/augmentation/albumentations_aug.yaml', 'data_train_dataset_volume_augmentations_path': 'conf/augmentation/volumentations_aug.yaml', 'data_train_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_train_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_train_dataset_data_percent': 1.0, 'data_train_dataset_mode': 'train', 'data_train_dataset_ignore_label': 255, 'data_train_dataset_num_labels': 20, 'data_train_dataset_add_raw_coordinates': True, 'data_train_dataset_add_colors': True, 'data_train_dataset_add_normals': False, 'data_train_dataset_add_instance': True, 'data_train_dataset_instance_oversampling': 0.0, 'data_train_dataset_place_around_existing': False, 'data_train_dataset_point_per_cut': 0, 'data_train_dataset_max_cut_region': 0, 'data_train_dataset_flip_in_center': False, 'data_train_dataset_noise_rate': 0, 'data_train_dataset_resample_points': 0, 'data_train_dataset_add_unlabeled_pc': False, 'data_train_dataset_cropping': False, 'data_train_dataset_cropping_args_min_points': 30000, 'data_train_dataset_cropping_args_aspect': 0.8, 'data_train_dataset_cropping_args_min_crop': 0.5, 'data_train_dataset_cropping_args_max_crop': 1.0, 'data_train_dataset_is_tta': False, 'data_train_dataset_crop_min_size': 20000, 'data_train_dataset_crop_length': 6.0, 'data_train_dataset_filter_out_classes': [0, 1], 'data_train_dataset_label_offset': 2, 'data_validation_dataset__target_': 'datasets.semseg.SemanticSegmentationDataset', 'data_validation_dataset_dataset_name': 'scannet', 'data_validation_dataset_data_dir': 'data/processed/scannet', 'data_validation_dataset_image_augmentations_path': None, 'data_validation_dataset_volume_augmentations_path': None, 'data_validation_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_validation_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_validation_dataset_data_percent': 1.0, 'data_validation_dataset_mode': 'validation', 'data_validation_dataset_ignore_label': 255, 'data_validation_dataset_num_labels': 20, 'data_validation_dataset_add_raw_coordinates': True, 'data_validation_dataset_add_colors': True, 'data_validation_dataset_add_normals': False, 'data_validation_dataset_add_instance': True, 'data_validation_dataset_cropping': False, 'data_validation_dataset_is_tta': False, 'data_validation_dataset_crop_min_size': 20000, 'data_validation_dataset_crop_length': 6.0, 'data_validation_dataset_filter_out_classes': [0, 1], 'data_validation_dataset_label_offset': 2, 'data_test_dataset__target_': 'datasets.semseg.SemanticSegmentationDataset', 'data_test_dataset_dataset_name': 'scannet', 'data_test_dataset_data_dir': 'data/processed/scannet', 'data_test_dataset_image_augmentations_path': None, 'data_test_dataset_volume_augmentations_path': None, 'data_test_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_test_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_test_dataset_data_percent': 1.0, 'data_test_dataset_mode': 'validation', 'data_test_dataset_ignore_label': 255, 'data_test_dataset_num_labels': 20, 'data_test_dataset_add_raw_coordinates': True, 'data_test_dataset_add_colors': True, 'data_test_dataset_add_normals': False, 'data_test_dataset_add_instance': True, 'data_test_dataset_cropping': False, 'data_test_dataset_is_tta': False, 'data_test_dataset_crop_min_size': 20000, 'data_test_dataset_crop_length': 6.0, 'data_test_dataset_filter_out_classes': [0, 1], 'data_test_dataset_label_offset': 2, 'data_train_collation__target_': 'datasets.utils.VoxelizeCollate', 'data_train_collation_ignore_label': 255, 'data_train_collation_voxel_size': 0.02, 'data_train_collation_mode': 'train', 'data_train_collation_small_crops': False, 'data_train_collation_very_small_crops': False, 'data_train_collation_batch_instance': False, 'data_train_collation_probing': False, 'data_train_collation_task': 'instance_segmentation', 'data_train_collation_ignore_class_threshold': 100, 'data_train_collation_filter_out_classes': [0, 1], 'data_train_collation_label_offset': 2, 'data_train_collation_num_queries': 150, 'data_validation_collation__target_': 'datasets.utils.VoxelizeCollate', 'data_validation_collation_ignore_label': 255, 'data_validation_collation_voxel_size': 0.02, 'data_validation_collation_mode': 'validation', 'data_validation_collation_batch_instance': False, 'data_validation_collation_probing': False, 'data_validation_collation_task': 'instance_segmentation', 'data_validation_collation_ignore_class_threshold': 100, 'data_validation_collation_filter_out_classes': [0, 1], 'data_validation_collation_label_offset': 2, 'data_validation_collation_num_queries': 150, 'data_test_collation__target_': 'datasets.utils.VoxelizeCollate', 'data_test_collation_ignore_label': 255, 'data_test_collation_voxel_size': 0.02, 'data_test_collation_mode': 'validation', 'data_test_collation_batch_instance': False, 'data_test_collation_probing': False, 'data_test_collation_task': 'instance_segmentation', 'data_test_collation_ignore_class_threshold': 100, 'data_test_collation_filter_out_classes': [0, 1], 'data_test_collation_label_offset': 2, 'data_test_collation_num_queries': 150, 'logging': [{'target': 'pytorch_lightning.loggers.WandbLogger', 'project': 'scannet_eval', 'name': 'test1_scannet_val_query_150_topk_500_dbscan_0.95', 'save_dir': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'entity': 'manjusaka_labs', 'resume': 'allow', 'id': 'test1_scannet_val_query_150_topk_500_dbscan_0.95'}], 'model__target_': 'models.Mask3D', 'model_hidden_dim': 128, 'model_dim_feedforward': 1024, 'model_num_queries': 150, 'model_num_heads': 8, 'model_num_decoders': 3, 'model_dropout': 0.0, 'model_pre_norm': False, 'model_use_level_embed': False, 'model_normalize_pos_enc': True, 'model_positional_encoding_type': 'fourier', 'model_gauss_scale': 1.0, 'model_hlevels': [0, 1, 2, 3], 'model_non_parametric_queries': True, 'model_random_query_both': False, 'model_random_normal': False, 'model_random_queries': False, 'model_use_np_features': False, 'model_sample_sizes': [200, 800, 3200, 12800, 51200], 'model_max_sample_size': False, 'model_shared_decoder': True, 'model_num_classes': 19, 'model_train_on_segments': True, 'model_scatter_type': 'mean', 'model_voxel_size': 0.02, 'model_config_backbone__target_': 'models.Res16UNet34C', 'model_config_backbone_config_dialations': [1, 1, 1, 1], 'model_config_backbone_config_conv1_kernel_size': 5, 'model_config_backbone_config_bn_momentum': 0.02, 'model_config_backbone_in_channels': 3, 'model_config_backbone_out_channels': 20, 'model_config_backbone_out_fpn': True, 'metrics__target_': 'models.metrics.ConfusionMatrix', 'metrics_num_classes': 20, 'metrics_ignore_label': 255, 'optimizer__target_': 'torch.optim.AdamW', 'optimizer_lr': 0.0001, 'scheduler_scheduler__target_': 'torch.optim.lr_scheduler.OneCycleLR', 'scheduler_scheduler_max_lr': 0.0001, 'scheduler_scheduler_epochs': 601, 'scheduler_scheduler_steps_per_epoch': -1, 'scheduler_pytorch_lightning_params_interval': 'step', 'trainer_deterministic': False, 'trainer_max_epochs': 601, 'trainer_min_epochs': 1, 'trainer_resume_from_checkpoint': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95/last-epoch.ckpt', 'trainer_check_val_every_n_epoch': 50, 'trainer_num_sanity_val_steps': 2, 'callbacks': [{'target': 'pytorch_lightning.callbacks.ModelCheckpoint', 'monitor': 'val_mean_ap_50', 'save_last': True, 'save_top_k': 1, 'mode': 'max', 'dirpath': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'filename': '{epoch}-{val_mean_ap_50:.3f}', 'every_n_epochs': 1}, {'target': 'pytorch_lightning.callbacks.LearningRateMonitor'}], 'matcher__target_': 'models.matcher.HungarianMatcher', 'matcher_cost_class': 2.0, 'matcher_cost_mask': 5.0, 'matcher_cost_dice': 2.0, 'matcher_num_points': -1, 'loss__target_': 'models.criterion.SetCriterion', 'loss_num_classes': 19, 'loss_eos_coef': 0.1, 'loss_losses': ['labels', 'masks'], 'loss_num_points': -1, 'loss_oversample_ratio': 3.0, 'loss_importance_sample_ratio': 0.75, 'loss_class_weights': -1}
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py:446: LightningDeprecationWarning: Setting Trainer(gpus=1) is deprecated in v1.7 and will be removed in v2.0. Please use Trainer(accelerator='gpu', devices=1) instead.
rank_zero_deprecation(
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py:52: LightningDeprecationWarning: Setting Trainer(resume_from_checkpoint=) is deprecated in v1.5 and will be removed in v1.7. Please pass Trainer.fit(ckpt_path=) directly instead.
rank_zero_deprecation(
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:57: LightningDeprecationWarning: Setting Trainer(weights_save_path=) has been deprecated in v1.6 and will be removed in v1.8. Please pass dirpath directly to the ModelCheckpoint callback
rank_zero_deprecation(
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/home/mylabs/Mask3D/datasets/semseg.py:696: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
file = yaml.load(f)
Traceback (most recent call last):
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/utils.py", line 63, in call
return _instantiate_class(type_or_callable, config, *args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 500, in _instantiate_class
return clazz(*args, **final_kwargs)
File "/home/mylabs/Mask3D/datasets/semseg.py", line 218, in init
self._labels = self._select_correct_labels(labels, num_labels)
File "/home/mylabs/Mask3D/datasets/semseg.py", line 724, in _select_correct_labels
raise ValueError(msg)
ValueError: not available number labels, select from:
200, 200

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/core/utils.py", line 128, in run_job
ret.return_value = task_function(task_cfg)
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 110, in main
test(cfg)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 27, in decorated_main
return task_function(cfg_passthrough)
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 100, in test
runner.test(model)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 864, in test
return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in _test_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1097, in _run
self._data_connector.prepare_data()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 120, in prepare_data
self.trainer._call_lightning_module_hook("prepare_data")
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1552, in _call_lightning_module_hook
output = fn(*args, **kwargs)
File "/home/mylabs/Mask3D/trainer/trainer.py", line 1269, in prepare_data
self.train_dataset = hydra.utils.instantiate(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/utils.py", line 70, in call
raise HydraException(f"Error calling '{cls}' : {e}") from e
hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from:
200, 200

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 114, in
main()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report
print_exception(etype=None, value=ex, tb=final_tb) # type: ignore
TypeError: print_exception() got an unexpected keyword argument 'etype'

Can you tell me how to solve the problem, thank you very much!

[xibi777]

I encountered the same question! Did you solve it?

[JonasSchult]

Hi!

Is it possible that your generated dataset is ScanNet200 and not ScanNet?

Best,
Jonas

[bh-cai]

Your reply and help are greatly appreciated!
However,
I regenrated the scannet data by " python python -m datasets.preprocessing.scannet_preprocessing preprocess
--data_dir="./data/raw/scannet/scannet"
--save_dir="./data/processed/scannet"
--git_repo="./data/raw/scannet/ScanNet"
--scannet200=False"
the result of data structure like flow:
ScanNet200
├── instance_gt /
│ ├── train /
│ │ ├── scene0000_00.txt
│ │ ├── ...
│ │ └── scene0706_00.txt
│ └── validation /
│ ├── scene0011_00.txt
│ ├── ...
│ └── scene0704_01.txt
├── train /
│ ├── 0000_00.npy
│ ├── ...
│ └── 0706_00.npy
├── validation /
│ ├── 0011_00.npy
│ ├── ...
│ └── 0704_01.npy
├── test /
│ ├── 0707_00.npy
│ ├── ...
│ └── 0806_00.npy
├── color_mean_std.yaml 137B
├── label_database.yaml 15KB
├── test_database.yaml 24KB
├── train_database.yaml 1.19M
├── validation_database.yaml 319KB
└── train_validation_database.yaml 1.50M

and regenerated the scannet200 data by

"python -m datasets.preprocessing.scannet_preprocessing preprocess
--data_dir="./data/raw/scannet/scannet"
--save_dir="./data/processed/scannet200"
--git_repo="./data/raw/scannet/ScanNet"
--scannet200=true"
the result of data structure like flow:
ScanNet200
├── instance_gt /
│ ├── train /
│ │ ├── scene0000_00.txt
│ │ ├── ...
│ │ └── scene0706_00.txt
│ └── validation /
│ ├── scene0011_00.txt
│ ├── ...
│ └── scene0704_01.txt
├── train /
│ ├── 0000_00.npy
│ ├── ...
│ └── 0706_00.npy
├── validation /
│ ├── 0011_00.npy
│ ├── ...
│ └── 0704_01.npy
├── test /
│ ├── 0707_00.npy
│ ├── ...
│ └── 0806_00.npy
├── color_mean_std.yaml 137B
├── label_database.yaml 15KB
├── test_database.yaml 23KB
├── train_database.yaml 1.17M
├── validation_database.yaml 319KB
└── train_validation_database.yaml 1.14M
I used the same raw dataset "--data_dir="./data/raw/scannet/scannet" ",and just make the different command like " --scannet200=False" or " --scannet200=true".
In the end, I got the same structure output data, and there just have the different on the size of data.
So, if I used a wrong command? I not very understand clearly.
Looking forward to your reply and help,best wish for you!

Hi!

Is it possible that your generated dataset is ScanNet200 and not ScanNet?

Best, Jonas

[bh-cai]

And when I run the train command as "python main_instance_segmentation.py" or "python main_instance_segmentation.py \ general.checkpoint='/home/mylabs/Mask3D/checkpoints/s3dis/from_scratch/area1.ckpt'
general.train_mode=false" there was the same error.

[bh-cai]

I encountered the same question! Did you solve it?

not yet.

[Haiyan-Chris-Wang]

I have the similar problem and hope the author could help to explain it. Many thanks! @JonasSchult

Mask3D/main_instance_segmentation.py", line 79, in train runner = Trainer( File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/argparse.py", line 345, in insert_env_defaults return fn(self, **kwargs)
File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 485, in init
self._callback_connector.on_trainer_init(
File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py", line 89, in on_trainer_init
self.trainer.callbacks.extend(_configure_external_callbacks())
File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py", line 265, in _configure_external_callbacks
factories = entry_points(group=group) # type: ignore[call-arg]
File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 1021, in entry_points return SelectableGroups.load(eps).select(**params) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 459, in load ordered = sorted(eps, key=by_group) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 1018, in eps = itertools.chain.from_iterable( File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen k = key(element) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 943, in _normalized_name or super().normalized_name
File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 622, in normalized_name return Prepared.normalize(self.name) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 871, in normalize return re.sub(r"[-.]+", "-", name).lower().replace('-', '') File "/opt/conda/envs/mask3d/lib/python3.10/re.py", line 209, in sub return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Mask3D/main_instance_segmentation.py", line 116, in
main()
File "/opt/conda/envs/mask3d/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/opt/conda/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/opt/conda/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report
print_exception(etype=None, value=ex, tb=final_tb) # type: ignore
TypeError: print_exception() got an unexpected keyword argument 'etype'

[Jiayuan-Gu]

It seems that the issue results from

Mask3D/datasets/semseg.py

Lines 699 to 720 in 3db966d

	def _select_correct_labels(self, labels, num_labels):
	number_of_validation_labels = 0
	number_of_all_labels = 0
	for (
	k,
	v,
	) in labels.items():
	number_of_all_labels += 1
	if v["validation"]:
	number_of_validation_labels += 1
	if num_labels == number_of_all_labels:
	return labels
	elif num_labels == number_of_validation_labels:
	valid_labels = dict()
	for (
	k,
	v,
	) in labels.items():
	if v["validation"]:
	valid_labels.update({k: v})
	return valid_labels

.
The label_database.yaml is the same for both ScanNet and ScanNet200 after preprocessing.

[bh-cai]

Thank you for your reply, but I don't understand how to change the code to solve this problem. May I get some advice from you?

[Jiayuan-Gu]

You just need to rerun scannet data preprocessing with --scannet200=False/True instead of --scannet200=false/true.
See my pull request: #111

[bh-cai]

Thank you very much, I have solved the issue. However, I have got another problem with the flowing:
"pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='val_mean_ap_50') could not find the monitored key in the returned metrics: ['train_loss_ce', 'train_loss_mask', 'train_loss_dice', 'train_loss_ce_0', 'train_loss_mask_0', 'train_loss_dice_0', 'train_loss_ce_1', 'train_loss_mask_1', 'train_loss_dice_1', 'train_loss_ce_2', 'train_loss_mask_2', 'train_loss_dice_2', 'train_loss_ce_3', 'train_loss_mask_3', 'train_loss_dice_3', 'train_loss_ce_4', 'train_loss_mask_4', 'train_loss_dice_4', 'train_loss_ce_5', 'train_loss_mask_5', 'train_loss_dice_5', 'train_loss_ce_6', 'train_loss_mask_6', 'train_loss_dice_6', 'train_loss_ce_7', 'train_loss_mask_7', 'train_loss_dice_7', 'train_loss_ce_8', 'train_loss_mask_8', 'train_loss_dice_8', 'train_loss_ce_9', 'train_loss_mask_9', 'train_loss_dice_9', 'train_loss_ce_10', 'train_loss_mask_10', 'train_loss_dice_10', 'train_loss_ce_11', 'train_loss_mask_11', 'train_loss_dice_11', 'train_mean_loss_ce', 'train_mean_loss_mask', 'train_mean_loss_dice', 'epoch', 'step']. HINT: Did you call log('val_mean_ap_50', value) in the LightningModule?
"

Epoch 49: 100%|████| 1513/1513 [1:56:39<00:00, 4.63s/it, loss=43.9, v_num=TION]
Traceback (most recent call last):
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/core/utils.py", line 128, in run_job
ret.return_value = task_function(task_cfg)
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 108, in main
train(cfg)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 27, in decorated_main
return task_function(cfg_passthrough)
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 84, in train
runner.fit(model)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit
self._call_and_handle_interrupt(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 737, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1168, in _run
results = self._run_stage()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1254, in _run_stage
return self._run_train()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1285, in _run_train
self.fit_loop.run()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 200, in run
self.advance(*args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 270, in advance
self._outputs = self.epoch_loop.run(self._data_fetcher)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 201, in run
self.on_advance_end()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 241, in on_advance_end
self._run_validation()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 299, in _run_validation
self.val_loop.run()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 207, in run
output = self.on_run_end()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 201, in on_run_end
self._on_evaluation_end()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 265, in _on_evaluation_end
self.trainer._call_callback_hooks(hook_name, *args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1599, in _call_callback_hooks
fn(self, self.lightning_module, *args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 319, in on_validation_end
self._save_topk_checkpoint(trainer, monitor_candidates)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 380, in _save_topk_checkpoint
raise MisconfigurationException(m)
pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='val_mean_ap_50') could not find the monitored key in the returned metrics: ['train_loss_ce', 'train_loss_mask', 'train_loss_dice', 'train_loss_ce_0', 'train_loss_mask_0', 'train_loss_dice_0', 'train_loss_ce_1', 'train_loss_mask_1', 'train_loss_dice_1', 'train_loss_ce_2', 'train_loss_mask_2', 'train_loss_dice_2', 'train_loss_ce_3', 'train_loss_mask_3', 'train_loss_dice_3', 'train_loss_ce_4', 'train_loss_mask_4', 'train_loss_dice_4', 'train_loss_ce_5', 'train_loss_mask_5', 'train_loss_dice_5', 'train_loss_ce_6', 'train_loss_mask_6', 'train_loss_dice_6', 'train_loss_ce_7', 'train_loss_mask_7', 'train_loss_dice_7', 'train_loss_ce_8', 'train_loss_mask_8', 'train_loss_dice_8', 'train_loss_ce_9', 'train_loss_mask_9', 'train_loss_dice_9', 'train_loss_ce_10', 'train_loss_mask_10', 'train_loss_dice_10', 'train_loss_ce_11', 'train_loss_mask_11', 'train_loss_dice_11', 'train_mean_loss_ce', 'train_mean_loss_mask', 'train_mean_loss_dice', 'epoch', 'step']. HINT: Did you call log('val_mean_ap_50', value) in the LightningModule?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 114, in
main()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report
print_exception(etype=None, value=ex, tb=final_tb) # type: ignore
TypeError: print_exception() got an unexpected keyword argument 'etype'
wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.

I am not very clear about the problem above, may I get some help from you, if so, it would my best greatest.

[asadjan1801]

I have the same problem. Did you ever find solution for this problem.

[zoeyliu1999]

Hey, I just found that the function traceback.print_exception() has been changed in python v3.10, where the 'etype' parameter has been renamed to 'exc' and is now positional-only.

So I rewrite the file "/Users/.../python3.10/site-packages/hydra/_internal/utils.py", line 267 from

print_exception(etype=None, value=ex, tb=final_tb)

to

print_exception(None, value=ex, tb=final_tb)

And things go well. The package's change log can be found in website).

[Xin200203]

You just need to rerun scannet data preprocessing with --scannet200=False/True instead of --scannet200=false/true.您只需使用 --scannet200=False/True 而不是 --scannet200=false/true 重新运行 scannet 数据预处理。 See my pull request: #111查看我的拉取请求： #111

Thank you soooooo much! This saved my day😭