You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Trying to trace function "__execute_sample_during_training" from https://github.com/Nerogar/OneTrainer
There are some errors at the end about <string> and <frozen importlib._bootstrap>:
__execute_sample_during_training - put_events: so far: 918571
finalizing native tracer
queue length 0
put_file_slice
tracing_did_complete
got evt EventsSlice
queue length 1
got evt FileContentSlice
error! at store_file_contents:
[Errno 22] Invalid argument: ''
error! at store_file_contents:
[Errno 22] Invalid argument: ''
skip_to_free_header_chunk
pos = 12
after pos = 28
queue length 0
got evt StopCommand
skip_to_free_header_chunk
pos = 12
after pos = 52
metadata saved to C:\Users\Max\Desktop\OneTrainer\pycrunch-recordings__execute_sample_during_training\pycrunch-trace.meta.json
C:\Users\Max\Desktop\OneTrainer\venv\lib\site-packages\torch\autograd\graph.py:744: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ..\aten\src\ATen\native\cudnn\Conv_v8.cpp:919.)
return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
step: 100%|████████████████████████████████████████████████| 1/1 [09:36<00:00, 576.06s/it, loss=1.49, smooth loss=1.49]
epoch: 100%|████████████████████████████████████████████████████████████████████████████| 1/1 [09:38<00:00, 578.23s/it]
But files pycrunch-trace.meta.json and session.chunked.pycrunch-trace appear as expected. When I open them in app.pytrace.com, I see nothing at screen but error in console:
google-protobuf.js:
{
"message": "Failure: Invalid wire type: 6 (at position 6)",
"reportErrorToServer": true,
"messagePattern": "Failure: Invalid wire type: %s (at position %s)"
}
pycrunch-trace v0.3
python 3.10
Something gone wrong?
The text was updated successfully, but these errors were encountered:
sorry, it seems these errors are not related to issues with protobuf. If I interrupt traced program execution early with a KeyboardInterrupt, the file opens without issues
Trying to trace function "__execute_sample_during_training" from https://github.com/Nerogar/OneTrainer
There are some errors at the end about <string> and <frozen importlib._bootstrap>:
__execute_sample_during_training - put_events: so far: 918571
finalizing native tracer
queue length 0
put_file_slice
tracing_did_complete
got evt EventsSlice
queue length 1
got evt FileContentSlice
error! at store_file_contents:
[Errno 22] Invalid argument: ''
error! at store_file_contents:
[Errno 22] Invalid argument: ''
skip_to_free_header_chunk
pos = 12
after pos = 28
queue length 0
got evt StopCommand
skip_to_free_header_chunk
pos = 12
after pos = 52
metadata saved to C:\Users\Max\Desktop\OneTrainer\pycrunch-recordings__execute_sample_during_training\pycrunch-trace.meta.json
C:\Users\Max\Desktop\OneTrainer\venv\lib\site-packages\torch\autograd\graph.py:744: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ..\aten\src\ATen\native\cudnn\Conv_v8.cpp:919.)
return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
step: 100%|████████████████████████████████████████████████| 1/1 [09:36<00:00, 576.06s/it, loss=1.49, smooth loss=1.49]
epoch: 100%|████████████████████████████████████████████████████████████████████████████| 1/1 [09:38<00:00, 578.23s/it]
But files pycrunch-trace.meta.json and session.chunked.pycrunch-trace appear as expected. When I open them in app.pytrace.com, I see nothing at screen but error in console:
google-protobuf.js:
{
"message": "Failure: Invalid wire type: 6 (at position 6)",
"reportErrorToServer": true,
"messagePattern": "Failure: Invalid wire type: %s (at position %s)"
}
pycrunch-trace v0.3
python 3.10
Something gone wrong?
The text was updated successfully, but these errors were encountered: