Enable profile for cpu userbenchmark #1759
Conversation
|
Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
WeizhuoZhang-intel
changed the title
chuanqi129
changed the title
| print("[INFO] Collecting Profiling logs...") | ||
| with profiler.profile(activities=[profiler.ProfilerActivity.CPU]) as prof, profiler.record_function("model_inference"): | ||
| result: TorchBenchModelMetrics = get_model_test_metrics(model, metrics=metrics) | ||
| print(prof.key_averages().table(sort_by="self_cpu_time_total", row_limit=-1)) |
There was a problem hiding this comment.
Can we also support to write the profiling results to a json file?
There was a problem hiding this comment.
Can we also support to write the profiling results to a json file?
It's not easy to transfer this table to Json file. As Json file is not good at showing table-content in format.
The torch.profile provide a function for chrome_trace json file, which can only open in Chrome Browser or VScode.
And prof.key_averages().table() is suggested print out directly, as the line breaker \n will be printed at the end each line.
It looks like the command line output is the most suggested way.
>>> prof.key_averages().table()
'------------------- ------------ ------------ ------------ ------------ ------------ ------------ \n Name Self CPU % Self CPU CPU total % CPU total CPU time avg # of Calls \n------------------- ------------ ------------ ------------ ------------ ------------ ------------ \n model_inference 100.00% 124.000us 100.00% 124.000us 124.000us 1 \n------------------- ------------ ------------ ------------ ------------ ------------ ------------ \nSelf CPU time total: 124.000us\n'
>>> prof.key_averages()
There was a problem hiding this comment.
ok, seems it profile for all iterations, is it our expected behavior? refer profile in run.py, https://github.com/pytorch/benchmark/blob/main/run.py#L187
| @@ -59,7 +60,13 @@ def run_config(config: TorchBenchModelConfig, metrics: List[str], dryrun: bool=F | |||
| # load the model instance within current process | |||
| model = load_model(config) | |||
| # get the model test metrics | |||
| result: TorchBenchModelMetrics = get_model_test_metrics(model, metrics=metrics) | |||
| if profile: | |||
There was a problem hiding this comment.
Better to move the import torch.profiler as profiler under this line
| result: TorchBenchModelMetrics = get_model_test_metrics(model, metrics=metrics) | ||
| if profile: | ||
| print("[INFO] Collecting Profiling logs...") | ||
| with profiler.profile(activities=[profiler.ProfilerActivity.CPU]) as prof, profiler.record_function("model_inference"): |
There was a problem hiding this comment.
Not suggest to use "model_inference" in here, it also support train test
| print("[INFO] Collecting Profiling logs...") | ||
| with profiler.profile(activities=[profiler.ProfilerActivity.CPU]) as prof, profiler.record_function("model_inference"): | ||
| result: TorchBenchModelMetrics = get_model_test_metrics(model, metrics=metrics) | ||
| print(prof.key_averages().table(sort_by="self_cpu_time_total", row_limit=-1)) |
There was a problem hiding this comment.
ok, seems it profile for all iterations, is it our expected behavior? refer profile in run.py, https://github.com/pytorch/benchmark/blob/main/run.py#L187
|
@WeizhuoZhang-intel Can you please rebase this PR on the latest main branch? |
Enable a new option
--profilein cpu userbenchmark for profiling content based ontorch.profiler.Works for #1293