Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't get rf_id in kineto trace!!!, trace_link.py can't find relation of cpu_op. #152

Open
32HD opened this issue Sep 13, 2024 · 2 comments

Comments

@32HD
Copy link

32HD commented Sep 13, 2024

chakra_device_trace_loader.py:

kineto_rf_id_to_kineto_op_map = {op.rf_id: op for op in kineto_cpu_ops if op.rf_id is not None}


kineto.jason:

{
    "ph": "X", "cat": "cpu_op", "name": "MseLossBackward0", "pid": 1182, "tid": 1331,
    "ts": 1726134043823422, "dur": 158,
    "args": {
      "External id": 2562,"Sequence number": 4583, "Fwd thread id": 1, "Ev Idx": 1
    }
  },
  {
    "ph": "X", "cat": "cpu_op", "name": "aten::mse_loss_backward", "pid": 1182, "tid": 1331,
    "ts": 1726134043823444, "dur": 135,
    "args": {
      "External id": 2563,"Ev Idx": 2
    }
  },
  {
    "ph": "X", "cat": "cpu_op", "name": "aten::zeros_like", "pid": 1182, "tid": 1331,
    "ts": 1726134043823452, "dur": 90,
    "args": {
      "External id": 2564,"Ev Idx": 3

**There is no rf_id entry, that means trace_linker.py can't find connection of TE.json and kineto.json. **

Maybe pytorch version is key

@32HD
Copy link
Author

32HD commented Sep 13, 2024

@32HD 32HD changed the title Can't get rf_id in kineto trace!!!, trace_link.py can't find relation with gpu_op with cpu_op. Can't get rf_id in kineto trace!!!, trace_link.py can't find relation of cpu_op. Sep 13, 2024
@JoongunPark
Copy link
Contributor

How did you collect the trace? Have you used the recent PyTorch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants