Skip to content

Conversation

@tcauth
Copy link

@tcauth tcauth commented Dec 9, 2025

Summary:
WHY:
Distributed testing in TorchRec requires the ability to mock functions and methods across multiple processes. Without this capability, tests that spawn multiple processes cannot easily mock expensive operations, external dependencies, or specific behaviors, making it difficult to write comprehensive unit tests for distributed PyTorch code. This leads to either incomplete test coverage or tests that require actual expensive resources.

This change is also for T245908194

WHAT:

  1. Created MultiProcessMock class in /data/sandcastle/boxes/fbsource/fbcode/torchrec/distributed/test_utils/multi_process.py:

    • Added mocks list to maintain mock configurations
    • Added add_mock(target, return_value, side_effect, **kwargs) to register new mocks
    • Added apply_mocks() to apply all registered mocks in child processes
    • Added clear_mocks() to clear all registered mocks
  2. Integrated MultiProcessMock with MultiProcessTestBase:

    • Added _mock_manager instance variable initialized in __init__
    • Added public add_mock(...) method for users to register mocks
    • Created _callable_wrapper_with_mocks() static method to apply mocks before calling test functions
    • Modified _run_multi_process_test() to pass mock manager to child processes and use the wrapper
    • Modified _run_multi_process_test_per_rank() to pass mock manager to child processes and use the wrapper
  3. Added comprehensive test suite in /data/sandcastle/boxes/fbsource/fbcode/torchrec/distributed/tests/test_multi_process_mock.py:

    • 6 unit tests for MultiProcessMock class functionality
    • 6 integration tests including baseline tests to ensure backward compatibility
    • Added BUCK target for the new test file

TEST:
All 12 tests pass successfully:

  • Unit tests verify mock storage, multiple mocks, side effects, clearing, and application
  • Integration tests verify cross-process mocking with return values, multiple mocks, and side effects
  • Baseline tests confirm backward compatibility - existing tests without mocks continue to work exactly as before
  • Fixed pickling issue by using module-level functions for side effects (required for multiprocessing)

Ran: buck2 test torchrec/distributed/tests:test_multi_process_mock
Result: Tests finished: Pass 12. Fail 0. Fatal 0. Skip 0.

Differential Revision: D88696058

Summary:
**WHY:**
Distributed testing in TorchRec requires the ability to mock functions and methods across multiple processes. Without this capability, tests that spawn multiple processes cannot easily mock expensive operations, external dependencies, or specific behaviors, making it difficult to write comprehensive unit tests for distributed PyTorch code. This leads to either incomplete test coverage or tests that require actual expensive resources.

This change is also for T245908194

**WHAT:**
1. Created `MultiProcessMock` class in `/data/sandcastle/boxes/fbsource/fbcode/torchrec/distributed/test_utils/multi_process.py`:
   - Added `mocks` list to maintain mock configurations
   - Added `add_mock(target, return_value, side_effect, **kwargs)` to register new mocks
   - Added `apply_mocks()` to apply all registered mocks in child processes
   - Added `clear_mocks()` to clear all registered mocks

2. Integrated `MultiProcessMock` with `MultiProcessTestBase`:
   - Added `_mock_manager` instance variable initialized in `__init__`
   - Added public `add_mock(...)` method for users to register mocks
   - Created `_callable_wrapper_with_mocks()` static method to apply mocks before calling test functions
   - Modified `_run_multi_process_test()` to pass mock manager to child processes and use the wrapper
   - Modified `_run_multi_process_test_per_rank()` to pass mock manager to child processes and use the wrapper

3. Added comprehensive test suite in `/data/sandcastle/boxes/fbsource/fbcode/torchrec/distributed/tests/test_multi_process_mock.py`:
   - 6 unit tests for `MultiProcessMock` class functionality
   - 6 integration tests including baseline tests to ensure backward compatibility
   - Added BUCK target for the new test file

**TEST:**
All 12 tests pass successfully:
- Unit tests verify mock storage, multiple mocks, side effects, clearing, and application
- Integration tests verify cross-process mocking with return values, multiple mocks, and side effects
- Baseline tests confirm backward compatibility - existing tests without mocks continue to work exactly as before
- Fixed pickling issue by using module-level functions for side effects (required for multiprocessing)

Ran: `buck2 test torchrec/distributed/tests:test_multi_process_mock`
Result: Tests finished: Pass 12. Fail 0. Fatal 0. Skip 0.

Differential Revision: D88696058
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 9, 2025
@meta-codesync
Copy link
Contributor

meta-codesync bot commented Dec 9, 2025

@tcauth has exported this pull request. If you are a Meta employee, you can view the originating Diff in D88696058.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant