Skip to content

Commit e2aea39

Browse files
mrunalhirve128Copilot
mrunalhirve128
and
Copilot
authored
Introduce Testing Plan and Implement Runtime Module Tests (#74)
* Test plan * Fix running tests instructions * fix Readme * fix * format fix * fix plan * Add min tests * ruff check * Fix test cases * fix tests * fix * Update tests/TEST_PLAN.md Co-authored-by: Copilot <[email protected]> * copilot comment * Apply suggestion from @Copilot Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]>
1 parent 4592dc0 commit e2aea39

File tree

8 files changed

+703
-309
lines changed

8 files changed

+703
-309
lines changed

‎.gitignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,15 @@ build/
7474
.pytest_cache/
7575
_version.py
7676

77+
# Test coverage and reports
78+
htmlcov/
79+
.coverage
80+
coverage.xml
81+
test-report.html
82+
test-results.xml
83+
*.cover
84+
.hypothesis/
85+
7786
# Virtual environments
7887
.venv/
7988
venv/

‎tests/README.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Microsoft Agent 365 SDK Tests
2+
3+
Unit and integration tests for the Microsoft Agent 365-Python SDK. This test suite ensures reliability, maintainability, and quality across all modules including runtime, tooling, notifications, and observability extensions.
4+
5+
## Usage
6+
7+
For detailed instructions on running tests and generating coverage reports, see:
8+
9+
- **[Test Plan](TEST_PLAN.md)** - Comprehensive testing strategy and implementation roadmap
10+
- **[Running Tests](RUNNING_TESTS.md)** - Complete guide for installation, running tests, generating coverage reports, and troubleshooting
11+
12+
## Support
13+
14+
For issues, questions, or feedback:
15+
16+
- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-python/issues) section
17+
- See the [main documentation](../README.md) for more information
18+
19+
## Trademarks
20+
21+
*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.*
22+
23+
## License
24+
25+
Copyright (c) Microsoft Corporation. All rights reserved.
26+
27+
Licensed under the MIT License - see the [LICENSE](../LICENSE.md) file for details.

‎tests/RUNNING_TESTS.md‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# Running Unit Tests for Agent365-Python SDK
2+
3+
This guide covers setting up and running tests.
4+
5+
---
6+
7+
## Prerequisites
8+
9+
### 1. Create Virtual Environment
10+
11+
```powershell
12+
# Create virtual environment using uv
13+
uv venv
14+
15+
# Activate the virtual environment
16+
.\.venv\Scripts\Activate.ps1
17+
```
18+
19+
### 2. Configure Python Environment
20+
21+
1. Press `Ctrl+Shift+P`
22+
2. Type "Python: Select Interpreter"
23+
3. Choose the `.venv` interpreter from the list
24+
25+
### 3. Install Dependencies
26+
27+
```powershell
28+
# Test framework and reporting
29+
uv pip install pytest pytest-asyncio pytest-mock pytest-cov pytest-html wrapt
30+
31+
# SDK core libraries
32+
uv pip install -e libraries/microsoft-agents-a365-runtime -e libraries/microsoft-agents-a365-notifications -e libraries/microsoft-agents-a365-observability-core -e libraries/microsoft-agents-a365-tooling
33+
34+
# Framework extension libraries
35+
uv pip install -e libraries/microsoft-agents-a365-observability-extensions-langchain -e libraries/microsoft-agents-a365-observability-extensions-openai -e libraries/microsoft-agents-a365-observability-extensions-semantickernel -e libraries/microsoft-agents-a365-observability-extensions-agentframework -e libraries/microsoft-agents-a365-tooling-extensions-agentframework -e libraries/microsoft-agents-a365-tooling-extensions-azureaifoundry -e libraries/microsoft-agents-a365-tooling-extensions-openai -e libraries/microsoft-agents-a365-tooling-extensions-semantickernel
36+
```
37+
38+
---
39+
40+
## Test Structure
41+
42+
> **Note:** This structure will be updated as new tests are added.
43+
44+
```plaintext
45+
tests/
46+
├── runtime/ # Runtime tests
47+
├── observability/ # Observability tests
48+
├── tooling/ # Tooling tests
49+
└── notifications/ # Notifications tests
50+
```
51+
52+
---
53+
54+
## Running Tests in VS Code (Optional)
55+
56+
### Test Explorer
57+
58+
1. Click the beaker icon in the Activity Bar or press `Ctrl+Shift+P` → "Test: Focus on Test Explorer View"
59+
2. Click the play button to run tests (all/folder/file/individual)
60+
3. Right-click → "Debug Test" to debug with breakpoints
61+
62+
### Command Palette
63+
64+
- `Test: Run All Tests`
65+
- `Test: Run Tests in Current File`
66+
- `Test: Debug Tests in Current File`
67+
68+
---
69+
70+
## Running Tests from Command Line
71+
72+
```powershell
73+
# Run all tests
74+
python -m pytest tests/
75+
76+
# Run specific module/file
77+
python -m pytest tests/runtime/
78+
python -m pytest tests/runtime/test_environment_utils.py
79+
80+
# Run with options
81+
python -m pytest tests/ -v # Verbose
82+
python -m pytest tests/ -x # Stop on first failure
83+
python -m pytest tests/ -k "environment" # Pattern matching
84+
python -m pytest --lf # Re-run failed tests
85+
```
86+
87+
---
88+
89+
## Generating Reports
90+
91+
### HTML Reports
92+
93+
```powershell
94+
# Coverage report
95+
python -m pytest tests/ --cov=libraries --cov-report=html -v
96+
97+
# View reports
98+
start htmlcov\index.html
99+
100+
# Test report (requires: uv pip install pytest-html)
101+
python -m pytest tests/ --html=test-report.html --self-contained-html
102+
103+
# View reports
104+
start test-report.html
105+
106+
# Combined (requires: uv pip install pytest-html)
107+
python -m pytest tests/ --cov=libraries --cov-report=html --html=test-report.html --self-contained-html -v
108+
109+
# View reports
110+
start htmlcov\index.html
111+
```
112+
113+
### CI/CD Reports
114+
115+
```powershell
116+
# XML reports for CI/CD pipelines
117+
python -m pytest tests/ --cov=libraries --cov-report=xml --junitxml=test-results.xml
118+
119+
# View reports
120+
start test-results.xml
121+
start coverage.xml
122+
```
123+
124+
---
125+
126+
## Troubleshooting
127+
128+
### Common Issues
129+
130+
| Issue | Solution |
131+
|-------|----------|
132+
| **Test loading failed** | Clean pyproject.toml, reinstall packages, restart VS Code |
133+
| **ImportError: No module named 'pytest'** | `uv pip install pytest pytest-asyncio pytest-mock` |
134+
| **ImportError: No module named 'microsoft_agents_a365'** | `uv pip install -e .` |
135+
| **Tests not discovered** | Refresh Test Explorer or restart VS Code |
136+
137+
### Fix Steps
138+
139+
If tests fail to discover or import errors occur:
140+
141+
**1. Clean pyproject.toml**
142+
143+
```powershell
144+
$content = Get-Content "pyproject.toml" -Raw
145+
$fixed = $content -replace "`r`n", "`n"
146+
$fixed | Set-Content "pyproject.toml" -NoNewline
147+
```
148+
149+
**2. Reinstall packages**
150+
151+
```powershell
152+
uv pip install -e .
153+
```
154+
155+
**3. Restart VS Code**
156+
157+
- Close completely and reopen
158+
- Wait for Python extension to reload
159+
- Refresh Test Explorer

‎tests/TEST_PLAN.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Test Plan for Agent365-Python SDK
2+
3+
> **Note:** This plan is under active development. Keep updating as testing progresses.
4+
5+
**Version:** 1.0
6+
**Date:** November 24, 2025
7+
**Status:** Draft
8+
9+
---
10+
11+
## Overview
12+
13+
### Current State
14+
- ✅ Unit tests exist for `observability` and `runtime` modules
15+
- ❌ Missing tests for `tooling` and `notifications` modules
16+
- ❌ No integration tests or CI/CD automation
17+
18+
### Goals
19+
- Achieve **80%+ code coverage** across all modules
20+
- Implement integration tests for cross-module functionality
21+
- Integrate testing into CI/CD pipeline with coverage enforcement
22+
23+
---
24+
25+
## Testing Strategy
26+
27+
**Framework:** `pytest`
28+
**Coverage:** `pytest-cov`
29+
**Mocking:** `unittest.mock`
30+
**Async:** `pytest-asyncio`
31+
32+
**Test Pattern:** AAA (Arrange → Act → Assert)
33+
**Naming Convention:** `test_<method>_<condition>_<expected_result>`
34+
35+
---
36+
37+
## Implementation Roadmap
38+
39+
| Phase | Deliverables | Priority |
40+
|-------|-------------|----------|
41+
| 1.1 | Runtime unit tests | ✅ Complete |
42+
| 1.2 | Tooling unit tests | HIGH |
43+
| 1.3 | Notifications unit tests | HIGH |
44+
| 1.4 | Expand observability tests | MEDIUM |
45+
| 1.5 | Tooling extension tests | LOW |
46+
| 2 | Integration tests | MEDIUM |
47+
| 3 | CI/CD automation | HIGH |
48+
49+
---
50+
51+
## Phase 1: Unit Tests
52+
53+
### 1.1 Runtime Module
54+
**Priority:** HIGH
55+
56+
| Module | Test File | Status |
57+
|--------|-----------|--------|
58+
| `power_platform_api_discovery.py` | `test_power_platform_api_discovery.py` | ✅ Complete |
59+
| `utility.py` | `test_utility.py` | ✅ Complete |
60+
| `environment_utils.py` | `test_environment_utils.py` | ✅ Complete |
61+
| `version_utils.py` | `test_version_utils.py` | ✅ Complete |
62+
63+
---
64+
65+
### 1.2 Tooling Module
66+
**Priority:** HIGH
67+
68+
| Module | Test File | Status |
69+
|--------|-----------|--------|
70+
| `utils/utility.py` | `test_utility.py` | ❌ Missing |
71+
| `models/mcp_server_config.py` | `test_mcp_server_config.py` | ❌ Missing |
72+
| `services/mcp_tool_server_configuration_service.py` | `test_mcp_tool_server_configuration_service.py` | ❌ Missing |
73+
74+
---
75+
76+
### 1.3 Notifications Module
77+
**Priority:** HIGH
78+
79+
| Module | Test File | Status |
80+
|--------|-----------|--------|
81+
| `models/agent_lifecycle_event.py` | `test_agent_lifecycle_event.py` | ❌ Missing |
82+
| `models/agent_notification_activity.py` | `test_agent_notification_activity.py` | ❌ Missing |
83+
| `models/email_reference.py` | `test_email_reference.py` | ❌ Missing |
84+
| `agent_notification.py` | `test_agent_notification.py` | ❌ Missing |
85+
86+
---
87+
88+
### 1.4 Observability Extensions
89+
**Priority:** MEDIUM
90+
91+
| Extension | Status |
92+
|-----------|--------|
93+
| `agentframework` | ✅ Expand existing |
94+
| `langchain` | ✅ Expand existing |
95+
| `openai` | ✅ Expand existing |
96+
| `semantickernel` | ✅ Expand existing |
97+
98+
---
99+
100+
### 1.5 Tooling Extensions
101+
**Priority:** LOW
102+
103+
| Extension | Status |
104+
|-----------|--------|
105+
| Agent Framework | ❌ Missing |
106+
| Azure AI Foundry | ❌ Missing |
107+
| OpenAI | ❌ Missing |
108+
| Semantic Kernel | ❌ Missing |
109+
110+
---
111+
112+
## Phase 2: Integration Tests
113+
114+
**Priority:** MEDIUM
115+
116+
| Integration | Status |
117+
|-------------|--------|
118+
| Runtime + Observability | ❌ Missing |
119+
| Tooling + Runtime | ❌ Missing |
120+
| Notifications + Runtime | ❌ Missing |
121+
| Agent Framework full flow | ❌ Missing |
122+
| LangChain full flow | ❌ Missing |
123+
124+
---
125+
126+
## Phase 3: CI/CD Integration
127+
128+
**Priority:** HIGH
129+
130+
| Component | Status |
131+
|-----------|--------|
132+
| GitHub Actions workflow | ❌ Missing |
133+
| Python matrix (3.9-3.12) | ❌ Missing |
134+
| Coverage enforcement (80%+) | ❌ Missing |
135+
| Codecov integration | ❌ Missing |
136+
| PR blocking on failures | ❌ Missing |
137+
138+
---
139+
140+
## Success Criteria
141+
142+
- ✅ 80%+ code coverage for all modules
143+
- ✅ All tests pass independently
144+
- ✅ Full suite completes in < 30 seconds (unit) / < 5 minutes (full)
145+
- ✅ Automated test execution on all PRs
146+
- ✅ Coverage reports visible and enforced

‎tests/runtime/test_environment_utils.py‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT License.
3+
4+
"""Unit tests for environment_utils module."""
5+
6+
import pytest
7+
8+
from microsoft_agents_a365.runtime.environment_utils import (
9+
PROD_OBSERVABILITY_SCOPE,
10+
get_observability_authentication_scope,
11+
is_development_environment,
12+
)
13+
14+
15+
def test_get_observability_authentication_scope():
16+
"""Test get_observability_authentication_scope returns production scope."""
17+
result = get_observability_authentication_scope()
18+
assert result == [PROD_OBSERVABILITY_SCOPE]
19+
20+
21+
@pytest.mark.parametrize(
22+
"env_value,expected",
23+
[
24+
(None, False),
25+
("Development", True),
26+
("production", False),
27+
("staging", False),
28+
],
29+
)
30+
def test_is_development_environment(monkeypatch, env_value, expected):
31+
"""Test is_development_environment returns correct value based on PYTHON_ENVIRONMENT."""
32+
if env_value is None:
33+
monkeypatch.delenv("PYTHON_ENVIRONMENT", raising=False)
34+
else:
35+
monkeypatch.setenv("PYTHON_ENVIRONMENT", env_value)
36+
37+
result = is_development_environment()
38+
assert result == expected

0 commit comments

Comments
 (0)