|
| 1 | +--- |
| 2 | +applyTo: '**' |
| 3 | +--- |
| 4 | + |
| 5 | +# Functional Tests Management Instructions |
| 6 | + |
| 7 | +## 📍 **Location** |
| 8 | +All functional tests are stored in: `.\simplechat\application\single_app\functional_tests\` |
| 9 | + |
| 10 | +## 📂 **Directory Structure** |
| 11 | +The functional tests directory contains: |
| 12 | +- **Python test files** (`test_*.py`) - Executable test scripts |
| 13 | +- **JavaScript test files** (`test_*.js`) - Client-side/browser test scripts |
| 14 | +- **Documentation files** (`*.md`) - Test documentation and fix summaries |
| 15 | +- **flask_session/** - Session data for tests requiring authenticated state |
| 16 | + |
| 17 | +## 🎯 **When to Create Functional Tests** |
| 18 | + |
| 19 | +### **Always Create Tests For:** |
| 20 | +✅ **Bug Fixes** - Validate the fix works and prevents regression |
| 21 | +✅ **New Features** - Ensure functionality works as designed |
| 22 | +✅ **API Changes** - Verify operation consistency and compatibility |
| 23 | +✅ **Plugin Integration** - Test plugin loading, operation calls, error handling |
| 24 | +✅ **Database Migration** - Validate data migration and container operations |
| 25 | +✅ **UI/UX Changes** - Test display logic, user interactions, data flow |
| 26 | +✅ **Authentication/Security** - Verify access controls and data isolation |
| 27 | + |
| 28 | +### **Test Categories:** |
| 29 | +- **Integration Tests** - End-to-end functionality across multiple components |
| 30 | +- **Regression Tests** - Prevent previously fixed bugs from returning |
| 31 | +- **Consistency Tests** - Validate behavior remains consistent across iterations |
| 32 | +- **Migration Tests** - Verify data and schema migrations work correctly |
| 33 | + |
| 34 | +## 📝 **Naming Conventions** |
| 35 | + |
| 36 | +### **File Naming:** |
| 37 | +- **Python Tests**: `test_{feature_area}_{specific_test}.py` |
| 38 | +- **JavaScript Tests**: `test_{feature_area}_{specific_test}.js` |
| 39 | +- **Documentation**: `{FEATURE_AREA}_{DESCRIPTION}.md` |
| 40 | + |
| 41 | +### **Examples:** |
| 42 | +``` |
| 43 | +test_agent_citations_fix.py # Agent citation bug fix |
| 44 | +test_semantic_kernel_operation_consistency.py # SK operation reliability |
| 45 | +test_openapi_operation_lookup.py # OpenAPI plugin testing |
| 46 | +test_conversation_id_display.py # UI feature testing |
| 47 | +test_migration.py # Database migration testing |
| 48 | +AGENT_MODEL_DISPLAY_FIXES.md # Fix documentation |
| 49 | +``` |
| 50 | + |
| 51 | +## 🏗️ **Test Structure Patterns** |
| 52 | + |
| 53 | +### **Python Test Template:** |
| 54 | +```python |
| 55 | +#!/usr/bin/env python3 |
| 56 | +""" |
| 57 | +Brief description of what this test validates. |
| 58 | +
|
| 59 | +This test ensures [specific functionality] works correctly and |
| 60 | +prevents regression of [specific issue/bug]. |
| 61 | +""" |
| 62 | + |
| 63 | +import sys |
| 64 | +import os |
| 65 | +sys.path.append(os.path.dirname(os.path.abspath(__file__))) |
| 66 | + |
| 67 | +def test_primary_functionality(): |
| 68 | + """Test the main functionality.""" |
| 69 | + print("🔍 Testing [Feature Name]...") |
| 70 | + |
| 71 | + try: |
| 72 | + # Setup |
| 73 | + # Test execution |
| 74 | + # Validation |
| 75 | + # Cleanup |
| 76 | + |
| 77 | + print("✅ Test passed!") |
| 78 | + return True |
| 79 | + |
| 80 | + except Exception as e: |
| 81 | + print(f"❌ Test failed: {e}") |
| 82 | + import traceback |
| 83 | + traceback.print_exc() |
| 84 | + return False |
| 85 | + |
| 86 | +if __name__ == "__main__": |
| 87 | + success = test_primary_functionality() |
| 88 | + sys.exit(0 if success else 1) |
| 89 | +``` |
| 90 | + |
| 91 | +### **Multi-Test Pattern:** |
| 92 | +```python |
| 93 | +def test_feature_a(): |
| 94 | + """Test specific aspect A.""" |
| 95 | + # Implementation |
| 96 | + |
| 97 | +def test_feature_b(): |
| 98 | + """Test specific aspect B.""" |
| 99 | + # Implementation |
| 100 | + |
| 101 | +if __name__ == "__main__": |
| 102 | + tests = [test_feature_a, test_feature_b] |
| 103 | + results = [] |
| 104 | + |
| 105 | + for test in tests: |
| 106 | + print(f"\n🧪 Running {test.__name__}...") |
| 107 | + results.append(test()) |
| 108 | + |
| 109 | + success = all(results) |
| 110 | + print(f"\n📊 Results: {sum(results)}/{len(results)} tests passed") |
| 111 | + sys.exit(0 if success else 1) |
| 112 | +``` |
| 113 | + |
| 114 | +## 🔍 **Test Discovery & Reuse** |
| 115 | + |
| 116 | +### **Before Creating New Tests:** |
| 117 | +1. **Search existing tests**: `grep -r "test_.*{feature}" functional_tests/` |
| 118 | +2. **Check for similar patterns**: Look for tests in the same feature area |
| 119 | +3. **Review related documentation**: Check for `*.md` files describing fixes |
| 120 | +4. **Examine imports**: See what modules/functions are already being tested |
| 121 | + |
| 122 | +### **Reusable Test Components:** |
| 123 | +- **OpenAPI Testing**: Use `OpenApiPluginFactory` patterns from `test_openapi_*.py` |
| 124 | +- **Agent Testing**: Reference citation and model display test patterns |
| 125 | +- **Database Testing**: Follow migration test patterns for Cosmos DB operations |
| 126 | +- **Plugin Testing**: Use plugin logging patterns for function call validation |
| 127 | + |
| 128 | +## 🔧 **Common Testing Utilities** |
| 129 | + |
| 130 | +### **Available Imports:** |
| 131 | +```python |
| 132 | +# OpenAPI Plugin Testing |
| 133 | +from semantic_kernel_plugins.openapi_plugin_factory import OpenApiPluginFactory |
| 134 | + |
| 135 | +# Database Operations (Personal Containers) |
| 136 | +from functions_personal_agents import get_personal_agents, save_personal_agent |
| 137 | +from functions_personal_actions import get_personal_actions, save_personal_action |
| 138 | + |
| 139 | +# Plugin Logging |
| 140 | +from semantic_kernel_plugins.plugin_logging import get_plugin_logger |
| 141 | + |
| 142 | +# Conversation Management |
| 143 | +from conversation_manager import ConversationManager |
| 144 | +``` |
| 145 | + |
| 146 | +### **Test Data Patterns:** |
| 147 | +```python |
| 148 | +# Test User ID |
| 149 | +test_user_id = "test-user-12345" |
| 150 | + |
| 151 | +# Test Agent Configuration |
| 152 | +test_agent = { |
| 153 | + "name": "TestAgent", |
| 154 | + "display_name": "Test Agent", |
| 155 | + "description": "A test agent for validation", |
| 156 | + "instructions": "You are a test agent", |
| 157 | + "azure_openai_gpt_deployment": "gpt-4o" |
| 158 | +} |
| 159 | + |
| 160 | +# Test OpenAPI Plugin Configuration |
| 161 | +test_config = { |
| 162 | + 'name': 'test_plugin', |
| 163 | + 'base_url': 'https://api.example.com', |
| 164 | + 'openapi_spec_content': { |
| 165 | + 'openapi': '3.0.0', |
| 166 | + 'info': {'title': 'Test API', 'version': '1.0.0'}, |
| 167 | + 'paths': { |
| 168 | + '/test': { |
| 169 | + 'get': { |
| 170 | + 'operationId': 'testOperation', |
| 171 | + 'summary': 'Test operation' |
| 172 | + } |
| 173 | + } |
| 174 | + } |
| 175 | + } |
| 176 | +} |
| 177 | +``` |
| 178 | + |
| 179 | +## 🎯 **Where to Store Tests** |
| 180 | + |
| 181 | +### **Test Categories by Directory Usage:** |
| 182 | +- **Core Functionality Tests** → Direct in `functional_tests/` |
| 183 | +- **Fix Validation Tests** → `functional_tests/` with accompanying `.md` documentation |
| 184 | +- **Plugin Integration Tests** → `functional_tests/` (follow `test_openapi_*.py` patterns) |
| 185 | +- **Migration Tests** → `functional_tests/` (follow `test_migration.py` pattern) |
| 186 | +- **UI/Display Tests** → `functional_tests/` (follow `test_*_display.py` patterns) |
| 187 | + |
| 188 | +### **Documentation Requirements:** |
| 189 | +- **For Bug Fixes**: Create accompanying `.md` file describing the issue and solution |
| 190 | +- **For New Features**: Include comprehensive test documentation in docstrings |
| 191 | +- **For Complex Integrations**: Add setup/teardown documentation |
| 192 | + |
| 193 | +## ⚡ **Execution Patterns** |
| 194 | + |
| 195 | +### **Standalone Execution:** |
| 196 | +```bash |
| 197 | +cd functional_tests |
| 198 | +python test_specific_feature.py |
| 199 | +``` |
| 200 | + |
| 201 | +### **Multiple Test Execution:** |
| 202 | +```bash |
| 203 | +# Run all Python tests |
| 204 | +for test in test_*.py; do python $test; done |
| 205 | + |
| 206 | +# Run specific test pattern |
| 207 | +python test_openapi_*.py |
| 208 | +``` |
| 209 | + |
| 210 | +### **Integration with Development Workflow:** |
| 211 | +- Run relevant tests after making changes in related areas |
| 212 | +- Create/update tests as part of bug fix or feature development |
| 213 | +- Use tests to validate fixes before marking issues as resolved |
| 214 | + |
| 215 | +## 📋 **Best Practices** |
| 216 | + |
| 217 | +### **Test Design:** |
| 218 | +✅ **Independent Tests** - Each test should run standalone without dependencies |
| 219 | +✅ **Clear Output** - Use emojis and descriptive messages for test progress |
| 220 | +✅ **Proper Cleanup** - Clean up test data to avoid pollution |
| 221 | +✅ **Error Handling** - Include comprehensive error reporting with stack traces |
| 222 | +✅ **Validation** - Test both positive and negative scenarios |
| 223 | + |
| 224 | +### **Code Organization:** |
| 225 | +✅ **Meaningful Names** - Test and function names should describe what they validate |
| 226 | +✅ **Documentation** - Include docstrings explaining test purpose and approach |
| 227 | +✅ **Imports** - Group imports logically and include only necessary dependencies |
| 228 | +✅ **Modularity** - Break complex tests into smaller, focused functions |
| 229 | + |
| 230 | +### **Maintenance:** |
| 231 | +✅ **Regular Review** - Periodically review and update tests for relevance |
| 232 | +✅ **Refactoring** - Extract common patterns into reusable utilities |
| 233 | +✅ **Documentation Updates** - Keep test documentation current with code changes |
0 commit comments