|
| 1 | +# Testing Documentation Guidelines |
| 2 | + |
| 3 | +This document defines the rules and requirements for writing test scenarios and documentation for the Rivet Inspector frontend. |
| 4 | + |
| 5 | +## Purpose |
| 6 | + |
| 7 | +Test documentation should: |
| 8 | +- Focus on **critical business logic** only |
| 9 | +- Be **simple, readable, and maintainable** |
| 10 | +- Avoid duplication through reference documents |
| 11 | +- Be **ambiguous enough** to allow implementation flexibility |
| 12 | +- Work for both **manual testing** and **e2e test development** |
| 13 | + |
| 14 | +## Core Principles |
| 15 | + |
| 16 | +### 1. Test What, Not How |
| 17 | + |
| 18 | +✅ **Good**: "User is informed about connection failure" |
| 19 | +❌ **Bad**: "Display error message: 'Connection failed. Please check your server is running at http://localhost:6420'" |
| 20 | + |
| 21 | +**Why**: We care that users are informed, not the exact wording or format. |
| 22 | + |
| 23 | +### 2. Avoid Specific Text or Formats |
| 24 | + |
| 25 | +✅ **Good**: "Error details are shown" |
| 26 | +❌ **Bad**: "Error message format: `Error: {message}`" |
| 27 | + |
| 28 | +**Why**: UI text changes frequently. Test the presence of information, not exact formatting. |
| 29 | + |
| 30 | +### 3. Focus on User Information and Actions |
| 31 | + |
| 32 | +Every test should verify: |
| 33 | +- **What the user sees/knows**: "User is informed about X" |
| 34 | +- **What the user can do**: "Available actions: Retry, Cancel" |
| 35 | + |
| 36 | +### 4. Business Logic Only |
| 37 | + |
| 38 | +Include: |
| 39 | +- ✅ Connection flows |
| 40 | +- ✅ Rivet Actor lifecycle |
| 41 | +- ✅ Data display and updates |
| 42 | +- ✅ Error handling |
| 43 | +- ✅ Critical user actions |
| 44 | + |
| 45 | +Exclude: |
| 46 | +- ❌ Accessibility testing |
| 47 | +- ❌ Performance testing |
| 48 | +- ❌ Visual styling |
| 49 | +- ❌ Browser-specific quirks |
| 50 | +- ❌ Animation timing |
| 51 | + |
| 52 | +### 5. Use Reference Documents |
| 53 | + |
| 54 | +When information is shared across multiple scenarios: |
| 55 | +- Create a reference document in `references/` |
| 56 | +- Link to it from main scenarios |
| 57 | +- Avoid duplicating the same information |
| 58 | + |
| 59 | +**Example**: Instead of repeating all Rivet Actor states in every test, reference `actor-states.md`. |
| 60 | + |
| 61 | +## Document Structure |
| 62 | + |
| 63 | +### Main Test Document Format |
| 64 | + |
| 65 | +```markdown |
| 66 | +# [Feature] Testing Scenarios |
| 67 | + |
| 68 | +## Quick Reference |
| 69 | +- Links to reference documents |
| 70 | + |
| 71 | +## Testing Environment |
| 72 | +- URLs, endpoints, prerequisites |
| 73 | + |
| 74 | +## Critical User Flows |
| 75 | + |
| 76 | +### Flow N: [Flow Name] |
| 77 | + |
| 78 | +**Scenario**: Brief description |
| 79 | + |
| 80 | +**Prerequisites** (if needed): What must be true before testing |
| 81 | + |
| 82 | +**Verify**: |
| 83 | +- Point 1 |
| 84 | +- Point 2 |
| 85 | + |
| 86 | +OR |
| 87 | + |
| 88 | +**Test Cases**: |
| 89 | + |
| 90 | +#### TCN.N: [Test Case Name] |
| 91 | +**Given**: Initial conditions |
| 92 | +**When**: User action |
| 93 | +**Then**: Expected outcome (high-level) |
| 94 | +``` |
| 95 | + |
| 96 | +### Reference Document Format |
| 97 | + |
| 98 | +```markdown |
| 99 | +# [Topic] Reference |
| 100 | + |
| 101 | +## [Category] |
| 102 | + |
| 103 | +### [Item Name] |
| 104 | + |
| 105 | +**When**: Condition when this applies |
| 106 | + |
| 107 | +**User is informed about**: |
| 108 | +- Information point 1 |
| 109 | +- Information point 2 |
| 110 | + |
| 111 | +**Available actions**: |
| 112 | +- Action 1 |
| 113 | +- Action 2 |
| 114 | + |
| 115 | +**Displays** (optional): |
| 116 | +- UI element 1 |
| 117 | +- UI element 2 |
| 118 | +``` |
| 119 | + |
| 120 | +## Writing Style |
| 121 | + |
| 122 | +### Language Rules |
| 123 | + |
| 124 | +1. **Use present tense**: "User is informed" not "User will be informed" |
| 125 | +2. **Be concise**: Avoid unnecessary words |
| 126 | +3. **Be specific about intent**: "User is informed about connection failure" not "Show error" |
| 127 | +4. **Avoid implementation details**: "Rivet Actor can be destroyed" not "Click destroy button and confirm in dialog" |
| 128 | + |
| 129 | +### Verification Points |
| 130 | + |
| 131 | +Use "Verify" sections for bullet-point checks: |
| 132 | + |
| 133 | +```markdown |
| 134 | +**Verify**: |
| 135 | +- Feature works as expected |
| 136 | +- Data is displayed |
| 137 | +- Actions are available |
| 138 | +``` |
| 139 | + |
| 140 | +Use "User is informed about" for error/information states: |
| 141 | + |
| 142 | +```markdown |
| 143 | +**User is informed about**: |
| 144 | +- What went wrong |
| 145 | +- How to fix it |
| 146 | +``` |
| 147 | + |
| 148 | +Use "Available actions" for what users can do: |
| 149 | + |
| 150 | +```markdown |
| 151 | +**Available actions**: |
| 152 | +- Retry connection |
| 153 | +- Modify settings |
| 154 | +``` |
| 155 | + |
| 156 | +### Scenario vs Test Case |
| 157 | + |
| 158 | +**Use Scenario** for: |
| 159 | +- High-level user flows |
| 160 | +- Integration testing |
| 161 | +- Multi-step processes |
| 162 | + |
| 163 | +**Use Test Case** for: |
| 164 | +- Specific conditions |
| 165 | +- Edge cases |
| 166 | +- Individual feature verification |
| 167 | + |
| 168 | +## Naming Conventions |
| 169 | + |
| 170 | +### Test Case IDs |
| 171 | + |
| 172 | +Format: `TC[Flow].[Case]` |
| 173 | + |
| 174 | +Examples: |
| 175 | +- `TC1.1` - Flow 1, Test Case 1 |
| 176 | +- `TC5.14` - Flow 5, Test Case 14 |
| 177 | + |
| 178 | +### Integration Test IDs |
| 179 | + |
| 180 | +Format: `I[Number]` |
| 181 | + |
| 182 | +Examples: |
| 183 | +- `I1` - Integration Test 1 |
| 184 | +- `I4` - Integration Test 4 |
| 185 | + |
| 186 | +### Edge Case IDs |
| 187 | + |
| 188 | +Format: `E[Number]` |
| 189 | + |
| 190 | +Examples: |
| 191 | +- `E1` - Edge Case 1 |
| 192 | +- `E3` - Edge Case 3 |
| 193 | + |
| 194 | +### Reference Document Names |
| 195 | + |
| 196 | +Use lowercase with hyphens: |
| 197 | +- `connection-states.md` |
| 198 | +- `actor-states.md` |
| 199 | +- `error-states.md` |
| 200 | +- `ui-components.md` |
| 201 | + |
| 202 | +## Common Patterns |
| 203 | + |
| 204 | +### Testing State Changes |
| 205 | + |
| 206 | +✅ **Good**: |
| 207 | +```markdown |
| 208 | +**Verify**: |
| 209 | +- Rivet Actor state updates when changed |
| 210 | +- Events appear in real-time |
| 211 | +``` |
| 212 | + |
| 213 | +❌ **Bad**: |
| 214 | +```markdown |
| 215 | +**Verify**: |
| 216 | +- When actor state changes from {old} to {new}, the UI updates within 100ms showing the new state in JSON format |
| 217 | +``` |
| 218 | + |
| 219 | +### Testing Error Handling |
| 220 | + |
| 221 | +✅ **Good**: |
| 222 | +```markdown |
| 223 | +**When**: Connection fails |
| 224 | +**User is informed about**: |
| 225 | +- Connection failure |
| 226 | +- Possible reasons |
| 227 | + |
| 228 | +**Available actions**: |
| 229 | +- Retry connection |
| 230 | +``` |
| 231 | + |
| 232 | +❌ **Bad**: |
| 233 | +```markdown |
| 234 | +**When**: Connection fails |
| 235 | +**Then**: Show red error banner with text "Connection failed: ERR_CONNECTION_REFUSED. Please check that your server is running." and a "Retry" button in blue |
| 236 | +``` |
| 237 | + |
| 238 | +### Testing User Actions |
| 239 | + |
| 240 | +✅ **Good**: |
| 241 | +```markdown |
| 242 | +**Verify**: |
| 243 | +- Rivet Actor can be created |
| 244 | +- Rivet Actor appears in list after creation |
| 245 | +``` |
| 246 | + |
| 247 | +❌ **Bad**: |
| 248 | +```markdown |
| 249 | +**Steps**: |
| 250 | +1. Click the "+" button next to "Game Session" |
| 251 | +2. Fill in the form fields: name, region, max_players |
| 252 | +3. Click "Create" button |
| 253 | +4. Wait for success toast message |
| 254 | +5. Verify actor appears in sidebar with green dot |
| 255 | +``` |
| 256 | + |
| 257 | +## Examples |
| 258 | + |
| 259 | +### Good Scenario Example |
| 260 | + |
| 261 | +```markdown |
| 262 | +### Flow 1: First-Time Visit |
| 263 | + |
| 264 | +**Scenario**: User visits inspector URL without a running RivetKit server |
| 265 | + |
| 266 | +**Verify**: |
| 267 | +- Getting started card is displayed |
| 268 | +- Connection form is shown with default endpoint |
| 269 | +- User can enter custom endpoint |
| 270 | +- Connect button is available |
| 271 | +``` |
| 272 | + |
| 273 | +### Bad Scenario Example |
| 274 | + |
| 275 | +```markdown |
| 276 | +### Flow 1: First-Time Visit Without Server Running on Default Port |
| 277 | + |
| 278 | +**Scenario**: When a user who has never used the inspector before opens the URL http://localhost:5173 in their Chrome browser and there is no RivetKit server running on port 6420 |
| 279 | + |
| 280 | +**Expected Behavior**: |
| 281 | +- The page should render with a white background |
| 282 | +- A card titled "Getting Started with Rivet Inspector" should appear centered on the screen |
| 283 | +- Below that, a form with a single input field labeled "Endpoint" containing the placeholder text "http://localhost:6420" should be visible |
| 284 | +- A blue button labeled "Connect" should be displayed below the input |
| 285 | +- When hovering over the button, it should turn a darker shade of blue |
| 286 | +``` |
| 287 | + |
| 288 | +## Checklist for New Test Documentation |
| 289 | + |
| 290 | +Before submitting test documentation, verify: |
| 291 | + |
| 292 | +- [ ] Focuses on business logic only (no accessibility, performance, styling) |
| 293 | +- [ ] Uses high-level verification points |
| 294 | +- [ ] Avoids specific UI text or message formats |
| 295 | +- [ ] Uses reference documents for shared concepts |
| 296 | +- [ ] Follows naming conventions |
| 297 | +- [ ] Uses "User is informed about" for information verification |
| 298 | +- [ ] Uses "Available actions" for action verification |
| 299 | +- [ ] Uses "Verify" for feature verification |
| 300 | +- [ ] Written in present tense |
| 301 | +- [ ] Concise and readable |
| 302 | +- [ ] No implementation details (no exact button clicks, colors, timing) |
| 303 | +- [ ] Works for both manual and automated testing |
| 304 | + |
| 305 | +## Updates and Maintenance |
| 306 | + |
| 307 | +When updating test documentation: |
| 308 | + |
| 309 | +1. **Keep it simple**: Remove details, don't add them |
| 310 | +2. **Use references**: If adding new information that applies to multiple tests, create/update a reference document |
| 311 | +3. **Stay high-level**: Focus on what users should know and do, not how the UI accomplishes it |
| 312 | +4. **Remove outdated info**: Delete sections that no longer apply rather than commenting them out |
| 313 | + |
| 314 | +## For AI Agents |
| 315 | + |
| 316 | +When writing or updating test documentation: |
| 317 | + |
| 318 | +1. Always check existing reference documents before creating new content |
| 319 | +2. Default to high-level verification points |
| 320 | +3. Never include specific UI text, error messages, or formatting |
| 321 | +4. Focus on "User is informed about X" rather than "Error message says X" |
| 322 | +5. Keep scenarios focused on single flows or features |
| 323 | +6. Use bullet points for multiple verification items |
| 324 | +7. Avoid adding notes, warnings, or future considerations |
| 325 | +8. When in doubt, make it more ambiguous, not more specific |
0 commit comments