v4
- rotating header
- multi-thread
- multi-target
- json dump
- windows
- file seeker
max_depth=4:
- How deep the crawler goes into links
- Example: Home → Page → Subpage → Deeper page (4 levels)
- Higher number = more pages but slower
- Lower number = faster but might miss content
max_concurrent=150:
- How many URLs it checks at once
- Like opening 150 browser tabs simultaneously
- Higher number = faster but might trigger site blocks
- Lower number = slower but more stealthy
Recommendation:
- For stealth: max_depth=3, max_concurrent=50
- For speed: max_depth=4, max_concurrent=150
- For deep search: max_depth=5, max_concurrent=100