What's new in StormCrawler 2.9
Disclaimer
This is a Pre-ASF release and did not undergo a formal review by the PMC.
What's Changed
- Change HttpProtocol to defer to configured values for retryOnConnectionFailure and followRedirects by @ndtreviv in #1056
- Cache redirected robots.txt for target host only if path is /robots.txt and query is empty by @sebastian-nagel in #1057
- Issue #1043: Fixing problems after restart of Frontier service by @michaeldinzinger in #1054
- #1049 Replace "Collapse and Expand Results" Solr query with "Result Grouping" query. by @syefimov in #1053
- OpenSearch 2.7.0 + renamed OpenSearchConnection by @jnioche in #1064
- BasicURLNormalizer .unmangleQueryString() returns invalid results if "&" symbol in a parents path #1059 by @syefimov in #1062
- Dependency upgrades. fixes #1066 by @jnioche in #1067
- Automatic creation of index definitions should use the bolt type by @jnioche in #1069
- mechanism to retrieve more generic value of configuration by @jnioche in #1071
- Create DeletionBolt.java for Solr. #1050 by @syefimov in #1073
- Increase the number of redirects to 5 for Robots.txt fetching by @michaeldinzinger in #1074
- Issue #1042: Adapt parsing of robots.txt files by @michaeldinzinger in #1055
- Test URL Filtering from the command line by @jnioche in #1081
New Contributors
- @michaeldinzinger made their first contribution in #1054
- @syefimov made their first contribution in #1053
Full Changelog: 2.8...2.9