Releases: EdJoPaTo/website-stalker
v0.9.0
v0.8.0
More generic config file format
Each site in the config file is now more generic. Before each entry was an html
or utf8
entry. Now each entry is basically the same.
Each entry has an URL and a file extension which is then used to save the resulting file.
Each site can also have editors. An editor manipulates the content before saving the result.
css_selector
and regex_replacer
are now editors. The default behavior of html
to prettify the content is now and editor too: html_prettify
.
Additionally this update includes a new editor html_text
which only returns text entries from the HTML.
To give an example:
sites:
- url: "https://edjopato.de/post/"
extension: html
editors:
- css_selector: article
- css_selector:
selector: .meta
remove: true
- html_prettify
If you want to see a config migration see this commit.
css_selector
remove elements
The css_selector
can now remove matching HTML elements from the result. This is already included in the example above.
html_text
Editor
This editor only returns text entries from the HTML.
To give an example: This will save every h1
heading to the resulting file.
- url: "https://edjopato.de/post/"
extension: txt
editors:
- css_selector: h1
- html_text
systemd improvements
v0.7.1
v0.7.0
systemd files
Adds a systemd service and timer to be used locally 3a210f2
libgit2
Migrate some functions from running git
as a commandline tool towards libgit2
.
This should make handling and detecting easier on the code side of things.
Not everything is migrated (yet?). Some outputs like the git diff are just fine currently via the commandline command.
0074611 bf08eb4 20f07d5 9398cd6
This also allows for running from within a subfolder of a git repo 9398cd6
Minor changes
v0.6.0
v0.5.0
v0.4.0
v0.3.0: fix(logger): use also all uppercase for hint
v0.2.0
- feat: run async and wait 5s between same domain 70f4a4e 70c0ba2
- feat: show time it took b130729
- feat: improve validation of FROM header 0bf3ba6
- fix(http): set user_agent once 58ad848
- perf(git): do git add once with all files e6f8334
- fix(git): remove version from git message 9af4cde
- fix(git): never use a pager d47fe63