Pull data from HTTP endpoint #10802
-
|
Hello, I would like periodically access specified HTTP endpoint providing a JSON document. The JSON document is being changed over the time and it is subject for logging/tracking history. What is the recommended plugin setup for this? I have not found suitable existing input plugin. Am I supposed to use webassembly to write what I need? Thanks. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
|
We provide a more generalised http API scraper support commercially and found doing it in golang was easier primarily to manage the handling of the http interactions and then using golang templating/etc. to handle the generic processing you need to do. You can quite easily use exec or http to ask for the JSON document then do some stuff to it, you could do it with LUA or WASM too, etc. |
Beta Was this translation helpful? Give feedback.
-
|
That's good to know, thank you. I also learned that WebAssembly currently has missed networking capabilities. Regarding your advanced commercial scraper, does this mean you would not accept a community contribution for an in_http_client plugin? |
Beta Was this translation helpful? Give feedback.
-
|
Oh no, go for it. I'm just saying we have one :) |
Beta Was this translation helpful? Give feedback.
We provide a more generalised http API scraper support commercially and found doing it in golang was easier primarily to manage the handling of the http interactions and then using golang templating/etc. to handle the generic processing you need to do.
It depends really on the specifics of what you need to do.
You can quite easily use exec or http to ask for the JSON document then do some stuff to it, you could do it with LUA or WASM too, etc.