Skip to content

Commit e603825

Browse files
francois2metzsparrc
francois2metz
authored and
sparrc
committed
Add new webhooks plugin that superseed github and rollbar plugins.
closes influxdata#1289 Signed-off-by: François de Metz <[email protected]> Signed-off-by: Cyril Duez <[email protected]> Rename internals struct. Signed-off-by: François de Metz <[email protected]> Signed-off-by: Cyril Duez <[email protected]> Update changelog. Signed-off-by: François de Metz <[email protected]> Signed-off-by: Cyril Duez <[email protected]> Update READMEs and CHANGELOG. Signed-off-by: François de Metz <[email protected]> Signed-off-by: Cyril Duez <[email protected]> Update SampleConfig. Update the config format. Update telegraf config. Update the webhooks README. Update changelog. Update the changelog with an upgrade path. Update default ports. Fix indent. Check for nil value on AvailableWebhooks. Check for CanInterface.
1 parent e344815 commit e603825

18 files changed

+302
-235
lines changed

‎CHANGELOG.md‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,36 @@
11
## v1.0
22

3+
### Release Notes
4+
5+
**Breaking Change**: users of github_webhooks must change to the new
6+
`[[inputs.webhooks]]` plugin.
7+
8+
This means that the default github_webhooks config:
9+
10+
```
11+
# A Github Webhook Event collector
12+
[[inputs.github_webhooks]]
13+
## Address and port to host Webhook listener on
14+
service_address = ":1618"
15+
```
16+
17+
should now look like:
18+
19+
```
20+
# A Webhooks Event collector
21+
[[inputs.webhooks]]
22+
## Address and port to host Webhook listener on
23+
service_address = ":1618"
24+
25+
[inputs.webhooks.github]
26+
path = "/"
27+
```
28+
329
### Features
430

31+
- [#1289](https://github.com/influxdata/telegraf/pull/1289): webhooks input plugin. Thanks @francois2metz and @cduez!
32+
- [#1247](https://github.com/influxdata/telegraf/pull/1247): rollbar webhook plugin.
33+
534
### Bugfixes
635

736
- [#1384](https://github.com/influxdata/telegraf/pull/1384): Fix datarace in apache input plugin.
@@ -50,11 +79,11 @@ in conjunction with wildcard dimension values as it will control the amount of
5079
time before a new metric is included by the plugin.
5180

5281
### Features
82+
5383
- [#1262](https://github.com/influxdata/telegraf/pull/1261): Add graylog input pluging.
5484
- [#1294](https://github.com/influxdata/telegraf/pull/1294): consul input plugin. Thanks @harnash
5585
- [#1164](https://github.com/influxdata/telegraf/pull/1164): conntrack input plugin. Thanks @robinpercy!
5686
- [#1165](https://github.com/influxdata/telegraf/pull/1165): vmstat input plugin. Thanks @jshim-xm!
57-
- [#1247](https://github.com/influxdata/telegraf/pull/1247): rollbar input plugin. Thanks @francois2metz and @cduez!
5887
- [#1208](https://github.com/influxdata/telegraf/pull/1208): Standardized AWS credentials evaluation & wildcard CloudWatch dimensions. Thanks @johnrengelman!
5988
- [#1264](https://github.com/influxdata/telegraf/pull/1264): Add SSL config options to http_response plugin.
6089
- [#1272](https://github.com/influxdata/telegraf/pull/1272): graphite parser: add ability to specify multiple tag keys, for consistency with influxdb parser.

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -217,8 +217,9 @@ Telegraf can also collect metrics via the following service plugins:
217217
* [mqtt_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mqtt_consumer)
218218
* [kafka_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/kafka_consumer)
219219
* [nats_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nats_consumer)
220-
* [github_webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/github_webhooks)
221-
* [rollbar_webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/rollbar_webhooks)
220+
* [webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks)
221+
* [github](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/github)
222+
* [rollbar](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/rollbar)
222223

223224
We'll be adding support for many more over the coming months. Read on if you
224225
want to add support for another service or third-party API.

‎etc/telegraf.conf‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1490,12 +1490,6 @@
14901490
# SERVICE INPUT PLUGINS #
14911491
###############################################################################
14921492

1493-
# # A Github Webhook Event collector
1494-
# [[inputs.github_webhooks]]
1495-
# ## Address and port to host Webhook listener on
1496-
# service_address = ":1618"
1497-
1498-
14991493
# # Read metrics from Kafka topic(s)
15001494
# [[inputs.kafka_consumer]]
15011495
# ## topic(s) to consume
@@ -1601,12 +1595,6 @@
16011595
# data_format = "influx"
16021596

16031597

1604-
# # A Rollbar Webhook Event collector
1605-
# [[inputs.rollbar_webhooks]]
1606-
# ## Address and port to host Webhook listener on
1607-
# service_address = ":1619"
1608-
1609-
16101598
# # Statsd Server
16111599
# [[inputs.statsd]]
16121600
# ## Address and port to host UDP listener on
@@ -1701,3 +1689,15 @@
17011689
# ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
17021690
# data_format = "influx"
17031691

1692+
1693+
# # A Webhooks Event collector
1694+
# [[inputs.webhooks]]
1695+
# ## Address and port to host Webhook listener on
1696+
# service_address = ":1619"
1697+
#
1698+
# [inputs.webhooks.github]
1699+
# path = "/github"
1700+
#
1701+
# [inputs.webhooks.rollbar]
1702+
# path = "/rollbar"
1703+

‎plugins/inputs/all/all.go‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ import (
1919
_ "github.com/influxdata/telegraf/plugins/inputs/elasticsearch"
2020
_ "github.com/influxdata/telegraf/plugins/inputs/exec"
2121
_ "github.com/influxdata/telegraf/plugins/inputs/filestat"
22-
_ "github.com/influxdata/telegraf/plugins/inputs/github_webhooks"
2322
_ "github.com/influxdata/telegraf/plugins/inputs/graylog"
2423
_ "github.com/influxdata/telegraf/plugins/inputs/haproxy"
2524
_ "github.com/influxdata/telegraf/plugins/inputs/http_response"
@@ -57,7 +56,6 @@ import (
5756
_ "github.com/influxdata/telegraf/plugins/inputs/redis"
5857
_ "github.com/influxdata/telegraf/plugins/inputs/rethinkdb"
5958
_ "github.com/influxdata/telegraf/plugins/inputs/riak"
60-
_ "github.com/influxdata/telegraf/plugins/inputs/rollbar_webhooks"
6159
_ "github.com/influxdata/telegraf/plugins/inputs/sensors"
6260
_ "github.com/influxdata/telegraf/plugins/inputs/snmp"
6361
_ "github.com/influxdata/telegraf/plugins/inputs/sqlserver"
@@ -70,6 +68,7 @@ import (
7068
_ "github.com/influxdata/telegraf/plugins/inputs/twemproxy"
7169
_ "github.com/influxdata/telegraf/plugins/inputs/udp_listener"
7270
_ "github.com/influxdata/telegraf/plugins/inputs/varnish"
71+
_ "github.com/influxdata/telegraf/plugins/inputs/webhooks"
7372
_ "github.com/influxdata/telegraf/plugins/inputs/win_perf_counters"
7473
_ "github.com/influxdata/telegraf/plugins/inputs/zfs"
7574
_ "github.com/influxdata/telegraf/plugins/inputs/zookeeper"

‎plugins/inputs/rollbar_webhooks/rollbar_webhooks.go‎

Lines changed: 0 additions & 119 deletions
This file was deleted.

‎plugins/inputs/webhooks/README.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Webhooks
2+
3+
This is a Telegraf service plugin that start an http server and register multiple webhook listeners.
4+
5+
```sh
6+
$ telegraf -sample-config -input-filter webhooks -output-filter influxdb > config.conf.new
7+
```
8+
9+
Change the config file to point to the InfluxDB server you are using and adjust the settings to match your environment. Once that is complete:
10+
11+
```sh
12+
$ cp config.conf.new /etc/telegraf/telegraf.conf
13+
$ sudo service telegraf start
14+
```
15+
16+
## Available webhooks
17+
18+
- [Github](github/)
19+
- [Rollbar](rollbar/)
20+
21+
## Adding new webhooks plugin
22+
23+
1. Add your webhook plugin inside the `webhooks` folder
24+
1. Your plugin must implement the `Webhook` interface
25+
1. Import your plugin in the `webhooks.go` file and add it to the `Webhooks` struct
26+
27+
Both [Github](github/) and [Rollbar](rollbar/) are good example to follow.

plugins/inputs/github_webhooks/README.md renamed to plugins/inputs/webhooks/github/README.md

Lines changed: 2 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,6 @@
1-
# github_webhooks
1+
# github webhooks
22

3-
This is a Telegraf service plugin that listens for events kicked off by Github's Webhooks service and persists data from them into configured outputs. To set up the listener first generate the proper configuration:
4-
```sh
5-
$ telegraf -sample-config -input-filter github_webhooks -output-filter influxdb > config.conf.new
6-
```
7-
Change the config file to point to the InfluxDB server you are using and adjust the settings to match your environment. Once that is complete:
8-
```sh
9-
$ cp config.conf.new /etc/telegraf/telegraf.conf
10-
$ sudo service telegraf start
11-
```
12-
Once the server is running you should configure your Organization's Webhooks to point at the `github_webhooks` service. To do this go to `github.com/{my_organization}` and click `Settings > Webhooks > Add webhook`. In the resulting menu set `Payload URL` to `http://<my_ip>:1618`, `Content type` to `application/json` and under the section `Which events would you like to trigger this webhook?` select 'Send me <b>everything</b>'. By default all of the events will write to the `github_webhooks` measurement, this is configurable by setting the `measurement_name` in the config file.
3+
You should configure your Organization's Webhooks to point at the `webhooks` service. To do this go to `github.com/{my_organization}` and click `Settings > Webhooks > Add webhook`. In the resulting menu set `Payload URL` to `http://<my_ip>:1619/github`, `Content type` to `application/json` and under the section `Which events would you like to trigger this webhook?` select 'Send me <b>everything</b>'. By default all of the events will write to the `github_webhooks` measurement, this is configurable by setting the `measurement_name` in the config file.
134

145
## Events
156

plugins/inputs/github_webhooks/github_webhooks.go renamed to plugins/inputs/webhooks/github/github_webhooks.go

Lines changed: 13 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,78 +1,27 @@
1-
package github_webhooks
1+
package github
22

33
import (
44
"encoding/json"
5-
"fmt"
65
"io/ioutil"
76
"log"
87
"net/http"
9-
"sync"
108

119
"github.com/gorilla/mux"
1210
"github.com/influxdata/telegraf"
13-
"github.com/influxdata/telegraf/plugins/inputs"
1411
)
1512

16-
func init() {
17-
inputs.Add("github_webhooks", func() telegraf.Input { return &GithubWebhooks{} })
13+
type GithubWebhook struct {
14+
Path string
15+
acc telegraf.Accumulator
1816
}
1917

20-
type GithubWebhooks struct {
21-
ServiceAddress string
22-
// Lock for the struct
23-
sync.Mutex
24-
// Events buffer to store events between Gather calls
25-
events []Event
18+
func (gh *GithubWebhook) Register(router *mux.Router, acc telegraf.Accumulator) {
19+
router.HandleFunc(gh.Path, gh.eventHandler).Methods("POST")
20+
log.Printf("Started the webhooks_github on %s\n", gh.Path)
21+
gh.acc = acc
2622
}
2723

28-
func NewGithubWebhooks() *GithubWebhooks {
29-
return &GithubWebhooks{}
30-
}
31-
32-
func (gh *GithubWebhooks) SampleConfig() string {
33-
return `
34-
## Address and port to host Webhook listener on
35-
service_address = ":1618"
36-
`
37-
}
38-
39-
func (gh *GithubWebhooks) Description() string {
40-
return "A Github Webhook Event collector"
41-
}
42-
43-
// Writes the points from <-gh.in to the Accumulator
44-
func (gh *GithubWebhooks) Gather(acc telegraf.Accumulator) error {
45-
gh.Lock()
46-
defer gh.Unlock()
47-
for _, event := range gh.events {
48-
p := event.NewMetric()
49-
acc.AddFields("github_webhooks", p.Fields(), p.Tags(), p.Time())
50-
}
51-
gh.events = make([]Event, 0)
52-
return nil
53-
}
54-
55-
func (gh *GithubWebhooks) Listen() {
56-
r := mux.NewRouter()
57-
r.HandleFunc("/", gh.eventHandler).Methods("POST")
58-
err := http.ListenAndServe(fmt.Sprintf("%s", gh.ServiceAddress), r)
59-
if err != nil {
60-
log.Printf("Error starting server: %v", err)
61-
}
62-
}
63-
64-
func (gh *GithubWebhooks) Start(_ telegraf.Accumulator) error {
65-
go gh.Listen()
66-
log.Printf("Started the github_webhooks service on %s\n", gh.ServiceAddress)
67-
return nil
68-
}
69-
70-
func (gh *GithubWebhooks) Stop() {
71-
log.Println("Stopping the ghWebhooks service")
72-
}
73-
74-
// Handles the / route
75-
func (gh *GithubWebhooks) eventHandler(w http.ResponseWriter, r *http.Request) {
24+
func (gh *GithubWebhook) eventHandler(w http.ResponseWriter, r *http.Request) {
7625
defer r.Body.Close()
7726
eventType := r.Header["X-Github-Event"][0]
7827
data, err := ioutil.ReadAll(r.Body)
@@ -85,9 +34,10 @@ func (gh *GithubWebhooks) eventHandler(w http.ResponseWriter, r *http.Request) {
8534
w.WriteHeader(http.StatusBadRequest)
8635
return
8736
}
88-
gh.Lock()
89-
gh.events = append(gh.events, e)
90-
gh.Unlock()
37+
38+
p := e.NewMetric()
39+
gh.acc.AddFields("github_webhooks", p.Fields(), p.Tags(), p.Time())
40+
9141
w.WriteHeader(http.StatusOK)
9242
}
9343

0 commit comments

Comments
 (0)