Skip to content

OutOfBedlam/tine

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TINE

latest CI codecov Go Report Card

TINE is not ETL

TINE a data pipeline runner.

Install

go install github.com/OutOfBedlam/tine@latest

Find documents from https://tine.thingsme.xyz/

Usage

Define pipeline in TOML

Set the pipeline's inputs and outputs.

[[inlets.cpu]]
    interval = "3s"
[[flows.select]]
    includes = ["#*", "*"]  # all tags and all fields
[[outlets.file]]
    path  = "-"
    decimal = 2

Run

tine run <config.toml>

It generates CPU usage in CSV format which is default format of 'outlets.file'.

1721635296,cpu,1.57
1721635299,cpu,0.66
1721635302,cpu,1.07
1721635305,cpu,2.08

Change output format to "json" from "csv", add format = "json" at the end of the file.

[[outlets.file]]
    path  = "-"
    decimal = 2
    format = "json"
{"_in":"cpu","_ts":1721780188,"total_percent":0.91}
{"_in":"cpu","_ts":1721780191,"total_percent":1.04}
{"_in":"cpu","_ts":1721780194,"total_percent":0.25}
{"_in":"cpu","_ts":1721780197,"total_percent":1.20}

Shebang

  1. Save this file as load.toml
#!/path/to/tine run
[[inlets.load]]
    loads = [1, 5]
    interval = "3s"
[[flows.select]]
    includes = ["**"]  # equivalent to ["#*", "*"]
[[outlets.file]]
    path  = "-"
    decimal = 2
  1. Chmod for executable.
chmod +x load.toml
  1. Run
$ ./load.toml

1721635438,load,0.03,0.08
1721635441,load,0.03,0.08
1721635444,load,0.03,0.08
^C

Embedding in Go

TINE is not only designed as a standalone application, but also allows for embedding in other Go applications.

Examples

How to use TINE as a library for your application.

How to set a custom inlet/outlet/flows.

How to use pipelines as a HTTP handler

How to collect metrics into RRD and display rrdgraph in a web page

image

How to collect metrics into Sqlite and display it on web page

This example also shows how to utilize HTTP query parameters as variables of Go Templates to build pipeline configuration.

image

Documents

Please visit https://tine.thingsme.xyz for the documents.

About

TINE a data pipeline runner.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages