Logscape - Large scale log analytics

Logscape.com | Logscape Blog | Logscape Support | Logscape Apps

Logscape provides real-time visibility of log streams, interactive dashboarding, scalable distributed search, file exploration and alerting. Install and run one agent as a Manager (server + web interface) and ship logs by installing agents as Forwarders; alternatively, send network data via Syslog.

Features

• Search log files at scale
• Ship using agents as LogForwarders
• Ship using Syslog
• Tail data in real-time
• Application/OS metric instrumentation via APPS
• Interactive Search and field discovery (extraction)
• Automatic JSON and CSV extraction
• Centrally manage all shipping rules
• Log Archive support (.snappy-compress or delete)
• Read all

Installation

Pre-requisites -

• JRE 8 or above

Quickstart using a pre-built binary and read our getting started guide.

Alternatively, you can download via curl

curl -O http://logscape.net/download/Release-Opensource-Latest/Logscape-Opensource-Latest.zip

Or run from docker images available on docker hub. This image is based upon a ubuntu base, and will run a Logscape manager.

Run Logscape by following these steps.

• Extract the .zip folder
• Run as Manager (default behaviour) or configure your agents role via the configure.sh/bat located in /logscape/scripts/ to run as a LogFowarder
• Start your agent via logscape.sh start or logscape.bat
• Navigate to port :8080 on the target machine in your browser
• Log in with the default credentials admin/ll4bs

Using Logscape

View the getting started guide

View different deployment configurations

Deployment

A Logscape manager can be deployed in isolation - or alongside forwarders and Indexstores. For more information read the full deployment guide.

Build from Source

Note: This repo includes IntelliJ project files for your convenience.

Pre-requisites -

• JDK 8 or above
• Ant build tool.

Then build from source by following these steps

• Navigate to your checkout directory
• Navigate to the master folder /LogScape/master
• Execute ant clean dist

On a successful build, this will generate -

• Distributable zip located in /Logscape/master/dist
• Expanded build located in Logscape/master/build

The artifacts of this build are as follows -

.
├── logscape.sh // Runs Logscape on Unix based systems
├── logscape.bat // Runs Logscape on Windows
├── readme.txt
├── agent-log4j.properties // Log4j configuration file
├── log4j.properties // Log4j configuration file
├── boot.properties // Configures boot parameters of this agent
├── boot.jar //Used by Logscape scripts
├── boot.sh //Used by Logscape scripts
├── boot.bat //Used by Logscape scripts
├── downloads
|   └── contains downloaded files which will be distributed across the entire deployment
├── apps
|   └── Jetty configuration
├── etc
|   └── Service configuration
├── libs
|   └── Common Libs
├── pids
|   └── Used in the tracking of the currently active Logscape pids
├── scripts
|   └── Contains groovy scripts, as well as utility shell and bash scripts
├── space
|   └── Contains data storage .`journal` files generated by system modules
├── ssl
|   └── Contains SSL key files
├── deployed-bundles
|   └── Contains the expanded contents of the `downloads` folder
└── system-bundles

Running Tests

Logscape uses the Ant build manager for testing purposes. Each module can run individual tests, or the master module can be used to test the code base. To test a single module

• Navigate to the specific module directory
• Run ant testSingle

To test the entire code base

• Navigate to the master directory
• Run ant cleanTestAll

Module Breakdown

Boot - bootstraps the agent process and managing its life cycle. Generates boot.jar.

Common - a series of commonly used low-level classes. Generates common.jar.

DashboardServer - Responsible for the web front-end. Generates dashboard.jar.

Play - Main web module. Generates play-lib.zip and play.war

Lib - commonly used third-party libraries.

Master - scripts to perform high-level operations i.e build project.

Print - Generates reports. Generates print.war.

Replicator - Manages the distribution of the contents of the download folder.

Root - Displays the Logscape loading page. Generates root.war.

SaaS-Meter - Allows for multi-tenanted environments with data quotas. Generates saas-meter.jar and test-saas-meter.jar

Transport - Non-blocking socket communication layer and RabbitMQ integration. Generates transport.jar.

vs-admin - User administration. Generates vs-admin.jar and test-vs-admin.jar.

vs-log - Stores and indexes searches, dashboards, alerts etc. Generates vs-log.jar, vs-log-agent.jar and test-vslog-log.jar.

vs-log-disco - Extracts fields from data at ingest and search time. Generates vs-log-disco.jar.

vs-log-server - Ships logs from Forwarders to Indexers. Generates vs-log-server.jar and test-vs-log.jar.

vso - VScape orchestration. Generates vso.jar and test-vso.jar.

vs-orm - Maps objects into the VSpace datastore. Generates vs-orm.jar.

vspace - Storage of space objects and VScape services. Generates vspace.jar.

vs-socket-server - Logscape Socket Server. Generates vs-socket-server.jar and test-vs-socket-server.jar.

vs-syslog-server - Logscape Syslog Server. Generates vs-syslog.server and test-vs-syslog-server.jar.

vs-util - Utility module for chargeback and costing. Generates vs-util.jar and test-vs-util.jar

Contact

Please direct any queries to [email protected]

License

Copyright 2017 Logscape

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License