Logscape - Large scale log analytics

Logscape.com | Logscape Blog | Logscape Support | Logscape Apps

Logscape provides real-time visibility of log streams, interactive dashboarding, scalable distributed search, file exploration and alerting. Install and run one agent as a Manager (server + web interface) and ship logs by installing agents as Forwarders; alternatively, send network data via Syslog.

Features

• Search log files at scale
• Ship using agents as LogForwarders
• Ship using Syslog
• Tail data in real-time
• Application/OS metric instrumentation via APPS
• Interactive Search and field discovery (extraction)
• Automatic JSON and CSV extraction
• Centrally manage all shipping rules
• Log Archive support (.snappy-compress or delete)
• Read all

Installation

Quickstart using a pre-built binary and read our getting started guide.

Pre-requisites -

• JRE 8 or above

Run Logscape by following these steps.

• Extract the .zip folder
• Run as Manager (default behaviour) or configure your agents role via the configure.sh/bat located in /logscape/scripts/ to run as a LogFowarder
• Start your agent via logscape.sh start or logscape.bat
• Navigate to port :8080 on the target machine in your browser
• Log in with the default credentials admin/ll4bs

Using Logscape

View the getting started guide

View different deployment configurations

Deployment

A Logscape manager can be deployed in isolation - or alongside forwarders and Indexstores. For more information read the full deployment guide.

Docker

Logscape docker images are available from docker hub.

Build from Source

Note: This repo includes IntelliJ project files for your convenience.

Pre-requisites -

• JDK 8 or above
• Ant build tool.

Then build from source by following these steps

• Navigate to your checkout directory
• Navigate to the master folder /LogScape/master
• Execute ant clean dist

On a successful build, this will generate -

• Distributable zip located in /Logscape/master/dist
• Expanded build located in Logscape/master/build

The artifacts of this build are as follows -

.
├── logscape.sh // Used for running Logscape on Unix based systems
├── logscape.bat // Used for running Logscape on Windows
├── readme.txt // Explains how to run and configure Logscape
├── agent-log4j.properties // Log4j configuration file
├── log4j.properties // Log4j configuration file
├── boot.properties // Used to configure boot parameters of this agent
├── boot.jar //Used by Logscape scripts
├── boot.sh //Used by Logscape scripts
├── boot.bat //Used by Logscape scripts
├── downloads
|   └── contains downloaded files which will be distributed across the entire deployment
├── apps
|   └── Jetty configuration
├── etc
|   └── Service configuration
├── libs
|   └── Common Libs
├── pids
|   └── Used in the tracking of the currently active Logscape pids
├── scripts
|   └── Contains groovy scripts, as well as utility shell and bash scripts
├── space
|   └── Contains data storage .`journal` files generated by system modules
├── ssl
|   └── Contains SSL key files
├── deployed-bundles
|   └── Contains the expanded contents of the `downloads` folder
└── system-bundles

Running Tests

Logscape uses the Ant build manager for testing purposes. Each module can run individual tests, or the master module can be used to test the code base. To test a single module

• Navigate to the specific module directory
• Run ant testSingle

To test the entire code base

• Navigate to the master directory
• Run ant cleanTestAll

Module Breakdown

Boot - Responsible for bootstrapping the agent process and managing its life cycle.

Common - Contains a series of commonly used low-level classes.

DashboardServer - Responsible for the web front-end.

Lib - Contains libraries commonly used throughout the project.

Master - Storage of scripts to perform high-level operations.

Play - Main web module.

Print - Generates reports.

Replicator - Manages the distribution of the contents of the download folder.

Root - Displays the Logscape loading page.

SaaS-Meter - Allows for multi-tenanted environments with data quotas.

Transport - Non-blocking socket communication layer and RabbitMQ integration.

vs-admin - User administration.

vs-log - Stores and indexes searches, dashboards, alerts etc

vs-log-disco - Extracts fields from data at ingest and search time.

vs-log-server - Ships logs from Forwarders to Indexers.

vso - VScape orchestration.

vs-orm - Maps objects into the VSpace datastore.

vspace - Storage of space objects and VScape services.

vs-socket-server - Logscape Socket Server

vs-syslog-server - Logscape Syslog Server

vs-util - Utility module for chargeback and costing.

Contact

Please direct any queries to [email protected]

License

Copyright 2017 Logscape

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License