Skip to content

Latest commit

 

History

History
67 lines (48 loc) · 3.09 KB

README.md

File metadata and controls

67 lines (48 loc) · 3.09 KB
Raw

GetClojure

Searchable Clojure examples programatically compiled from lots of scraping. GetClojure extracts valid s-expressions from a set of logfiles, runs them in a sandbox, and captures the value and output of every s-expression. The result is then made searchable.

Usage

Go to GetClojure and start searching.

Project Overview

namespace purpose
getclojure.config Where any globally-necessary project configuration and functions live
getclojure.elastic Houses configuration, query, and seeding from files for elasticsearch
getclojure.extract Contains logic related to parsing clojure IRC logfiles and spitting out files for later consumption
getclojure.format Logic for formatting and syntax highlighting s-expressions
getclojure.layout Static HTML generation via hiccup
getclojure.routes Routes for the application
getclojure.seed Runs the full ETL pipeline from logs to seeding elasticsearch
getclojure.server The entrypoint for the server
getclojure.sexp Logic related to running s-expressions: sandboxing via SCI
getclojure.util Everyone has a junk drawer. This is ours.

Developers

In order to run locally in development mode you'll need to do the following:

  • OpenJDK 17. No guarantees it works with older JVMs.
  • pip install pygments: There are a variety of options for keeping your python envs separate. virtualenv, pyenv, etc. Use whatever works for you. Note that you'll need python 3.X or later.
  • docker-compose up -d: Runs elasticsearch.
  • Download the logs and extract them into the resources/logs directory.
  • Set the appropriate env vars in your .envrc. If you don't use direnv, you'll need to export APP_ENV=development and INDEX_NAME=getclojure.
  • Run lein seed 25 to get the s-expressions found in the first 25 logfiles, or lein seed :all to run the entire pipeline across all local logfiles.
  • Start the server: lein ring server-headless
  • Visit localhost:8080 and search.

Contributions

Contributions welcome. This project uses lein test-refresh for rerunning tests. Run lein test-refresh :all to run all tests, including the integration tests. If you have questions about contributing, please reach out to me on Clojurians Slack (@devn) or Libera IRC (devn).

Thanks

  • To borkdude for providing so much great open source software to the Clojure community.
  • To Chris Houser for giving me a treasure trove of logs to harvest.

YourKit

YourKit is kindly supporting open source projects with its full-featured Java Profiler.

YourKit, LLC is the creator of innovative and intelligent tools for profiling Java and .NET applications. Take a look at YourKit's leading software products:

License

Copyright © 2022 Devin Walters

Distributed under the Eclipse Public License, the same as Clojure.