X_Elasticsearch 5.6 setup and test locally

These are the information for FEC seven legal documents

No	Name	ElasticSearch Index	Document Type (ElasticSearch type)	Case Type	DB Schema	Refresh Schedule
1	Advisory opinions	docs	advisory_opinions	--	aouser	1)changed/5min; 2)1amGMT/Sun; 3)changed/1amGMT/daily excepted Sun.
2	Current MUR	docs	murs	MUR	fecmur	modified/5min
3	Administrative fines	docs	admin_fines	AF	fecmur	modified/5min
4	Alternative dispute resolution	docs	adrs	ADR	fecmur	modified/5min
5	statutes	docs	statutes	--	--	http://uscode.house.gov
6	regulations	docs	regulations	--	fec-eregs-db-rdn	yearly
7	Archived MUR	archived_murs	murs	--	mur_arch	--

ElasticSearch Index / Aliases

Aliases	Index
docs_index	docs
docs_search	docs
archived_murs_index	archived_murs
docs_search	archived_murs

Test upload archived MUR to ElasticSearch 5.6 locally

1. Install Elasticsearch5.6 locally

a) Check Java version 1.8.0_16 (java -version)
b) you can install ES parallel with openFEC folder
c) run these two commands to install
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.8.tar.gz
tar -xvf elasticsearch-5.6.8.tar.gz
d) start Elasticsearch
cd elasticsearch-5.6.8/bin
./elasticsearch
e) check elasticsearch server on browser: http://127.0.0.1:9200/

2. Get your local openFEC repo ready

a) Check out dev branch
b) enter virtual env
c) point to dev db

3. Create es-index, alias, mapping (run commands on terminal)

a) create index: docs
./manage.py create_docs_index
b) create index: archived_murs
./manage.py create_archived_murs_index

4. Check you local Elasticsearch resources on browser

a) two indexes : http://localhost:9200/_cat/indices?v
b) aliases under each index: http://localhost:9200/_aliases?pretty
c) mappings under each index: http://localhost:9200/_mappings

5. Run: upload single archived MUR on terminal:

a) set DEBUG mode: uncomment webservices/legal_docs/archived_murs.py line 17)
b) Run command to upload a single archived MUR on terminal:
python manage.py load_archived_murs -m 179
python manage.py load_archived_murs -m 1252

Check the MURs you just upload from browser:
(total archived_murs count:) http://localhost:9200/archived_murs/_count?q=:
(single pdf:) http://localhost:9200/archived_murs/murs/mur_179/?pretty
(multi pdf:) http://localhost:9200/archived_murs/murs/mur_1252/?pretty

6. Run: upload all archived MURs on terminal:

(take about 3.5 hours locally, you can ctr+c to stop any time)
python manage.py load_archived_murs
(total archived murs: 4448)