GitHub - DataSF/fetch-metadata-fields: Utility node and python scripts to grab and update socrata field and metadata information

DataSF / fetch-metadata-fields Public

Notifications You must be signed in to change notification settings
Fork 0
Star 1

Utility node and python scripts to grab and update socrata field and metadata information

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 51 Commits
configs		configs
pydev		pydev
scripts		scripts
.gitignore		.gitignore
README		README
fetch_metadata_fields.sh		fetch_metadata_fields.sh
package.json		package.json
setup.sh		setup.sh

Repository files navigation

// To download fields from socrata

node scripts/processTables
node scripts/processGeo
node scripts/combineOutput
node scripts/tocsv


// To get attachments

node scripts/getAttachmentList


// Or to run all the node scripts and get resulting csvs

npm run output_csvs


// to update the field and attachment datasets on the portal

python pydev/update_metadata_fields.py -c fieldConfig.yaml -d /Users/j9/Desktop/fetch-socrata-fields/configs/

// update the master data dictionary dataset

python pydev/update_master_data_dictionary.py -c fieldConfigMasterDD.yaml -d /Users/j9/Desktop/fetch-socrata-fields/configs/


// to run the total job

./fetch_metadata_fields.sh


// Additional Installation notes

// You will need node and npm installed --> we used node v.6 or greater
// run the install script, setup_dirs to set up the dirctory structure:
`setup_dirs.sh`