We use a set of loosely coupled technologies to manage our publishing process.
Rather than developing something monolithic, we leverage commodity software that does one aspect well and connect the tools through APIs and automation.
The tools that support our process are detailed on subsequent pages.
Our stack includes:
- Screendoor. The platform for accepting publishing submissions at https://datasf.org/publishing
- Trello. Kanban board for managing the process queues and team communication about data publishing.
- Desk.com. Help desk for managing communication with departments during the course of publishing.
- PowerBI. The dashboard platform for measuring and monitoring our publishing processes.
- Airtable. Cloud-based data system used to actively manage the inventory, including logging published datasets at end of process.
- Google Sheets. Used as a lightweight data store for logging activity in Trello and Desk.com, which is then used in our PowerBI dashboard.
- Zapier. An API automation platform for connecting multiple cloud tools together (e.g. Screendoor -> Desk.com -> Trello)
- SAFE FME. ETL Framework used for some of our automation jobs.
- Socrata Publica. The open data platform where users access data. The output of the publishing process lives here.