Skip to content
This repository was archived by the owner on May 16, 2019. It is now read-only.

Commit 64fc8d2

Browse files
danhunsakerGargron
authored andcommitted
[Nanobox] Stream backups to the warehouse (mastodon#6799)
The `curl` docs are terrible. Use `-X POST -T` instead of `--data-binary`, to avoid loading entire backups into memory _before_ transferring to the warehouse, and just stream the data across as it comes in.
1 parent fd385e2 commit 64fc8d2

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

boxfile.yml

+3-3
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ data.db:
195195
command: |
196196
PGPASSWORD=${DATA_DB_PASS} pg_dump -U ${DATA_DB_USER} -w -Fc -O gonano |
197197
gzip |
198-
curl -k -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/backup-${HOSTNAME}-$(date -u +%Y-%m-%d.%H-%M-%S).sql.gz --data-binary @- &&
198+
curl -k -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/backup-${HOSTNAME}-$(date -u +%Y-%m-%d.%H-%M-%S).sql.gz -X POST -T - >&2
199199
curl -k -s -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/ |
200200
sed 's/,/\n/g' |
201201
grep ${HOSTNAME} |
@@ -215,7 +215,7 @@ data.redis:
215215
- id: backup
216216
schedule: '0 3 * * *'
217217
command: |
218-
curl -k -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/backup-${HOSTNAME}-$(date -u +%Y-%m-%d.%H-%M-%S).rdb --data-binary @/data/var/db/redis/dump.rdb &&
218+
curl -k -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/backup-${HOSTNAME}-$(date -u +%Y-%m-%d.%H-%M-%S).rdb -X POST -T /data/var/db/redis/dump.rdb >&2
219219
curl -k -s -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/ |
220220
sed 's/,/\n/g' |
221221
grep ${HOSTNAME} |
@@ -236,7 +236,7 @@ data.storage:
236236
schedule: '0 3 * * *'
237237
command: |
238238
tar cz -C /data/var/db/unfs/ . |
239-
curl -k -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/backup-${HOSTNAME}-$(date -u +%Y-%m-%d.%H-%M-%S).tgz --data-binary @- &&
239+
curl -k -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/backup-${HOSTNAME}-$(date -u +%Y-%m-%d.%H-%M-%S).tgz -X POST -T - >&2
240240
curl -k -s -H "X-AUTH-TOKEN: ${WAREHOUSE_DATA_HOARDER_TOKEN}" https://${WAREHOUSE_DATA_HOARDER_HOST}:7410/blobs/ |
241241
sed 's/,/\n/g' |
242242
grep ${HOSTNAME} |

0 commit comments

Comments
 (0)