problem with G4-upload cgm values don't upload and pumploop says 'cgm extras failed'

[PieterGit]

With devww branch and G4-upload my loop works fine, but the uploading of cgm values does not seem to work. I can't find the problem, and I noticed there is a problem with the maybe-extras call.

$ openaps maybe-extras
CGM Data 1.58 mins ago is fresh (< 3)
cgm://JSON/nightscout_calibrations/monitor/cal.json
monitor/cal.json  raised  Could not parse Calibration
Traceback (most recent call last):
  File "/usr/local/bin/openaps-report", line 4, in <module>
    __import__('pkg_resources').run_script('openaps==0.2.0-dev', 'openaps-report')
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 534, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1438, in run_script
    execfile(script_filename, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/EGG-INFO/scripts/openaps-report", line 82, in <module>
    app( )
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/openaps/cli/__init__.py", line 51, in __call__
    self.run(self.args)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/EGG-INFO/scripts/openaps-report", line 75, in run
    output = app(args, self)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/openaps/cli/subcommand.py", line 52, in __call__
    return self.method.main(args, app)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/openaps/reports/invoke.py", line 40, in main
    output = task.method(args, app)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/openaps/uses/use.py", line 45, in __call__
    output = self.main(args, app)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/openaps/vendors/dexcom.py", line 830, in main
    results = super(iter_calibrations, self).main(args, app)
  File "/usr/local/lib/python2.7/dist-packages/openaps-0.2.0_dev-py2.7.egg/openaps/vendors/dexcom.py", line 420, in main
    for item in candidates:
  File "/usr/local/lib/python2.7/dist-packages/dexcom_reader/readdata.py", line 301, in iter_records
    records = list(self.ReadDatabasePage(record_type, x))
  File "/usr/local/lib/python2.7/dist-packages/dexcom_reader/readdata.py", line 267, in GenericRecordYielder
    yield record_type.Create(data, x)
  File "/usr/local/lib/python2.7/dist-packages/dexcom_reader/database_records.py", line 158, in Create
    return cls(unpacked_data, raw_data)
  File "/usr/local/lib/python2.7/dist-packages/dexcom_reader/database_records.py", line 180, in __init__
    self.check_crc()
  File "/usr/local/lib/python2.7/dist-packages/dexcom_reader/database_records.py", line 47, in check_crc
    raise constants.CrcError('Could not parse %s' % self.__class__.__name__)
dexcom_reader.constants.CrcError: Could not parse Calibration

The pumploop says:

reporting monitor/glucose-zoned.json
merge://JSON/shell/monitor/glucose-zoned-merge.json
reporting monitor/glucose-zoned-merge.json
raw://JSON/shell/monitor/glucose-raw-merge.json
reporting monitor/glucose-raw-merge.json
>>>>RESULTS<<<<
90  2017-03-22T22:18:18+01:00
90  2017-03-22T22:13:19+01:00
90  2017-03-22T22:08:17+01:00
first://JSON/shell/monitor/glucose-zoned-first.json
reporting monitor/glucose-zoned-first.json
ns://JSON/shell/nightscout/uploaded-first.json
reporting nightscout/uploaded-first.json
ns://JSON/shell/nightscout/recent-missing-entries.json
reporting nightscout/recent-missing-entries.json
ns://JSON/shell/nightscout/uploaded-entries.json
reporting nightscout/uploaded-entries.json
CGM Data 0.34 mins ago is fresh (< 3)
cgm://JSON/nightscout_calibrations/monitor/cal.json
cgm extras failed
Wed 22 Mar 22:19:01 CET 2017
CGM Data 0.76 mins ago is fresh (< 5.1), 4.34 mins > max wait (1 mins) waiting for next attempt

@scottleibrand any idea?

[PieterGit]

uploading from CLI works fine with ns-upload-entries monitor/glucose-raw-merge.json

[scottleibrand]

It looks like you're having trouble uploading the Dexcom calibration records rather than the actual glucose records. Not sure why that'd be: have you calibrated the Dexcom recently?

[PieterGit]

I think these are two problems. the calibration records via openaps/oref0 don't work for me. They do seem to work with the Nightscout uploader app. The upload works from the CLI, but not from the loop for some reason. Will debug more later.

[PieterGit]

The calibration error seems to be an openaps or dexcom_reader issue. It also fails with the underlying
openaps use cgm nightscout_calibrations 1 call. I created issue openaps/openaps#117 to track that issue.

I found the culpritt of the cgm uploading problem. The ~/.profile had a good API_SECRET and the crontab had a wrong API_SECRET. Problem disappeared when correcting the API_SECRET.

Closing this issue.

[PieterGit]

There were in fact two problems in this issue:

• the wrong API_SECRET
  (I solved this)

But, the the root issue of why calibrations don't work with certain G4 receivers seems to be not oref0 related but dexcom_related, see
openaps/dexcom_reader#16

[PieterGit]

I closed this issue, because the root issue is a dexcom_reader issue (and not and oref0 issue)