Skip to content

Conversation

@r2evans
Copy link
Contributor

@r2evans r2evans commented Jan 7, 2026

A few facets, all related to round-trip handling of numbers and their precision.

Before this patch:

bq_auth(path="my_token.json")
tbl <- data.frame(dbl=pi, psx=Sys.time())
ds <- bigrquery::bq_table(...)
bq_table_upload(ds, tbl); tbl2 <- bq_table_download(ds); bq_table_delete(ds)
tbl$psx - tbl2$psx
# Time difference of 5.000016 hours

The "5 hours" issue is due to missing the tzone on upload. By appending the timezone on uploaded POSIXt data, we get then

bq_table_upload(ds, tbl); tbl3 <- bq_table_download(ds); bq_table_delete(ds)
tbl$psx - tbl3$psx
# Time difference of 0.0591259 secs

where 59 milliseconds can be an issue for those needing sub-second precision. (It crushes my work completely.)

Additionally,

tbl$dbl - tbl2$dbl
# [1] 2.65359e-06

That's not as troublesome, but it has been discussed and requested that there be a mechanism to set the digits=/precision of uploaded data. After this patch:

options(bigrquery.digits=22, bigrquery.digits.secs=6)
bq_table_upload(ds, tbl); tbl4 <- bq_table_download(ds); bq_table_delete(ds)
tbl$dbl - tbl4$dbl
# [1] 0
tbl$psx - tbl4$psx
# Time difference of 9.536743e-07 secs

@r2evans
Copy link
Contributor Author

r2evans commented Jan 9, 2026

@hadley @jennybc

Is there a way to escalate discussing this or the related bugs #320 and #660? I have premium Posit Support (I'm the POC for Ford Racing's use of Connect). This bug has significant impact, there's no way we can write anything to BQ when our client's data requires millisecond resolution.

@hadley
Copy link
Member

hadley commented Jan 9, 2026

@r2evans Jenny and I are now aware of the problem, which is the first step. As I'm sure you're aware, a premium posit support client doesn't entitled you to expedited open source support, but we are of course happy to try and help. It would be much easier to review your work if you did one PR per issue; as it is, there are a lot of changes making it hard to review your PR.

@r2evans
Copy link
Contributor Author

r2evans commented Jan 9, 2026

Thanks for your quick reply, @hadley. I was hoping the priority support would give at least an iota, I apologize for over-stepping.

I thought combining them into one was useful b/c they're all about round-trip data, and the changes seemed minor. Since it will help things, I will break these into multiple 1-change PRs, one for each issue. (And likely #663 as well, almost a one-line change.)

@hadley
Copy link
Member

hadley commented Jan 9, 2026

Anything you can do to make it faster for us to review a PR is always appreciated 😄

@r2evans
Copy link
Contributor Author

r2evans commented Jan 9, 2026

Done, now there's #664, #665, #666, and #667.

@r2evans r2evans closed this Jan 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants