Skip to content

Conversation

@r2evans
Copy link
Contributor

@r2evans r2evans commented Jan 9, 2026

Two steps here for POSIXt value preservation:

  • sets "tzone" attribute when missing/empty
  • add options(bigrquery.digits.secs=6) to allow sending up to 6 sub-second digits to POSIXt-class objects

Before this patch,

bq_auth(path="my_token.json")
tbl <- data.frame(dbl=pi, psx=Sys.time())
ds <- bigrquery::bq_table(...)
bq_table_upload(ds, tbl); tbl2 <- bq_table_download(ds); bq_table_delete(ds)
tbl$psx - tbl2$psx
# Time difference of 5.000016 hours

Just fixing "tzone", we can get

bq_table_upload(ds, tbl); tbl3 <- bq_table_download(ds); bq_table_delete(ds)
tbl$psx - tbl3$psx
# Time difference of 0.0591259 secs

To support microsecond precision,

options(bigrquery.digits=22, bigrquery.digits.secs=6)
bq_table_upload(ds, tbl); tbl4 <- bq_table_download(ds); bq_table_delete(ds)
tbl$psx - tbl4$psx
# Time difference of 9.536743e-07 secs

Includes NEWS, code, and unit-tests.

@r2evans r2evans changed the title fix #660, POSIXt precision and tzone Fix POSIXt precision and tzone Jan 10, 2026
R/bq-perform.R Outdated
# Convert times to canonical format
is_time <- vapply(values, function(x) inherits(x, "POSIXt"), logical(1))
values[is_time] <- lapply(values[is_time], format, "%Y-%m-%d %H:%M:%S")
digsec <- getOption("bigrquery.digits.secs")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we need to make this an option? i.e. why not always use 6 digits?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair question. Two reasons, for better or worse:

  1. Backward compatibility, so that revdeps don't see starkly-changing behavior.
  2. Consistency with jsonlite's notion of "smaller is better" (regardless of lossy-ness).

Some (perhaps even most) people may not care about millisecond-resolution, I don't assume everybody is as concerned about it as I am. If it's hard-coded 6 then I'll be fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd prefer making it a constant. I don't think it should negatively affect revdeps; it might change some results in very small ways, but those changes should generally be for the better.

Copy link
Member

@hadley hadley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big picture now looks good, so it's time for the nit-picking 😄 Thanks for working on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants