Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add dt.replace #19708

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

feat: Add dt.replace #19708

wants to merge 14 commits into from

Conversation

mcrumiller
Copy link
Contributor

Closes #8879.

Probably can use some improvements on the Rust side (seems to be some code duplication but I'm not sure how to best fix it), so any suggestions are welcome please! @MarcoGorelli I moved around a bit of code that overlapped with some of yours but I think the movements all make sense.

Examples

import polars as pl
from polars import col
from datetime import date, datetime

s = pl.Series([date(2024, 1, 1), date(2024, 1, 2), date(2024, 1, 3)])
s.dt.replace(month=2, day=15)
# shape: (3,)
# Series: '' [date]
# [
#         2024-02-15
#         2024-02-15
#         2024-02-15
# ]

df = pl.DataFrame({
    "a": [datetime(1, 1, 1, 1, 1, 1, 1)],
})
df.with_columns(
    col("a").dt.replace(year=2024).alias("year"),
    col("a").dt.replace(year=2015, day=23, second=45, microsecond=5000).alias("many"),
)
# shape: (1, 3)
# ┌────────────────────────────┬────────────────────────────┬─────────────────────────┐
# │ a                          ┆ year                       ┆ many                    │
# │ ---                        ┆ ---                        ┆ ---                     │
# │ datetime[μs]               ┆ datetime[μs]               ┆ datetime[μs]            │
# ╞════════════════════════════╪════════════════════════════╪═════════════════════════╡
# │ 0001-01-01 01:01:01.000001 ┆ 2024-01-01 01:01:01.000001 ┆ 2015-01-23 01:01:45.005 │
# └────────────────────────────┴────────────────────────────┴─────────────────────────┘

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Nov 9, 2024
Comment on lines -139 to -178
let ca: Int64Chunked = year
.into_iter()
.zip(month)
.zip(day)
.zip(hour)
.zip(minute)
.zip(second)
.zip(microsecond)
.map(|((((((y, m), d), h), mnt), s), us)| {
if let (Some(y), Some(m), Some(d), Some(h), Some(mnt), Some(s), Some(us)) =
(y, m, d, h, mnt, s, us)
{
NaiveDate::from_ymd_opt(y, m, d)
.and_then(|nd| nd.and_hms_micro_opt(h, mnt, s, us))
.map(|ndt| match time_unit {
TimeUnit::Milliseconds => ndt.and_utc().timestamp_millis(),
TimeUnit::Microseconds => ndt.and_utc().timestamp_micros(),
TimeUnit::Nanoseconds => ndt.and_utc().timestamp_nanos_opt().unwrap(),
})
} else {
None
}
})
.collect_trusted();

let ca = match time_zone {
#[cfg(feature = "timezones")]
Some(_) => {
let mut ca = ca.into_datetime(*time_unit, None);
ca = replace_time_zone(&ca, time_zone, _ambiguous, NonExistent::Raise)?;
ca
},
_ => {
polars_ensure!(
time_zone.is_none(),
ComputeError: "cannot make use of the `time_zone` argument without the 'timezones' feature enabled."
);
ca.into_datetime(*time_unit, None)
},
};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this code into datetime_series_from_parts in polars_time/src/series/mod.rs for re-use.

@mcrumiller
Copy link
Contributor Author

@MarcoGorelli I could also perhaps use some help creating tests to cover the non_existent parameter and the ambiguous parameter, neither of which my tests cover.

@MarcoGorelli
Copy link
Collaborator

@MarcoGorelli I could also perhaps use some help creating tests to cover the non_existent parameter and the ambiguous parameter, neither of which my tests cover.

sure - would it work to start with

datetime(2020, 10, 25)

and then do .dt.replace(hour=1)?

@mcrumiller
Copy link
Contributor Author

sure - would it work to start with

datetime(2020, 10, 25)

and then do .dt.replace(hour=1)?

Thanks--looks like I need to use Europe/London for this to work (found in another unit test).

Copy link

codecov bot commented Nov 13, 2024

Codecov Report

Attention: Patch coverage is 75.00000% with 80 lines in your changes missing coverage. Please review.

Project coverage is 79.55%. Comparing base (8cb7839) to head (a1933a3).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-time/src/replace.rs 62.18% 45 Missing ⚠️
crates/polars-time/src/series/mod.rs 66.17% 23 Missing ⚠️
...ates/polars-plan/src/dsl/function_expr/datetime.rs 84.21% 6 Missing ⚠️
.../polars-python/src/lazyframe/visitor/expr_nodes.rs 0.00% 5 Missing ⚠️
py-polars/polars/expr/datetime.py 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #19708      +/-   ##
==========================================
- Coverage   79.55%   79.55%   -0.01%     
==========================================
  Files        1544     1545       +1     
  Lines      213240   213514     +274     
  Branches     2441     2442       +1     
==========================================
+ Hits       169643   169851     +208     
- Misses      43048    43113      +65     
- Partials      549      550       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add .dt.replace
2 participants