Skip to content

FEAT - Extend the ToDatetime transformer so that it can take a list of formats #1719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
amirakahub wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

FEAT - Extend the ToDatetime transformer so that it can take a list of formats #1719

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7801852
shuffle the df before generating the html doc for reporting
amirakahub Oct 29, 2025
fa9dca4
shuffle the df before generating the html doc for reporting
amirakahub Oct 29, 2025
8d042e1
Update HTML table to match the visual and structural format of the sh…
amirakahub Oct 29, 2025
1b1e574
keep the index information while updating the html table
amirakahub Oct 29, 2025
1398d13
Enable the ToDatetime transformer to accept a list of format and chec…
amirakahub Oct 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 28 additions & 28 deletions doc/_templates/demo_tabular_pipeline.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,26 +47,26 @@
</thead>
<tbody>
<tr>
<th>0</th>
<td>F</td>
<td>POL</td>
<td>Department of Police</td>
<td>MSB Information Mgmt and...</td>
<th>1</th>
<td>M</td>
<td>FRS</td>
<td>Fire and Rescue Services</td>
<td>Third Battalion - Administration</td>
<td>Fulltime-Regular</td>
<td>Office Services Coordinator</td>
<td>09/22/1986</td>
<td>1986</td>
<td>Fire/Rescue Lieutenant</td>
<td>06/07/2004</td>
<td>2004</td>
</tr>
<tr>
<th>1</th>
<th>2</th>
<td>M</td>
<td>POL</td>
<td>Department of Police</td>
<td>ISB Major Crimes...</td>
<td>HHS</td>
<td>Department of Health and Human Services</td>
<td>Environmental Health and Regulatory Services</td>
<td>Fulltime-Regular</td>
<td>Master Police Officer</td>
<td>09/12/1988</td>
<td>1988</td>
<td>Environmental Health Specialist III</td>
<td>02/20/2007</td>
<td>2007</td>
</tr>
<tr>
<th>...</th>
Expand All @@ -82,24 +82,24 @@
<tr>
<th>9226</th>
<td>M</td>
<td>CCL</td>
<td>County Council</td>
<td>Council Central Staff</td>
<td>DGS</td>
<td>Department of General Services</td>
<td>Facilities Maintenance</td>
<td>Fulltime-Regular</td>
<td>Manager II</td>
<td>09/05/2006</td>
<td>2006</td>
<td>Master Plumber</td>
<td>03/26/2001</td>
<td>2001</td>
</tr>
<tr>
<th>9227</th>
<td>M</td>
<td>DLC</td>
<td>Department of Liquor Control</td>
<td>Licensure, Regulation...</td>
<td>F</td>
<td>HHS</td>
<td>Department of Health and Human Services</td>
<td>Infants and Toddlers</td>
<td>Fulltime-Regular</td>
<td>Alcohol/Tobacco Enforcement Specialist II</td>
<td>01/30/2012</td>
<td>2012</td>
<td>Program Specialist II</td>
<td>03/25/2013</td>
<td>2013</td>
</tr>
</tbody>
</table>
Expand Down
2 changes: 2 additions & 0 deletions doc/table_report.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

def generate_demo():
X = fetch_employee_salaries().X
X = X.sample(frac=1, random_state=145).reset_index(drop=True)

with open(
"_templates/demo_table_report_generated.html", "w", encoding="utf-8"
) as f:
Expand Down
11 changes: 9 additions & 2 deletions skrub/_to_datetime.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ class ToDatetime(SingleColumnTransformer):

Parameters
----------
format : str or None, optional, default=None
format : str or None or a given list of str, optional, default=None
Format to use for parsing dates that are stored as strings, e.g.
``"%Y-%m-%dT%H:%M%S"``.
If not specified, the format is inferred from the data when possible.
Expand Down Expand Up @@ -424,14 +424,21 @@ def transform(self, column):
return sbd.cast(column, self.output_dtype_)

def _get_datetime_format(self, column):
if self.format is not None:
if self.format is not None and isinstance(self.format, str) :
return self.format
not_null = sbd.drop_nulls(column)
sample = sbd.sample(
not_null, n=min(_SAMPLE_SIZE, sbd.shape(not_null)[0]), seed=0
)
if not sbd.is_string(sample):
return None
if isinstance(self.format, list):
for format_option in self.format:
try:
sbd.to_datetime(column, format=format_option, strict=False)
return proposed_format
except Exception:
continue
return _guess_datetime_format(sample)


Expand Down
Loading