Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add data type and time zone considerations #6335

Merged
merged 8 commits into from
Oct 23, 2024
Merged

add data type and time zone considerations #6335

Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
efb3561
add data type and time zone considerations
mirnawong1 Oct 22, 2024
19199a3
Update website/docs/docs/build/metricflow-time-spine.md
mirnawong1 Oct 22, 2024
e689c8a
Update website/docs/docs/build/metricflow-time-spine.md
mirnawong1 Oct 22, 2024
742e216
Update metricflow-time-spine.md
mirnawong1 Oct 22, 2024
33a795d
Merge branch 'current' into mirnawong1-patch-27
mirnawong1 Oct 22, 2024
086761c
Merge branch 'current' into mirnawong1-patch-27
mirnawong1 Oct 22, 2024
74a351e
Update website/docs/docs/build/metricflow-time-spine.md
mirnawong1 Oct 23, 2024
180cb85
Update website/docs/docs/build/metricflow-time-spine.md
mirnawong1 Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion website/docs/docs/build/metricflow-time-spine.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -463,9 +463,22 @@ For example, if you use a custom calendar in your organization, such as a fiscal

- This is useful for calculating metrics based on a custom calendar, such as fiscal quarters or weeks.
- Use the `custom_granularities` key to define a non-standard time period for querying data, such as a `retail_month` or `fiscal_week`, instead of standard options like `day`, `month`, or `year`.
- Ensure the the `standard_granularity_column` is a date time type.
- This feature provides more control over how time-based metrics are calculated.

<Expandable alt_header="Data types and time zone considerations">

When working with custom calendars in MetricFlow, it's important to ensure:

- Consistent data types &mdash; Both your dimension column and the time spine column should use the same data type to allow accurate comparisons. Functions like `DATE_TRUNC` don't change the data type of the input in some databases (like Snowflake). Using different data types can lead to mismatches and inaccurate results.

We recommend using `DATETIME` or `TIMESTAMP` data types for your time dimensions and time spine, as they support all granularities. The `DATE` data type may not support higher granularities like hours or minutes.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

- Consistent time zones &mdash; Ensure that all your time-related data uses the same time zone. MetricFlow supports UTC and currently doesn't perform any timezone manipulation. This means inconsistent time zones can cause unexpected results during aggregations and comparisons.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would change this section slightly:
Time zones &mdash; MetricFlow currently doesn't perform any timezone manipulation. When working with timezone-aware data, inconsistent time zones may lead to unexpected results during aggregations and comparisons.
Reasons for the change:

  • MF doesn't support UTC per se, it just ignores timezones entirely. UTC is just the most common way that people are likely to store their data for timezone consistency.
  • I don't think we should ask them to ensure all their data uses the same timezone since they might have a valid reason for not doing that. We just want to warn them that there might be weird interactions in that case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good callout!

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

For example, if your time spine column is `TIMESTAMP` type and your dimension column is `DATE` type, comparisons between these columns might not work as intended. To fix this, convert your `DATE` column to `TIMESTAMP`, or make sure both columns are the same data type.

</Expandable>

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
### Add custom granularities

To add custom granularities, the Semantic Layer supports custom calendar configurations that allow users to query data using non-standard time periods like `fiscal_year` or `retail_month`. You can define these custom granularities (all lowercased) by modifying your model's YAML configuration like this:
Expand Down
Loading