Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-37239][docs] Add examples for unhex, btrim, translate, elt and percentile #26091

Merged
merged 4 commits into from
Feb 2, 2025
Merged

[FLINK-37239][docs] Add examples for unhex, btrim, translate, elt and percentile #26091

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9f480dc
[FLINK-37239][docs] Add examples for unhex, btrim, translate, elt and…
gustavodemorais Jan 30, 2025
3c08458
[FLINK-37239][docs] Improve example for unhex
gustavodemorais Jan 30, 2025
8f5a88d
[FLINK-37239][docs] Add examples to chinese document
gustavodemorais Jan 30, 2025
771c400
[FLINK-37239][docs] Improve percentile example
gustavodemorais Jan 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions docs/data/sql_functions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,9 +241,9 @@ arithmetic:
- sql: UNHEX(expr)
table: expr.unhex()
description: |
Converts hexadecimal string expr to BINARY.
Converts hexadecimal string expr to BINARY. If the length of expr is odd, the first character is discarded and the result is left padded with a null byte.

If the length of expr is odd, the first character is discarded and the result is left padded with a null byte.
E.g., SELECT DECODE(UNHEX('466C696E6B') , 'UTF-8' ) or '466C696E6B'.unhex().decode('UTF-8') returns "Flink".

expr <CHAR | VARCHAR>

Expand All @@ -256,7 +256,9 @@ arithmetic:
description: |
Returns the exact percentile value of expr at the specified percentage in a group.

percentage must be a literal numeric value between `[0.0, 1.0]` or an array of such values.
E.g., SELECT PERCENTILE(age, 0.5) FROM (VALUES (0), (50), (100)) AS age or $('age').percentile(0.5) returns 50.0.

The percentage must be a literal numeric value between `[0.0, 1.0]` or an array of such values.
Copy link
Contributor

@snuyanzin snuyanzin Jan 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, during the first iteration I didn't test the examples...

Now it took me almost half an hour to realize that this is based on a continuous distribution...
I think we need to think about other example here...
It returns same result for the case of 0.5 for both
however in case of discrete distribution

SELECT PERCENTILE(age, 0.25) FROM (VALUES (0), (25), (50), (100)) AS age

I would expect output 25.0 however it shows different result (since the function is for continuous)

It should be explicitly mentioned in the doc since in SQL Standard there are PERCENTILE_CONT and PERCENTILE_DISC.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returns the exact percentile value of expr at the specified percentage in a group.

looks like description is also incorrect since usually by group it is meant within a limited group (discrete)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into it and makes sense. I can update the example to a non ambiguous one and complement the description. Suggestion:

Returns the percentile value of expr at the specified percentage using continous distribution.
      
E.g., SELECT PERCENTILE(age, 0.25) FROM (VALUES  (10), (20), (30), (40)) AS age returns 17.5
      
The percentage must be a literal numeric value between `[0.0, 1.0]` or an array of such values.
...

Btw, there's another paragraph right below that points to the behavior being of a continuous percentile function and explain to the user how the value is calculated when there's no exact match:

If no expr lies exactly at the desired percentile, the result is calculated using linear interpolation of the two nearest exprs. If expr or frequency is NULL, or frequency is not positive, the input row will be ignored.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good to me, thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks for the input @snuyanzin

If a variable expression is passed to this function, the result will be calculated using any one of them.
frequency describes how many times expr should be counted, the default value is 1.

Expand Down Expand Up @@ -335,6 +337,8 @@ string:
description: |
Removes any leading and trailing characters within trimStr from str. trimStr is set to whitespace by default.

E.g., BTRIM(' www.apache.org ') or ' www.apache.org '.btrim() returns "www.apache.org", BTRIM('/www.apache.org/', '/') or ' www.apache.org '.btrim() returns "www.apache.org".

str <CHAR | VARCHAR>, trimStr <CHAR | VARCHAR>

Returns a STRING representation of the trimmed str. `NULL` if any of the arguments are `NULL`.
Expand Down Expand Up @@ -431,10 +435,10 @@ string:
- sql: TRANSLATE(expr, fromStr, toStr)
table: expr.translate(fromStr, toStr)
description: |
Translate an expr where all characters in fromStr have been replaced with those in toStr.

If toStr has a shorter length than fromStr, unmatched characters are removed.
Translate an expr where all characters in fromStr have been replaced with those in toStr. If toStr has a shorter length than fromStr, unmatched characters are removed.

E.g., SELECT TRANSLATE3('www.apache.org', 'wapcheorg', ' APCHEcom') or 'www.apache.org'.translate('wapcheorg', ' APCHEcom') returns " .APACHE.com".

`expr <CHAR | VARCHAR>, fromStr <CHAR | VARCHAR>, toStr <CHAR | VARCHAR>`

Returns a `STRING` of translated expr.
Expand Down Expand Up @@ -531,6 +535,8 @@ string:
description: |
Returns the index-th expression. index must be an integer between 1 and the number of expressions.

E.g., SELECT ELT(2, 'scala-1', 'java-2', 'go-3') or 2.elt('scala-1', 'java-2', 'go-3') returns "java-2".

index <TINYINT | SMALLINT | INTEGER | BIGINT>, expr <CHAR | VARCHAR>, exprs <CHAR | VARCHAR>

index <TINYINT | SMALLINT | INTEGER | BIGINT>, expr <BINARY | VARBINARY>, exprs <BINARY | VARBINARY>
Expand Down
16 changes: 11 additions & 5 deletions docs/data/sql_functions_zh.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,9 +299,9 @@ arithmetic:
- sql: UNHEX(expr)
table: expr.unhex()
description: |
将十六进制字符串 expr 转换为 BINARY。
将十六进制字符串 expr 转换为 BINARY。如果 expr 的长度为奇数，则会舍弃第一个字符，并在结果开头补充一个空字节。

如果 expr 的长度为奇数，则会舍弃第一个字符，并在结果开头补充一个空字节。
E.g., SELECT DECODE(UNHEX('466C696E6B') , 'UTF-8' ) or '466C696E6B'.unhex().decode('UTF-8') returns "Flink".

expr <CHAR | VARCHAR>

Expand All @@ -317,6 +317,8 @@ arithmetic:
table: expr.percentile(percentage[, frequency])
description: |
返回 expr 的 percentage 百分位值。

E.g., SELECT PERCENTILE(age, 0.5) FROM (VALUES (0), (50), (100)) AS age or $('age').percentile(0.5) returns 50.0.

percentage 必须是一个在 `[0.0, 1.0]` 之间的字面数值，或者是一个该范围的数组。
如果传递了一个可变表达式给这个函数，结果将根据其中的任意一个值进行计算。
Expand Down Expand Up @@ -397,6 +399,8 @@ string:
description: |
从 str 的开头和结尾删除 trimStr 中的字符。trimStr 默认设置为空格。

E.g., BTRIM(' www.apache.org ') or ' www.apache.org '.btrim() returns "www.apache.org", BTRIM('/www.apache.org/', '/') or ' www.apache.org '.btrim() returns "www.apache.org".

str <CHAR | VARCHAR>, trimStr <CHAR | VARCHAR>

返回一个 STRING 格式的裁剪后的 str。如果任何参数为 `NULL`，则返回 `NULL`。
Expand Down Expand Up @@ -498,10 +502,10 @@ string:
- sql: TRANSLATE(expr, fromStr, toStr)
table: expr.translate(fromStr, toStr)
description: |
将 expr 中所有出现在 fromStr 之中的字符替换为 toStr 中的相应字符。

如果 toStr 的长度短于 fromStr，则未匹配的字符将被移除。
将 expr 中所有出现在 fromStr 之中的字符替换为 toStr 中的相应字符。如果 toStr 的长度短于 fromStr，则未匹配的字符将被移除。

E.g., SELECT TRANSLATE3('www.apache.org', 'wapcheorg', ' APCHEcom') or 'www.apache.org'.translate('wapcheorg', ' APCHEcom') returns " .APACHE.com".

`expr <CHAR | VARCHAR>, fromStr <CHAR | VARCHAR>, toStr <CHAR | VARCHAR>`

返回 `STRING` 格式的转换结果。
Expand Down Expand Up @@ -628,6 +632,8 @@ string:
description: |
返回第 index 个表达式。index 必须是介于 1 和 表达式数量之间的整数。

E.g., SELECT ELT(2, 'scala-1', 'java-2', 'go-3') or 2.elt('scala-1', 'java-2', 'go-3') returns "java-2".

index <TINYINT | SMALLINT | INTEGER | BIGINT>, expr <CHAR | VARCHAR>, exprs <CHAR | VARCHAR>

index <TINYINT | SMALLINT | INTEGER | BIGINT>, expr <BINARY | VARBINARY>, exprs <BINARY | VARBINARY>
Expand Down