Reject CREATE TABLE/VIEW with duplicate column names #13517
+790
−160
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
sql
DFSchema checks column for uniqueness, but allows duplicate column names when they are qualified differently. This is because DFSchema plays central role during query planning as a identifier resolution scope. Those checks in their current form should not be there, since they prevent execution of queries with duplicate column aliases, which is legal in SQL. But even with these checks present, they are not sufficient to ensure CREATE TABLE/VIEW is well structured. Table or view columns need to have unique names and there is no qualification involved. This commit adds necessary checks in CREATE TABLE/VIEW DDL structs, ensuring that CREATE TABLE/VIEW logical plans are valid in that regard.
findepi
force-pushed
the
findepi/protect-create-table
branch
from
57aba78
to
65e6c46
Compare
findepi
commented
Nov 21, 2024
comphead
reviewed
Nov 21, 2024
| @@ -150,6 +150,11 @@ pub enum SchemaError { | |||
| qualifier: Box<TableReference>, | |||
| name: String, | |||
| }, | |||
| /// Schema duplicate qualified fields with duplicate unqualified names | |||
There was a problem hiding this comment.
Probably add a small text example here what is considered as duplicates?
comphead
reviewed
Nov 21, 2024
| /// Whether the table is an infinite streams | ||
| pub unbounded: bool, | ||
| /// Table(provider) specific options | ||
| pub options: HashMap<String, String>, |
There was a problem hiding this comment.
maybe we can add comments what is a key and what is the value in the hashmaps?
comphead
reviewed
Nov 21, 2024
| ])) | ||
| )?) | ||
| .unwrap_err().strip_backtrace().to_string(), | ||
| "Schema error: Schema contains qualified fields with duplicate unqualified names t1.c1" |
There was a problem hiding this comment.
🤔 I feel lower case col names are not easy to read in the error message. Will tweak the schema_err! output
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reject CREATE TABLE/VIEW with duplicate column names
DFSchema checks column for uniqueness, but allows duplicate column names
when they are qualified differently. This is because DFSchema plays
central role during query planning as a identifier resolution scope.
Those checks in their current form should not be there, since they
prevent execution of queries with duplicate column aliases, which is
legal in SQL. But even with these checks present, they are not
sufficient to ensure CREATE TABLE/VIEW is well structured. Table or
view columns need to have unique names and there is no qualification
involved.
This commit adds necessary checks in CREATE TABLE/VIEW DDL structs,
ensuring that CREATE TABLE/VIEW logical plans are valid in that regard.
This PR includes
Encapsulate create table/view construction (to be able to add these checks)
Checks in create table/view construction to validate schema has no duplicate names
fixes CREATE TABLE succeeds when schema has duplicate names, resulting in a table that cannot be selected from #13487
extracted from Support duplicate column aliases in queries #13489