Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: panic when adding new field in schema registry proto definition #19736

Open
fuyufjh opened this issue Dec 10, 2024 · 2 comments · May be fixed by #19828
Open

bug: panic when adding new field in schema registry proto definition #19736

fuyufjh opened this issue Dec 10, 2024 · 2 comments · May be fixed by #19828
Assignees
@BugenZhao
Labels
type/bug Something isn't working
Milestone
release-2.2

Comments

@fuyufjh
Copy link
Member

fuyufjh commented Dec 10, 2024 •
edited
Loading

Describe the bug

After updating the schema in schema registry and run REFRESH from RisingWave, the compute nodes start crash loop

Possible reason: Currently, RisingWave doesn’t support alter a struct column i.e. adding a new field inside the struct column. If doing this with SQL, it will return an “unsupported” error. But this schema change caused exactly this kind of column schema change, and RW fails to handle it correctly.

Error message/log

22:     0x5ad9a35c8cd2 - risingwave_common::util::value_encoding::inner_deserialize_datum::h5d12937549c43ae1
                               at /risingwave/src/common/src/util/value_encoding/mod.rs:195:20
  23:     0x5ad9a35c8cd2 - risingwave_common::util::value_encoding::deserialize_struct::h9d502dba455bb3c1
                               at /risingwave/src/common/src/util/value_encoding/mod.rs:371:27
  24:     0x5ad9a35c8cd2 - risingwave_common::util::value_encoding::deserialize_value::hf4b8c836e0f09fb4
                               at /risingwave/src/common/src/util/value_encoding/mod.rs:357:41
  25:     0x5ad9a35a1306 - <risingwave_common::util::value_encoding::column_aware_row_encoding::Deserializer as risingwave_common::util::value_encoding::ValueRowDeserializer>::deserialize::hc70588ccd8708ede
                               at /risingwave/src/common/src/util/value_encoding/column_aware_row_encoding.rs:232:30
  26:     0x5ad9abd7c00d - <risingwave_common::util::value_encoding::column_aware_row_encoding::ColumnAwareSerde as risingwave_common::util::value_encoding::ValueRowDeserializer>::deserialize::ha0cde7b4168459e3
                               at /risingwave/src/common/src/util/value_encoding/column_aware_row_encoding.rs:278:9
  27:     0x5ad9abd7c00d - risingwave_stream::common::table::state_table::StateTableInner<S,SD,_,_>::get_row::{{closure}}::hfc439b1e9f62e24f
                               at /risingwave/src/stream/src/common/table/state_table.rs:818:27
  28:     0x5ad9abd7c00d - risingwave_stream::common::table::state_table::StateTableInner<S,SD,_,_>::get_compacted_row::{{closure}}::hc7a971c6f3feee86
                               at /risingwave/src/stream/src/common/table/state_table.rs:877:18
  29:     0x5ad9abd7c00d - risingwave_stream::executor::mview::materialize::MaterializeCache<SD>::fetch_keys::{{closure}}::{{closure}}::h52a099359679c498
                               at /risingwave/src/stream/src/executor/mview/materialize.rs:736:66

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

Full Logs: https://files.slack.com/files-pri/T030LTU38S2-F084MFP37PB/panic.log
@fuyufjh fuyufjh added the type/bug Something isn't working label Dec 10, 2024
@github-actions github-actions bot added this to the release-2.2 milestone Dec 10, 2024
@st1page
Copy link
Contributor

st1page commented Dec 10, 2024

Could it be a break changes becasue we need to change the encoding of the struct?

@fuyufjh
Copy link
Member Author

fuyufjh commented Dec 11, 2024 •
edited
Loading

Could it be a break changes becasue we need to change the encoding of the struct?

True. For the existing tables, at least we need to ban this action, e.g. returning an error when alter table refresh schema

I think this issue should be targeted at this. While for the full support, let's open another issue. #19755

This was referenced Dec 11, 2024
Open
Open
@fuyufjh fuyufjh mentioned this issue Dec 18, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BugenZhao BugenZhao

Labels
type/bug Something isn't working
Projects
None yet
Milestone
release-2.2
Development

Successfully merging a pull request may close this issue.

fix(frontend): check data type in column id generator risingwavelabs/risingwave
3 participants
@fuyufjh @BugenZhao @st1page