Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Update document, remove the experimental word for spark.gluten.enabled #8615

Merged
merged 2 commits into from
Jan 29, 2025
Merged

[VL] Update document, remove the experimental word for spark.gluten.enabled #8615

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
33623f3
Update Configuration.md
FelixYBW Jan 25, 2025
186ac21
Update Configuration.md
FelixYBW Jan 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/Configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ You can add these configurations into spark-defaults.conf to enable or disable t
| spark.sql.join.preferSortMergeJoin | When true, prefer sort merge join over shuffled hash join. <br /> Note: Please turn off preferSortMergeJoin. | false |
| spark.plugins | To load Gluten's components by Spark's plug-in loader | org.apache.gluten.GlutenPlugin |
| spark.shuffle.manager | To turn on Gluten Columnar Shuffle Plugin | org.apache.spark.shuffle.sort.ColumnarShuffleManager |
| spark.gluten.enabled | Enable Gluten, default is true. Just an experimental property. Recommend to enable/disable Gluten through the setting for `spark.plugins`. | true |
| spark.gluten.enabled | Enable Gluten at runtime, default is true. It fallbacks to vanilla Spark for all query plans. Recommend to enable/disable Gluten through the setting for `spark.plugins`. | true |
FelixYBW marked this conversation as resolved.
Show resolved Hide resolved
| spark.gluten.memory.isolation | (Experimental) Enable isolated memory mode. If true, Gluten controls the maximum off-heap memory can be used by each task to X, X = executor memory / max task slots. It's recommended to set true if Gluten serves concurrent queries within a single session, since not all memory Gluten allocated is guaranteed to be spillable. In the case, the feature should be enabled to avoid OOM. Note when true, setting spark.memory.storageFraction to a lower value is suggested since storage memory is considered non-usable by Gluten. | false |
| spark.gluten.ras.enabled | Enables RAS (relation algebra selector) during physical planning to generate more efficient query plan. Note, this feature doesn't bring performance profits by default. Try exploring option `spark.gluten.ras.costModel` for advanced usage. | false |
| spark.gluten.sql.columnar.maxBatchSize | Number of rows to be processed in each batch. Default value is 4096. | 4096 |
Expand Down
Loading