Skip to content

Commit

Permalink
[VL] Update document, remove the experimental word for spark.gluten.e…
Browse files Browse the repository at this point in the history
…nabled (apache#8615)

spark.gluten.enabled is stable for long time. It can enable/disable gluten at run time. With it disable, all query plans fallback to vanilla spark. Compare to plugin setting, the config may have a little overhead when generating query plan.
  • Loading branch information
FelixYBW authored and baibaichen committed Feb 1, 2025
1 parent e34e32a commit ea9f9a1
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ You can add these configurations into spark-defaults.conf to enable or disable t
| spark.sql.join.preferSortMergeJoin | When true, prefer sort merge join over shuffled hash join. <br /> Note: Please turn off preferSortMergeJoin. | false |
| spark.plugins | To load Gluten's components by Spark's plug-in loader | org.apache.gluten.GlutenPlugin |
| spark.shuffle.manager | To turn on Gluten Columnar Shuffle Plugin | org.apache.spark.shuffle.sort.ColumnarShuffleManager |
| spark.gluten.enabled | Enable Gluten, default is true. Just an experimental property. Recommend to enable/disable Gluten through the setting for `spark.plugins`. | true |
| spark.gluten.enabled | Enable Gluten at runtime, default is true. It fallbacks to vanilla Spark for all query plans if set to false. Recommend to enable/disable Gluten through the setting for `spark.plugins`. | true |
| spark.gluten.memory.isolation | (Experimental) Enable isolated memory mode. If true, Gluten controls the maximum off-heap memory can be used by each task to X, X = executor memory / max task slots. It's recommended to set true if Gluten serves concurrent queries within a single session, since not all memory Gluten allocated is guaranteed to be spillable. In the case, the feature should be enabled to avoid OOM. Note when true, setting spark.memory.storageFraction to a lower value is suggested since storage memory is considered non-usable by Gluten. | false |
| spark.gluten.ras.enabled | Enables RAS (relation algebra selector) during physical planning to generate more efficient query plan. Note, this feature doesn't bring performance profits by default. Try exploring option `spark.gluten.ras.costModel` for advanced usage. | false |
| spark.gluten.sql.columnar.maxBatchSize | Number of rows to be processed in each batch. Default value is 4096. | 4096 |
Expand Down

0 comments on commit ea9f9a1

Please sign in to comment.