Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support loading hiveConf from hive-site.xml with customized location for hive connector #193

Open
numbernumberone opened this issue Nov 21, 2022 · 3 comments · May be fixed by #390
Open

Support loading hiveConf from hive-site.xml with customized location for hive connector #193

numbernumberone opened this issue Nov 21, 2022 · 3 comments · May be fixed by #390
Labels
difficulty-easy Easy difficulty to fix this issue help wanted Extra attention is needed

Comments

@numbernumberone
Copy link

No description provided.

@garyli1019
Copy link
Collaborator

hi, currently we didn't support this feature yet. I think this is a very good feature to have though. Converting this to a feature request for the contributor motivation program with prize~

@garyli1019 garyli1019 added help wanted Extra attention is needed difficulty-easy Easy difficulty to fix this issue labels Nov 21, 2022
@garyli1019 garyli1019 changed the title How to customize hive-site.xml in bitsail project Support loading hiveConf from hive-site.xml with customized location for hive connector Nov 21, 2022
@garyli1019
Copy link
Collaborator

Currently we need to pass hive configuration through job.writer.metastore_properties to the hive connector, but for the user already have hive-site.xml in their environment, passing those configs manually could be avoided. So we could add a configuration and support loading hive-site.xml automatically.
Note: We need to support two scenarios. hive-site.xml only exists on the client side and hive-site.xml exists on both client side and yarn.

@numbernumberone
Copy link
Author

Currently we need to pass hive configuration through job.writer.metastore_properties to the hive connector, but for the user already have hive-site.xml in their environment, passing those configs manually could be avoided. So we could add a configuration and support loading hive-site.xml automatically. Note: We need to support two scenarios. hive-site.xml only exists on the client side and hive-site.xml exists on both client side and yarn.

@garyli1019 yeah，but i found that the configuration job.writer.metastore_properties priority in the json is not as high as the environment hive-site.xml.

@garyli1019 garyli1019 linked a pull request Feb 13, 2023 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
difficulty-easy Easy difficulty to fix this issue help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[BitSail] support load hive conf from specified location garyli1019/bitsail
2 participants
@numbernumberone @garyli1019