Skip to content

Athena query not taking Glue documentatioin #2062

@Davidbackx

Description

@Davidbackx

Describe the bug
With the WrenAI Athena connection it runs a query to create the catalog (based of glue) but if you have Glue documentation for your tables/columns it currently is not ingested.

Expected behavior
Currently the Athena connection is running the following query:

SELECT 
                t.table_catalog,
                t.table_schema,
                t.table_name,
                c.column_name,
                c.ordinal_position,
                c.is_nullable,
                c.data_type
            FROM 
                information_schema.tables AS t 
            JOIN 
                information_schema.columns AS c 
                ON t.table_catalog = c.table_catalog
                AND t.table_schema = c.table_schema
                AND t.table_name = c.table_name
            WHERE t.table_schema = '<schema>'
            ORDER BY t.table_name

If we were to add the column c.comment it would include the documentation for the table/columns

Here is the query defined: https://github.com/Canner/wren-engine/blob/51a622f02d603c58a87d3757704fada82d5f5851/ibis-server/app/model/metadata/athena.py#L54

It should accept the c.comment field, and update the description as well

Wren AI Information

  • Version: 0.28.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions