Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

新增DAG时,算子设置PSI同样的字段属性,但前端选择框不一致 #20

Closed
WandQ opened this issue May 10, 2024 · 14 comments
Assignees

Comments

@WandQ
Copy link

WandQ commented May 10, 2024

Issue Type

Others

Source

binary

Secretflow Version

v1.5.0.dev240319

OS Platform and Distribution

centos

Python version

3.10.14

Bazel version

No response

GCC/Compiler version

No response

What happend and What you expected to happen.

新增自己的DAG组件时,前端展示上遇到了一个疑问,请教一下。
我参考PSI算子写了我的算子DAG组件,但是输入按钮的前端风格上不一样,是为什么呢?
PSI DAG的key选择方式:
![4A1CC307-BA4B-4814-AEAC-390AC8439E78](https://github.com/secretflow/secretflow/assets/56994734/0ffa253a-1400-4ebb-bd87-03182100ef24)
其会自主选择连线的输入表,并且是一个下拉框用于选择ID。
而我产生的DAG:
![C766ACEF-E730-4E6B-B05E-9B417F44D105](https://github.com/secretflow/secretflow/assets/56994734/dc7c0ee0-4703-48ee-9b0d-6b63d87a05c3)
其并不会有默认选择,并且点击后是个弹窗。

Reproduction code to reproduce the issue.

我的算子的输入部分代码:
xx.io(
    io_type=IoType.INPUT,
    name="receiver_input",
    desc="Individual table for receiver",
    types=[DistDataType.INDIVIDUAL_TABLE],
    col_params=[
        TableColParam(
            name="key",
            desc="Column(s) used to join.",
            col_min_cnt_inclusive=1,
            col_max_cnt_inclusive=1,
        ),
        TableColParam(
            name="labels",
            desc="Column(s) used to compute.",
        ),
    ],
)
xx.io(
    io_type=IoType.INPUT,
    name="sender_input",
    desc="Individual table for sender",
    types=[DistDataType.INDIVIDUAL_TABLE],
    col_params=[
        TableColParam(
            name="key",
            desc="Column(s) used to join.",
            col_min_cnt_inclusive=1,
            col_max_cnt_inclusive=1,
        ),
        TableColParam(
            name="labels",
            desc="Column(s) used to compute.",
        ),
    ],
)

PSI该部分:
psi_comp.io(
    io_type=IoType.INPUT,
    name="receiver_input",
    desc="Individual table for receiver",
    types=[DistDataType.INDIVIDUAL_TABLE],
    col_params=[
        TableColParam(
            name="key",
            desc="Column(s) used to join.",
            col_min_cnt_inclusive=1,
        )
    ],
)
psi_comp.io(
    io_type=IoType.INPUT,
    name="sender_input",
    desc="Individual table for sender",
    types=[DistDataType.INDIVIDUAL_TABLE],
    col_params=[
        TableColParam(
            name="key",
            desc="Column(s) used to join.",
            col_min_cnt_inclusive=1,
        )
    ],
)
@WandQ
Copy link
Author

WandQ commented May 10, 2024

新增自己的DAG组件时,前端展示上遇到了一个疑问,请教一下。
我参考PSI算子写了我的算子DAG组件,但是输入按钮的前端风格上不一样,是为什么呢?
PSI DAG的key选择方式:
4A1CC307-BA4B-4814-AEAC-390AC8439E78
其会自主选择连线的输入表,并且是一个下拉框用于选择ID。
而我产生的DAG:
C766ACEF-E730-4E6B-B05E-9B417F44D105
其并不会有默认选择,并且点击后是个弹窗。

@tonywu6 tonywu6 transferred this issue from secretflow/secretflow May 11, 2024
@Chrisdehe
Copy link
Member

收到 我们内部看下哈~

@WandQ
Copy link
Author

WandQ commented May 13, 2024

收到 我们内部看下哈~

有结论是什么原因了吗?我怎么才能使用类似于PSI key的那种DAG风格呢?

@Angelii
Copy link
Contributor

Angelii commented May 13, 2024

您好,这边先给您提供问题排查思路:检查【修改后的】渲染组件的 json 配置(组件定义信息)是否和【原来的】有出入,然后我们再通过这个出入进一步排查是否是渲染规则的问题。
image

@WandQ
Copy link
Author

WandQ commented May 13, 2024

4348557F-7FE7-4492-B662-C3EBF12CD0AB
我看起来没什么区别,按官方给出的教程添加的compare算子,其DAG组件也是不能像原始PSI一样

@Angelii
Copy link
Contributor

Angelii commented May 13, 2024

4348557F-7FE7-4492-B662-C3EBF12CD0AB 我看起来没什么区别,按官方给出的教程添加的compare算子,其DAG组件也是不能像原始PSI一样

image

正常来看这里的 attrs 里边不会存在两个元素(多了一个名为 labels 的元素),是否改动过其他内容?

@WandQ
Copy link
Author

WandQ commented May 13, 2024

xx.io(
io_type=IoType.INPUT,
name="receiver_input",
desc="Individual table for receiver",
types=[DistDataType.INDIVIDUAL_TABLE],
col_params=[
TableColParam(
name="key",
desc="Column(s) used to join.",
col_min_cnt_inclusive=1,
col_max_cnt_inclusive=1,
),
TableColParam(
name="labels",
desc="Column(s) used to compute.",
),
],
)
我新增的算子需要两个输入,所以这里是正常的,其他的内容没有修改

@Angelii
Copy link
Contributor

Angelii commented May 13, 2024

xx.io( io_type=IoType.INPUT, name="receiver_input", desc="Individual table for receiver", types=[DistDataType.INDIVIDUAL_TABLE], col_params=[ TableColParam( name="key", desc="Column(s) used to join.", col_min_cnt_inclusive=1, col_max_cnt_inclusive=1, ), TableColParam( name="labels", desc="Column(s) used to compute.", ), ], ) 我新增的算子需要两个输入,所以这里是正常的,其他的内容没有修改

你好,能麻烦把生成的这个 json 完整地粘贴一下吗?
image

@WandQ
Copy link
Author

WandQ commented May 13, 2024

补充一个情况,我将psi.py只修改名字的情况下拷贝成psi2.py。然后生成的dag变成了
image
所以使用psi自己的配置也产生不了原始的组件,你们是不是对名为psi的组件做了特殊处理呢?

@WandQ
Copy link
Author

WandQ commented May 13, 2024

xx.io( io_type=IoType.INPUT, name="receiver_input", desc="Individual table for receiver", types=[DistDataType.INDIVIDUAL_TABLE], col_params=[ TableColParam( name="key", desc="Column(s) used to join.", col_min_cnt_inclusive=1, col_max_cnt_inclusive=1, ), TableColParam( name="labels", desc="Column(s) used to compute.", ), ], ) 我新增的算子需要两个输入,所以这里是正常的,其他的内容没有修改

你好,能麻烦把生成的这个 json 完整地粘贴一下吗? image

{
"domain": "data_prep",
"name": "anony_psi",
"desc": "anony PSI between two parties.",
"version": "0.0.1",
"inputs": [
{
"name": "receiver_input",
"desc": "Individual table for receiver",
"types": [
"sf.table.individual"
],
"attrs": [
{
"name": "key",
"desc": "Column(s) used to join.",
"col_min_cnt_inclusive": 1,
"col_max_cnt_inclusive": 1
},
{
"name": "labels",
"desc": "Column(s) used to compute."
}
]
},
{
"name": "sender_input",
"desc": "Individual table for sender",
"types": [
"sf.table.individual"
],
"attrs": [
{
"name": "key",
"desc": "Column(s) used to join.",
"col_min_cnt_inclusive": 1,
"col_max_cnt_inclusive": 1
},
{
"name": "labels",
"desc": "Column(s) used to compute."
}
]
}
],
"outputs": [
{
"name": "receiver_output",
"desc": "Output for receiver",
"types": [
"sf.table.individual"
]
},
{
"name": "sender_output",
"desc": "Output for sender",
"types": [
"sf.table.individual"
]
}
]
}

@Angelii
Copy link
Contributor

Angelii commented May 13, 2024

xx.io( io_type=IoType.INPUT, name="receiver_input", desc="Individual table for receiver", types=[DistDataType.INDIVIDUAL_TABLE], col_params=[ TableColParam( name="key", desc="Column(s) used to join.", col_min_cnt_inclusive=1, col_max_cnt_inclusive=1, ), TableColParam( name="labels", desc="Column(s) used to compute.", ), ], ) 我新增的算子需要两个输入,所以这里是正常的,其他的内容没有修改

你好,能麻烦把生成的这个 json 完整地粘贴一下吗? image

{ "domain": "data_prep", "name": "anony_psi", "desc": "anony PSI between two parties.", "version": "0.0.1", "inputs": [ { "name": "receiver_input", "desc": "Individual table for receiver", "types": [ "sf.table.individual" ], "attrs": [ { "name": "key", "desc": "Column(s) used to join.", "col_min_cnt_inclusive": 1, "col_max_cnt_inclusive": 1 }, { "name": "labels", "desc": "Column(s) used to compute." } ] }, { "name": "sender_input", "desc": "Individual table for sender", "types": [ "sf.table.individual" ], "attrs": [ { "name": "key", "desc": "Column(s) used to join.", "col_min_cnt_inclusive": 1, "col_max_cnt_inclusive": 1 }, { "name": "labels", "desc": "Column(s) used to compute." } ] } ], "outputs": [ { "name": "receiver_output", "desc": "Output for receiver", "types": [ "sf.table.individual" ] }, { "name": "sender_output", "desc": "Output for sender", "types": [ "sf.table.individual" ] } ] }

您好,经过排查,需要麻烦补充一下注册的逻辑即可。

// component-config-protocol.ts
export const codeNameRenderKey = {
  'read_data/datatable': 'DATA_TABLE_SELECT',
  'data_prep/psi': 'UNION_KEY_SELECT',
  'data_prep/anony_psi': 'UNION_KEY_SELECT', //  👈 您需要补充的
  'preprocessing/psi': 'UNION_KEY_SELECT',
  'preprocessing/sqlite': 'SQL',
};

注册了这个组件之后,才会走进相应的渲染规则:

// config-render-contribution.ts
{
    canHandle: (node: AtomicConfigNode, renderKey?: string) => {
      return renderKey === 'UNION_KEY_SELECT' && node.type === 'AT_SF_TABLE_COL'
        ? 3
        : false;
    },
    component: DefaultColSelection,
  },

@WandQ
Copy link
Author

WandQ commented May 13, 2024

需要修改前端代码指定下拉框风格是吗?

@Angelii
Copy link
Contributor

Angelii commented May 14, 2024

需要修改前端代码指定下拉框风格是吗?

并非单纯的下拉框风格,建议根据具体的组件类型、数据结构,选择对应的渲染组件;

export const codeNameRenderKey = {
  //...
  'data_prep/psi': 'UNION_KEY_SELECT',
  //...
};

上述约定的意思是:如果遇到 psi (codeName: data_prep/psi)组件,会注册特殊的 renderKey:UNION_KEY_SELECT,并结合 AT_SF_TABLE_COL 类型,把 [输入桩] 在配置面板中渲染成下拉式的、可选择多关联键的配置项组件:DefaultColSelection

@WandQ
Copy link
Author

WandQ commented May 14, 2024

好的,明白了,感谢解答

@WandQ WandQ closed this as completed May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants