oneOf Performance Issues

[sam-heilbron]

I've noticed that when we use cue to generate open api schemas from an api defined via protocol buffers, schema generation takes increasingly longer with the existence of each oneof. I created a contrived proto to outline this:

message ComplicatedMockResource {

    // comment
    NestedOneOf nested_one_of = 3;

    string name = 4;
}

// message comment
message NestedOneOf {
    // oneof comment
    oneof color_option {
        GreenNestedOneOf green = 1;
        BlueNestedOneOf blue = 2;
    }

    oneof type_option {
        string type_option_a = 11;
        int64 type_option_b = 12;
        bool type_option_c = 13;
    }

    oneof string_option {
        string and_another_option_a = 21;
        string and_another_option_b = 22;
        string and_another_option_c = 23;
    }

    string field = 4;

    repeated string multiple_a = 5;
    repeated string multiple_b = 6;
    repeated string multiple_c = 7;

    bool choice = 8;
}

message GreenNestedOneOf {

    // oneof comment
    oneof option {
        string option_a = 1;
        string option_b = 2;
        string option_c = 3;
    }

    oneof another_option {
        string another_option_a = 11;
        string another_option_b = 12;
        string another_option_c = 13;
    }

    oneof and_another_option {
        string and_another_option_a = 21;
        string and_another_option_b = 22;
        string and_another_option_c = 23;
    }
}

message BlueNestedOneOf {

    // oneof comment
    oneof option {
        string option_a = 1;
        string option_b = 2;
        string option_c = 3;
    }

    oneof another_option {
        string another_option_a = 11;
        string another_option_b = 12;
        string another_option_c = 13;
    }

    oneof and_another_option {
        string and_another_option_a = 21;
        string and_another_option_b = 22;
        string and_another_option_c = 23;
    }
}

The above proto alone takes 4-5 minutes to generate an open api schema.

Do you have any tips for navigating this issue? Is this a known issue?

[mpvl]

Yes this is a known issue. There are some limitations to the current disjunction implementation. Must disjunctions that occur in practice can be evaluated in linear time, but the current implementation is very slow indeed, though this particular amount of slowdown is somewhat unexpected. We know what to do to fix it and it is a fairly high-priority item, but it may take some time.

But before getting to this fix, this would also be addressed with the proposed numconcrete builtin, which captures the semantics of oneof slightly better as well. See #943. I expect this to be rolled out fairly soon.

[sam-heilbron]

Thanks for following up! How would the numconcrete builtin be used to mirror the behavior of oneof? Also when you say that you expect the core builtin extensions to be rolled out soon and the disjunction evaluation improvements to take some time, what sort of timeline are you imagining for both? (days? weeks? months?).

[mpvl]

Weeks probably.

See the section for numconcrete in #822. Basically, you reference a set of fields and specify how many of those (e.g. 1 or <=1) should be concrete.

FWIW, this representation is very close in form to the structural form in OpenAPI, used for CRDs.

[mpvl]

The implementation of this is not terribly hard. It is just the reworking of the proto parser and OpenAPI generator that is a bit laborious. But if it helps, this can be fast tracked a bit by enabling this as an option (e.g. enabled by CUE_ONEOF_EXPERIMENT=1).

[cueckoo]

This discussion has been migrated to cue-lang/cue#944.

For more details about CUE's migration to a new home, please see cue-lang/cue#1078.