Proto file for tapi-gnmi-streaming

[roshan-joyce-fujitsu]

Hi @nigel-r-davis ,

I couldn't find a proto file corresponding to tapi-gnmi-streaming in this repo.
Basically I was looking for a ".proto" file with the definitions in sec 6.3.1.2 of TR548.

BTW, the snippet in 6.3.1.2 has syntax issues as well.

Raising this as an issue here for tracking and closure...

Thanks,
Roshan
CC: @amazzini

[roshan-joyce-fujitsu]

Hi @nigel-r-davis , attached below is my attempt at:

• copy pasting from section 6.3.1.2
• fixing syntax errors:
• • Missing '{'
• • Added proto3 syntax
• • Removed hyphenated field names with names using underscores (CEP, MEP, MIP refs)
• Added copyright and some comments
• Added some option statements for code generation
• Added json_name to the fields to allow automatic conversion from JSON complying with tapi-gnmi-streaming.proto into a protobuf
• Added missing additionalInfo field as per "Figure 42 Basic measurement augment" and "Figure 43 Support for multiple measurements in one record" in TR548 v3.1 pages 133, 134.
• Validated codegen using protoc.

CC: @bcjohnso99

[roshan-joyce-fujitsu]

Here is the proposed proto file with corrections applied:

//
// Copyright (c) 2024 Linux Foundation (LF). All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//    http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

syntax = "proto3";

//
// This package defines message types used to represent a performance measurement record
// in a TAPI stream as per the specifications in TR548.
//
package tapi;

option java_multiple_files = true;
option java_outer_classname = "TapiGnmi";
option java_package = "org.linuxfoundation.onmi.tapi";

message StreamDetails {
    MeasurementDetails measurement_details = 1;
}

message MeasurementDetails {
    uint64 time_measurement_was_sampled = 1 [json_name = "time-measurement-was-sampled"];
    QualifiedMeasuredValue qualified_measured_value = 2 [json_name = "qualified-measured-value"];
    string native_resource_id = 3 [json_name = "native-resource-id"];
    SampleQualifier sample_qualifier = 4 [json_name = "sample-qualifier"];
    RelativePosition relative_position_of_measurement = 5 [json_name = "relative-position-of-measurement"];
    DirectionOfMeasuredSignal direction_of_measured_signal = 6 [json_name = "direction-of-measured-signal"];
    uint64 sample_interval = 7 [json_name = "sample-interval"];
    string abstract_resource_ref = 8 [json_name = "abstract-resource-ref"];
    string native_measurement_type = 9 [json_name = "native-measurement-type"];
    NormalizedMeasurementType normalized_measurement_type = 10 [json_name = "normalized-measurement-type"];
    repeated QualifiedMeasurement qualified_measurement = 11 [json_name = "qualified-measurement"];
    string connection_end_point = 12 [json_name = "connection-end-point"];
    string maintenance_intermediate_point = 13 [json_name = "maintenance-intermediate-point"];
    string maintenance_end_point = 14 [json_name = "maintenance-end-point"];
    uint64 measurement_start_time = 15 [json_name = "measurement-start-time"];
    repeated QualifiedMeasuredValue qualified_measured_value_set = 16 [json_name = "qualified-measured-value-set"];
    repeated NameAndValue additionalInfo = 17 [json_name = "additional-info"];
}

message QualifiedMeasurement {
    uint64 time_measurement_was_sampled = 1 [json_name = "time-measurement-was-sampled"];
    QualifiedMeasuredValue qualified_measured_value = 2 [json_name = "qualified-measured-value"];
    string native_resource_id = 3 [json_name = "native-resource-id"];
    SampleQualifier sample_qualifier = 4 [json_name = "sample-qualifier"];
    RelativePosition relative_position_of_measurement = 5 [json_name = "relative-position-of-measurement"];
    DirectionOfMeasuredSignal direction_of_measured_signal = 6 [json_name = "direction-of-measured-signal"];
    uint64 sample_interval = 7 [json_name = "sample-interval"];
    string abstract_resource_ref = 8 [json_name = "abstract-resource-ref"];
    string native_measurement_type = 9 [json_name = "native-measurement-type"];
    NormalizedMeasurementType normalized_measurement_type = 10 [json_name = "normalized-measurement-type"];
    uint64 list_index = 11 [json_name ="list-index"];
    string connection_end_point = 12 [json_name = "connection-end-point"];
    string maintenance_intermediate_point = 13 [json_name = "maintenance-intermediate-point"];
    string maintenance_end_point = 14 [json_name = "maintenance-end-point"];
    uint64 measurement_start_time = 15 [json_name = "measurement-start-time"];
    repeated QualifiedMeasuredValue qualified_measured_value_set = 16 [json_name = "qualified-measured-value-set"];
    repeated NameAndValue additionalInfo = 17 [json_name = "additional-info"];
}

message QualifiedMeasuredValue {
    double value = 1;
    ValueQualifier value_qualifier = 2 [json_name = "value-qualifier"];
    string units = 3;
    string qualified_value_name = 4 [json_name = "qualified-value-name"];
    string qualified_location_name = 5 [json_name = "qualified-location-name"];
    string qualified_measurement_type = 6 [json_name = "qualified-measurement-type"];
}

message NameAndValue {
    string valueName = 1 [json_name = "value-name"];
    string value = 2;
}

enum ValueQualifier {
    INVALID = 0;
    SUSPECT = 1;
    MISSING = 2;
    PARTIAL = 3;
}

enum RelativePosition {
    NEAR_END = 0;
    FAR_END = 1;
    NOT_APPLICABLE = 2;
    ALL_LOCATIONS = 3;
}

enum DirectionOfMeasuredSignal {
    NO_DIRECTION = 0;
    RECEIVE =1;
    TRANSMIT = 2;
    RECEIVE_AND_TRANSMIT = 3;
    CONTRA_RECEIVE = 4;
    CONTRA_TRANSMIT = 5;
    CONTRA_RECEIVE_AND_CONTRA_TRANSMIT = 6;
}

enum SampleQualifier {
    DELAYED = 0;
    FINAL = 1;
    FIRST = 2;
    BASELINE=3;
}

enum NormalizedMeasurementType {
    BBE = 0;
    CHROM_DISP = 1;
    DELAY = 2;
    DIFF_GROUP_DELAY = 3;
    FEC_CORRECTABLE_BLOCKS = 4;
    FEC_CORRECTED_BITS = 5;
    FEC_CORRECTED_BYTES = 6;
    FEC_CORRECTED_CODEWORDS = 7;
    FEC_CORRECTED_ERRORS = 8;
    FEC_POST_FEC_BER = 9;
    FEC_PRE_FEC_BER = 10;
    FEC_SYMBOL_ERRORS = 11;
    FEC_UNCORRECTABLE_BITS = 12;
    FEC_UNCORRECTABLE_BLOCKS = 13;
    FEC_UNCORRECTABLE_BYTES = 14;
    FEC_UNCORRECTABLE_WORDS = 15;
    FEC_UNCORRECTED_CODEWORDS = 16;
    FREQ_OFFS = 17;
    OPTICAL_GAIN = 18;
    OPTICAL_POWER_INPUT = 19;
    OPTICAL_POWER_OUTPUT = 20;
    OPTICAL_TILT = 21;
    POL_MODE_DISP = 22;
    SES = 23;
    UAS = 24;
}

[nigel-r-davis]

Hi @roshan-joyce-fujitsu

Apologies for the delay. Your proposal to publish a proto file and the additions you propose to the raw protobuf structure provided in TR-548 are good. We should publish a proto file as part of the delivery. I will take an action to do that for the next delivery.

Thanks for correcting the missing "{" in the line message StreamDetails, correcting the field names to not use "_", adding the additionalInfo which is shown in the UML and adding the JSON name fields.

I noticed that some of the protobuf you provide above does not align with the protobuf in TR-548 [v2.6.0-RC1]. For example, definitions of enums ValueQualifier, SampleQualifier and NormalizedMeasurementType are not aligned. ValueQualifier should be:
OK = 0;
INVALID = 1;
SUSPECT = 2;
MISSING = 3;
PARTIAL = 4;
I assume there was no intention to change the protobuf structures in any way. I will apply the structures from TR-548 v3.1 in the proto file combining with your header details and corrections etc.

Regards

[roshan-joyce-fujitsu]

Hi @nigel-r-davis ,

Thank you for looking into this.

There was no intention to change the structures in any way. I think the changes were because I may have copy-pasted from section 6.3.1.2 of an older version of TR548!

Thanks
Roshan

[nigel-r-davis]

Hi @roshan-joyce-fujitsu, as, I suspected, but just wanted to be sure :)

[nigel-r-davis]

@bcjohnso99 we should schedule a review of the proto file and updated text on the call 11 March for deliver in 2.6. We need to determine where we put the file during that review.

@bcjohnso99 Note that we will need to have the deliverables ready by end of that week (i.e., 14 March) so we can go into the 2 week steering review for final delivery at the end of March).

[nigel-r-davis]

Note that I have removed the protobuf content from TR-548 and simply referenced to delivery folder. TR-548 draft should be ready later today, 10 March.

[roshan-joyce-fujitsu]

Hi @nigel-r-davis @bcjohnso99

I added one comment - it is related to missing json_name.

#614 (comment)

I noticed that the new file does not include my addition of "json_name" for the fields in the message definitions.
I had found that they were needed to facilitate automatic conversion from a JSON that confirms to the tapi-gnmi-streaming.yang module, into protobuf messages using google's https://protobuf.dev/reference/java/api-docs/com/google/protobuf/util/JsonFormat.html.

[nigel-r-davis]

Apologies, my bad. I fixed the issue of the changes structure by simply extracting the proto from the document and forgot the JSON names were not there. I will fix that and resubmit. The JSON names are indeed useful.

[nigel-r-davis]

Updates made to add JSON names.

[roshan-joyce-fujitsu]

Hi @nigel-r-davis , @bcjohnso99 ,

I reviewed your latest changes and they look good.
I see that you've corrected some mistakes and typos I had made.

Thank you for taking care of these.

We can close this issue.