WaitUntil feature on MG #1068

lorbax · 2024-07-19T10:13:11Z

fix to issue 1059.
To see a description of the changes apported to this PR, look at the commit message and the README changes.
The feature introduced is backward compatible, as it is introduced optionally.

How to test it:

easy way. look at translation-proxy.json MG test. YOu can see that it is launched TProxy. Go the config used and change it in order that the channel_nominal_hashrate is few magnitude order higher than the min_individual_miner_hashrate. This forces the channel to be updated (possibly more than once) and an UpdateChannel. Then you should see RECEIVED 22, WAITING MESSAGE TYPE 27 several times (i.e. UpdateChannel) and finally MATCHED WAITED MESSAGE TYPE 27, namely SubmitSharesExtended, resulting in success of the MG test. Reducing the miner_num_submits_before_update to 1 may help testing. The hashrates depend on your hardware, on my PC this worked

min_individual_miner_hashrate=1000000.0
channel_nominal_hashrate = 50000000000.0

hard way. test PR 1001: checkit out and rebase it on top of the current PR. Then test it following instructions in the two top commits. Finally change this MG-test with the following

{
    "version": "2",
    "doc": [
        "This test does",
        "Acts like a pool that refuses share with JobDeclatarion",
        "Waits for SetupConnection",
        "Sends a SetupConnection.Success and waits that a SetCustomMiningJob is received",
        "Sends a SetCustomMiningJob.Success and waits that UpdateChannel is received",
        "Waits that SubmitShare is received",
        "Responds with SubmitShare.Error"
    ],
    "frame_builders": [
        {
            "type": "automatic",
            "message_id": "test/message-generator/messages/mining_messages.json::open_extended_mining_channel_success"
        },
        {
            "type": "automatic",
            "message_id": "test/message-generator/messages/common_messages.json::setup_connection_success_tproxy"
        },
        {
            "type": "automatic",
            "message_id": "test/message-generator/messages/mining_messages.json::set_custom_mining_job_success"
        },
        {
            "type": "automatic",
            "message_id": "test/message-generator/messages/mining_messages.json::submit_shares_error"
        }
    ],
    "actions": [
       {
            "message_ids": [],
            "role": "server",
            "results": [
                {
                    "type": "match_message_type",
                    "value": "0x00"
                }
            ],
            "actiondoc": "This action checks that a Setupconnection message is received"
        },
        {
            "message_ids": ["setup_connection_success_tproxy"],
            "role": "server",
            "results": [
                {
                    "type": "match_message_type",
                    "value": "0x13"
                }
            ],
          "actiondoc": "This action sends SetupConnection.Success and check that a OpenExtendedfMiningChannel is received"
        },
        {
            "message_ids": ["open_extended_mining_channel_success"],
            "role": "server",
            "results": [
                {
                    "type": "get_message_field",
                    "value": [
                        "MiningProtocol",
                        "SetCustomMiningJob",
                        [
                            [
                                "request_id",
                                "custom_job_req_id"
                            ]
                        ]
                    ]
                }
            ],
          "actiondoc": "This action sends open_extended_mining_channel_success, and a wait for a SetCustomMiningJoband get the req id"
        },
        {
            "message_ids": [],
            "role": "server",
            "results": [
                {
                    "type": "match_message_type",
                    "value": "0x1b",
                    "condition": {"WaitUntil": {"WaitUntilConfig": {"timeout": 120, "allowed_messages": ["0x1b", "0x16", "0x22"] } } }
                }
            ],
            "actiondoc": "This action check submit_shares is received, allowed messages: update channel, submit shares, set custom miningjob"
        },
        {
            "message_ids": ["submit_shares_error"],
            "role": "server",
            "results": [],
            "actiondoc": "This action sends a submit_shares_error"
        }
    ],
    "setup_commands": [
    ],
    "execution_commands": [
    ],
    "cleanup_commands": [
        {
            "command": "sleep",
            "args": ["100000000000"],
            "conditions": "None"
        }
    ],
    "role": "server",
    "upstream": {
        "ip": "127.0.0.1",
        "port": 44254,
        "pub_key": "9auqWEzQDVyd2oe1JVGFLMLHZtCo2FFqZwtKA5gd9xbuEu7PH72",
        "secret_key": "mkDLTBBRxdBv998612qipDYoTK3YUrqLe8uWw7gu3iXbSrn2n"
    }
}

github-actions · 2024-07-19T10:20:07Z

Bencher

Report	Mon, July 29, 2024 at 19:45:11 UTC
Project	Stratum v2 (SRI)
Branch	1068/merge
Testbed	sv1

🚨 1 ALERT: Threshold Boundary Limit exceeded!

Benchmark	Measure (units)	View	Value	Lower Boundary	Upper Boundary
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle	Latency (nanoseconds (ns))	🚨 (view plot \| view alert)	801.53 (+6.43%)		793.41 (101.02%)

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
client-submit-serialize	✅ (view plot)	6,516.40 (-4.93%)	7,380.09 (88.30%)
client-submit-serialize-deserialize	✅ (view plot)	7,396.00 (-4.86%)	8,343.75 (88.64%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle	✅ (view plot)	7,987.10 (-4.31%)	8,858.89 (90.16%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle	✅ (view plot)	885.06 (-1.54%)	929.28 (95.24%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize	✅ (view plot)	696.99 (-0.16%)	719.06 (96.93%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize	✅ (view plot)	247.36 (-0.46%)	258.28 (95.77%)
client-sv1-get-authorize/client-sv1-get-authorize	✅ (view plot)	156.73 (-0.41%)	163.07 (96.11%)
client-sv1-get-submit	✅ (view plot)	6,189.80 (-6.61%)	7,148.92 (86.58%)
client-sv1-get-subscribe/client-sv1-get-subscribe	✅ (view plot)	281.15 (+0.69%)	291.87 (96.33%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle	🚨 (view plot \| view alert)	801.53 (+6.43%)	793.41 (101.02%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize	✅ (view plot)	624.00 (+1.53%)	636.67 (98.01%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize	✅ (view plot)	206.80 (+0.29%)	218.84 (94.50%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-07-19T10:20:07Z

Bencher

Report	Mon, July 29, 2024 at 19:45:11 UTC
Project	Stratum v2 (SRI)
Branch	MG-new-feature-for-actions
Testbed	sv2

🚨 2 ALERTS: Threshold Boundary Limits exceeded!

Benchmark	Measure (units)	View	Value	Lower Boundary	Upper Boundary
client_sv2_mining_message_submit_standard	Latency (nanoseconds (ns))	🚨 (view plot \| view alert)	14.70 (+0.30%)		14.69 (100.02%)
client_sv2_open_channel	Latency (nanoseconds (ns))	🚨 (view plot \| view alert)	174.28 (+4.75%)		174.18 (100.06%)

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
client_sv2_handle_message_common	✅ (view plot)	44.16 (-0.90%)	45.22 (97.66%)
client_sv2_handle_message_mining	✅ (view plot)	74.67 (+2.34%)	80.10 (93.23%)
client_sv2_mining_message_submit_standard	🚨 (view plot \| view alert)	14.70 (+0.30%)	14.69 (100.02%)
client_sv2_mining_message_submit_standard_serialize	✅ (view plot)	261.47 (-0.93%)	283.15 (92.34%)
client_sv2_mining_message_submit_standard_serialize_deserialize	✅ (view plot)	611.05 (+2.90%)	626.33 (97.56%)
client_sv2_open_channel	🚨 (view plot \| view alert)	174.28 (+4.75%)	174.18 (100.06%)
client_sv2_open_channel_serialize	✅ (view plot)	277.80 (-1.39%)	294.11 (94.45%)
client_sv2_open_channel_serialize_deserialize	✅ (view plot)	380.36 (+0.62%)	420.91 (90.37%)
client_sv2_setup_connection	✅ (view plot)	157.00 (-4.16%)	174.33 (90.06%)
client_sv2_setup_connection_serialize	✅ (view plot)	441.22 (-6.44%)	507.74 (86.90%)
client_sv2_setup_connection_serialize_deserialize	✅ (view plot)	957.13 (-1.49%)	1,036.04 (92.38%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-07-19T10:20:08Z

Bencher

Report	Mon, July 29, 2024 at 19:45:10 UTC
Project	Stratum v2 (SRI)
Branch	MG-new-feature-for-actions
Testbed	sv1

Click to view all benchmark results

Benchmark	Estimated Cycles	Estimated Cycles Results estimated cycles \| (Δ%)	Estimated Cycles Upper Boundary estimated cycles \| (%)	Instructions	Instructions Results instructions \| (Δ%)	Instructions Upper Boundary instructions \| (%)	L1 Accesses	L1 Accesses Results accesses \| (Δ%)	L1 Accesses Upper Boundary accesses \| (%)	L2 Accesses	L2 Accesses Results accesses \| (Δ%)	L2 Accesses Upper Boundary accesses \| (%)	RAM Accesses	RAM Accesses Results accesses \| (Δ%)	RAM Accesses Upper Boundary accesses \| (%)
get_authorize	✅ (view plot)	8,538.00 (+1.07%)	8,727.88 (97.82%)	✅ (view plot)	3,746.00 (+0.13%)	3,849.39 (97.31%)	✅ (view plot)	5,248.00 (+0.05%)	5,392.90 (97.31%)	✅ (view plot)	7.00 (-10.88%)	10.23 (68.40%)	✅ (view plot)	93.00 (+2.91%)	94.61 (98.30%)
get_submit	✅ (view plot)	95,509.00 (-0.04%)	96,103.44 (99.38%)	✅ (view plot)	59,439.00 (-0.05%)	59,758.05 (99.47%)	✅ (view plot)	85,359.00 (-0.05%)	85,807.63 (99.48%)	✅ (view plot)	49.00 (-9.99%)	62.50 (78.40%)	✅ (view plot)	283.00 (+0.29%)	287.51 (98.43%)
get_subscribe	✅ (view plot)	7,991.00 (+0.16%)	8,255.22 (96.80%)	✅ (view plot)	2,841.00 (+0.35%)	2,934.94 (96.80%)	✅ (view plot)	3,971.00 (+0.39%)	4,093.49 (97.01%)	✅ (view plot)	13.00 (-17.92%)	19.81 (65.61%)	✅ (view plot)	113.00 (+0.29%)	116.73 (96.80%)
serialize_authorize	✅ (view plot)	12,291.00 (+0.61%)	12,501.11 (98.32%)	✅ (view plot)	5,317.00 (+0.09%)	5,420.39 (98.09%)	✅ (view plot)	7,411.00 (+0.04%)	7,556.24 (98.08%)	✅ (view plot)	10.00 (-6.77%)	13.20 (75.74%)	✅ (view plot)	138.00 (+1.59%)	140.52 (98.21%)
serialize_deserialize_authorize	✅ (view plot)	24,588.00 (+0.43%)	24,721.65 (99.46%)	✅ (view plot)	9,898.00 (-0.02%)	10,018.76 (98.79%)	✅ (view plot)	13,958.00 (-0.05%)	14,139.62 (98.72%)	✅ (view plot)	33.00 (-8.82%)	41.45 (79.60%)	✅ (view plot)	299.00 (+1.24%)	299.25 (99.91%)
serialize_deserialize_handle_authorize	✅ (view plot)	30,331.00 (+0.54%)	30,389.88 (99.81%)	✅ (view plot)	12,101.00 (+0.04%)	12,204.39 (99.15%)	✅ (view plot)	17,116.00 (-0.01%)	17,270.54 (99.11%)	✅ (view plot)	60.00 (+1.89%)	64.49 (93.04%)	✅ (view plot)	369.00 (+1.24%)	369.06 (99.98%)
serialize_deserialize_handle_submit	✅ (view plot)	126,406.00 (-0.00%)	126,993.42 (99.54%)	✅ (view plot)	73,224.00 (-0.03%)	73,596.46 (99.49%)	✅ (view plot)	104,946.00 (-0.04%)	105,477.96 (99.50%)	✅ (view plot)	120.00 (-0.37%)	130.61 (91.88%)	✅ (view plot)	596.00 (+0.18%)	599.07 (99.49%)
serialize_deserialize_handle_subscribe	✅ (view plot)	27,529.00 (+0.25%)	27,605.25 (99.72%)	✅ (view plot)	9,643.00 (+0.10%)	9,736.94 (99.04%)	✅ (view plot)	13,639.00 (+0.10%)	13,769.01 (99.06%)	✅ (view plot)	62.00 (-5.20%)	73.39 (84.48%)	✅ (view plot)	388.00 (+0.52%)	388.87 (99.78%)
serialize_deserialize_submit	✅ (view plot)	114,999.00 (-0.06%)	115,613.77 (99.47%)	✅ (view plot)	68,001.00 (-0.07%)	68,378.84 (99.45%)	✅ (view plot)	97,559.00 (-0.08%)	98,115.63 (99.43%)	✅ (view plot)	65.00 (-5.71%)	75.18 (86.45%)	✅ (view plot)	489.00 (+0.16%)	492.27 (99.34%)
serialize_deserialize_subscribe	✅ (view plot)	22,922.00 (+0.18%)	23,105.16 (99.21%)	✅ (view plot)	8,195.00 (+0.10%)	8,292.26 (98.83%)	✅ (view plot)	11,542.00 (+0.10%)	11,675.60 (98.86%)	✅ (view plot)	36.00 (-7.77%)	43.95 (81.92%)	✅ (view plot)	320.00 (+0.41%)	321.65 (99.49%)
serialize_submit	✅ (view plot)	99,826.00 (-0.06%)	100,430.87 (99.40%)	✅ (view plot)	61,483.00 (-0.05%)	61,807.12 (99.48%)	✅ (view plot)	88,206.00 (-0.05%)	88,660.77 (99.49%)	✅ (view plot)	49.00 (-10.86%)	62.54 (78.35%)	✅ (view plot)	325.00 (+0.09%)	328.96 (98.80%)
serialize_subscribe	✅ (view plot)	11,328.00 (+0.01%)	11,590.37 (97.74%)	✅ (view plot)	4,188.00 (+0.24%)	4,281.94 (97.81%)	✅ (view plot)	5,828.00 (+0.24%)	5,952.45 (97.91%)	✅ (view plot)	15.00 (-7.55%)	18.82 (79.70%)	✅ (view plot)	155.00 (-0.12%)	159.47 (97.20%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-07-19T10:20:11Z

Bencher

Report	Mon, July 29, 2024 at 19:45:13 UTC
Project	Stratum v2 (SRI)
Branch	MG-new-feature-for-actions
Testbed	sv2

🚨 6 ALERTS: Threshold Boundary Limits exceeded!

Benchmark	Measure (units)	View	Value	Upper Boundary
client_sv2_mining_message_submit_standard_serialize_deserialize	Instructions (instructions)	🚨 (view plot \| view alert)	10,585.00 (+0.35%)	10,583.99 (100.01%)
client_sv2_mining_message_submit_standard_serialize_deserialize	L1 Accesses (accesses)	🚨 (view plot \| view alert)	15,399.00 (+0.33%)	15,397.14 (100.01%)
client_sv2_open_channel_serialize_deserialize	Instructions (instructions)	🚨 (view plot \| view alert)	8,027.00 (+0.47%)	8,026.31 (100.01%)
client_sv2_open_channel_serialize_deserialize	L1 Accesses (accesses)	🚨 (view plot \| view alert)	11,672.00 (+0.43%)	11,670.82 (100.01%)
client_sv2_setup_connection_serialize_deserialize	Instructions (instructions)	🚨 (view plot \| view alert)	14,855.00 (+0.26%)	14,854.12 (100.01%)
client_sv2_setup_connection_serialize_deserialize	L1 Accesses (accesses)	🚨 (view plot \| view alert)	21,811.00 (+0.24%)	21,809.04 (100.01%)

Click to view all benchmark results

Benchmark	Estimated Cycles	Estimated Cycles Results estimated cycles \| (Δ%)	Estimated Cycles Upper Boundary estimated cycles \| (%)	Instructions	Instructions Results instructions \| (Δ%)	Instructions Upper Boundary instructions \| (%)	L1 Accesses	L1 Accesses Results accesses \| (Δ%)	L1 Accesses Upper Boundary accesses \| (%)	L2 Accesses	L2 Accesses Results accesses \| (Δ%)	L2 Accesses Upper Boundary accesses \| (%)	RAM Accesses	RAM Accesses Results accesses \| (Δ%)	RAM Accesses Upper Boundary accesses \| (%)
client_sv2_handle_message_common	✅ (view plot)	2,111.00 (+2.65%)	2,138.86 (98.70%)	✅ (view plot)	473.00 (+0.43%)	485.77 (97.37%)	✅ (view plot)	731.00 (-0.14%)	753.67 (96.99%)	✅ (view plot)	10.00 (+35.56%)	11.65 (85.86%)	✅ (view plot)	38.00 (+3.30%)	38.77 (98.02%)
client_sv2_handle_message_mining	✅ (view plot)	8,219.00 (+0.24%)	8,331.34 (98.65%)	✅ (view plot)	2,137.00 (+0.41%)	2,170.26 (98.47%)	✅ (view plot)	3,159.00 (+0.40%)	3,214.04 (98.29%)	✅ (view plot)	39.00 (+1.06%)	43.36 (89.95%)	✅ (view plot)	139.00 (+0.09%)	141.80 (98.03%)
client_sv2_mining_message_submit_standard	✅ (view plot)	6,316.00 (+0.58%)	6,388.70 (98.86%)	✅ (view plot)	1,750.00 (+0.02%)	1,762.40 (99.30%)	✅ (view plot)	2,546.00 (-0.27%)	2,574.28 (98.90%)	✅ (view plot)	26.00 (+39.49%)	26.49 (98.13%)	✅ (view plot)	104.00 (+0.17%)	106.73 (97.44%)
client_sv2_mining_message_submit_standard_serialize	✅ (view plot)	14,657.00 (-0.69%)	15,017.43 (97.60%)	✅ (view plot)	4,694.00 (+0.01%)	4,706.40 (99.74%)	✅ (view plot)	6,752.00 (-0.03%)	6,773.76 (99.68%)	✅ (view plot)	55.00 (+13.81%)	55.61 (98.91%)	✅ (view plot)	218.00 (-1.72%)	229.62 (94.94%)
client_sv2_mining_message_submit_standard_serialize_deserialize	✅ (view plot)	27,469.00 (-0.03%)	27,820.31 (98.74%)	🚨 (view plot \| view alert)	10,585.00 (+0.35%)	10,583.99 (100.01%)	🚨 (view plot \| view alert)	15,399.00 (+0.33%)	15,397.14 (100.01%)	✅ (view plot)	90.00 (+6.44%)	91.53 (98.33%)	✅ (view plot)	332.00 (-0.73%)	344.52 (96.37%)
client_sv2_open_channel	✅ (view plot)	4,451.00 (-0.78%)	4,602.75 (96.70%)	✅ (view plot)	1,461.00 (+0.05%)	1,473.68 (99.14%)	✅ (view plot)	2,156.00 (+0.13%)	2,172.25 (99.25%)	✅ (view plot)	11.00 (-7.45%)	15.00 (73.34%)	✅ (view plot)	64.00 (-1.46%)	67.98 (94.15%)
client_sv2_open_channel_serialize	✅ (view plot)	14,040.00 (-1.08%)	14,457.03 (97.12%)	✅ (view plot)	5,064.00 (+0.01%)	5,076.68 (99.75%)	✅ (view plot)	7,320.00 (+0.03%)	7,338.29 (99.75%)	✅ (view plot)	42.00 (+10.77%)	43.60 (96.33%)	✅ (view plot)	186.00 (-2.63%)	198.94 (93.50%)
client_sv2_open_channel_serialize_deserialize	✅ (view plot)	22,557.00 (-0.32%)	22,994.87 (98.10%)	🚨 (view plot \| view alert)	8,027.00 (+0.47%)	8,026.31 (100.01%)	🚨 (view plot \| view alert)	11,672.00 (+0.43%)	11,670.82 (100.01%)	✅ (view plot)	84.00 (+12.39%)	85.90 (97.79%)	✅ (view plot)	299.00 (-1.59%)	314.42 (95.10%)
client_sv2_setup_connection	✅ (view plot)	4,693.00 (-0.08%)	4,760.46 (98.58%)	✅ (view plot)	1,502.00 (+0.05%)	1,514.68 (99.16%)	✅ (view plot)	2,273.00 (-0.15%)	2,297.99 (98.91%)	✅ (view plot)	15.00 (+51.74%)	15.96 (94.00%)	✅ (view plot)	67.00 (-1.09%)	69.60 (96.27%)
client_sv2_setup_connection_serialize	✅ (view plot)	15,976.00 (-1.54%)	16,525.59 (96.67%)	✅ (view plot)	5,963.00 (+0.01%)	5,975.68 (99.79%)	✅ (view plot)	8,661.00 (+0.06%)	8,677.38 (99.81%)	✅ (view plot)	49.00 (+8.34%)	50.46 (97.10%)	✅ (view plot)	202.00 (-3.73%)	218.82 (92.31%)
client_sv2_setup_connection_serialize_deserialize	✅ (view plot)	35,416.00 (-0.29%)	35,736.70 (99.10%)	🚨 (view plot \| view alert)	14,855.00 (+0.26%)	14,854.12 (100.01%)	🚨 (view plot \| view alert)	21,811.00 (+0.24%)	21,809.04 (100.01%)	✅ (view plot)	110.00 (+8.89%)	115.41 (95.31%)	✅ (view plot)	373.00 (-1.51%)	385.41 (96.78%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

utils/message-generator/src/main.rs

utils/message-generator/src/parser/actions.rs

plebhash

Looks good to me.

I left some suggestion for small improvements.

It's also worth noting that the current implementation of this new feature introduces the possibility of MG entering an infinite loop, which is a potential footgun.

That would happen if some MG test defined a match_message_type with a "condition": "WaitUntil" but the expected message never arrived.

In that scenario, Executor::execute would enter an infinite loop and the test would never return TEST FAILED nor TEST OK.

So maybe in the future we might realize we need something like this:

pub enum Condition {
    WaitUntil(u32), // timeout
    ...
}

where the u32 represents a timeout that breaks the infinite loop inside Executor::execute

but for now, I think this is good to merge so we can unblock the other work that depends on it

lorbax · 2024-07-21T08:31:42Z

Looks good to me.

I left some suggestion for small improvements.

It's also worth noting that the current implementation of this new feature introduces the possibility of MG entering an infinite loop, which is a potential footgun.

That would happen if some MG test defined a match_message_type with a "condition": "WaitUntil" but the expected message never arrived.

In that scenario, Executor::execute would enter an infinite loop and the test would never return TEST FAILED nor TEST OK.

So maybe in the future we might realize we need something like this:
pub enum Condition {
    WaitUntil(u32), // timeout
    ...
}
where the u32 represents a timeout that breaks the infinite loop inside Executor::execute

but for now, I think this is good to merge so we can unblock the other work that depends on it

you are right I thought that there was a timer for the entire MG test. BTW this should be a nice-to-have and an easy fix
Adding an u32 is a a parameter is not a big deal, it is just a matter of adding a timer and unsuccessfully exit if the timer has elapsed. Should't be a bit deal to add

plebhash · 2024-07-21T17:25:44Z

I thought that there was a timer for the entire MG test

you're right @lorbax

this is documented on the MG README.md:

stratum/utils/message-generator/README.md

Lines 281 to 294 in 7841021

    
           **Definition of commands mentioned above** 
        
           A command is an object with the following fields: 
        
           1. `command`: the bash command that we want to execute 
        
           2. `args`: the command's args 
        
           3. `condition`: can be either `WithConditions` or `None` in the first case the command executor 
        
              will wait for some condition before return in the latter it just launch the command and return 
        
           `WithConditions` is an object composed by the following fields: 
        
           1. `conditions`: an array of object that must be true or false on order for the executor to 
        
              return 
        
           2. `timer_secs`: number of seconds after then the command is considered stuck and the test fail 
        
           3. `warn_no_panic`: `bool` if true the test do not fail with panic if timer secs terminate but it 
        
              just exit (passing) and emit a warning

so if some MG test (or mock) has the following conditions:

command has WithConditions with a timer_secs parameter
action has match_message_type with a "condition": "WaitUntil"
the expected message never arrives

then eventually the original command will run out of time and the test will fail.

but if some MG test (or mock) has the following conditions:

command does not have WithConditions with a timer_secs parameter
action has match_message_type with a "condition": "WaitUntil"
the expected message never arrives

then the test would get stuck.

however, this can be easily solved with good documentation, so that the user is aware that WaitUntil must be used in conjunction with WithConditions + timer_secs

here's a commit where I add the docs for this new feature, feel free to cherry pick it into your branch so it's included in this PR: plebhash@2be0191

Shourya742

Shouldn't we also add this for other related ActionResults? I tested it locally and it looks good. I agree with @plebhash's view on having a timeout for this action due to the possibility of staleness.

Shourya742 · 2024-07-22T05:11:29Z

utils/message-generator/src/executor.rs

+                                        let message: Sv2Frame<AnyMessage<'static>, _> =
+                                            message.try_into().unwrap();
+                                        debug!("RECV {:#?}", message);
+                                        let header = message.get_header().unwrap();


Suggested change

let header = message.get_header().unwrap();

let header = match sv2_frame.get_header() {

Some(hdr) => hdr,

None => {

error!("Failed to get header from Sv2Frame");

success = false;

break;

}

};

don't know if I can commit this, the get_header() produces an Option<T>

@lorbax can you elaborate? this match works exactly on a Option, what's wrong with it?

added this part of code

lorbax · 2024-07-22T06:51:17Z

I thought that there was a timer for the entire MG test

you're right @lorbax

this is documented on the MG README.md:

stratum/utils/message-generator/README.md

Lines 281 to 294 in 7841021

**Definition of commands mentioned above**

A command is an object with the following fields:

1. `command`: the bash command that we want to execute

2. `args`: the command's args

3. `condition`: can be either `WithConditions` or `None` in the first case the command executor

will wait for some condition before return in the latter it just launch the command and return

`WithConditions` is an object composed by the following fields:

1. `conditions`: an array of object that must be true or false on order for the executor to

return

2. `timer_secs`: number of seconds after then the command is considered stuck and the test fail

3. `warn_no_panic`: `bool` if true the test do not fail with panic if timer secs terminate but it

just exit (passing) and emit a warning

so if some MG test (or mock) has the following conditions:
* `command` has `WithConditions` with a `timer_secs` parameter

* action has `match_message_type` with a `"condition": "WaitUntil"`

* the expected message never arrives
then eventually the original command will run out of time and the test will fail.

but if some MG test (or mock) has the following conditions:
* `command` **does not** have `WithConditions` with a `timer_secs` parameter

* action has `match_message_type` with a `"condition": "WaitUntil"`

* the expected message never arrives
then the test would get stuck.

however, this can be easily solved with good documentation, so that the user is aware that WaitUntil must be used in conjunction with WithConditions + timer_secs

here's a commit where I add the docs for this new feature, feel free to cherry pick it into your branch so it's included in this PR: plebhash@2be0191

This timeout seems to be valid only for commands, while we are talking about a timeout for the action. I proposed also a global timeout, longer then all the other timeouts.

lorbax · 2024-07-22T07:26:06Z

Shouldn't we also add this for other related ActionResults? I tested it locally and it looks good. I agree with @plebhash's view on having a timeout for this action due to the possibility of staleness.

I think that it make sense with GetMessageField and MatchMessageField, but I would prefer to introduce this feature for those results when it is needed.

lorbax · 2024-07-22T12:11:27Z

Pushed a commit for applying the review.
@plebhash @Shourya742

plebhash · 2024-07-22T18:27:53Z

@lorbax we need docs for this new feature, please add something to utils/message-generator/README.md

as already stated, I added some to plebhash@2be0191

you can either cherry-pick this commit, or just use it as inspiration to write something on your own words

new feature: it is possible to add the condition "WaitUntil" to a `MatchMessageType` action result. The condition idea is to wait until the desired message arrives. In the condition config there is a timeout, that avoids the MG test getting stuck. There is also a list of allowed messages. If the test awaits a certain message, then all the other messages arrived must belong to this user specified list of allowed messages. This config is passed to the MG in the json file. A `MatchMessageField` with the `WaitUntil` condition has the following shape: ```json "results": [ { "type": "match_message_type", "value": "0x1b", "condition": {"WaitUntil": {"WaitUntilConfig": {"timeout": 120, "allowed_messages": ["0x1b", "0x16"] } } } } ], ``` This action ignores a message with id `0x16`, fails if arrives a message with id `0x17` (for example) and succeeds if arrive the message with desired id `0x1b`. Add README notes for WaitUntil feature

lorbax force-pushed the MG-new-feature-for-actions branch from 1466f93 to bbfdc23 Compare July 19, 2024 10:26

GitGab19 linked an issue Jul 19, 2024 that may be closed by this pull request

New feature for the Message Generator #1059

Closed

plebhash reviewed Jul 20, 2024

View reviewed changes

utils/message-generator/src/main.rs Outdated Show resolved Hide resolved

plebhash reviewed Jul 20, 2024

View reviewed changes

utils/message-generator/src/parser/actions.rs Outdated Show resolved Hide resolved

plebhash approved these changes Jul 20, 2024

View reviewed changes

Shourya742 suggested changes Jul 22, 2024

View reviewed changes

lorbax force-pushed the MG-new-feature-for-actions branch 2 times, most recently from b957b3b to ac836f4 Compare July 22, 2024 12:17

plebhash mentioned this pull request Jul 23, 2024

New feature for the Message Generator #1059

Closed

lorbax force-pushed the MG-new-feature-for-actions branch 3 times, most recently from 404b694 to 4036214 Compare July 29, 2024 19:34

lorbax closed this Jul 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WaitUntil feature on MG #1068

WaitUntil feature on MG #1068

lorbax commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

plebhash left a comment •

edited

Loading

lorbax commented Jul 21, 2024 •

edited

Loading

plebhash commented Jul 21, 2024 •

edited

Loading

Shourya742 left a comment

Shourya742 Jul 22, 2024 •

edited by plebhash

Loading

lorbax Jul 22, 2024

plebhash Jul 22, 2024

lorbax Jul 29, 2024

lorbax commented Jul 22, 2024

lorbax commented Jul 22, 2024

lorbax commented Jul 22, 2024

plebhash commented Jul 22, 2024

-                                        let header = message.get_header().unwrap();
+                                        let header = match sv2_frame.get_header() {
+                                              Some(hdr) => hdr,
+                                              None => {
+                                                  error!("Failed to get header from Sv2Frame");
+                                                  success = false;
+                                                  break;
+                                              }
+                                          };

WaitUntil feature on MG #1068

WaitUntil feature on MG #1068

Conversation

lorbax commented Jul 19, 2024 • edited Loading

github-actions bot commented Jul 19, 2024 • edited Loading

Bencher

github-actions bot commented Jul 19, 2024 • edited Loading

Bencher

github-actions bot commented Jul 19, 2024 • edited Loading

Bencher

github-actions bot commented Jul 19, 2024 • edited Loading

Bencher

plebhash left a comment • edited Loading

Choose a reason for hiding this comment

lorbax commented Jul 21, 2024 • edited Loading

plebhash commented Jul 21, 2024 • edited Loading

Shourya742 left a comment

Choose a reason for hiding this comment

Shourya742 Jul 22, 2024 • edited by plebhash Loading

Choose a reason for hiding this comment

lorbax Jul 22, 2024

Choose a reason for hiding this comment

plebhash Jul 22, 2024

Choose a reason for hiding this comment

lorbax Jul 29, 2024

Choose a reason for hiding this comment

lorbax commented Jul 22, 2024

lorbax commented Jul 22, 2024

lorbax commented Jul 22, 2024

plebhash commented Jul 22, 2024

lorbax commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

github-actions bot commented Jul 19, 2024 •

edited

Loading

plebhash left a comment •

edited

Loading

lorbax commented Jul 21, 2024 •

edited

Loading

plebhash commented Jul 21, 2024 •

edited

Loading

Shourya742 Jul 22, 2024 •

edited by plebhash

Loading