Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull out template engine and improve performance #268

Open
nikita-petko opened this issue Feb 14, 2023 · 0 comments
Open

Pull out template engine and improve performance #268

nikita-petko opened this issue Feb 14, 2023 · 0 comments
Assignees

Comments

@nikita-petko
Copy link
Contributor

It currently performs a hell ton of RegEx calls that overtime, with more added parts of templates added, majorly decreases performance. The following template code can be used as an example:

# FourHours: 14400 (last week)
# OneDay: 86400 (last month)
# OneMinute: 60 (Last 15 minutes)
# ThirtyMinutes: 1800 (last day)
# ThreeDays: 259200 (last 3 months)
# TwoMinutes: 120 (Last hour)

# var granularities = {
#   OneMinute: 60,
#   TwoMinutes: 120,
#   FourHours: 14400,
#   OneDay: 86400,
#   ThirtyMinutes: 1800,
#   ThreeDays: 259200,
# };
#
# var actualTimings = {
#   Last15Minutes: 900,
#   LastHour: 3600,
#   LastDay: 86400,
#   LastWeek: 604800,
#   LastMonth: 2592000,
#   Last3Months: 7776000,
# };
#
# var granularityToActualTiming = {
#   OneMinute: "Last15Minutes",
#   TwoMinutes: "LastHour",
#   FourHours: "LastWeek",
#   OneDay: "LastMonth",
#   ThirtyMinutes: "LastDay",
#   ThreeDays: "Last3Months",
# };
#
# var granularity = granularities[req.query.granularity];
# var actualTiming = actualTimings[granularityToActualTiming[req.query.granularity]];
# var iterations = actualTiming / granularity;
#
# OUTPUT:
# Granularity   Actual Timing    Iterations
# OneMinute     Last15Minutes    16
# TwoMinutes    LastHour         31
# FourHours     LastWeek         169
# OneDay        LastMonth        31 -- depends on month
# ThirtyMinutes LastDay          49
# ThreeDays     Last3Months      91

- routeTemplate: /compute-telemetry-service/v1/universe/(?<universe_id>\d+)/place/(?<place_id>.+)/kpi-names/(?<kpi_name>[a-zA-Z]+)
  hostname: "arbitrary"
  statusCode: 200
  method: "*"
  templateVariables:
    # Constants
    oneMinute: 60
    twoMinutes: 120
    fourHours: 14400
    oneDay: 86400
    thirtyMinutes: 1800
    threeDays: 259200

    # Variable Expressions
    incrementExpr: >
      {{empty | 
         setVarSwitch increment number 
         ${request.query.granularity} eq OneMinute then ${oneMinute}; 
         ${request.query.granularity} eq TwoMinutes then ${twoMinutes}; 
         ${request.query.granularity} eq FourHours then ${fourHours}; 
         ${request.query.granularity} eq OneDay then ${oneDay}; 
         ${request.query.granularity} eq ThirtyMinutes then ${thirtyMinutes}; 
         ${request.query.granularity} eq ThreeDays then ${threeDays}}}
    iterationExpr: >
      {{empty | 
         setVarSwitch iteration number 
         ${request.query.granularity} eq OneMinute then 16; 
         ${request.query.granularity} eq TwoMinutes then 31; 
         ${request.query.granularity} eq FourHours then 169; 
         ${request.query.granularity} eq OneDay then 31; 
         ${request.query.granularity} eq ThirtyMinutes then 49; 
         ${request.query.granularity} eq ThreeDays then 91}}
    keyExpr: query.startTime | setVar increment number ${increment} * ${__index__} | fromIso | div 1000 | floor | add ${increment}
    valueExpr: randRange 0.0 5.0 | parseFloat | toFixed 2 | parseFloat
    limitValueExpr: randRange 0.0 5.0 | parseFloat | toFixed 2 | parseFloat | add 5.0 | toFixed 2 | parseFloat
    percentileValueExpr: randRange 0.0 1.0 | parseFloat | toFixed 2 | parseFloat | mul 100 | toFixed 2 | parseFloat

  body:
    {
      "placeId": "{{path.place_id | parseInt}}",
      "kpiName": "{{path.kpi_name}}",
      "startTime": "{{query.startTime | default number ${now_} | toIso}}",
      "endTime": "{{query.endTime | default number ${now_} + ${oneDay} | toIso}}",
      "data":
        {
          "Avg":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${valueExpr}}}",
            },
          "Limit":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${limitValueExpr}}}",
            },
          "P10":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${percentileValueExpr}}}",
            },
          "P50":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${percentileValueExpr}}}",
            },
          "P75":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${percentileValueExpr}}}",
            },
          "P90":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${percentileValueExpr}}}",
            },
          "P99":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${percentileValueExpr}}}",
            },
          "P999":
            {
              "data": "{{kvExpression ${iteration} ${keyExpr} --- ${percentileValueExpr}}}",
            },
        },
    }
  formatBody: true
  headers:
    Content-Type: application/json

The KV expression here will take more than 2 minutes to execute at base

@nikita-petko nikita-petko self-assigned this Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant