Limit of number of metrics per workflow run?

[jduan-highnote]

Is there a limit of how many metrics that can be sent to datadog per workflow run? We have a pretty complicated workflow and I'm noticing that there are no metrics for certain jobs. Here's an example:

2022-10-02T22:01:09.3441781Z Sending 3288 metrics to Datadog
2022-10-02T22:01:09.3441869Z Sent as {"status":"ok"}
2022-10-02T22:01:09.3689665Z Cleaning up orphan processes

Even with 3288 metrics, there are still no metrics for certain jobs. Thanks!

[jduan-highnote]

@int128 can you take a look? Our workflow keeps getting larger and larger. It would be great if this action can send all the metrics of a workflow run, even when there are lots of metrics at the job or step level. Can the metrics be split into small chunks so they don't exceed Datadog's limit? Thanks!

[jduan-highnote]

@int128 can you take a look? can you point me to the code that sends metrics to datadog? I might take a stab at making a change for this.

[int128]

This function sends the metrics to Datadog API. Does it need to split them to smaller chunks?

datadog-actions-metrics/src/client.ts

Lines 44 to 52 in 05c61e8

	async submitMetrics(series: v1.Series[], description: string): Promise<void> {
	series = this.metricsFilter(series)
	core.startGroup(`Metrics payload (${description})`)
	core.info(JSON.stringify(series, undefined, 2))
	core.endGroup()
	core.info(`Sending ${series.length} metrics to Datadog`)
	const accepted = await this.metricsApi.submitMetrics({ body: { series } })
	core.info(`Sent ${JSON.stringify(accepted)}`)
	}

It also needs to paginate the GitHub API calls here:

datadog-actions-metrics/src/workflowRun/handler.ts

Line 36 in 05c61e8

checkSuite = await getCompletedCheckSuite(octokit, { node_id: e.workflow_run.check_suite_node_id })

datadog-actions-metrics/src/workflowRun/handler.ts

Lines 49 to 55 in 05c61e8

	workflowJobs = await octokit.rest.actions.listJobsForWorkflowRunAttempt({
	owner: e.workflow_run.repository.owner.login,
	repo: e.workflow_run.repository.name,
	run_id: e.workflow_run.id,
	attempt_number: e.workflow_run.run_attempt,
	per_page: 100,
	})

[jduan-highnote]

Hi @int128,

Thanks for your response! I just looked at one of our largest workflows and I didn't see any problems. You can close this issue now!

These are the logs if it helps.

Got workflow run completed event: https://github.com/bay1inc/bayone/actions/runs/11732060213
Finding the check suite CS_kwDOEoUU4c8AAAAHIi_qMg
setServerVariables is deprecated, please use Configuration.setServerVariables instead.
Found the check suite with 50 check run(s)
Finding the jobs for the workflow run 11732060213
Found 50 job(s)
Metrics payload (workflow run)
Sending 3 metrics to Datadog
Sent {"status":"ok"}
Distribution points payload (workflow run)
Sending 0 distribution points to Datadog
Sent {"status":"ok"}
Metrics payload (job)
Sending 227 metrics to Datadog
Sent {"status":"ok"}
Distribution points payload (job)
Sending 0 distribution points to Datadog
Sent {"status":"ok"}
Metrics payload (rate-limit)
Sending 6 metrics to Datadog
Sent {"status":"ok"}