Meaning of "samples/sec" for DLRM offline

[calvinqi]

Hi! I was looking at the results in the table here (https://mlperf.org/inference-results-0-7/) and had a question about the meaning of the unit for DLRM offline samples/s.

Is one "sample" taken to be a single data point, i.e. input features for (user, item, metadata) to generate one single prediction probability, or are they somehow batched according to a distribution like this one https://github.com/mlperf/inference/blob/master/recommendation/dlrm/pytorch/tools/dist_quantile.txt? I'm a little confused on the terminology and what's being measured in these benchmarks. Some clarification would be amazing, thank you!

[TheKanter]

Clarifications here: mlcommons/inference_policies#203

[cayleyhamilton]

The term 'user-item pair' is not defined. The definition of sample hinges on this and even the Ad Criteo dataset does not explicitly define it. Could a definition of 'user-item pair' be added? Thanks.