You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We do not account for the memory size of the hierarchical results normalizer.
When it was first introduced it was fairly close to being a fixed size object, so could reasonably be considered part of the "process overhead" - fixed data structures for which we add 10MB to the required memory of each job.
However, now that we do per partition scoring it is no longer reasonable to consider the normalizer part of the fixed overhead. For a job with a large number of partition field values it can be a non-negligible size, and this memory should be properly accounted for.
The text was updated successfully, but these errors were encountered:
We do not account for the memory size of the hierarchical results normalizer.
When it was first introduced it was fairly close to being a fixed size object, so could reasonably be considered part of the "process overhead" - fixed data structures for which we add 10MB to the required memory of each job.
However, now that we do per partition scoring it is no longer reasonable to consider the normalizer part of the fixed overhead. For a job with a large number of partition field values it can be a non-negligible size, and this memory should be properly accounted for.
The text was updated successfully, but these errors were encountered: