[core][stats-die/08] kill metric_defs

Signed-off-by: Cuong Nguyen <[email protected]>

Author: can-anyscale

doc/source/ray-observability/reference/system-metrics.rst

@@ -32,10 +32,10 @@ Ray exports a number of system metrics, which provide introspection into the sta
      - Current number of actors in each state described in `rpc::ActorTableData::ActorState <https://github.com/ray-project/ray/blob/b3799a53dcabd8d1a4d20f22faa98e781b0059c7/src/ray/protobuf/gcs.proto#L79>`. ALIVE has two sub-states: ALIVE_IDLE, and ALIVE_RUNNING_TASKS. An actor is considered ALIVE_IDLE if it is not running any tasks.
    * - `ray_resources`
      - `Name`, `State`, `instance`
-     - Logical resource usage for each node of the cluster. Each resource has some quantity that is in either `USED or AVAILABLE state <https://github.com/ray-project/ray/blob/9eab65ed77bdd9907989ecc3e241045954a09cb4/src/ray/stats/metric_defs.cc#L188>`_. The Name label defines the resource name (e.g., CPU, GPU).
+     - Logical resource usage for each node of the cluster. Each resource has some quantity that is in either USED or AVAILABLE state. The Name label defines the resource name (e.g., CPU, GPU).
    * - `ray_object_store_memory`
      - `Location`, `ObjectState`, `instance`
-     - Object store memory usage in bytes, `broken down <https://github.com/ray-project/ray/blob/9eab65ed77bdd9907989ecc3e241045954a09cb4/src/ray/stats/metric_defs.cc#L231>`_ by logical Location (SPILLED, MMAP_DISK, MMAP_SHM, and WORKER_HEAP). Definitions are as follows. SPILLED--Objects that have spilled to disk or a remote Storage solution (for example, AWS S3). The default is the disk. MMAP_DISK--Objects stored on a memory-mapped page on disk. This mode very slow and only happens under severe memory pressure. MMAP_SHM--Objects store on a memory-mapped page in Shared Memory. This mode is the default, in the absence of memory pressure. WORKER_HEAP--Objects, usually smaller, stored in the memory of the Ray Worker process itself. Small objects are stored in the worker heap.
+     - Object store memory usage in bytes, broken down by logical Location (SPILLED, MMAP_DISK, MMAP_SHM, and WORKER_HEAP). Definitions are as follows. SPILLED--Objects that have spilled to disk or a remote Storage solution (for example, AWS S3). The default is the disk. MMAP_DISK--Objects stored on a memory-mapped page on disk. This mode very slow and only happens under severe memory pressure. MMAP_SHM--Objects store on a memory-mapped page in Shared Memory. This mode is the default, in the absence of memory pressure. WORKER_HEAP--Objects, usually smaller, stored in the memory of the Ray Worker process itself. Small objects are stored in the worker heap.
    * - `ray_placement_groups`
      - `State`
      - Current number of placement groups by state. The State label (e.g., PENDING, CREATED, REMOVED) describes the state of the placement group. See `rpc::PlacementGroupTable <https://github.com/ray-project/ray/blob/e85355b9b593742b4f5cb72cab92051980fa73d3/src/ray/protobuf/gcs.proto#L517>`_ for more information.

python/ray/_private/metrics_agent.py

@@ -596,7 +596,7 @@ def collect(self):  # pragma: NO COVER
                         # metrics. The aggregation of histogram metrics from worker
                         # level to node level is not well defined. In addition, we
                         # currently have very few histogram metrics in Ray
-                        # (metric_defs.cc) so the impact of them is negligible.
+                        # so the impact of them is negligible.
                         to_lower_cardinality[metric.name].append(metric)
                     else:
                         open_cencus_metrics.append(metric)

python/ray/_private/telemetry/metric_cardinality.py

@@ -5,7 +5,7 @@
 
 # Keep in sync with the WorkerIdKey in src/ray/stats/tag_defs.cc
 WORKER_ID_TAG_KEY = "WorkerId"
-# Keep in sync with the NameKey in src/ray/stats/metric_defs.cc
+# Keep in sync with the NameKey in src/ray/stats/tag_defs.cc
 TASK_OR_ACTOR_NAME_TAG_KEY = "Name"
 HIGH_CARDINALITY_METRICS_TO_AGGREGATION: Dict[str, Callable[[List[float]], float]] = {
     "tasks": lambda values: sum(values),

python/ray/dashboard/modules/metrics/dashboards/default_dashboard_panels.py

@@ -215,7 +215,7 @@ def max_plus_pending(max_resource, pending_resource):
     Panel(
         id=29,
         title="Object Store Memory by Location",
-        description="Object store memory usage by location. The dotted line indicates the object store memory capacity. This metric can go over the max capacity in case of spillage to disk.\n\nLocation: where the memory was allocated, which is MMAP_SHM or MMAP_DISK to indicate memory-mapped page, SPILLED to indicate spillage to disk, and WORKER_HEAP for objects small enough to be inlined in worker memory. Refer to metric_defs.cc for more information.",
+        description="Object store memory usage by location. The dotted line indicates the object store memory capacity. This metric can go over the max capacity in case of spillage to disk.\n\nLocation: where the memory was allocated, which is MMAP_SHM or MMAP_DISK to indicate memory-mapped page, SPILLED to indicate spillage to disk, and WORKER_HEAP for objects small enough to be inlined in worker memory.",
         unit="bytes",
         targets=[
             Target(

python/ray/tests/test_metrics_agent.py

@@ -54,7 +54,7 @@
 except ImportError:
     prometheus_client = None
 
-# This list of metrics should be kept in sync with src/ray/stats/metric_defs.h
+# This list of metrics should be kept in sync with metric definitions across the codebase
 # NOTE: Commented out metrics are not available in this test.
 # TODO(Clark): Find ways to trigger commented out metrics in cluster setup.
 _METRICS = [

src/ray/core_worker/store_provider/memory_store/memory_store.cc

@@ -22,7 +22,6 @@
 
 #include "ray/common/ray_config.h"
 #include "ray/raylet_ipc_client/raylet_ipc_client_interface.h"
-#include "ray/stats/metric_defs.h"
 #include "ray/stats/tag_defs.h"
 
 namespace ray {

src/ray/gcs/gcs_actor_manager.cc

@@ -26,7 +26,6 @@
 #include "ray/common/protobuf_utils.h"
 #include "ray/common/ray_config.h"
 #include "ray/common/task/task_spec.h"
-#include "ray/stats/metric_defs.h"
 #include "ray/util/logging.h"
 #include "ray/util/time.h"
 

src/ray/gcs/gcs_node_manager.h

@@ -29,7 +29,6 @@
 #include "ray/observability/ray_event_recorder_interface.h"
 #include "ray/pubsub/gcs_publisher.h"
 #include "ray/raylet_rpc_client/raylet_client_pool.h"
-#include "ray/stats/metric_defs.h"
 #include "ray/util/event.h"
 #include "src/ray/protobuf/autoscaler.pb.h"
 #include "src/ray/protobuf/gcs.pb.h"

src/ray/gcs/gcs_placement_group_manager.cc

@@ -22,7 +22,6 @@
 #include "ray/common/asio/asio_util.h"
 #include "ray/common/bundle_spec.h"
 #include "ray/common/ray_config.h"
-#include "ray/stats/metric_defs.h"
 #include "src/ray/protobuf/gcs.pb.h"
 
 namespace ray {

src/ray/gcs/gcs_task_manager.cc

@@ -29,7 +29,6 @@
 #include "ray/common/ray_config.h"
 #include "ray/common/status.h"
 #include "ray/gcs/gcs_ray_event_converter.h"
-#include "ray/stats/metric_defs.h"
 
 namespace ray {
 namespace gcs {