> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trace0hq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# JVM Metrics

> Monitor JVM metrics including request latency, memory usage, GC times, etc.


## How are Metrics Calculated?

In Trace0, JVM metrics come from the following two OpenTelemetry sources:

* **Requests metrics** are calculated from the service's `SPAN_KIND_SERVER` spans, one per request the service handles. Request count, requests per second, latency percentiles and errors are all derived from these spans and their durations and statuses.
* **JVM metrics** (Memory, Garbage Collection, Threads, Buffers, Classes, CPU and File Descriptors) are calculated from the OpenTelemetry metrics the JVM reports, as defined by the [OpenTelemetry semantic conventions for JVM metrics](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/). Each metric below links to its section of that spec.

## Aggregation Temporality

OpenTelemetry Sum metrics (Counters and UpDownCounters) and Histogram metrics are reported with an [aggregation temporality](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#temporality), which decides what each reported data point means. We use the most recent aggregation temporality the service reported for the metric in the selected period.

An aggregation temporality can either be `Cumulative` or `Delta`.

| Metric Type | Cumulative | Delta |
| - | - | - |
| `Sum (Counter, UpDownCounter)` | Each data point is a running total since the process started. We show the most recent value reported in each interval. | Each data point is the change since the previous export. We add up the values reported in each interval. |
| `Histogram` | Each data point holds each bucket's running total since the process started. We show each bucket's most recent count in the selected period. | Each data point holds each bucket's count since the previous export. We add up each bucket's counts across the selected period. |

`Gauge` and `Summary` metrics don't have an aggregation temporality, so this doesn't apply to them.

## Metrics

We currently display the following application metrics for the JVM:

### Requests

**Single Value**

| Metric | Unit | How is it Calculated? |
| - | - | - |
| Request Count | Number | TODO |
| Requests per Second | Number | TODO |
| Request Latency P99 | Time (ms, s) | TODO |
| Error Rate | Percentage | TODO |

**Chart**

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| Request Latency P99, P95, P90, P75, P50 | Latency percentile distribution | Line chart | TODO |
| Requests & Errors | Request volume vs error volume | Histogram | TODO |
| Errors | Error volume over time | Histogram | TODO |
| Requests By Endpoint | Request details (count, latency, errors) per endpoint. | Table, one row per endpoint | TODO |

### JVM Memory

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| Total Heap Memory Used | Total memory used (in bytes) across all heap memory pools. | Line chart | From the [`jvm.memory.used`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryused) metric. We take the series where [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) is `heap`, and add up their values across every memory pool at each interval. |
| Heap Memory Used by Pool | Heap memory used (in bytes), per memory pool. | Line chart, one line per memory pool | From the [`jvm.memory.used`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryused) metric. Split by the [`jvm.memory.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/). Only includes pools where the [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute is `heap`. Each interval shows the most recent value reported. |
| Non-Heap Memory Used by Pool | Non-heap memory used (in bytes), per memory pool. | Line chart, one line per memory pool | From the [`jvm.memory.used`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryused) metric. Split by the [`jvm.memory.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/). Only includes pools where the [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute is `non_heap`. Each interval shows the most recent value reported. |
| Heap Memory Committed by Pool | Memory committed (in bytes), per memory pool. | Line chart, one line per memory pool | From the [`jvm.memory.committed`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemorycommitted) UpDownCounter, obtained from `MemoryPoolMXBean#getUsage()`. Split by the [`jvm.memory.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) and [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attributes. Each interval shows the most recent value reported. |
| Memory Limit | Maximum memory (in bytes) that can be obtained, per memory pool. | Line chart, one line per memory pool | From the [`jvm.memory.limit`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemorylimit) UpDownCounter, obtained from `MemoryPoolMXBean#getUsage()`. Split by the [`jvm.memory.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) and [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attributes. Each interval shows the most recent value reported. |
| Memory Used After Last GC | Memory used (in bytes) after the most recent garbage collection on each memory pool. | Line chart, one line per memory pool | From the [`jvm.memory.used_after_last_gc`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryused_after_last_gc) UpDownCounter, obtained from `MemoryPoolMXBean#getCollectionUsage()`. Split by the [`jvm.memory.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) and [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attributes. Each interval shows the most recent value reported. |
| Memory Init | Initial memory requested (in bytes), per memory pool. | Line chart, one line per memory pool | From the [`jvm.memory.init`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmmemoryinit) UpDownCounter (Development status), obtained from `MemoryPoolMXBean#getUsage()`. Split by the [`jvm.memory.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) and [`jvm.memory.type`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attributes. Each interval shows the most recent value reported. |

### JVM Garbage Collection

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| GC Duration | Distribution of how long (in seconds) garbage collection actions take, per garbage collector. | OpenTelemetry histogram, one per garbage collector | From the [`jvm.gc.duration`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmgcduration) Histogram. Its values come from `GcInfo#getDuration()` on `GarbageCollectionNotificationInfo` events. Split by the [`jvm.gc.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute (e.g. `G1 Young Generation`, `G1 Old Generation`). For each bucket, we add up the counts across the whole selected period. The spec recommends the bucket boundaries `[0.01, 0.1, 1, 10]`. |

### JVM Threads

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| Thread Count | Number of executing platform threads, per thread state. | Line chart, one line per thread state | From the [`jvm.thread.count`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmthreadcount) UpDownCounter, obtained from `ThreadMXBean` / `ThreadInfo#getThreadState()`. Split by the [`jvm.thread.state`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute (e.g. `runnable`, `blocked`, `waiting`, `timed_waiting`). Each interval shows the most recent value reported. |

### JVM Buffers

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| Buffer Memory Used | Memory used by buffers (in bytes), per buffer pool. | Line chart, one line per buffer pool | From the [`jvm.buffer.memory.used`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmbuffermemoryused) UpDownCounter (Development status), obtained from `BufferPoolMXBean#getMemoryUsed()`. Split by the [`jvm.buffer.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute (e.g. `mapped`, `direct`). Each interval shows the most recent value reported. |
| Buffer Memory Limit | Total memory capacity of buffers (in bytes), per buffer pool. | Line chart, one line per buffer pool | From the [`jvm.buffer.memory.limit`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmbuffermemorylimit) UpDownCounter (Development status), obtained from `BufferPoolMXBean#getTotalCapacity()`. Split by the [`jvm.buffer.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute. Each interval shows the most recent value reported. |
| Buffer Count | Number of buffers, per buffer pool. | Line chart, one line per buffer pool | From the [`jvm.buffer.count`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmbuffercount) UpDownCounter (Development status), obtained from `BufferPoolMXBean#getCount()`. Split by the [`jvm.buffer.pool.name`](https://opentelemetry.io/docs/specs/semconv/registry/attributes/jvm/) attribute. Each interval shows the most recent value reported. |

### JVM Classes

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| Classes Loaded | Number of classes loaded since the JVM started. | Line chart | From the [`jvm.class.loaded`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmclassloaded) Counter, obtained from `ClassLoadingMXBean#getTotalLoadedClassCount()`. Each interval shows the most recent value reported (cumulative), or the sum of the values reported during the interval (delta). |
| Classes Unloaded | Number of classes unloaded since the JVM started. | Line chart | From the [`jvm.class.unloaded`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmclassunloaded) Counter, obtained from `ClassLoadingMXBean#getUnloadedClassCount()`. Each interval shows the most recent value reported (cumulative), or the sum of the values reported during the interval (delta). |
| Class Count | Number of classes currently loaded. | Line chart | From the [`jvm.class.count`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmclasscount) UpDownCounter, obtained from `ClassLoadingMXBean#getLoadedClassCount()`. Each interval shows the most recent value reported. |

### JVM CPU

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| CPU Time | CPU time used by the process (in seconds), as reported by the JVM. | Line chart | From the [`jvm.cpu.time`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmcputime) Counter. It is obtained from `OperatingSystemMXBean#getProcessCpuTime()` on HotSpot and OpenJ9. Each interval shows the most recent value reported (cumulative), or the sum of the values reported during the interval (delta). |
| CPU Count | Number of processors available to the JVM. | Line chart | From the [`jvm.cpu.count`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmcpucount) UpDownCounter, obtained from `Runtime#availableProcessors()`. It's always a whole number, since fractional cores and millicores aren't represented. Each interval shows the most recent value reported. |

### JVM File Descriptors

| Metric | Description | Chart Type | How is it Calculated? |
| - | - | - | - |
| File Descriptor Count | Number of open file descriptors, as reported by the JVM. | Line chart | From the [`jvm.file_descriptor.count`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmfile_descriptorcount) UpDownCounter (Development status, Opt-In), obtained from `UnixOperatingSystemMXBean#getOpenFileDescriptorCount()`. Each interval shows the most recent value reported. |
| File Descriptor Limit | Maximum number of open file descriptors, as reported by the JVM. | Line chart | From the [`jvm.file_descriptor.limit`](https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmfile_descriptorlimit) UpDownCounter (Development status, Opt-In), obtained from `UnixOperatingSystemMXBean#getMaxFileDescriptorCount()`. Each interval shows the most recent value reported. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.