You are viewing docs on Elastic's new documentation system, currently in technical preview. For all other Elastic docs, visit elastic.co/guide.

Elastic Agent

Collect logs and metrics from Elastic Agents.

What is an Elastic integration?
Version
1.18.0 (View all)
Compatible Kibana version(s)
8.11.2 or higher
Supported Serverless project types

What's this?
Security
Observability
Subscription level
What's this?
Basic

This integration provides observability for Elastic Agent metrics. It provides a dashboard to visualize the status of your agents so you can troubleshoot problems and determine when to add capacity.

You can enable or disable agent monitoring in the agent policy settings.

Metrics

Core

Exported fields

FieldDescriptionType
@timestamp
Event timestamp.
date
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host is running.
keyword
cloud.image.id
Image ID for the cloud instance.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.instance.name
Instance name of the host machine.
keyword
cloud.machine.type
Machine type of the host machine.
keyword
cloud.project.id
Name of the project in Google Cloud.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host is running.
keyword
container.id
Unique container id.
keyword
container.image.name
Name of the image the container was built on.
keyword
container.labels
Image labels.
object
container.name
Container name.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
host.architecture
Operating system architecture.
keyword
host.containerized
If the host is a container.
boolean
host.domain
Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider.
keyword
host.hostname
Hostname of the host. It normally contains what the hostname command returns on the host machine.
keyword
host.id
Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name.
keyword
host.ip
Host ip address.
ip
host.mac
Host mac address.
keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.
keyword
host.os.build
OS build information.
keyword
host.os.codename
OS codename, if any.
keyword
host.os.family
OS family (such as redhat, debian, freebsd, windows).
keyword
host.os.full
Operating system name, including the version or code name.
keyword
host.os.kernel
Operating system kernel version as a raw string.
keyword
host.os.name
Operating system name, without the version.
keyword
host.os.platform
Operating system platform (such centos, ubuntu, windows).
keyword
host.os.version
Operating system version as a raw string.
keyword
host.type
Type of host.
keyword
elastic_agent.id
Elastic agent id.
elastic_agent.process
Elastic agent process (elastic-agent, metricbeat, ...).
elastic_agent.version
Elastic version as a raw string.

Process

The Elastic Agent process dataset provides process statistics about Elastic Agent processes. One document is provided for each process.

FieldDescriptionType
system.process.cpu.system.ticks
The amount of CPU time the process spent in kernel space.
long
system.process.cpu.system.time.me
The time when the process was started.
date
system.process.cpu.total.ticks
The total CPU time spent by the process.
long
system.process.cpu.total.value
The value of CPU usage since starting the process.
long
system.process.cpu.total.time.me
The time when the process was started.
date
system.process.cpu.user.ticks
The amount of CPU time the process spent in user space.
long
system.process.cpu.user.time.me
The time when the process was started.
date
system.process.env
The environment variables used to start the process. The data is available on FreeBSD, Linux, and OS X.
object
system.process.fd.limit.soft
The soft limit on the number of file descriptors opened by the process. The soft limit can be changed by the process at any time.
long
system.process.fd.open
The number of file descriptors open by the process.
long
system.process.memory.size
The total virtual memory the process has. On Windows this represents the Commit Charge (the total amount of memory that the memory manager has committed for a running process) value in bytes for this process.
long
system.process.cgroup.blkio.id
ID of the cgroup.
keyword
system.process.cgroup.blkio.path
Path to the cgroup relative to the cgroup subsystems mountpoint.
keyword
system.process.cgroup.blkio.total.bytes
Total number of bytes transferred to and from all block devices by processes in the cgroup.
long
system.process.cgroup.blkio.total.ios
Total number of I/O operations performed on all devices by processes in the cgroup as seen by the throttling policy.
long
system.process.cgroup.cpu.cfs.period.us
Period of time in microseconds for how regularly a cgroup's access to CPU resources should be reallocated.
long
system.process.cgroup.cpu.cfs.quota.us
Total amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by cfs.period.us).
long
system.process.cgroup.cpu.cfs.shares
An integer value that specifies a relative share of CPU time available to the tasks in a cgroup. The value specified in the cpu.shares file must be 2 or higher.
long
system.process.cgroup.cpu.id
ID of the cgroup.
keyword
system.process.cgroup.cpu.path
Path to the cgroup relative to the cgroup subsystem's mountpoint.
keyword
system.process.cgroup.cpu.rt.period.us
Period of time in microseconds for how regularly a cgroup's access to CPU resources is reallocated.
long
system.process.cgroup.cpu.rt.runtime.us
Period of time in microseconds for the longest continuous period in which the tasks in a cgroup have access to CPU resources.
long
system.process.cgroup.cpu.stats.periods
Number of period intervals (as specified in cpu.cfs.period.us) that have elapsed.
long
system.process.cgroup.cpu.stats.throttled.ns
The total time duration (in nanoseconds) for which tasks in a cgroup have been throttled.
long
system.process.cgroup.cpu.stats.throttled.periods
Number of times tasks in a cgroup have been throttled (that is, not allowed to run because they have exhausted all of the available time as specified by their quota).
long
system.process.cgroup.cpuacct.id
ID of the cgroup.
keyword
system.process.cgroup.cpuacct.path
Path to the cgroup relative to the cgroup subsystem's mountpoint.
keyword
system.process.cgroup.cpuacct.percpu
CPU time (in nanoseconds) consumed on each CPU by all tasks in this cgroup.
object
system.process.cgroup.cpuacct.stats.system.ns
CPU time consumed by tasks in user (kernel) mode.
long
system.process.cgroup.cpuacct.stats.user.ns
CPU time consumed by tasks in user mode.
long
system.process.cgroup.cpuacct.total.ns
Total CPU time in nanoseconds consumed by all tasks in the cgroup.
long
system.process.cgroup.id
The ID common to all cgroups associated with this task. If there isn't a common ID used by all cgroups this field will be absent.
keyword
system.process.cgroup.memory.id
ID of the cgroup.
keyword
system.process.cgroup.memory.kmem.failures
The number of times that the memory limit (kmem.limit.bytes) was reached.
long
system.process.cgroup.memory.kmem.limit.bytes
The maximum amount of kernel memory that tasks in the cgroup are allowed to use.
long
system.process.cgroup.memory.kmem.usage.bytes
Total kernel memory usage by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.kmem.usage.max.bytes
The maximum kernel memory used by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.kmem_tcp.failures
The number of times that the memory limit (kmem_tcp.limit.bytes) was reached.
long
system.process.cgroup.memory.kmem_tcp.limit.bytes
The maximum amount of memory for TCP buffers that tasks in the cgroup are allowed to use.
long
system.process.cgroup.memory.kmem_tcp.usage.bytes
Total memory usage for TCP buffers in bytes.
long
system.process.cgroup.memory.kmem_tcp.usage.max.bytes
The maximum memory used for TCP buffers by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.mem.failures
The number of times that the memory limit (mem.limit.bytes) was reached.
long
system.process.cgroup.memory.mem.limit.bytes
The maximum amount of user memory in bytes (including file cache) that tasks in the cgroup are allowed to use.
long
system.process.cgroup.memory.mem.usage.bytes
Total memory usage by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.mem.usage.max.bytes
The maximum memory used by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.memsw.failures
The number of times that the memory plus swap space limit (memsw.limit.bytes) was reached.
long
system.process.cgroup.memory.memsw.limit.bytes
The maximum amount for the sum of memory and swap usage that tasks in the cgroup are allowed to use.
long
system.process.cgroup.memory.memsw.usage.bytes
The sum of current memory usage plus swap space used by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.memsw.usage.max.bytes
The maximum amount of memory and swap space used by processes in the cgroup (in bytes).
long
system.process.cgroup.memory.path
Path to the cgroup relative to the cgroup subsystem's mountpoint.
keyword
system.process.cgroup.memory.stats.active_anon.bytes
Anonymous and swap cache on active least-recently-used (LRU) list, including tmpfs (shmem), in bytes.
long
system.process.cgroup.memory.stats.active_file.bytes
File-backed memory on active LRU list, in bytes.
long
system.process.cgroup.memory.stats.cache.bytes
Page cache, including tmpfs (shmem), in bytes.
long
system.process.cgroup.memory.stats.hierarchical_memory_limit.bytes
Memory limit for the hierarchy that contains the memory cgroup, in bytes.
long
system.process.cgroup.memory.stats.hierarchical_memsw_limit.bytes
Memory plus swap limit for the hierarchy that contains the memory cgroup, in bytes.
long
system.process.cgroup.memory.stats.inactive_anon.bytes
Anonymous and swap cache on inactive LRU list, including tmpfs (shmem), in bytes
long
system.process.cgroup.memory.stats.inactive_file.bytes
File-backed memory on inactive LRU list, in bytes.
long
system.process.cgroup.memory.stats.major_page_faults
Number of times that a process in the cgroup triggered a major fault. "Major" faults happen when the kernel actually has to read the data from disk.
long
system.process.cgroup.memory.stats.mapped_file.bytes
Size of memory-mapped mapped files, including tmpfs (shmem), in bytes.
long
system.process.cgroup.memory.stats.page_faults
Number of times that a process in the cgroup triggered a page fault.
long
system.process.cgroup.memory.stats.pages_in
Number of pages paged into memory. This is a counter.
long
system.process.cgroup.memory.stats.pages_out
Number of pages paged out of memory. This is a counter.
long
system.process.cgroup.memory.stats.rss.bytes
Anonymous and swap cache (includes transparent hugepages), not including tmpfs (shmem), in bytes.
long
system.process.cgroup.memory.stats.rss_huge.bytes
Number of bytes of anonymous transparent hugepages.
long
system.process.cgroup.memory.stats.swap.bytes
Swap usage, in bytes.
long
system.process.cgroup.memory.stats.unevictable.bytes
Memory that cannot be reclaimed, in bytes.
long
system.process.cgroup.path
The path to the cgroup relative to the cgroup subsystem's mountpoint. If there isn't a common path used by all cgroups this field will be absent.
keyword

Changelog

VersionDetailsKibana version(s)

1.18.0

Enhancement View pull request
Add metrics dashboard for httpjson, http_endpoint, filestream and CEL, fix decimal numbers on certain counters, add field mappings for filebeat_input.id, and component fields to filebeat_input logs

8.11.2 or higher

1.17.0

Enhancement View pull request
Exposing beat.stats.libbeat.pipeline.queue.max_events metrics.

8.9.0 or higher

1.16.0

Bug fix View pull request
Improve and unify dimensions for Elastic-Agent and Beats metrics, this avoids duplicated TSDB entries.

8.9.0 or higher

1.15.0

Enhancement View pull request
Add data stream for logs of Universal Profiling services.

8.9.0 or higher

1.14.0

Enhancement View pull request
Modify field mappings to reference ECS fields where possible and remove duplicate field declarations.

8.9.0 or higher

1.13.1

Bug fix View pull request
Fix mapping and description for the system.process.cpu.{system,user,total}.time.ms fields.

Bug fix View pull request
Align mapping for the beat.stats.libbeat.config.{running,starts,stops} fields with the beat integration.

Bug fix View pull request
For the message field, consistently use the ECS defined mapping type of match_only_text.

8.9.0 or higher

1.13.0

Enhancement View pull request
Remove metric mappings from the filebeat_input_logs data stream

8.9.0 or higher

1.12.1

Enhancement View pull request
Add a new dataset to include both metrics and logs for the active integrations and agent charts

8.9.0 or higher

1.12.0

Enhancement View pull request
Add metrics for queue depth, output batch size and output batch rate.

8.9.0 or higher

1.11.2

Enhancement View pull request
Adding hyperlinks that works when installed on different spaces.

8.9.0 or higher

1.11.1

Bug fix View pull request
Fix agent health dashboard links to work when installed in other spaces.

8.9.0 or higher

1.11.0

Enhancement View pull request
Enable time series data streams for the metrics datastreams except for endpoint security metrics and filebeat input metrics. This improves storage usage and query performance. For more details, see https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html.

8.9.0 or higher

1.10.1

Enhancement View pull request
Set metric type for all metric fields.

8.9.0 or higher

1.10.0

Enhancement View pull request
Set dimension fields for metrics data streams APM Server, Auditbeat, Cloudbeat, Elastic Agent, Heartbeat, Filebeat, Metricbeat, Osquery and Packetbeat.

8.7.1 or higher

1.9.1

Bug fix View pull request
For the filebeat_input metrics data stream, prevent dynamic mapping rules designed for long values from matching objects. This fixes mapping issues for the 'filebeat_input.httpjson_interval_pages_total' histogram.

8.7.1 or higher

1.9.0

Enhancement View pull request
Add fleet-server attributes to log.

8.7.1 or higher

1.8.0

Enhancement View pull request
Added new Health dashboards for Input Metrics

8.7.1 or higher

1.7.0

Enhancement View pull request
Added agent.* field mappings and updated filters on certain dashboards

8.6.1 or higher

1.6.0

Enhancement View pull request
Adding new Agent Health dashboards, and remaking Agent Metrics.

8.6.1 or higher

1.5.2

Enhancement View pull request
Add datastreams for cloud_defend service logs

7.16.0 or higher
8.0.0 or higher

1.5.1

Bug fix View pull request
Add dataset filters for agent metrics

7.16.0 or higher
8.0.0 or higher

1.5.0

Enhancement View pull request
Add filebeat input metrics

7.16.0 or higher
8.0.0 or higher

1.4.1

Enhancement View pull request
Cloudbeat decision logs support

7.16.0 or higher
8.0.0 or higher

1.4.0

Enhancement View pull request
Add new fields for Elastic Agent v2 components and units

7.16.0 or higher
8.0.0 or higher

1.3.5

Bug fix View pull request
Fix the external ECS fields not being properly resolved during the package build

7.16.0 or higher
8.0.0 or higher

1.3.4

Enhancement View pull request
Cloudbeat logs search support

7.16.0 or higher
8.0.0 or higher

1.3.3

Enhancement View pull request
Add configuration for cloudbeat logs and metrics.

7.16.0 or higher
8.0.0 or higher

1.3.2

Bug fix View pull request
Fix some CPU elastic_agent_metrics mapping from date to long

—

1.3.1

Bug fix View pull request
Fix missing ecs.version mapping

7.16.0 or higher
8.0.0 or higher

1.3.0

Enhancement View pull request
Update compatibility of package to be compatible with 8.0.x

7.16.0 or higher
8.0.0 or higher

1.2.2

Enhancement View pull request
Uniform with guidelines

—

1.2.1

Bug fix View pull request
Fix dashboard default filter

7.15.0 or higher

1.2.0

Enhancement View pull request
Update dashboard to CGroup CPU usage and events rates visualization and add Elastic Agent logo

7.15.0 or higher

1.1.1

Bug fix View pull request
Fix missing support for heartbeat metrics and logs

7.15.0 or higher

1.1.0

Enhancement View pull request
Add mappings for all metrics and logs shipped by Elastic Agent and its sub processes.

—

1.0.0

Enhancement View pull request
Make integration GA.

7.14.0 or higher

0.1.0

Enhancement View pull request
Update integration description

—

0.0.7

Bug fix View pull request
Fix typo in dashboard

—

0.0.6

Bug fix View pull request
Fix README, icons and add screenshot

—

0.0.5

Enhancement View pull request
initial release

—

On this page