You are viewing docs on Elastic's new documentation system, currently in technical preview. For all other Elastic docs, visit elastic.co/guide.

STAN

Collect logs and metrics from STAN servers with Elastic Agent.

What is an Elastic integration?
Version
1.5.0 (View all)
Compatible Kibana version(s)
8.10.2 or higher
Supported Serverless project types

What's this?
Security
Observability
Subscription level
What's this?
Basic

This integration is used to collect logs and metrics from STAN servers. The integration collects metrics from STAN monitoring server APIs.

Compatibility

The STAN package is tested with Stan 0.15.1.

Logs

log

The log dataset collects the STAN logs.

An example event for log looks as following:

{
    "@timestamp": "2022-01-12T04:11:35.529Z",
    "agent": {
        "ephemeral_id": "8d87b679-d308-4954-a88f-fdac22706bb7",
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.0.0-beta1"
    },
    "data_stream": {
        "dataset": "stan.log",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "snapshot": false,
        "version": "8.0.0-beta1"
    },
    "event": {
        "agent_id_status": "verified",
        "created": "2022-01-12T04:11:50.063Z",
        "dataset": "stan.log",
        "ingested": "2022-01-12T04:11:50Z",
        "kind": "event",
        "type": [
            "info"
        ]
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/stan.log"
        },
        "level": "info",
        "offset": 0
    },
    "message": "STREAM: Starting nats-streaming-server[test-cluster] version 0.15.1",
    "process": {
        "pid": 7
    },
    "stan": {
        "log": {
            "msg": {}
        }
    },
    "tags": [
        "forwarded",
        "stan-log"
    ]
}

Exported fields

FieldDescriptionType
@timestamp
Event timestamp.
date
client.ip
IP address of the client (IPv4 or IPv6).
ip
client.port
Port of the client.
long
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
ecs.version
ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.
keyword
error.message
Error message.
match_only_text
event.created
event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used.
date
event.dataset
Event dataset
constant_keyword
event.kind
This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not.
keyword
event.module
Event module
constant_keyword
event.type
This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types.
keyword
input.type
Type of Filebeat input.
keyword
log.file.path
Full path to the log file this event came from.
keyword
log.level
Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in log.level. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). Some examples are warn, err, i, informational.
keyword
log.offset
Offset of the entry in the log file.
long
message
For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message.
match_only_text
network.direction
Direction of the network traffic. Recommended values are: * ingress * egress * inbound * outbound * internal * external * unknown When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers.
keyword
process.pid
Process id.
long
related.ip
All of the IPs seen on your event.
ip
stan.log.client.id
The id of the client
integer
stan.log.msg.bytes
Size of the payload in bytes
long
stan.log.msg.error.message
Details about the error occurred
text
stan.log.msg.max_messages
An optional number of messages to wait for before automatically unsubscribing
integer
stan.log.msg.queue_group
The queue group which subscriber will join
text
stan.log.msg.reply_to
The inbox subject on which the publisher is listening for responses
keyword
stan.log.msg.sid
The unique alphanumeric subscription ID of the subject
integer
stan.log.msg.subject
Subject name this message was received on
keyword
stan.log.msg.type
The protocol message type
keyword
tags
List of keywords used to tag each event.
keyword

Metrics

The default datasets are stats, channels, and subscriptions.

stats

This is the stats dataset of the STAN package, in charge of retrieving generic metrics from a STAN instance.

An example event for stats looks as following:

{
    "@timestamp": "2022-01-12T04:12:59.760Z",
    "agent": {
        "ephemeral_id": "6e4beb8a-ccda-438d-b3f4-e89b98e870d8",
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.0.0-beta1"
    },
    "data_stream": {
        "dataset": "stan.stats",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "1.12.0"
    },
    "elastic_agent": {
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "snapshot": false,
        "version": "8.0.0-beta1"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.stats",
        "duration": 15794254,
        "ingested": "2022-01-12T04:13:00Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "4ccba669f0df47fa3f57a9e4169ae7f1",
        "ip": [
            "172.18.0.4"
        ],
        "mac": [
            "02:42:ac:12:00:04"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "Core",
            "family": "redhat",
            "kernel": "5.11.0-44-generic",
            "name": "CentOS Linux",
            "platform": "centos",
            "type": "linux",
            "version": "7 (Core)"
        }
    },
    "metricset": {
        "name": "stats",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/serverz",
        "type": "stan"
    },
    "stan": {
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "JQCbrpPJGBxuQGsQ9Yx4Xs"
        },
        "stats": {
            "bytes": 0,
            "channels": 1,
            "clients": 100,
            "messages": 0,
            "state": "STANDALONE",
            "subscriptions": 100
        }
    }
}

Exported fields

FieldDescriptionTypeMetric Type
@timestamp
Event timestamp.
date
agent.id
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.
keyword
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host, resource, or service is located.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host, resource, or service is located.
keyword
container.id
Unique container id.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
ecs.version
ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.
keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.
keyword
service.address
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).
keyword
service.type
The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, service.type would be elasticsearch.
keyword
stan.cluster.id
The cluster ID
keyword
stan.server.id
The server ID
keyword
stan.stats.bytes
Number of bytes consumed across all STAN queues
long
counter
stan.stats.channels
The number of STAN channels
integer
gauge
stan.stats.clients
The number of STAN clients
integer
gauge
stan.stats.messages
Number of messages across all STAN queues
long
counter
stan.stats.role
If clustered, role of this node in the cluster (Leader, Follower, Candidate)
keyword
stan.stats.state
The cluster / streaming configuration state (STANDALONE, CLUSTERED)
keyword
stan.stats.subscriptions
The number of STAN streaming subscriptions
integer
gauge

channels

This is the channels dataset of the STAN package, in charge of retrieving metrics about channels from a STAN instance.

An example event for channels looks as following:

{
    "@timestamp": "2022-01-12T04:11:05.571Z",
    "agent": {
        "ephemeral_id": "02c989d8-8cf2-4e65-bf07-a8e93785fdaa",
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.0.0-beta1"
    },
    "data_stream": {
        "dataset": "stan.channels",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "1.12.0"
    },
    "elastic_agent": {
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "snapshot": false,
        "version": "8.0.0-beta1"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.channels",
        "duration": 22264899,
        "ingested": "2022-01-12T04:11:06Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "4ccba669f0df47fa3f57a9e4169ae7f1",
        "ip": [
            "172.18.0.4"
        ],
        "mac": [
            "02:42:ac:12:00:04"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "Core",
            "family": "redhat",
            "kernel": "5.11.0-44-generic",
            "name": "CentOS Linux",
            "platform": "centos",
            "type": "linux",
            "version": "7 (Core)"
        }
    },
    "metricset": {
        "name": "channels",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
        "type": "stan"
    },
    "stan": {
        "channels": {
            "bytes": 0,
            "depth": 0,
            "first_seq": 0,
            "last_seq": 0,
            "messages": 0,
            "name": "foo"
        },
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "dEvzTKomxEioLU6oP1VuXM"
        }
    }
}

Exported fields

FieldDescriptionTypeMetric Type
@timestamp
Event timestamp.
date
agent.id
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.
keyword
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host, resource, or service is located.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host, resource, or service is located.
keyword
container.id
Unique container id.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
ecs.version
ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.
keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.
keyword
service.address
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).
keyword
service.type
The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, service.type would be elasticsearch.
keyword
stan.channels.bytes
The number of STAN bytes in the channel
long
counter
stan.channels.depth
Queue depth based upon current sequence number and highest reported subscriber sequence number
long
gauge
stan.channels.first_seq
First sequence number stored in the channel. If first_seq > min([seq in subscriptions]) data loss has possibly occurred
long
stan.channels.last_seq
Last sequence number stored in the channel
long
counter
stan.channels.messages
The number of STAN streaming messages
long
counter
stan.channels.name
The name of the STAN streaming channel
keyword
stan.cluster.id
The cluster ID
keyword
stan.server.id
The server ID
keyword

subscriptions

This is the subscriptions dataset of the STAN package, in charge of retrieving metrics about subscriptions from a STAN instance.

An example event for subscriptions looks as following:

{
    "@timestamp": "2022-01-12T04:13:52.133Z",
    "agent": {
        "ephemeral_id": "edb669a5-3b36-43d7-8190-d485d6517f69",
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.0.0-beta1"
    },
    "data_stream": {
        "dataset": "stan.subscriptions",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "1.12.0"
    },
    "elastic_agent": {
        "id": "9878d192-22ad-49b6-a6c2-9959b0815d04",
        "snapshot": false,
        "version": "8.0.0-beta1"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.subscriptions",
        "duration": 6243276,
        "ingested": "2022-01-12T04:13:53Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "4ccba669f0df47fa3f57a9e4169ae7f1",
        "ip": [
            "172.18.0.4"
        ],
        "mac": [
            "02:42:ac:12:00:04"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "Core",
            "family": "redhat",
            "kernel": "5.11.0-44-generic",
            "name": "CentOS Linux",
            "platform": "centos",
            "type": "linux",
            "version": "7 (Core)"
        }
    },
    "metricset": {
        "name": "subscriptions",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
        "type": "stan"
    },
    "stan": {
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "NevWjtY7nB1yzea8TtJaGb"
        },
        "subscriptions": {
            "channel": "foo",
            "id": "benchmark-sub-25",
            "last_sent": 0,
            "offline": false,
            "pending": 0,
            "queue": "T",
            "stalled": false
        }
    }
}

Exported fields

FieldDescriptionTypeMetric Type
@timestamp
Event timestamp.
date
agent.id
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.
keyword
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host, resource, or service is located.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host, resource, or service is located.
keyword
container.id
Unique container id.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
ecs.version
ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.
keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.
keyword
service.address
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).
keyword
service.type
The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, service.type would be elasticsearch.
keyword
stan.cluster.id
The cluster ID
keyword
stan.server.id
The server ID
keyword
stan.subscriptions.channel
The name of the STAN channel the subscription is associated with
keyword
stan.subscriptions.id
The name of the STAN channel subscription (client_id)
keyword
stan.subscriptions.last_sent
Last known sequence number of the subscription that was acked
long
counter
stan.subscriptions.offline
Is the subscriber marked as offline?
boolean
stan.subscriptions.pending
Number of pending messages from / to the subscriber
long
gauge
stan.subscriptions.queue
The name of the NATS queue that the STAN channel subscription is associated with, if any
keyword
stan.subscriptions.stalled
Is the subscriber known to be stalled?
boolean

Changelog

VersionDetailsKibana version(s)

1.5.0

Enhancement View pull request
Enable time series data for metrics data streams. This dramatically reduces storage for metrics and is expected to progressively improve query performance. For more details, see https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html.

8.10.2 or higher

1.4.2

Enhancement View pull request
Migrate Client IP Count Timeline visualization to lens.

8.10.2 or higher

1.4.1

Enhancement View pull request
Add subscriptions data stream metric types mappings.

8.10.2 or higher

1.4.0

Enhancement View pull request
Add subscriptions data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.9

Enhancement View pull request
Add stats data stream metric types mappings.

8.10.2 or higher

1.3.8

Enhancement View pull request
Add stats data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.7

Enhancement View pull request
Add channels data stream metric types mappings.

8.10.2 or higher

1.3.6

Enhancement View pull request
Add channels data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.5

Enhancement View pull request
Migrate Logs Overview dashboard visualizations to lens.

8.10.2 or higher

1.3.4

Enhancement View pull request
Migrate Metrics Overview dashboard visualizations to lens.

8.10.2 or higher

1.3.3

Bug fix View pull request
Fix reference error in Channel Overview dashboard.

7.14.0 or higher
8.0.0 or higher

1.3.2

Bug fix View pull request
Add null check to the rename processor

7.14.0 or higher
8.0.0 or higher

1.3.1

Enhancement View pull request
Added categories and/or subcategories.

7.14.0 or higher
8.0.0 or higher

1.3.0

Enhancement View pull request
Update to ECS 8.0

7.14.0 or higher
8.0.0 or higher

1.2.0

Enhancement View pull request
Release stan package for v8.0.0

7.14.0 or higher
8.0.0 or higher

1.1.2

Enhancement View pull request
Uniform with guidelines

—

1.1.1

Bug fix View pull request
Fix logic that checks for the 'forwarded' tag

—

1.1.0

Enhancement View pull request
Update to ECS 1.12.0

7.14.0 or higher

1.0.0

Enhancement View pull request
Release Stan as GA

—

0.5.3

Enhancement View pull request
Convert to generated ECS fields

—

0.5.2

Enhancement View pull request
update to ECS 1.11.0

—

0.5.1

Enhancement View pull request
Escape special characters in docs

—

0.5.0

Enhancement View pull request
Update integration description

—

0.4.0

Enhancement View pull request
Set "event.module" and "event.dataset"

—

0.3.0

Enhancement View pull request
update to ECS 1.10.0 and adding event.original options

—

0.2.0

Bug fix View pull request
Fix stack compatability

—

0.1.3

Enhancement View pull request
update to ECS 1.9.0

—

0.1.2

Bug fix View pull request
Change kibana.version constraint to be more conservative.

—

0.1.0

Enhancement View pull request
initial release

—

On this page