Network Packet Capture
Capture and analyze network traffic from a host with Elastic Agent.
Version |
1.30.0 (View all) |
Compatible Kibana version(s) |
8.6.2 or higher |
Supported Serverless project types |
Security Observability |
Subscription level |
Basic |
Level of support |
Elastic |
This integration sniffs network packets on a host and dissects known protocols.
Monitoring your network traffic is critical to gaining observability and securing your environment — ensuring high levels of performance and security. The Network Packet Capture integration captures the network traffic between your application servers, decodes common application layer protocols and records the interesting fields for each transaction.
Supported Protocols
Currently, Network Packet Capture supports the following protocols:
- ICMP (v4 and v6)
- DHCP (v4)
- DNS
- HTTP
- AMQP 0.9.1
- Cassandra
- Mysql
- PostgreSQL
- Redis
- Thrift-RPC
- MongoDB
- Memcache
- NFS
- TLS
- SIP/SDP (beta)
Common protocol options
The following options are available for all protocols:
map_to_ecs
Remap any non-ECS Packetbeat fields in root to their correct ECS fields. This will rename fields that are moved so the fields will not be present at the root of the document and so any rules that depend on the fields will need to be updated.
The legacy behaviour of this option is to not remap to ECS. This behaviour is still the default, but is deprecated and users are encouraged to set this option to true.
ECS remapping may have an impact on workflows that depend on the identity
of non-ECS fields, and users should assess their use of these fields before
making the change. Users who need to retain data collected with the legacy
mappings may need to re-index their older documents. Instructions for doing
this are available here.
The pipeline used to perform ECS remapping for each data stream can be found
in Stack Management›Ingest Pipelines and and searching for
"logs-network_traffic compatibility".
The deprecation and retirement timeline for legacy behavior is available here.
enabled
The enabled setting is a boolean setting to enable or disable protocols without having to comment out configuration sections. If set to false, the protocol is disabled.
The default value is true.
ports
Exception: For ICMP the option enabled has to be used instead.
The ports where Network Packet Capture will look to capture traffic for specific protocols. Network Packet Capture installs a BPF filter based on the ports specified in this section. If a packet doesn’t match the filter, very little CPU is required to discard the packet. Network Packet Capture also uses the ports specified here to determine which parser to use for each packet.
monitor_processes
If this option is enabled then network traffic events will be enriched with information about the process associated with the events.
The default value is false.
send_request
If this option is enabled, the raw message of the request (request
field) is sent to Elasticsearch. The default is false. This option is
useful when you want to index the whole request. Note that for HTTP, the
body is not included by default, only the HTTP headers.
send_response
If this option is enabled, the raw message of the response (response
field) is sent to Elasticsearch. The default is false. This option is
useful when you want to index the whole response. Note that for HTTP,
the body is not included by default, only the HTTP headers.
transaction_timeout
The per protocol transaction timeout. Expired transactions will no longer be correlated to incoming responses, but sent to Elasticsearch immediately.
tags
A list of tags that will be sent with the transaction event. This setting is optional.
processors
A list of processors to apply to the data generated by the protocol.
keep_null
If this option is set to true, fields with null values will be
published in the output document. By default, keep_null is set to
false.
Network Flows
Overall flow information about the network connections on a host.
You can configure Network Packet Capture to collect and report statistics on network flows. A flow is a group of packets sent over the same time period that share common properties, such as the same source and destination address and protocol. You can use this feature to analyze network traffic over specific protocols on your network.
For each flow, Network Packet Capture reports the number of packets and the total number of bytes sent from the source to the destination. Each flow event also contains information about the source and destination hosts, such as their IP address. For bi-directional flows, Network Packet Capture reports statistics for the reverse flow.
Network Packet Capture collects and reports statistics up to and including the transport layer.
Configuration options
You can specify the following options for capturing flows.
enabled
Enables flows support if set to true. Set to false to disable network flows support without having to delete or comment out the flows section. The default value is true.
timeout
Timeout configures the lifetime of a flow. If no packets have been received for a flow within the timeout time window, the flow is killed and reported. The default value is 30s.
period
Configure the reporting interval. All flows are reported at the very same point in time. Periodical reporting can be disabled by setting the value to -1. If disabled, flows are still reported once being timed out. The default value is 10s.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.mac | MAC address of the destination. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
destination.packets | Packets sent from the destination to the source. | long |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.packets | Total packets transferred in both directions. If source.packets and destination.packets are known, network.packets is their sum. | long |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
network_traffic.flow.id | Internal flow ID based on connection meta data and address. | keyword |
network_traffic.flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.mac | MAC address of the source. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
source.packets | Packets sent from the source to the destination. | long |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for flow looks as following:
{
"@timestamp": "2023-10-16T22:40:20.005Z",
"agent": {
"ephemeral_id": "005dde79-7459-4b47-ae00-972086b4f5db",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"data_stream": {
"dataset": "network_traffic.flow",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 64,
"ip": "::1",
"packets": 1,
"port": 8000
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"action": "network_flow",
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.flow",
"duration": 73561,
"end": "2023-10-16T22:39:45.677Z",
"ingested": "2023-10-16T22:40:21Z",
"kind": "event",
"start": "2023-10-16T22:39:45.677Z",
"type": [
"connection",
"end"
]
},
"flow": {
"final": true,
"id": "QAT///////8A////IP8AAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAUAfeMg"
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"network": {
"bytes": 152,
"community_id": "1:5y9AkdbV9U8xqD9dhlj6obkubHg=",
"packets": 2,
"transport": "tcp",
"type": "ipv6"
},
"source": {
"bytes": 88,
"ip": "::1",
"packets": 1,
"port": 51320
},
"type": "flow"
}
Protocols
AMQP
Configuration options
Also see Common protocol options.
max_body_length
The maximum size in bytes of the message displayed in the request or
response fields. Messages that are bigger than the specified size are
truncated. Use this option to avoid publishing huge messages when
send_request or
send_response is enabled. The default is
1000 bytes.
parse_headers
If set to true, Network Packet Capture parses the additional arguments specified in the headers field of a message. Those arguments are key-value pairs that specify information such as the content type of the message or the message priority. The default is true.
parse_arguments
If set to true, Network Packet Capture parses the additional arguments specified in AMQP methods. Those arguments are key-value pairs specified by the user and can be of any length. The default is true.
hide_connection_information
If set to false, the connection layer methods of the protocol are also displayed, such as the opening and closing of connections and channels by clients, or the quality of service negotiation. The default is true.
Fields published for AMQP packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
amqp.app-id | Creating application id. | keyword |
amqp.arguments | Optional additional arguments passed to some methods. Can be of various types. | flattened |
amqp.auto-delete | If set, auto-delete queue when unused. | boolean |
amqp.class-id | Failing method class. | long |
amqp.consumer-count | The number of consumers of a queue. | long |
amqp.consumer-tag | Identifier for the consumer, valid within the current channel. | keyword |
amqp.content-encoding | MIME content encoding. | keyword |
amqp.content-type | MIME content type. | keyword |
amqp.correlation-id | Application correlation identifier. | keyword |
amqp.delivery-mode | Non-persistent (1) or persistent (2). | keyword |
amqp.delivery-tag | The server-assigned and channel-specific delivery tag. | long |
amqp.durable | If set, request a durable exchange/queue. | boolean |
amqp.exchange | Name of the exchange. | keyword |
amqp.exchange-type | Exchange type. | keyword |
amqp.exclusive | If set, request an exclusive queue. | boolean |
amqp.expiration | Message expiration specification. | keyword |
amqp.headers | Message header field table. | object |
amqp.if-empty | Delete only if empty. | boolean |
amqp.if-unused | Delete only if unused. | boolean |
amqp.immediate | Request immediate delivery. | boolean |
amqp.mandatory | Indicates mandatory routing. | boolean |
amqp.message-count | The number of messages in the queue, which will be zero for newly-declared queues. | long |
amqp.message-id | Application message identifier. | keyword |
amqp.method-id | Failing method ID. | long |
amqp.multiple | Acknowledge multiple messages. | boolean |
amqp.no-ack | If set, the server does not expect acknowledgements for messages. | boolean |
amqp.no-local | If set, the server will not send messages to the connection that published them. | boolean |
amqp.no-wait | If set, the server will not respond to the method. | boolean |
amqp.passive | If set, do not create exchange/queue. | boolean |
amqp.priority | Message priority, 0 to 9. | long |
amqp.queue | The queue name identifies the queue within the vhost. | keyword |
amqp.redelivered | Indicates that the message has been previously delivered to this or another client. | boolean |
amqp.reply-code | AMQP reply code to an error, similar to http reply-code | long |
amqp.reply-text | Text explaining the error. | keyword |
amqp.reply-to | Address to reply to. | keyword |
amqp.routing-key | Message routing key. | keyword |
amqp.timestamp | Message timestamp. | keyword |
amqp.type | Message type name. | keyword |
amqp.user-id | Creating user id. | keyword |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.action | The action captured by the event. This describes the information in the event. It is more specific than event.category. Examples are group-add, process-started, file-created. The value is normally defined by the implementer. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.amqp.app-id | Creating application id. | keyword |
network_traffic.amqp.arguments | Optional additional arguments passed to some methods. Can be of various types. | flattened |
network_traffic.amqp.auto-delete | If set, auto-delete queue when unused. | boolean |
network_traffic.amqp.class-id | Failing method class. | long |
network_traffic.amqp.consumer-count | The number of consumers of a queue. | long |
network_traffic.amqp.consumer-tag | Identifier for the consumer, valid within the current channel. | keyword |
network_traffic.amqp.content-encoding | MIME content encoding. | keyword |
network_traffic.amqp.content-type | MIME content type. | keyword |
network_traffic.amqp.correlation-id | Application correlation identifier. | keyword |
network_traffic.amqp.delivery-mode | Non-persistent (1) or persistent (2). | keyword |
network_traffic.amqp.delivery-tag | The server-assigned and channel-specific delivery tag. | long |
network_traffic.amqp.durable | If set, request a durable exchange/queue. | boolean |
network_traffic.amqp.exchange | Name of the exchange. | keyword |
network_traffic.amqp.exchange-type | Exchange type. | keyword |
network_traffic.amqp.exclusive | If set, request an exclusive queue. | boolean |
network_traffic.amqp.expiration | Message expiration specification. | keyword |
network_traffic.amqp.headers | Message header field table. | object |
network_traffic.amqp.if-empty | Delete only if empty. | boolean |
network_traffic.amqp.if-unused | Delete only if unused. | boolean |
network_traffic.amqp.immediate | Request immediate delivery. | boolean |
network_traffic.amqp.mandatory | Indicates mandatory routing. | boolean |
network_traffic.amqp.message-count | The number of messages in the queue, which will be zero for newly-declared queues. | long |
network_traffic.amqp.message-id | Application message identifier. | keyword |
network_traffic.amqp.method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network_traffic.amqp.method-id | Failing method ID. | long |
network_traffic.amqp.multiple | Acknowledge multiple messages. | boolean |
network_traffic.amqp.no-ack | If set, the server does not expect acknowledgements for messages. | boolean |
network_traffic.amqp.no-local | If set, the server will not send messages to the connection that published them. | boolean |
network_traffic.amqp.no-wait | If set, the server will not respond to the method. | boolean |
network_traffic.amqp.passive | If set, do not create exchange/queue. | boolean |
network_traffic.amqp.priority | Message priority, 0 to 9. | long |
network_traffic.amqp.queue | The queue name identifies the queue within the vhost. | keyword |
network_traffic.amqp.redelivered | Indicates that the message has been previously delivered to this or another client. | boolean |
network_traffic.amqp.reply-code | AMQP reply code to an error, similar to http reply-code | long |
network_traffic.amqp.reply-text | Text explaining the error. | keyword |
network_traffic.amqp.reply-to | Address to reply to. | keyword |
network_traffic.amqp.routing-key | Message routing key. | keyword |
network_traffic.amqp.timestamp | Message timestamp. | keyword |
network_traffic.amqp.type | Message type name. | keyword |
network_traffic.amqp.user-id | Creating user id. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for amqp looks as following:
{
"@timestamp": "2023-10-16T22:25:39.072Z",
"agent": {
"ephemeral_id": "0749f3ad-7bc9-4e3a-9ffc-90eaefc86763",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"amqp": {
"auto-delete": false,
"consumer-count": 0,
"durable": false,
"exclusive": false,
"message-count": 0,
"no-wait": false,
"passive": false,
"queue": "hello"
},
"client": {
"bytes": 25,
"ip": "127.0.0.1",
"port": 34222
},
"data_stream": {
"dataset": "network_traffic.amqp",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 26,
"ip": "127.0.0.1",
"port": 5672
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"action": "amqp.queue.declare",
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.amqp",
"duration": 1265764,
"end": "2023-10-16T22:25:39.073Z",
"ingested": "2023-10-16T22:25:40Z",
"kind": "event",
"start": "2023-10-16T22:25:39.072Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "queue.declare",
"network": {
"bytes": 51,
"community_id": "1:i6J4zz0FGnZMYLIy8kabND2W/XE=",
"direction": "ingress",
"protocol": "amqp",
"transport": "tcp",
"type": "ipv4"
},
"related": {
"ip": [
"127.0.0.1"
]
},
"server": {
"bytes": 26,
"ip": "127.0.0.1",
"port": 5672
},
"source": {
"bytes": 25,
"ip": "127.0.0.1",
"port": 34222
},
"status": "OK",
"type": "amqp"
}
Cassandra
Configuration options
Also see Common protocol options.
send_request_header
If this option is enabled, the raw message of the response
(cassandra_request.request_headers field) is sent to Elasticsearch.
The default is true. Enable send_request first before enabling this
option.
send_response_header
If this option is enabled, the raw message of the response
(cassandra_response.response_headers field) is included in published
events. The default is true. enable send_response first before enable
this option.
ignored_ops
This option indicates which Operator/Operators captured will be ignored.
currently support: ERROR ,STARTUP ,READY ,AUTHENTICATE
,OPTIONS ,SUPPORTED , QUERY ,RESULT ,PREPARE ,EXECUTE
,REGISTER ,EVENT , BATCH ,AUTH_CHALLENGE,AUTH_RESPONSE
,AUTH_SUCCESS .
compressor
Configures the default compression algorithm being used to uncompress
compressed frames by name. Currently only snappy is can be configured.
By default no compressor is configured.
Fields published for Apache Cassandra packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
cassandra.no_request | Indicates that there is no request because this is a PUSH message. | boolean |
cassandra.request.headers.flags | Flags applying to this frame. | keyword |
cassandra.request.headers.length | A integer representing the length of the body of the frame (a frame is limited to 256MB in length). | long |
cassandra.request.headers.op | An operation type that distinguishes the actual message. | keyword |
cassandra.request.headers.stream | A frame has a stream id. If a client sends a request message with the stream id X, it is guaranteed that the stream id of the response to that message will be X. | keyword |
cassandra.request.headers.version | The version of the protocol. | keyword |
cassandra.request.query | The CQL query which client send to cassandra. | keyword |
cassandra.response.authentication.class | Indicates the full class name of the IAuthenticator in use | keyword |
cassandra.response.error.code | The error code of the Cassandra response. | long |
cassandra.response.error.details.alive | Representing the number of replicas that were known to be alive when the request had been processed (since an unavailable exception has been triggered). | long |
cassandra.response.error.details.arg_types | One string for each argument type (as CQL type) of the failed function. | keyword |
cassandra.response.error.details.blockfor | Representing the number of replicas whose acknowledgement is required to achieve consistency level. | long |
cassandra.response.error.details.data_present | It means the replica that was asked for data had responded. | boolean |
cassandra.response.error.details.function | The name of the failed function. | keyword |
cassandra.response.error.details.keyspace | The keyspace of the failed function. | keyword |
cassandra.response.error.details.num_failures | Representing the number of nodes that experience a failure while executing the request. | keyword |
cassandra.response.error.details.read_consistency | Representing the consistency level of the query that triggered the exception. | keyword |
cassandra.response.error.details.received | Representing the number of nodes having acknowledged the request. | long |
cassandra.response.error.details.required | Representing the number of nodes that should be alive to respect consistency level. | long |
cassandra.response.error.details.stmt_id | Representing the unknown ID. | keyword |
cassandra.response.error.details.table | The keyspace of the failed function. | keyword |
cassandra.response.error.details.write_type | Describe the type of the write that timed out. | keyword |
cassandra.response.error.msg | The error message of the Cassandra response. | keyword |
cassandra.response.error.type | The error type of the Cassandra response. | keyword |
cassandra.response.event.change | The message corresponding respectively to the type of change followed by the address of the new/removed node. | keyword |
cassandra.response.event.host | Representing the node ip. | keyword |
cassandra.response.event.port | Representing the node port. | long |
cassandra.response.event.schema_change.args | One string for each argument type (as CQL type). | keyword |
cassandra.response.event.schema_change.change | Representing the type of changed involved. | keyword |
cassandra.response.event.schema_change.keyspace | This describes which keyspace has changed. | keyword |
cassandra.response.event.schema_change.name | The function/aggregate name. | keyword |
cassandra.response.event.schema_change.object | This describes the name of said affected object (either the table, user type, function, or aggregate name). | keyword |
cassandra.response.event.schema_change.table | This describes which table has changed. | keyword |
cassandra.response.event.schema_change.target | Target could be "FUNCTION" or "AGGREGATE", multiple arguments. | keyword |
cassandra.response.event.type | Representing the event type. | keyword |
cassandra.response.headers.flags | Flags applying to this frame. | keyword |
cassandra.response.headers.length | A integer representing the length of the body of the frame (a frame is limited to 256MB in length). | long |
cassandra.response.headers.op | An operation type that distinguishes the actual message. | keyword |
cassandra.response.headers.stream | A frame has a stream id. If a client sends a request message with the stream id X, it is guaranteed that the stream id of the response to that message will be X. | keyword |
cassandra.response.headers.version | The version of the protocol. | keyword |
cassandra.response.result.keyspace | Indicating the name of the keyspace that has been set. | keyword |
cassandra.response.result.prepared.prepared_id | Representing the prepared query ID. | keyword |
cassandra.response.result.prepared.req_meta.col_count | Representing the number of columns selected by the query that produced this result. | long |
cassandra.response.result.prepared.req_meta.flags | Provides information on the formatting of the remaining information. | keyword |
cassandra.response.result.prepared.req_meta.keyspace | Only present after set Global_tables_spec, the keyspace name. | keyword |
cassandra.response.result.prepared.req_meta.paging_state | The paging_state is a bytes value that should be used in QUERY/EXECUTE to continue paging and retrieve the remainder of the result for this query. | keyword |
cassandra.response.result.prepared.req_meta.pkey_columns | Representing the PK columns index and counts. | long |
cassandra.response.result.prepared.req_meta.table | Only present after set Global_tables_spec, the table name. | keyword |
cassandra.response.result.prepared.resp_meta.col_count | Representing the number of columns selected by the query that produced this result. | long |
cassandra.response.result.prepared.resp_meta.flags | Provides information on the formatting of the remaining information. | keyword |
cassandra.response.result.prepared.resp_meta.keyspace | Only present after set Global_tables_spec, the keyspace name. | keyword |
cassandra.response.result.prepared.resp_meta.paging_state | The paging_state is a bytes value that should be used in QUERY/EXECUTE to continue paging and retrieve the remainder of the result for this query. | keyword |
cassandra.response.result.prepared.resp_meta.pkey_columns | Representing the PK columns index and counts. | long |
cassandra.response.result.prepared.resp_meta.table | Only present after set Global_tables_spec, the table name. | keyword |
cassandra.response.result.rows.meta.col_count | Representing the number of columns selected by the query that produced this result. | long |
cassandra.response.result.rows.meta.flags | Provides information on the formatting of the remaining information. | keyword |
cassandra.response.result.rows.meta.keyspace | Only present after set Global_tables_spec, the keyspace name. | keyword |
cassandra.response.result.rows.meta.paging_state | The paging_state is a bytes value that should be used in QUERY/EXECUTE to continue paging and retrieve the remainder of the result for this query. | keyword |
cassandra.response.result.rows.meta.pkey_columns | Representing the PK columns index and counts. | long |
cassandra.response.result.rows.meta.table | Only present after set Global_tables_spec, the table name. | keyword |
cassandra.response.result.rows.num_rows | Representing the number of rows present in this result. | long |
cassandra.response.result.schema_change.args | One string for each argument type (as CQL type). | keyword |
cassandra.response.result.schema_change.change | Representing the type of changed involved. | keyword |
cassandra.response.result.schema_change.keyspace | This describes which keyspace has changed. | keyword |
cassandra.response.result.schema_change.name | The function/aggregate name. | keyword |
cassandra.response.result.schema_change.object | This describes the name of said affected object (either the table, user type, function, or aggregate name). | keyword |
cassandra.response.result.schema_change.table | This describes which table has changed. | keyword |
cassandra.response.result.schema_change.target | Target could be "FUNCTION" or "AGGREGATE", multiple arguments. | keyword |
cassandra.response.result.type | Cassandra result type. | keyword |
cassandra.response.supported | Indicates which startup options are supported by the server. This message comes as a response to an OPTIONS message. | flattened |
cassandra.response.warnings | The text of the warnings, only occur when Warning flag was set. | keyword |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.cassandra.no_request | Indicates that there is no request because this is a PUSH message. | boolean |
network_traffic.cassandra.request.headers.flags | Flags applying to this frame. | keyword |
network_traffic.cassandra.request.headers.length | A integer representing the length of the body of the frame (a frame is limited to 256MB in length). | long |
network_traffic.cassandra.request.headers.op | An operation type that distinguishes the actual message. | keyword |
network_traffic.cassandra.request.headers.stream | A frame has a stream id. If a client sends a request message with the stream id X, it is guaranteed that the stream id of the response to that message will be X. | keyword |
network_traffic.cassandra.request.headers.version | The version of the protocol. | keyword |
network_traffic.cassandra.request.query | The CQL query which client send to cassandra. | keyword |
network_traffic.cassandra.response.authentication.class | Indicates the full class name of the IAuthenticator in use | keyword |
network_traffic.cassandra.response.error.code | The error code of the Cassandra response. | long |
network_traffic.cassandra.response.error.details.alive | Representing the number of replicas that were known to be alive when the request had been processed (since an unavailable exception has been triggered). | long |
network_traffic.cassandra.response.error.details.arg_types | One string for each argument type (as CQL type) of the failed function. | keyword |
network_traffic.cassandra.response.error.details.blockfor | Representing the number of replicas whose acknowledgement is required to achieve consistency level. | long |
network_traffic.cassandra.response.error.details.data_present | It means the replica that was asked for data had responded. | boolean |
network_traffic.cassandra.response.error.details.function | The name of the failed function. | keyword |
network_traffic.cassandra.response.error.details.keyspace | The keyspace of the failed function. | keyword |
network_traffic.cassandra.response.error.details.num_failures | Representing the number of nodes that experience a failure while executing the request. | keyword |
network_traffic.cassandra.response.error.details.read_consistency | Representing the consistency level of the query that triggered the exception. | keyword |
network_traffic.cassandra.response.error.details.received | Representing the number of nodes having acknowledged the request. | long |
network_traffic.cassandra.response.error.details.required | Representing the number of nodes that should be alive to respect consistency level. | long |
network_traffic.cassandra.response.error.details.stmt_id | Representing the unknown ID. | keyword |
network_traffic.cassandra.response.error.details.table | The keyspace of the failed function. | keyword |
network_traffic.cassandra.response.error.details.write_type | Describe the type of the write that timed out. | keyword |
network_traffic.cassandra.response.error.msg | The error message of the Cassandra response. | keyword |
network_traffic.cassandra.response.error.type | The error type of the Cassandra response. | keyword |
network_traffic.cassandra.response.event.change | The message corresponding respectively to the type of change followed by the address of the new/removed node. | keyword |
network_traffic.cassandra.response.event.host | Representing the node ip. | keyword |
network_traffic.cassandra.response.event.port | Representing the node port. | long |
network_traffic.cassandra.response.event.schema_change.args | One string for each argument type (as CQL type). | keyword |
network_traffic.cassandra.response.event.schema_change.change | Representing the type of changed involved. | keyword |
network_traffic.cassandra.response.event.schema_change.keyspace | This describes which keyspace has changed. | keyword |
network_traffic.cassandra.response.event.schema_change.name | The function/aggregate name. | keyword |
network_traffic.cassandra.response.event.schema_change.object | This describes the name of said affected object (either the table, user type, function, or aggregate name). | keyword |
network_traffic.cassandra.response.event.schema_change.table | This describes which table has changed. | keyword |
network_traffic.cassandra.response.event.schema_change.target | Target could be "FUNCTION" or "AGGREGATE", multiple arguments. | keyword |
network_traffic.cassandra.response.event.type | Representing the event type. | keyword |
network_traffic.cassandra.response.headers.flags | Flags applying to this frame. | keyword |
network_traffic.cassandra.response.headers.length | A integer representing the length of the body of the frame (a frame is limited to 256MB in length). | long |
network_traffic.cassandra.response.headers.op | An operation type that distinguishes the actual message. | keyword |
network_traffic.cassandra.response.headers.stream | A frame has a stream id. If a client sends a request message with the stream id X, it is guaranteed that the stream id of the response to that message will be X. | keyword |
network_traffic.cassandra.response.headers.version | The version of the protocol. | keyword |
network_traffic.cassandra.response.result.keyspace | Indicating the name of the keyspace that has been set. | keyword |
network_traffic.cassandra.response.result.prepared.prepared_id | Representing the prepared query ID. | keyword |
network_traffic.cassandra.response.result.prepared.req_meta.col_count | Representing the number of columns selected by the query that produced this result. | long |
network_traffic.cassandra.response.result.prepared.req_meta.flags | Provides information on the formatting of the remaining information. | keyword |
network_traffic.cassandra.response.result.prepared.req_meta.keyspace | Only present after set Global_tables_spec, the keyspace name. | keyword |
network_traffic.cassandra.response.result.prepared.req_meta.paging_state | The paging_state is a bytes value that should be used in QUERY/EXECUTE to continue paging and retrieve the remainder of the result for this query. | keyword |
network_traffic.cassandra.response.result.prepared.req_meta.pkey_columns | Representing the PK columns index and counts. | long |
network_traffic.cassandra.response.result.prepared.req_meta.table | Only present after set Global_tables_spec, the table name. | keyword |
network_traffic.cassandra.response.result.prepared.resp_meta.col_count | Representing the number of columns selected by the query that produced this result. | long |
network_traffic.cassandra.response.result.prepared.resp_meta.flags | Provides information on the formatting of the remaining information. | keyword |
network_traffic.cassandra.response.result.prepared.resp_meta.keyspace | Only present after set Global_tables_spec, the keyspace name. | keyword |
network_traffic.cassandra.response.result.prepared.resp_meta.paging_state | The paging_state is a bytes value that should be used in QUERY/EXECUTE to continue paging and retrieve the remainder of the result for this query. | keyword |
network_traffic.cassandra.response.result.prepared.resp_meta.pkey_columns | Representing the PK columns index and counts. | long |
network_traffic.cassandra.response.result.prepared.resp_meta.table | Only present after set Global_tables_spec, the table name. | keyword |
network_traffic.cassandra.response.result.rows.meta.col_count | Representing the number of columns selected by the query that produced this result. | long |
network_traffic.cassandra.response.result.rows.meta.flags | Provides information on the formatting of the remaining information. | keyword |
network_traffic.cassandra.response.result.rows.meta.keyspace | Only present after set Global_tables_spec, the keyspace name. | keyword |
network_traffic.cassandra.response.result.rows.meta.paging_state | The paging_state is a bytes value that should be used in QUERY/EXECUTE to continue paging and retrieve the remainder of the result for this query. | keyword |
network_traffic.cassandra.response.result.rows.meta.pkey_columns | Representing the PK columns index and counts. | long |
network_traffic.cassandra.response.result.rows.meta.table | Only present after set Global_tables_spec, the table name. | keyword |
network_traffic.cassandra.response.result.rows.num_rows | Representing the number of rows present in this result. | long |
network_traffic.cassandra.response.result.schema_change.args | One string for each argument type (as CQL type). | keyword |
network_traffic.cassandra.response.result.schema_change.change | Representing the type of changed involved. | keyword |
network_traffic.cassandra.response.result.schema_change.keyspace | This describes which keyspace has changed. | keyword |
network_traffic.cassandra.response.result.schema_change.name | The function/aggregate name. | keyword |
network_traffic.cassandra.response.result.schema_change.object | This describes the name of said affected object (either the table, user type, function, or aggregate name). | keyword |
network_traffic.cassandra.response.result.schema_change.table | This describes which table has changed. | keyword |
network_traffic.cassandra.response.result.schema_change.target | Target could be "FUNCTION" or "AGGREGATE", multiple arguments. | keyword |
network_traffic.cassandra.response.result.type | Cassandra result type. | keyword |
network_traffic.cassandra.response.supported | Indicates which startup options are supported by the server. This message comes as a response to an OPTIONS message. | flattened |
network_traffic.cassandra.response.warnings | The text of the warnings, only occur when Warning flag was set. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for cassandra looks as following:
{
"@timestamp": "2023-10-16T22:31:00.694Z",
"agent": {
"ephemeral_id": "c013fddf-67ee-4638-8676-393fc70318cc",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"cassandra": {
"request": {
"headers": {
"flags": "Default",
"length": 98,
"op": "QUERY",
"stream": 49,
"version": "4"
},
"query": "CREATE TABLE users (\n user_id int PRIMARY KEY,\n fname text,\n lname text\n);"
},
"response": {
"headers": {
"flags": "Default",
"length": 39,
"op": "RESULT",
"stream": 49,
"version": "4"
},
"result": {
"schema_change": {
"change": "CREATED",
"keyspace": "mykeyspace",
"object": "users",
"target": "TABLE"
},
"type": "schemaChanged"
}
}
},
"client": {
"bytes": 107,
"ip": "127.0.0.1",
"port": 52749
},
"data_stream": {
"dataset": "network_traffic.cassandra",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 48,
"ip": "127.0.0.1",
"port": 9042
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.cassandra",
"duration": 131789052,
"end": "2023-10-16T22:31:00.826Z",
"ingested": "2023-10-16T22:31:04Z",
"kind": "event",
"start": "2023-10-16T22:31:00.694Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"network": {
"bytes": 155,
"community_id": "1:bCORHZnGIk6GWYaE3Kn0DOpQCKE=",
"direction": "ingress",
"protocol": "cassandra",
"transport": "tcp",
"type": "ipv4"
},
"related": {
"ip": [
"127.0.0.1"
]
},
"server": {
"bytes": 48,
"ip": "127.0.0.1",
"port": 9042
},
"source": {
"bytes": 107,
"ip": "127.0.0.1",
"port": 52749
},
"status": "OK",
"type": "cassandra"
}
DHCP
Configuration options
Fields published for DHCPv4 packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
dhcpv4.assigned_ip | The IP address that the DHCP server is assigning to the client. This field is also known as "your" IP address. | ip |
dhcpv4.client_ip | The current IP address of the client. | ip |
dhcpv4.client_mac | The client's MAC address (layer two). | keyword |
dhcpv4.flags | Flags are set by the client to indicate how the DHCP server should its reply -- either unicast or broadcast. | keyword |
dhcpv4.hardware_type | The type of hardware used for the local network (Ethernet, LocalTalk, etc). | keyword |
dhcpv4.hops | The number of hops the DHCP message went through. | long |
dhcpv4.op_code | The message op code (bootrequest or bootreply). | keyword |
dhcpv4.option.boot_file_name | This option is used to identify a bootfile when the 'file' field in the DHCP header has been used for DHCP options. | keyword |
dhcpv4.option.broadcast_address | This option specifies the broadcast address in use on the client's subnet. | ip |
dhcpv4.option.class_identifier | This option is used by DHCP clients to optionally identify the vendor type and configuration of a DHCP client. Vendors may choose to define specific vendor class identifiers to convey particular configuration or other identification information about a client. For example, the identifier may encode the client's hardware configuration. | keyword |
dhcpv4.option.dns_servers | The domain name server option specifies a list of Domain Name System servers available to the client. | ip |
dhcpv4.option.domain_name | This option specifies the domain name that client should use when resolving hostnames via the Domain Name System. | keyword |
dhcpv4.option.hostname | This option specifies the name of the client. | keyword |
dhcpv4.option.ip_address_lease_time_sec | This option is used in a client request (DHCPDISCOVER or DHCPREQUEST) to allow the client to request a lease time for the IP address. In a server reply (DHCPOFFER), a DHCP server uses this option to specify the lease time it is willing to offer. | long |
dhcpv4.option.max_dhcp_message_size | This option specifies the maximum length DHCP message that the client is willing to accept. | long |
dhcpv4.option.message | This option is used by a DHCP server to provide an error message to a DHCP client in a DHCPNAK message in the event of a failure. A client may use this option in a DHCPDECLINE message to indicate the why the client declined the offered parameters. | text |
dhcpv4.option.message_type | The specific type of DHCP message being sent (e.g. discover, offer, request, decline, ack, nak, release, inform). | keyword |
dhcpv4.option.ntp_servers | This option specifies a list of IP addresses indicating NTP servers available to the client. | ip |
dhcpv4.option.parameter_request_list | This option is used by a DHCP client to request values for specified configuration parameters. | keyword |
dhcpv4.option.rebinding_time_sec | This option specifies the time interval from address assignment until the client transitions to the REBINDING state. | long |
dhcpv4.option.renewal_time_sec | This option specifies the time interval from address assignment until the client transitions to the RENEWING state. | long |
dhcpv4.option.requested_ip_address | This option is used in a client request (DHCPDISCOVER) to allow the client to request that a particular IP address be assigned. | ip |
dhcpv4.option.router | The router option specifies a list of IP addresses for routers on the client's subnet. | ip |
dhcpv4.option.server_identifier | IP address of the individual DHCP server which handled this message. | ip |
dhcpv4.option.subnet_mask | The subnet mask that the client should use on the currnet network. | ip |
dhcpv4.option.time_servers | The time server option specifies a list of RFC 868 time servers available to the client. | ip |
dhcpv4.option.utc_time_offset_sec | The time offset field specifies the offset of the client's subnet in seconds from Coordinated Universal Time (UTC). | long |
dhcpv4.option.vendor_identifying_options.data | Additional vendor data, encoded in hexadecimal format. | keyword |
dhcpv4.option.vendor_identifying_options.id | Device identifier. | keyword |
dhcpv4.relay_ip | The relay IP address used by the client to contact the server (i.e. a DHCP relay server). | ip |
dhcpv4.seconds | Number of seconds elapsed since client began address acquisition or renewal process. | long |
dhcpv4.server_ip | The IP address of the DHCP server that the client should use for the next step in the bootstrap process. | ip |
dhcpv4.server_name | The name of the server sending the message. Optional. Used in DHCPOFFER or DHCPACK messages. | keyword |
dhcpv4.transaction_id | Transaction ID, a random number chosen by the client, used by the client and server to associate messages and responses between a client and a server. | keyword |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.dhcpv4.assigned_ip | The IP address that the DHCP server is assigning to the client. This field is also known as "your" IP address. | ip |
network_traffic.dhcpv4.client_ip | The current IP address of the client. | ip |
network_traffic.dhcpv4.client_mac | The client's MAC address (layer two). | keyword |
network_traffic.dhcpv4.flags | Flags are set by the client to indicate how the DHCP server should its reply -- either unicast or broadcast. | keyword |
network_traffic.dhcpv4.hardware_type | The type of hardware used for the local network (Ethernet, LocalTalk, etc). | keyword |
network_traffic.dhcpv4.hops | The number of hops the DHCP message went through. | long |
network_traffic.dhcpv4.op_code | The message op code (bootrequest or bootreply). | keyword |
network_traffic.dhcpv4.option.boot_file_name | This option is used to identify a bootfile when the 'file' field in the DHCP header has been used for DHCP options. | keyword |
network_traffic.dhcpv4.option.broadcast_address | This option specifies the broadcast address in use on the client's subnet. | ip |
network_traffic.dhcpv4.option.class_identifier | This option is used by DHCP clients to optionally identify the vendor type and configuration of a DHCP client. Vendors may choose to define specific vendor class identifiers to convey particular configuration or other identification information about a client. For example, the identifier may encode the client's hardware configuration. | keyword |
network_traffic.dhcpv4.option.dns_servers | The domain name server option specifies a list of Domain Name System servers available to the client. | ip |
network_traffic.dhcpv4.option.domain_name | This option specifies the domain name that client should use when resolving hostnames via the Domain Name System. | keyword |
network_traffic.dhcpv4.option.hostname | This option specifies the name of the client. | keyword |
network_traffic.dhcpv4.option.ip_address_lease_time_sec | This option is used in a client request (DHCPDISCOVER or DHCPREQUEST) to allow the client to request a lease time for the IP address. In a server reply (DHCPOFFER), a DHCP server uses this option to specify the lease time it is willing to offer. | long |
network_traffic.dhcpv4.option.max_dhcp_message_size | This option specifies the maximum length DHCP message that the client is willing to accept. | long |
network_traffic.dhcpv4.option.message | This option is used by a DHCP server to provide an error message to a DHCP client in a DHCPNAK message in the event of a failure. A client may use this option in a DHCPDECLINE message to indicate the why the client declined the offered parameters. | text |
network_traffic.dhcpv4.option.message_type | The specific type of DHCP message being sent (e.g. discover, offer, request, decline, ack, nak, release, inform). | keyword |
network_traffic.dhcpv4.option.ntp_servers | This option specifies a list of IP addresses indicating NTP servers available to the client. | ip |
network_traffic.dhcpv4.option.parameter_request_list | This option is used by a DHCP client to request values for specified configuration parameters. | keyword |
network_traffic.dhcpv4.option.rebinding_time_sec | This option specifies the time interval from address assignment until the client transitions to the REBINDING state. | long |
network_traffic.dhcpv4.option.renewal_time_sec | This option specifies the time interval from address assignment until the client transitions to the RENEWING state. | long |
network_traffic.dhcpv4.option.requested_ip_address | This option is used in a client request (DHCPDISCOVER) to allow the client to request that a particular IP address be assigned. | ip |
network_traffic.dhcpv4.option.router | The router option specifies a list of IP addresses for routers on the client's subnet. | ip |
network_traffic.dhcpv4.option.server_identifier | IP address of the individual DHCP server which handled this message. | ip |
network_traffic.dhcpv4.option.subnet_mask | The subnet mask that the client should use on the currnet network. | ip |
network_traffic.dhcpv4.option.time_servers | The time server option specifies a list of RFC 868 time servers available to the client. | ip |
network_traffic.dhcpv4.option.utc_time_offset_sec | The time offset field specifies the offset of the client's subnet in seconds from Coordinated Universal Time (UTC). | long |
network_traffic.dhcpv4.option.vendor_identifying_options.data | Additional vendor data, encoded in hexadecimal format. | keyword |
network_traffic.dhcpv4.option.vendor_identifying_options.id | Device identifier. | keyword |
network_traffic.dhcpv4.relay_ip | The relay IP address used by the client to contact the server (i.e. a DHCP relay server). | ip |
network_traffic.dhcpv4.seconds | Number of seconds elapsed since client began address acquisition or renewal process. | long |
network_traffic.dhcpv4.server_ip | The IP address of the DHCP server that the client should use for the next step in the bootstrap process. | ip |
network_traffic.dhcpv4.server_name | The name of the server sending the message. Optional. Used in DHCPOFFER or DHCPACK messages. | keyword |
network_traffic.dhcpv4.transaction_id | Transaction ID, a random number chosen by the client, used by the client and server to associate messages and responses between a client and a server. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for dhcpv4 looks as following:
{
"@timestamp": "2023-10-16T22:31:47.648Z",
"agent": {
"ephemeral_id": "a1bdc581-8ac7-4f07-a78a-656bceaa0c91",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 272,
"ip": "0.0.0.0",
"port": 68
},
"data_stream": {
"dataset": "network_traffic.dhcpv4",
"namespace": "ep",
"type": "logs"
},
"destination": {
"ip": "255.255.255.255",
"port": 67
},
"dhcpv4": {
"client_mac": "00-0B-82-01-FC-42",
"flags": "unicast",
"hardware_type": "Ethernet",
"hops": 0,
"op_code": "bootrequest",
"option": {
"message_type": "discover",
"parameter_request_list": [
"Subnet Mask",
"Router",
"Domain Name Server",
"NTP Servers"
],
"requested_ip_address": "0.0.0.0"
},
"seconds": 0,
"transaction_id": "0x00003d1d"
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.dhcpv4",
"ingested": "2023-10-16T22:31:48Z",
"kind": "event",
"start": "2023-10-16T22:31:47.648Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"network": {
"bytes": 272,
"community_id": "1:t9O1j0qj71O4wJM7gnaHtgmfev8=",
"direction": "unknown",
"protocol": "dhcpv4",
"transport": "udp",
"type": "ipv4"
},
"related": {
"ip": [
"0.0.0.0",
"255.255.255.255"
]
},
"server": {
"ip": "255.255.255.255",
"port": 67
},
"source": {
"bytes": 272,
"ip": "0.0.0.0",
"port": 68
},
"status": "OK",
"type": "dhcpv4"
}
DNS
The DNS protocol supports processing DNS messages on TCP and UDP.
Configuration options
Also see Common protocol options.
include_authorities
If this option is enabled, dns.authority fields (authority resource records) are added to DNS events. The default is false.
include_additionals
If this option is enabled, dns.additionals fields (additional resource records) are added to DNS events. The default is false.
Fields published for DNS packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
dns.additionals | An array containing a dictionary for each additional section from the answer. | flattened |
dns.additionals.class | The class of DNS data contained in this resource record. | keyword |
dns.additionals.data | The data describing the resource. The meaning of this data depends on the type and class of the resource record. | keyword |
dns.additionals.name | The domain name to which this resource record pertains. | keyword |
dns.additionals.ttl | The time interval in seconds that this resource record may be cached before it should be discarded. Zero values mean that the data should not be cached. | long |
dns.additionals.type | The type of data contained in this resource record. | keyword |
dns.additionals_count | The number of resource records contained in the dns.additionals field. The dns.additionals field may or may not be included depending on the configuration of Packetbeat. | long |
dns.answers | An array containing an object for each answer section returned by the server. The main keys that should be present in these objects are defined by ECS. Records that have more information may contain more keys than what ECS defines. Not all DNS data sources give all details about DNS answers. At minimum, answer objects must contain the data key. If more information is available, map as much of it to ECS as possible, and add any additional fields to the answer objects as custom fields. | group |
dns.answers.class | The class of DNS data contained in this resource record. | keyword |
dns.answers.data | The data describing the resource. The meaning of this data depends on the type and class of the resource record. | keyword |
dns.answers.name | The domain name to which this resource record pertains. If a chain of CNAME is being resolved, each answer's name should be the one that corresponds with the answer's data. It should not simply be the original question.name repeated. | keyword |
dns.answers.ttl | The time interval in seconds that this resource record may be cached before it should be discarded. Zero values mean that the data should not be cached. | long |
dns.answers.type | The type of data contained in this resource record. | keyword |
dns.answers_count | The number of resource records contained in the dns.answers field. | long |
dns.authorities | An array containing a dictionary for each authority section from the answer. | flattened |
dns.authorities.class | The class of DNS data contained in this resource record. | keyword |
dns.authorities.name | The domain name to which this resource record pertains. | keyword |
dns.authorities.type | The type of data contained in this resource record. | keyword |
dns.authorities_count | The number of resource records contained in the dns.authorities field. The dns.authorities field may or may not be included depending on the configuration of Packetbeat. | long |
dns.flags.authentic_data | A DNS flag specifying that the recursive server considers the response authentic. | boolean |
dns.flags.authoritative | A DNS flag specifying that the responding server is an authority for the domain name used in the question. | boolean |
dns.flags.checking_disabled | A DNS flag specifying that the client disables the server signature validation of the query. | boolean |
dns.flags.recursion_available | A DNS flag specifying whether recursive query support is available in the name server. | boolean |
dns.flags.recursion_desired | A DNS flag specifying that the client directs the server to pursue a query recursively. Recursive query support is optional. | boolean |
dns.flags.truncated_response | A DNS flag specifying that only the first 512 bytes of the reply were returned. | boolean |
dns.header_flags | Array of 2 letter DNS header flags. | keyword |
dns.id | The DNS packet identifier assigned by the program that generated the query. The identifier is copied to the response. | keyword |
dns.op_code | The DNS operation code that specifies the kind of query in the message. This value is set by the originator of a query and copied into the response. | keyword |
dns.opt.do | If set, the transaction uses DNSSEC. | boolean |
dns.opt.ext_rcode | Extended response code field. | keyword |
dns.opt.udp_size | Requestor's UDP payload size (in bytes). | long |
dns.opt.version | The EDNS version. | keyword |
dns.question.class | The class of records being queried. | keyword |
dns.question.etld_plus_one | The effective top-level domain (eTLD) plus one more label. For example, the eTLD+1 for "foo.bar.golang.org." is "golang.org.". The data for determining the eTLD comes from an embedded copy of the data from http://publicsuffix.org. | keyword |
dns.question.name | The name being queried. If the name field contains non-printable characters (below 32 or above 126), those characters should be represented as escaped base 10 integers (\DDD). Back slashes and quotes should be escaped. Tabs, carriage returns, and line feeds should be converted to \t, \r, and \n respectively. | keyword |
dns.question.registered_domain | The highest registered domain, stripped of the subdomain. For example, the registered domain for "foo.example.com" is "example.com". This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". | keyword |
dns.question.subdomain | The subdomain is all of the labels under the registered_domain. If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. | keyword |
dns.question.top_level_domain | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". | keyword |
dns.question.type | The type of record being queried. | keyword |
dns.resolved_ip | Array containing all IPs seen in answers.data. The answers array can be difficult to use, because of the variety of data formats it can contain. Extracting all IP addresses seen in there to dns.resolved_ip makes it possible to index them as IP addresses, and makes them easier to visualize and query for. | ip |
dns.response_code | The DNS response code. | keyword |
dns.type | The type of DNS event captured, query or answer. If your source of DNS events only gives you DNS queries, you should only create dns events of type dns.type:query. If your source of DNS events gives you answers as well, you should create one event per query (optionally as soon as the query is seen). And a second event containing all query details as well as an array of answers. | keyword |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.dns.additionals | An array containing a dictionary for each additional section from the answer. | flattened |
network_traffic.dns.additionals.class | The class of DNS data contained in this resource record. | keyword |
network_traffic.dns.additionals.data | The data describing the resource. The meaning of this data depends on the type and class of the resource record. | keyword |
network_traffic.dns.additionals.name | The domain name to which this resource record pertains. | keyword |
network_traffic.dns.additionals.ttl | The time interval in seconds that this resource record may be cached before it should be discarded. Zero values mean that the data should not be cached. | long |
network_traffic.dns.additionals.type | The type of data contained in this resource record. | keyword |
network_traffic.dns.additionals_count | The number of resource records contained in the dns.additionals field. The dns.additionals field may or may not be included depending on the configuration of Packetbeat. | long |
network_traffic.dns.answers_count | The number of resource records contained in the dns.answers field. | long |
network_traffic.dns.authorities | An array containing a dictionary for each authority section from the answer. | flattened |
network_traffic.dns.authorities.class | The class of DNS data contained in this resource record. | keyword |
network_traffic.dns.authorities.name | The domain name to which this resource record pertains. | keyword |
network_traffic.dns.authorities.type | The type of data contained in this resource record. | keyword |
network_traffic.dns.authorities_count | The number of resource records contained in the dns.authorities field. The dns.authorities field may or may not be included depending on the configuration of Packetbeat. | long |
network_traffic.dns.flags.authentic_data | A DNS flag specifying that the recursive server considers the response authentic. | boolean |
network_traffic.dns.flags.authoritative | A DNS flag specifying that the responding server is an authority for the domain name used in the question. | boolean |
network_traffic.dns.flags.checking_disabled | A DNS flag specifying that the client disables the server signature validation of the query. | boolean |
network_traffic.dns.flags.recursion_available | A DNS flag specifying whether recursive query support is available in the name server. | boolean |
network_traffic.dns.flags.recursion_desired | A DNS flag specifying that the client directs the server to pursue a query recursively. Recursive query support is optional. | boolean |
network_traffic.dns.flags.truncated_response | A DNS flag specifying that only the first 512 bytes of the reply were returned. | boolean |
network_traffic.dns.method | The command/verb/method of the transaction. | keyword |
network_traffic.dns.opt.do | If set, the transaction uses DNSSEC. | boolean |
network_traffic.dns.opt.ext_rcode | Extended response code field. | keyword |
network_traffic.dns.opt.udp_size | Requestor's UDP payload size (in bytes). | long |
network_traffic.dns.opt.version | The EDNS version. | keyword |
network_traffic.dns.query | The query in a human readable format. | keyword |
network_traffic.dns.question.etld_plus_one | The effective top-level domain (eTLD) plus one more label. For example, the eTLD+1 for "foo.bar.golang.org." is "golang.org.". The data for determining the eTLD comes from an embedded copy of the data from http://publicsuffix.org. | keyword |
network_traffic.dns.resource | The logical resource that this transaction refers to. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for dns looks as following:
{
"@timestamp": "2023-10-16T22:36:55.594Z",
"agent": {
"ephemeral_id": "1aa050cd-250a-42b2-88cc-25d4a1e3b123",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 28,
"ip": "192.168.238.68",
"port": 53765
},
"data_stream": {
"dataset": "network_traffic.dns",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 167,
"ip": "8.8.8.8",
"port": 53
},
"dns": {
"additionals_count": 0,
"answers": [
{
"class": "IN",
"data": "ns-1183.awsdns-19.org",
"name": "elastic.co",
"ttl": "21599",
"type": "NS"
},
{
"class": "IN",
"data": "ns-2007.awsdns-58.co.uk",
"name": "elastic.co",
"ttl": "21599",
"type": "NS"
},
{
"class": "IN",
"data": "ns-66.awsdns-08.com",
"name": "elastic.co",
"ttl": "21599",
"type": "NS"
},
{
"class": "IN",
"data": "ns-835.awsdns-40.net",
"name": "elastic.co",
"ttl": "21599",
"type": "NS"
}
],
"answers_count": 4,
"authorities_count": 0,
"flags": {
"authentic_data": false,
"authoritative": false,
"checking_disabled": false,
"recursion_available": true,
"recursion_desired": true,
"truncated_response": false
},
"header_flags": [
"RD",
"RA"
],
"id": 26187,
"op_code": "QUERY",
"question": {
"class": "IN",
"etld_plus_one": "elastic.co",
"name": "elastic.co",
"registered_domain": "elastic.co",
"top_level_domain": "co",
"type": "NS"
},
"response_code": "NOERROR",
"type": "answer"
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.dns",
"duration": 68791650,
"end": "2023-10-16T22:36:55.663Z",
"ingested": "2023-10-16T22:36:56Z",
"kind": "event",
"start": "2023-10-16T22:36:55.594Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "QUERY",
"network": {
"bytes": 195,
"community_id": "1:3P4ruI0bVlqxiTAs0WyBhnF74ek=",
"direction": "unknown",
"protocol": "dns",
"transport": "udp",
"type": "ipv4"
},
"query": "class IN, type NS, elastic.co",
"related": {
"ip": [
"192.168.238.68",
"8.8.8.8"
]
},
"resource": "elastic.co",
"server": {
"bytes": 167,
"ip": "8.8.8.8",
"port": 53
},
"source": {
"bytes": 28,
"ip": "192.168.238.68",
"port": 53765
},
"status": "OK",
"type": "dns"
}
HTTP
Configuration options
Also see Common protocol options.
hide_keywords
A list of query parameters that Network Packet Capture will automatically censor in
the transactions that it saves. The values associated with these
parameters are replaced by 'xxxxx'. By default, no changes are made to
the HTTP messages.
Network Packet Capture has this option because, unlike SQL traffic, which typically only contains the hashes of the passwords, HTTP traffic may contain sensitive data. To reduce security risks, you can configure this option to avoid sending the contents of certain HTTP POST parameters.
This option replaces query parameters from GET requests and top-level
parameters from POST requests. If sensitive data is encoded inside a
parameter that you don’t specify here, Network Packet Capture cannot censor it.
Also, note that if you configure Network Packet Capture to save the raw request and
response fields (see the send_request and
the send_response options), sensitive data
may be present in those fields.
redact_authorization
When this option is enabled, Network Packet Capture obscures the value of
Authorization and Proxy-Authorization HTTP headers, and censors
those strings in the response.
You should set this option to true for transactions that use Basic Authentication because they may contain the base64 unencrypted username and password.
send_headers
A list of header names to capture and send to Elasticsearch. These
headers are placed under the headers dictionary in the resulting JSON.
send_all_headers
Instead of sending a white list of headers to Elasticsearch, you can send all headers by setting this option to true. The default is false.
redact_headers
A list of headers to redact if present in the HTTP request. This will keep the header field present, but will redact it’s value to show the header’s presence.
include_body_for
The list of content types for which Network Packet Capture exports the full HTTP
payload. The HTTP body is available under http.request.body.content
and http.response.body.content for these Content-Types.
In addition, if send_response option is
enabled, then the HTTP body is exported together with the HTTP headers
under response and if send_request
enabled, then request contains the entire HTTP message including the
body.
In the following example, the HTML attachments of the HTTP responses are
exported under the response field and under
http.request.body.content or http.response.body.content:
Network Packet Capture.protocols:
- type: http
ports: [80, 8080]
send_response: true
include_body_for: ["text/html"]decode_body
A boolean flag that controls decoding of HTTP payload. It interprets the
Content-Encoding and Transfer-Encoding headers and uncompresses the
entity body. Supported encodings are gzip and deflate. This option
is only applicable in the cases where the HTTP payload is exported, that
is, when one of the include_*_body_for options is specified or a POST
request contains url-encoded parameters.
split_cookie
If the Cookie or Set-Cookie headers are sent, this option controls
whether they are split into individual values. For example, with this
option set, an HTTP response might result in the following JSON:
"response": {
"code": 200,
"headers": {
"connection": "close",
"content-language": "en",
"content-type": "text/html; charset=utf-8",
"date": "Fri, 21 Nov 2014 17:07:34 GMT",
"server": "gunicorn/19.1.1",
"set-cookie": {
"csrftoken": "S9ZuJF8mvIMT5CL4T1Xqn32wkA6ZSeyf",
"expires": "Fri, 20-Nov-2015 17:07:34 GMT",
"max-age": "31449600",
"path": "/"
},
"vary": "Cookie, Accept-Language"
},
"status_phrase": "OK"
}- Note that
set-cookieis a map containing the cookie names as keys.
The default is false.
real_ip_header
The header field to extract the real IP from. This setting is useful
when you want to capture traffic behind a reverse proxy, but you want to
get the geo-location information. If this header is present and contains
a valid IP addresses, the information is used for the
network.forwarded_ip field.
max_message_size
If an individual HTTP message is larger than this setting (in bytes), it will be trimmed to this size. Unless this value is very small (less than 1.5K), Network Packet Capture is able to still correctly follow the transaction and create an event for it. The default is 10485760 (10 MB).
Fields published for HTTP packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.domain | The domain name of the destination system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
http.request.body.bytes | Size in bytes of the request body. | long |
http.request.bytes | Total size in bytes of the request (body and headers). | long |
http.request.headers | A map containing the captured header fields from the request. Which headers to capture is configurable. If headers with the same header name are present in the message, they will be separated by commas. | flattened |
http.request.method | HTTP request method. The value should retain its casing from the original event. For example, GET, get, and GeT are all considered valid values for this field. | keyword |
http.request.referrer | Referrer for this HTTP request. | keyword |
http.response.body.bytes | Size in bytes of the response body. | long |
http.response.bytes | Total size in bytes of the response (body and headers). | long |
http.response.headers | A map containing the captured header fields from the response. Which headers to capture is configurable. If headers with the same header name are present in the message, they will be separated by commas. | flattened |
http.response.status_code | HTTP response status code. | long |
http.response.status_phrase | The HTTP status phrase. | keyword |
http.version | HTTP version. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.http.query | The query in a human readable format, be something like GET /users/_search?name=test. | keyword |
network_traffic.http.request.headers | A map containing the captured header fields from the request. Which headers to capture is configurable. If headers with the same header name are present in the message, they will be separated by commas. | flattened |
network_traffic.http.response.headers | A map containing the captured header fields from the response. Which headers to capture is configurable. If headers with the same header name are present in the message, they will be separated by commas. | flattened |
network_traffic.http.response.status_phrase | The HTTP status phrase. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.domain | The domain name of the server system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
url.domain | Domain of the url, such as "www.elastic.co". In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the domain field. If the URL contains a literal IPv6 address enclosed by [ and ] (IETF RFC 2732), the [ and ] characters should also be captured in the domain field. | keyword |
url.extension | The field contains the file extension from the original request url, excluding the leading dot. The file extension is only set if it exists, as not every url has a file extension. The leading period must not be included. For example, the value must be "png", not ".png". Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz"). | keyword |
url.full | If full URLs are important to your use case, they should be stored in url.full, whether this field is reconstructed or present in the event source. | wildcard |
url.full.text | Multi-field of url.full. | match_only_text |
url.path | Path of the request, such as "/search". | wildcard |
url.port | Port of the request, such as 443. | long |
url.query | The query field describes the query string of the request, such as "q=elasticsearch". The ? is excluded from the query string. If a URL contains no ?, there is no query field. If there is a ? but no query, the query field exists with an empty string. The exists query can be used to differentiate between the two cases. | keyword |
url.scheme | Scheme of the request, such as "https". Note: The : is not part of the scheme. | keyword |
user_agent.original | Unparsed user_agent string. | keyword |
user_agent.original.text | Multi-field of user_agent.original. | match_only_text |
An example event for http looks as following:
{
"@timestamp": "2023-10-16T23:41:13.068Z",
"agent": {
"ephemeral_id": "e4d5d369-0170-43e1-9a37-89bddea96654",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 211,
"ip": "192.168.238.50",
"port": 64770
},
"data_stream": {
"dataset": "network_traffic.http",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 9108,
"domain": "packetbeat.com",
"ip": "107.170.1.22",
"port": 80
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.http",
"duration": 141073381,
"end": "2023-10-16T23:41:13.209Z",
"ingested": "2023-10-16T23:41:14Z",
"kind": "event",
"start": "2023-10-16T23:41:13.068Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"http": {
"request": {
"body": {
"bytes": 55
},
"bytes": 211,
"headers": {
"content-length": 55,
"content-type": "application/x-www-form-urlencoded"
},
"method": "POST"
},
"response": {
"body": {
"bytes": 8936
},
"bytes": 9108,
"headers": {
"content-length": 8936,
"content-type": "text/html; charset=utf-8"
},
"mime_type": "text/html; charset=utf-8",
"status_code": 404,
"status_phrase": "not found"
},
"version": "1.1"
},
"method": "POST",
"network": {
"bytes": 9319,
"community_id": "1:LREAuuDqOAxXEbzF064U0QX5FBs=",
"direction": "unknown",
"protocol": "http",
"transport": "tcp",
"type": "ipv4"
},
"query": "POST /register",
"related": {
"hosts": [
"packetbeat.com"
],
"ip": [
"192.168.238.50",
"107.170.1.22"
]
},
"server": {
"bytes": 9108,
"domain": "packetbeat.com",
"ip": "107.170.1.22",
"port": 80
},
"source": {
"bytes": 211,
"ip": "192.168.238.50",
"port": 64770
},
"status": "Error",
"type": "http",
"url": {
"domain": "packetbeat.com",
"full": "http://packetbeat.com/register?address=anklamerstr.14b&telephon=8932784368&user=monica",
"path": "/register",
"query": "address=anklamerstr.14b&telephon=8932784368&user=monica",
"scheme": "http"
},
"user_agent": {
"original": "curl/7.37.1"
}
}
ICMP
Configuration options
Also see Common protocol options.
enabled
The ICMP protocol can be enabled/disabled via this option. The default is true.
If enabled Network Packet Capture will generate the following BPF filter:
"icmp or icmp6".
Fields published for ICMP packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
icmp.request.code | The request code. | long |
icmp.request.message | A human readable form of the request. | keyword |
icmp.request.type | The request type. | long |
icmp.response.code | The response code. | long |
icmp.response.message | A human readable form of the response. | keyword |
icmp.response.type | The response type. | long |
icmp.version | The version of the ICMP protocol. | long |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.icmp.path | The path the transaction refers to. | keyword |
network_traffic.icmp.request.code | The request code. | long |
network_traffic.icmp.request.message | A human readable form of the request. | keyword |
network_traffic.icmp.request.type | The request type. | long |
network_traffic.icmp.response.code | The response code. | long |
network_traffic.icmp.response.message | A human readable form of the response. | keyword |
network_traffic.icmp.response.type | The response type. | long |
network_traffic.icmp.version | The version of the ICMP protocol. | long |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for icmp looks as following:
{
"@timestamp": "2023-10-16T23:44:40.347Z",
"agent": {
"ephemeral_id": "c420a35b-6aba-40f9-a69e-19e063419439",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 4,
"ip": "::1"
},
"data_stream": {
"dataset": "network_traffic.icmp",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 4,
"ip": "::2"
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.icmp",
"duration": 13567591,
"end": "2023-10-16T23:44:40.360Z",
"ingested": "2023-10-16T23:44:44Z",
"kind": "event",
"start": "2023-10-16T23:44:40.347Z",
"type": [
"connection"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"icmp": {
"request": {
"code": 0,
"message": "EchoRequest",
"type": 128
},
"response": {
"code": 0,
"message": "EchoReply",
"type": 129
},
"version": 6
},
"network": {
"bytes": 8,
"community_id": "1:9UpHcZHFAOl8WqZVOs5YRQ5wDGE=",
"direction": "egress",
"protocol": "icmp",
"transport": "ipv6-icmp",
"type": "ipv6"
},
"path": "::2",
"related": {
"ip": [
"::1",
"::2"
]
},
"server": {
"bytes": 4,
"ip": "::2"
},
"source": {
"bytes": 4,
"ip": "::1"
},
"status": "OK",
"type": "icmp"
}
Memcached
Configuration options
Also see Common protocol options.
parseunknown
When this option is enabled, it forces the memcache text protocol parser to accept unknown commands.
The unknown commands MUST NOT contain a data part.
maxvalues
The maximum number of values to store in the message (multi-get). All values will be base64 encoded.
The possible settings for this option are:
maxvalue: -1, which stores all values (text based protocol multi-get)maxvalue: 0, which stores no values (default)maxvalue: N, which stores up to N values
maxbytespervalue
The maximum number of bytes to be copied for each value element.
Values will be base64 encoded, so the actual size in the JSON document
will be 4 times the value that you specify for maxbytespervalue.
udptransactiontimeout
The transaction timeout in milliseconds. The defaults is 10000 milliseconds.
Quiet messages in UDP binary protocol get responses only if there is an
error. The memcache protocol analyzer will wait for the number of
milliseconds specified by udptransactiontimeout before publishing
quiet messages. Non-quiet messages or quiet requests with an error
response are published immediately.
Fields published for Memcached packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.action | The action captured by the event. This describes the information in the event. It is more specific than event.category. Examples are group-add, process-started, file-created. The value is normally defined by the implementer. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.outcome | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. event.outcome simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of event.outcome, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with event.type:info, or any events for which an outcome does not make logical sense. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
memcache.protocol_type | The memcache protocol implementation. The value can be "binary" for binary-based, "text" for text-based, or "unknown" for an unknown memcache protocol type. | keyword |
memcache.request.automove | The automove mode in the 'slab automove' command expressed as a string. This value can be "standby"(=0), "slow"(=1), "aggressive"(=2), or the raw value if the value is unknown. | keyword |
memcache.request.bytes | The byte count of the values being transferred. | long |
memcache.request.cas_unique | The CAS (compare-and-swap) identifier if present. | long |
memcache.request.command | The memcache command being requested in the memcache text protocol. For example "set" or "get". The binary protocol opcodes are translated into memcache text protocol commands. | keyword |
memcache.request.count_values | The number of values found in the memcache request message. If the command does not send any data, this field is missing. | long |
memcache.request.delta | The counter increment/decrement delta value. | long |
memcache.request.dest_class | The destination class id in 'slab reassign' command. | long |
memcache.request.exptime | The data expiry time in seconds sent with the memcache command (if present). If the value is \< 30 days, the expiry time is relative to "now", or else it is an absolute Unix time in seconds (32-bit). | long |
memcache.request.flags | The memcache command flags sent in the request (if present). | long |
memcache.request.initial | The counter increment/decrement initial value parameter (binary protocol only). | long |
memcache.request.keys | The list of keys sent in the store or load commands. | keyword |
memcache.request.line | The raw command line for unknown commands ONLY. | keyword |
memcache.request.noreply | Set to true if noreply was set in the request. The memcache.response field will be missing. | boolean |
memcache.request.opaque | The binary protocol opaque header value used for correlating request with response messages. | long |
memcache.request.opcode | The binary protocol message opcode name. | keyword |
memcache.request.opcode_value | The binary protocol message opcode value. | long |
memcache.request.quiet | Set to true if the binary protocol message is to be treated as a quiet message. | boolean |
memcache.request.raw_args | The text protocol raw arguments for the "stats ..." and "lru crawl ..." commands. | keyword |
memcache.request.sleep_us | The sleep setting in microseconds for the 'lru_crawler sleep' command. | long |
memcache.request.source_class | The source class id in 'slab reassign' command. | long |
memcache.request.type | The memcache command classification. This value can be "UNKNOWN", "Load", "Store", "Delete", "Counter", "Info", "SlabCtrl", "LRUCrawler", "Stats", "Success", "Fail", or "Auth". | keyword |
memcache.request.values | The list of base64 encoded values sent with the request (if present). | keyword |
memcache.request.vbucket | The vbucket index sent in the binary message. | long |
memcache.request.verbosity | The value of the memcache "verbosity" command. | long |
memcache.response.bytes | The byte count of the values being transferred. | long |
memcache.response.cas_unique | The CAS (compare-and-swap) identifier to be used with CAS-based updates (if present). | long |
memcache.response.command | Either the text based protocol response message type or the name of the originating request if binary protocol is used. | keyword |
memcache.response.count_values | The number of values found in the memcache response message. If the command does not send any data, this field is missing. | long |
memcache.response.error_msg | The optional error message in the memcache response (text based protocol only). | keyword |
memcache.response.flags | The memcache message flags sent in the response (if present). | long |
memcache.response.keys | The list of keys returned for the load command (if present). | keyword |
memcache.response.opaque | The binary protocol opaque header value used for correlating request with response messages. | long |
memcache.response.opcode | The binary protocol message opcode name. | keyword |
memcache.response.opcode_value | The binary protocol message opcode value. | long |
memcache.response.stats | The statistic values returned. | flattened |
memcache.response.status | The textual representation of the response error code (binary protocol only). | keyword |
memcache.response.status_code | The status code value returned in the response (binary protocol only). | long |
memcache.response.type | The memcache command classification. This value can be "UNKNOWN", "Load", "Store", "Delete", "Counter", "Info", "SlabCtrl", "LRUCrawler", "Stats", "Success", "Fail", or "Auth". The text based protocol will employ any of these, whereas the binary based protocol will mirror the request commands only (see memcache.response.status for binary protocol). | keyword |
memcache.response.value | The counter value returned by a counter operation. | long |
memcache.response.values | The list of base64 encoded values sent with the response (if present). | keyword |
memcache.response.version | The returned memcache version string. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.memcached.protocol_type | The memcache protocol implementation. The value can be "binary" for binary-based, "text" for text-based, or "unknown" for an unknown memcache protocol type. | keyword |
network_traffic.memcached.request.automove | The automove mode in the 'slab automove' command expressed as a string. This value can be "standby"(=0), "slow"(=1), "aggressive"(=2), or the raw value if the value is unknown. | keyword |
network_traffic.memcached.request.bytes | The byte count of the values being transferred. | long |
network_traffic.memcached.request.cas_unique | The CAS (compare-and-swap) identifier if present. | long |
network_traffic.memcached.request.command | The memcache command being requested in the memcache text protocol. For example "set" or "get". The binary protocol opcodes are translated into memcache text protocol commands. | keyword |
network_traffic.memcached.request.count_values | The number of values found in the memcache request message. If the command does not send any data, this field is missing. | long |
network_traffic.memcached.request.delta | The counter increment/decrement delta value. | long |
network_traffic.memcached.request.dest_class | The destination class id in 'slab reassign' command. | long |
network_traffic.memcached.request.exptime | The data expiry time in seconds sent with the memcache command (if present). If the value is \< 30 days, the expiry time is relative to "now", or else it is an absolute Unix time in seconds (32-bit). | long |
network_traffic.memcached.request.flags | The memcache command flags sent in the request (if present). | long |
network_traffic.memcached.request.initial | The counter increment/decrement initial value parameter (binary protocol only). | long |
network_traffic.memcached.request.keys | The list of keys sent in the store or load commands. | keyword |
network_traffic.memcached.request.line | The raw command line for unknown commands ONLY. | keyword |
network_traffic.memcached.request.noreply | Set to true if noreply was set in the request. The memcache.response field will be missing. | boolean |
network_traffic.memcached.request.opaque | The binary protocol opaque header value used for correlating request with response messages. | long |
network_traffic.memcached.request.opcode | The binary protocol message opcode name. | keyword |
network_traffic.memcached.request.opcode_value | The binary protocol message opcode value. | long |
network_traffic.memcached.request.quiet | Set to true if the binary protocol message is to be treated as a quiet message. | boolean |
network_traffic.memcached.request.raw_args | The text protocol raw arguments for the "stats ..." and "lru crawl ..." commands. | keyword |
network_traffic.memcached.request.sleep_us | The sleep setting in microseconds for the 'lru_crawler sleep' command. | long |
network_traffic.memcached.request.source_class | The source class id in 'slab reassign' command. | long |
network_traffic.memcached.request.type | The memcache command classification. This value can be "UNKNOWN", "Load", "Store", "Delete", "Counter", "Info", "SlabCtrl", "LRUCrawler", "Stats", "Success", "Fail", or "Auth". | keyword |
network_traffic.memcached.request.values | The list of base64 encoded values sent with the request (if present). | keyword |
network_traffic.memcached.request.vbucket | The vbucket index sent in the binary message. | long |
network_traffic.memcached.request.verbosity | The value of the memcache "verbosity" command. | long |
network_traffic.memcached.response.bytes | The byte count of the values being transferred. | long |
network_traffic.memcached.response.cas_unique | The CAS (compare-and-swap) identifier to be used with CAS-based updates (if present). | long |
network_traffic.memcached.response.command | Either the text based protocol response message type or the name of the originating request if binary protocol is used. | keyword |
network_traffic.memcached.response.count_values | The number of values found in the memcache response message. If the command does not send any data, this field is missing. | long |
network_traffic.memcached.response.error_msg | The optional error message in the memcache response (text based protocol only). | keyword |
network_traffic.memcached.response.flags | The memcache message flags sent in the response (if present). | long |
network_traffic.memcached.response.keys | The list of keys returned for the load command (if present). | keyword |
network_traffic.memcached.response.opaque | The binary protocol opaque header value used for correlating request with response messages. | long |
network_traffic.memcached.response.opcode | The binary protocol message opcode name. | keyword |
network_traffic.memcached.response.opcode_value | The binary protocol message opcode value. | long |
network_traffic.memcached.response.stats | The statistic values returned. | flattened |
network_traffic.memcached.response.status | The textual representation of the response error code (binary protocol only). | keyword |
network_traffic.memcached.response.status_code | The status code value returned in the response (binary protocol only). | long |
network_traffic.memcached.response.type | The memcache command classification. This value can be "UNKNOWN", "Load", "Store", "Delete", "Counter", "Info", "SlabCtrl", "LRUCrawler", "Stats", "Success", "Fail", or "Auth". The text based protocol will employ any of these, whereas the binary based protocol will mirror the request commands only (see memcache.response.status for binary protocol). | keyword |
network_traffic.memcached.response.value | The counter value returned by a counter operation. | long |
network_traffic.memcached.response.values | The list of base64 encoded values sent with the response (if present). | keyword |
network_traffic.memcached.response.version | The returned memcache version string. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for memcached looks as following:
{
"@timestamp": "2023-10-16T23:03:48.222Z",
"agent": {
"ephemeral_id": "7b5b07cc-deb1-4c1d-87f5-ea6f49b216fc",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"ip": "192.168.188.37",
"port": 65195
},
"data_stream": {
"dataset": "network_traffic.memcached",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 1064,
"ip": "192.168.188.38",
"port": 11211
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.memcached",
"ingested": "2023-10-16T23:03:59Z",
"kind": "event",
"start": "2023-10-16T23:03:48.222Z",
"type": [
"connection",
"protocol"
]
},
"event.action": "memcache.store",
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"memcache": {
"protocol_type": "binary",
"request": {
"bytes": 1024,
"command": "set",
"count_values": 1,
"exptime": 0,
"flags": 0,
"keys": [
"test_key"
],
"opaque": 65536,
"opcode": "SetQ",
"opcode_value": 17,
"quiet": true,
"type": "Store",
"vbucket": 0
}
},
"network": {
"bytes": 1064,
"community_id": "1:QMbWqXK5vGDDbp48SEFuFe8Z1lQ=",
"direction": "unknown",
"protocol": "memcache",
"transport": "udp",
"type": "ipv4"
},
"related": {
"ip": [
"192.168.188.37",
"192.168.188.38"
]
},
"server": {
"bytes": 1064,
"ip": "192.168.188.38",
"port": 11211
},
"source": {
"ip": "192.168.188.37",
"port": 65195
},
"status": "OK",
"type": "memcache"
}
MongoDB
Configuration options
The max_docs and max_doc_length settings are useful for limiting the
amount of data Network Packet Capture indexes in the response fields.
Also see Common protocol options.
max_docs
The maximum number of documents from the response to index in the
response field. The default is 10. You can set this to 0 to index an
unlimited number of documents.
Network Packet Capture adds a [...] line at the end to signify that there were
additional documents that weren’t saved because of this setting.
max_doc_length
The maximum number of characters in a single document indexed in the
response field. The default is 5000. You can set this to 0 to index an
unlimited number of characters per document.
If the document is trimmed because of this setting, Network Packet Capture adds the
string ... at the end of the document.
Note that limiting documents in this way means that they are no longer correctly formatted JSON objects.
Fields published for MongoDB packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
mongodb.cursorId | The cursor identifier returned in the OP_REPLY. This must be the value that was returned from the database. | keyword |
mongodb.error | If the MongoDB request has resulted in an error, this field contains the error message returned by the server. | keyword |
mongodb.fullCollectionName | The full collection name. The full collection name is the concatenation of the database name with the collection name, using a dot (.) for the concatenation. For example, for the database foo and the collection bar, the full collection name is foo.bar. | keyword |
mongodb.numberReturned | The number of documents in the reply. | long |
mongodb.numberToReturn | The requested maximum number of documents to be returned. | long |
mongodb.numberToSkip | Sets the number of documents to omit - starting from the first document in the resulting dataset - when returning the result of the query. | long |
mongodb.query | A JSON document that represents the query. The query will contain one or more elements, all of which must match for a document to be included in the result set. Possible elements include $query, $orderby, $hint, $explain, and $snapshot. | keyword |
mongodb.returnFieldsSelector | A JSON document that limits the fields in the returned documents. The returnFieldsSelector contains one or more elements, each of which is the name of a field that should be returned, and the integer value 1. | keyword |
mongodb.selector | A BSON document that specifies the query for selecting the document to update or delete. | keyword |
mongodb.startingFrom | Where in the cursor this reply is starting. | keyword |
mongodb.update | A BSON document that specifies the update to be performed. For information on specifying updates, see the Update Operations documentation from the MongoDB Manual. | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.mongodb.cursorId | The cursor identifier returned in the OP_REPLY. This must be the value that was returned from the database. | keyword |
network_traffic.mongodb.error | If the MongoDB request has resulted in an error, this field contains the error message returned by the server. | keyword |
network_traffic.mongodb.fullCollectionName | The full collection name. The full collection name is the concatenation of the database name with the collection name, using a dot (.) for the concatenation. For example, for the database foo and the collection bar, the full collection name is foo.bar. | keyword |
network_traffic.mongodb.method | The command/verb/method of the transaction. | keyword |
network_traffic.mongodb.numberReturned | The number of documents in the reply. | long |
network_traffic.mongodb.numberToReturn | The requested maximum number of documents to be returned. | long |
network_traffic.mongodb.numberToSkip | Sets the number of documents to omit - starting from the first document in the resulting dataset - when returning the result of the query. | long |
network_traffic.mongodb.query | A JSON document that represents the query. The query will contain one or more elements, all of which must match for a document to be included in the result set. Possible elements include $query, $orderby, $hint, $explain, and $snapshot. | keyword |
network_traffic.mongodb.resource | The logical resource that this transaction refers to. | keyword |
network_traffic.mongodb.returnFieldsSelector | A JSON document that limits the fields in the returned documents. The returnFieldsSelector contains one or more elements, each of which is the name of a field that should be returned, and the integer value 1. | keyword |
network_traffic.mongodb.selector | A BSON document that specifies the query for selecting the document to update or delete. | keyword |
network_traffic.mongodb.startingFrom | Where in the cursor this reply is starting. | keyword |
network_traffic.mongodb.update | A BSON document that specifies the update to be performed. For information on specifying updates, see the Update Operations documentation from the MongoDB Manual. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for mongodb looks as following:
{
"@timestamp": "2023-10-16T23:10:00.771Z",
"agent": {
"ephemeral_id": "ba8a356f-2bd0-4dd5-927d-a149f0e78281",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 50,
"ip": "127.0.0.1",
"port": 57203
},
"data_stream": {
"dataset": "network_traffic.mongodb",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 514,
"ip": "127.0.0.1",
"port": 27017
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.mongodb",
"duration": 1130257,
"end": "2023-10-16T23:10:00.772Z",
"ingested": "2023-10-16T23:10:01Z",
"kind": "event",
"start": "2023-10-16T23:10:00.771Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "find",
"mongodb": {
"cursorId": 0,
"fullCollectionName": "test.restaurants",
"numberReturned": 1,
"numberToReturn": 1,
"numberToSkip": 0,
"startingFrom": 0
},
"network": {
"bytes": 564,
"community_id": "1:mYSTZ4QZBfvJO05Em9TnPwrae6g=",
"direction": "ingress",
"protocol": "mongodb",
"transport": "tcp",
"type": "ipv4"
},
"query": "test.restaurants.find().limit(1)",
"related": {
"ip": [
"127.0.0.1"
]
},
"resource": "test.restaurants",
"server": {
"bytes": 514,
"ip": "127.0.0.1",
"port": 27017
},
"source": {
"bytes": 50,
"ip": "127.0.0.1",
"port": 57203
},
"status": "OK",
"type": "mongodb"
}
MySQL
Configuration options
Also see Common protocol options.
max_rows
The maximum number of rows from the SQL message to publish to Elasticsearch. The default is 10 rows.
max_row_length
The maximum length in bytes of a row from the SQL message to publish to Elasticsearch. The default is 1024 bytes.
statement_timeout
The duration for which prepared statements are cached after their last
use. Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". The
default is 1h.
Fields published for MySQL packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
mysql.affected_rows | If the MySQL command is successful, this field contains the affected number of rows of the last statement. | long |
mysql.error_code | The error code returned by MySQL. | long |
mysql.error_message | The error info message returned by MySQL. | keyword |
mysql.insert_id | If the INSERT query is successful, this field contains the id of the newly inserted row. | keyword |
mysql.num_fields | If the SELECT query is successful, this field is set to the number of fields returned. | long |
mysql.num_rows | If the SELECT query is successful, this field is set to the number of rows returned. | long |
mysql.query | The row mysql query as read from the transaction's request. | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.mysql.affected_rows | If the MySQL command is successful, this field contains the affected number of rows of the last statement. | long |
network_traffic.mysql.error_code | The error code returned by MySQL. | long |
network_traffic.mysql.error_message | The error info message returned by MySQL. | keyword |
network_traffic.mysql.insert_id | If the INSERT query is successful, this field contains the id of the newly inserted row. | keyword |
network_traffic.mysql.method | The command/verb/method of the transaction. | keyword |
network_traffic.mysql.num_fields | If the SELECT query is successful, this field is set to the number of fields returned. | long |
network_traffic.mysql.num_rows | If the SELECT query is successful, this field is set to the number of rows returned. | long |
network_traffic.mysql.path | The table name the transaction refers to. | keyword |
network_traffic.mysql.query | The row mysql query as read from the transaction's request. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for mysql looks as following:
{
"@timestamp": "2023-10-16T23:14:45.124Z",
"agent": {
"ephemeral_id": "bfa018bc-c1e8-45ea-b4ff-e8d2436764be",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 23,
"ip": "127.0.0.1",
"port": 41517
},
"data_stream": {
"dataset": "network_traffic.mysql",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 3629,
"ip": "127.0.0.1",
"port": 3306
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.mysql",
"duration": 4771069,
"end": "2023-10-16T23:14:45.129Z",
"ingested": "2023-10-16T23:14:46Z",
"kind": "event",
"start": "2023-10-16T23:14:45.124Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "SELECT",
"mysql": {
"affected_rows": 0,
"insert_id": 0,
"num_fields": 3,
"num_rows": 15
},
"network": {
"bytes": 3652,
"community_id": "1:goIcZn7CMIJ6W7Yf8JRV618zzxA=",
"direction": "ingress",
"protocol": "mysql",
"transport": "tcp",
"type": "ipv4"
},
"path": "test.test",
"query": "select * from test",
"related": {
"ip": [
"127.0.0.1"
]
},
"server": {
"bytes": 3629,
"ip": "127.0.0.1",
"port": 3306
},
"source": {
"bytes": 23,
"ip": "127.0.0.1",
"port": 41517
},
"status": "OK",
"type": "mysql"
}
NFS
Configuration options
Fields published for NFS packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.domain | The domain name of the client system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.action | The action captured by the event. This describes the information in the event. It is more specific than event.category. Examples are group-add, process-started, file-created. The value is normally defined by the implementer. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
group.id | Unique identifier for the group on the system/platform. | keyword |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.nfs.host.hostname | The hostname of the NFS host. | keyword |
network_traffic.nfs.minor_version | NFS protocol minor version number. | long |
network_traffic.nfs.opcode | NFS operation name, or main operation name, in case of COMPOUND calls. | keyword |
network_traffic.nfs.rpc.auth_flavor | RPC authentication flavor. | keyword |
network_traffic.nfs.rpc.cred.gid | RPC caller's group id, in case of auth-unix. | long |
network_traffic.nfs.rpc.cred.gids | RPC caller's secondary group ids, in case of auth-unix. | long |
network_traffic.nfs.rpc.cred.machinename | The name of the caller's machine. | keyword |
network_traffic.nfs.rpc.cred.stamp | Arbitrary ID which the caller machine may generate. | long |
network_traffic.nfs.rpc.cred.uid | RPC caller's user id, in case of auth-unix. | long |
network_traffic.nfs.rpc.status | RPC message reply status. | keyword |
network_traffic.nfs.rpc.xid | RPC message transaction identifier. | keyword |
network_traffic.nfs.status | NFS operation reply status. | keyword |
network_traffic.nfs.tag | NFS v4 COMPOUND operation tag. | keyword |
network_traffic.nfs.version | NFS protocol version number. | long |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
nfs.minor_version | NFS protocol minor version number. | long |
nfs.opcode | NFS operation name, or main operation name, in case of COMPOUND calls. | keyword |
nfs.status | NFS operation reply status. | keyword |
nfs.tag | NFS v4 COMPOUND operation tag. | keyword |
nfs.version | NFS protocol version number. | long |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
rpc.auth_flavor | RPC authentication flavor. | keyword |
rpc.cred.gid | RPC caller's group id, in case of auth-unix. | long |
rpc.cred.gids | RPC caller's secondary group ids, in case of auth-unix. | long |
rpc.cred.machinename | The name of the caller's machine. | keyword |
rpc.cred.stamp | Arbitrary ID which the caller machine may generate. | long |
rpc.cred.uid | RPC caller's user id, in case of auth-unix. | long |
rpc.status | RPC message reply status. | keyword |
rpc.xid | RPC message transaction identifier. | keyword |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.domain | The domain name of the source system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
user.id | Unique identifier of the user. | keyword |
An example event for nfs looks as following:
{
"@timestamp": "2023-10-16T23:18:26.753Z",
"agent": {
"ephemeral_id": "d177e674-4168-4b25-bceb-5113c0bb88b0",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 208,
"domain": "desycloud03.desy.de",
"ip": "131.169.5.156",
"port": 907
},
"data_stream": {
"dataset": "network_traffic.nfs",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 176,
"ip": "131.169.192.35",
"port": 2049
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"action": "nfs.CLOSE",
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.nfs",
"duration": 5463467,
"end": "2023-10-16T23:18:26.758Z",
"ingested": "2023-10-16T23:18:27Z",
"kind": "event",
"start": "2023-10-16T23:18:26.753Z",
"type": [
"connection",
"protocol"
]
},
"group.id": 48,
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"host.hostname": "desycloud03.desy.de",
"network": {
"bytes": 384,
"community_id": "1:cd5eLXemAsSPMdXwCbdDUWWud4M=",
"direction": "unknown",
"protocol": "nfsv4",
"transport": "tcp",
"type": "ipv4"
},
"nfs": {
"minor_version": 1,
"opcode": "CLOSE",
"status": "NFS_OK",
"tag": "",
"version": 4
},
"related": {
"ip": [
"131.169.5.156",
"131.169.192.35"
]
},
"rpc": {
"auth_flavor": "unix",
"cred": {
"gid": 48,
"gids": [
48
],
"machinename": "desycloud03.desy.de",
"stamp": 4308441,
"uid": 48
},
"status": "success",
"xid": "c3103fc1"
},
"server": {
"bytes": 176,
"ip": "131.169.192.35",
"port": 2049
},
"source": {
"bytes": 208,
"domain": "desycloud03.desy.de",
"ip": "131.169.5.156",
"port": 907
},
"status": "OK",
"type": "nfs",
"user.id": 48
}
PostgreSQL
Configuration options
Also see Common protocol options.
max_rows
The maximum number of rows from the SQL message to publish to Elasticsearch. The default is 10 rows.
max_row_length
The maximum length in bytes of a row from the SQL message to publish to Elasticsearch. The default is 1024 bytes.
Fields published for PostgreSQL packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.pgsql.error_code | The PostgreSQL error code. | keyword |
network_traffic.pgsql.error_message | The PostgreSQL error message. | keyword |
network_traffic.pgsql.error_severity | The PostgreSQL error severity. | keyword |
network_traffic.pgsql.method | The command/verb/method of the transaction. | keyword |
network_traffic.pgsql.num_fields | If the SELECT query if successful, this field is set to the number of fields returned. | long |
network_traffic.pgsql.num_rows | If the SELECT query if successful, this field is set to the number of rows returned. | long |
network_traffic.pgsql.query | The query in a human readable format. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
pgsql.error_code | The PostgreSQL error code. | keyword |
pgsql.error_message | The PostgreSQL error message. | keyword |
pgsql.error_severity | The PostgreSQL error severity. | keyword |
pgsql.num_fields | If the SELECT query if successful, this field is set to the number of fields returned. | long |
pgsql.num_rows | If the SELECT query if successful, this field is set to the number of rows returned. | long |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for pgsql looks as following:
{
"@timestamp": "2023-10-16T23:22:18.142Z",
"agent": {
"ephemeral_id": "6125f32f-943d-4b83-81a2-ca5dd7152657",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 34,
"ip": "127.0.0.1",
"port": 34936
},
"data_stream": {
"dataset": "network_traffic.pgsql",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 3186,
"ip": "127.0.0.1",
"port": 5432
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.pgsql",
"duration": 2454138,
"end": "2023-10-16T23:22:18.145Z",
"ingested": "2023-10-16T23:22:19Z",
"kind": "event",
"start": "2023-10-16T23:22:18.142Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "SELECT",
"network": {
"bytes": 3220,
"community_id": "1:WUuTzESSpZnUwZ2tuZKZtNOdHSU=",
"direction": "ingress",
"protocol": "pgsql",
"transport": "tcp",
"type": "ipv4"
},
"pgsql": {
"num_fields": 3,
"num_rows": 15
},
"query": "select * from long_response",
"related": {
"ip": [
"127.0.0.1"
]
},
"server": {
"bytes": 3186,
"ip": "127.0.0.1",
"port": 5432
},
"source": {
"bytes": 34,
"ip": "127.0.0.1",
"port": 34936
},
"status": "OK",
"type": "pgsql"
}
Redis
Configuration options
Also see Common protocol options.
queue_max_bytes and queue_max_messages
store requests in memory until a response is received. These settings
impose a limit on the number of bytes (queue_max_bytes) and number of
requests (queue_max_messages) that can be stored. These limits are
per-connection. The default is to queue up to 1MB or 20.000 requests per
connection, which allows to use request pipelining while at the same
time limiting the amount of memory consumed by replication sessions.
Fields published for Redis packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.action | The action captured by the event. This describes the information in the event. It is more specific than event.category. Examples are group-add, process-started, file-created. The value is normally defined by the implementer. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.outcome | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. event.outcome simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of event.outcome, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with event.type:info, or any events for which an outcome does not make logical sense. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.redis.error | If the Redis command has resulted in an error, this field contains the error message returned by the Redis server. | keyword |
network_traffic.redis.method | The command/verb/method of the transaction. | keyword |
network_traffic.redis.query | The query in a human readable format. | keyword |
network_traffic.redis.return_value | The return value of the Redis command in a human readable format. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
redis.error | If the Redis command has resulted in an error, this field contains the error message returned by the Redis server. | keyword |
redis.return_value | The return value of the Redis command in a human readable format. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for redis looks as following:
{
"@timestamp": "2023-10-16T23:23:39.505Z",
"agent": {
"ephemeral_id": "187d82c4-b575-4dba-83bf-4950cb7435d9",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 31,
"ip": "127.0.0.1",
"port": 32810
},
"data_stream": {
"dataset": "network_traffic.redis",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 5,
"ip": "127.0.0.1",
"port": 6380
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"action": "redis.set",
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.redis",
"duration": 1300522,
"end": "2023-10-16T23:23:39.506Z",
"ingested": "2023-10-16T23:23:43Z",
"kind": "event",
"start": "2023-10-16T23:23:39.505Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "SET",
"network": {
"bytes": 36,
"community_id": "1:GuHlyWpX6bKkMXy19YkvZSNPTS4=",
"direction": "ingress",
"protocol": "redis",
"transport": "tcp",
"type": "ipv4"
},
"query": "set key3 me",
"redis": {
"return_value": "OK"
},
"related": {
"ip": [
"127.0.0.1"
]
},
"resource": "key3",
"server": {
"bytes": 5,
"ip": "127.0.0.1",
"port": 6380
},
"source": {
"bytes": 31,
"ip": "127.0.0.1",
"port": 32810
},
"status": "OK",
"type": "redis"
}
SIP
Configuration options
Also see Common protocol options.
parse_authorization
If set to true Network Packet Capture will parse the authorization headers and include them in events. The default is true.
parse_body
If set to true, Network Packet Capture parses the SIP body when the body contains Session Description Protocol data. The default is true.
Fields published for SIP packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.action | The action captured by the event. This describes the information in the event. It is more specific than event.category. Examples are group-add, process-started, file-created. The value is normally defined by the implementer. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.original | Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from _source. If users wish to override this and index this field, please see Field data types in the Elasticsearch Reference. | keyword |
event.outcome | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. event.outcome simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of event.outcome, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with event.type:info, or any events for which an outcome does not make logical sense. | keyword |
event.reason | Reason why this event happened, according to the source. This describes the why of a particular action or outcome captured in the event. Where event.action captures the action from the event, event.reason describes why that action was taken. For example, a web proxy with an event.action which denied the request may also populate event.reason with the reason why (e.g. blocked site). | keyword |
event.sequence | Sequence number of the event. The sequence number is a value published by some event sources, to make the exact ordering of events unambiguous, regardless of the timestamp precision. | long |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.application | When a specific application or service is identified from network connection details (source/dest IPs, ports, certificates, or wire format), this field captures the application's or service's name. For example, the original event identifies the network connection being from a specific web service in a https network connection, like facebook or twitter. The field value must be normalized to lowercase for querying. | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.iana_number | IANA Protocol Number (https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml). Standardized list of protocols. This aligns well with NetFlow and sFlow related logs which use the IANA Protocol Number. | keyword |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.sip.accept | Accept header value. | keyword |
network_traffic.sip.allow | Allowed methods. | keyword |
network_traffic.sip.auth.realm | Auth realm | keyword |
network_traffic.sip.auth.scheme | Auth scheme | keyword |
network_traffic.sip.auth.uri.host | Auth URI host | keyword |
network_traffic.sip.auth.uri.original | Auth original URI | keyword |
network_traffic.sip.auth.uri.original.text | Multi-field of network_traffic.sip.auth.uri.original. | text |
network_traffic.sip.auth.uri.port | Auth URI port | long |
network_traffic.sip.auth.uri.scheme | Auth URI scheme | keyword |
network_traffic.sip.call_id | Call ID. | keyword |
network_traffic.sip.code | Response status code. | long |
network_traffic.sip.contact.display_info | Contact display info | keyword |
network_traffic.sip.contact.expires | Contact expires | keyword |
network_traffic.sip.contact.line | Contact line | keyword |
network_traffic.sip.contact.q | Contact Q | keyword |
network_traffic.sip.contact.transport | Contact transport | keyword |
network_traffic.sip.contact.uri.host | Contact URI host | keyword |
network_traffic.sip.contact.uri.original | Contact original URI | keyword |
network_traffic.sip.contact.uri.original.text | Multi-field of network_traffic.sip.contact.uri.original. | text |
network_traffic.sip.contact.uri.port | Contact URI port | long |
network_traffic.sip.contact.uri.scheme | Contat URI scheme | keyword |
network_traffic.sip.contact.uri.username | Contact URI user name | keyword |
network_traffic.sip.content_length | long | |
network_traffic.sip.content_type | keyword | |
network_traffic.sip.cseq.code | Sequence code. | long |
network_traffic.sip.cseq.method | Sequence method. | keyword |
network_traffic.sip.from.display_info | From display info | keyword |
network_traffic.sip.from.tag | From tag | keyword |
network_traffic.sip.from.uri.host | From URI host | keyword |
network_traffic.sip.from.uri.original | From original URI | keyword |
network_traffic.sip.from.uri.original.text | Multi-field of network_traffic.sip.from.uri.original. | text |
network_traffic.sip.from.uri.port | From URI port | long |
network_traffic.sip.from.uri.scheme | From URI scheme | keyword |
network_traffic.sip.from.uri.username | From URI user name | keyword |
network_traffic.sip.max_forwards | long | |
network_traffic.sip.method | Request method. | keyword |
network_traffic.sip.private.uri.host | Private URI host. | keyword |
network_traffic.sip.private.uri.original | Private original URI. | keyword |
network_traffic.sip.private.uri.original.text | Multi-field of network_traffic.sip.private.uri.original. | text |
network_traffic.sip.private.uri.port | Private URI port. | long |
network_traffic.sip.private.uri.scheme | Private URI scheme. | keyword |
network_traffic.sip.private.uri.username | Private URI user name. | keyword |
network_traffic.sip.sdp.body.original | SDP original body | keyword |
network_traffic.sip.sdp.body.original.text | Multi-field of network_traffic.sip.sdp.body.original. | text |
network_traffic.sip.sdp.connection.address | SDP connection address | keyword |
network_traffic.sip.sdp.connection.info | SDP connection info | keyword |
network_traffic.sip.sdp.owner.ip | SDP owner IP | ip |
network_traffic.sip.sdp.owner.session_id | SDP owner session ID | keyword |
network_traffic.sip.sdp.owner.username | SDP owner user name | keyword |
network_traffic.sip.sdp.owner.version | SDP owner version | keyword |
network_traffic.sip.sdp.session.name | SDP session name | keyword |
network_traffic.sip.sdp.version | SDP version | keyword |
network_traffic.sip.status | Response status phrase. | keyword |
network_traffic.sip.supported | Supported methods. | keyword |
network_traffic.sip.to.display_info | To display info | keyword |
network_traffic.sip.to.tag | To tag | keyword |
network_traffic.sip.to.uri.host | To URI host | keyword |
network_traffic.sip.to.uri.original | To original URI | keyword |
network_traffic.sip.to.uri.original.text | Multi-field of network_traffic.sip.to.uri.original. | text |
network_traffic.sip.to.uri.port | To URI port | long |
network_traffic.sip.to.uri.scheme | To URI scheme | keyword |
network_traffic.sip.to.uri.username | To URI user name | keyword |
network_traffic.sip.type | Either request or response. | keyword |
network_traffic.sip.uri.host | The URI host. | keyword |
network_traffic.sip.uri.original | The original URI. | keyword |
network_traffic.sip.uri.original.text | Multi-field of network_traffic.sip.uri.original. | text |
network_traffic.sip.uri.port | The URI port. | long |
network_traffic.sip.uri.scheme | The URI scheme. | keyword |
network_traffic.sip.uri.username | The URI user name. | keyword |
network_traffic.sip.user_agent.original | keyword | |
network_traffic.sip.user_agent.original.text | Multi-field of network_traffic.sip.user_agent.original. | text |
network_traffic.sip.version | SIP protocol version. | keyword |
network_traffic.sip.via.original | The original Via value. | keyword |
network_traffic.sip.via.original.text | Multi-field of network_traffic.sip.via.original. | text |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
related.user | All the user names or other user identifiers seen on the event. | keyword |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
sip.accept | Accept header value. | keyword |
sip.allow | Allowed methods. | keyword |
sip.auth.realm | Auth realm | keyword |
sip.auth.scheme | Auth scheme | keyword |
sip.auth.uri.host | Auth URI host | keyword |
sip.auth.uri.original | Auth original URI | keyword |
sip.auth.uri.original.text | Multi-field of sip.auth.uri.original. | text |
sip.auth.uri.port | Auth URI port | long |
sip.auth.uri.scheme | Auth URI scheme | keyword |
sip.call_id | Call ID. | keyword |
sip.code | Response status code. | long |
sip.contact.display_info | Contact display info | keyword |
sip.contact.expires | Contact expires | keyword |
sip.contact.line | Contact line | keyword |
sip.contact.q | Contact Q | keyword |
sip.contact.transport | Contact transport | keyword |
sip.contact.uri.host | Contact URI host | keyword |
sip.contact.uri.original | Contact original URI | keyword |
sip.contact.uri.original.text | Multi-field of sip.contact.uri.original. | text |
sip.contact.uri.port | Contact URI port | long |
sip.contact.uri.scheme | Contat URI scheme | keyword |
sip.contact.uri.username | Contact URI user name | keyword |
sip.content_length | long | |
sip.content_type | keyword | |
sip.cseq.code | Sequence code. | long |
sip.cseq.method | Sequence method. | keyword |
sip.from.display_info | From display info | keyword |
sip.from.tag | From tag | keyword |
sip.from.uri.host | From URI host | keyword |
sip.from.uri.original | From original URI | keyword |
sip.from.uri.original.text | Multi-field of sip.from.uri.original. | text |
sip.from.uri.port | From URI port | long |
sip.from.uri.scheme | From URI scheme | keyword |
sip.from.uri.username | From URI user name | keyword |
sip.max_forwards | long | |
sip.method | Request method. | keyword |
sip.private.uri.host | Private URI host. | keyword |
sip.private.uri.original | Private original URI. | keyword |
sip.private.uri.original.text | Multi-field of sip.private.uri.original. | text |
sip.private.uri.port | Private URI port. | long |
sip.private.uri.scheme | Private URI scheme. | keyword |
sip.private.uri.username | Private URI user name. | keyword |
sip.sdp.body.original | SDP original body | keyword |
sip.sdp.body.original.text | Multi-field of sip.sdp.body.original. | text |
sip.sdp.connection.address | SDP connection address | keyword |
sip.sdp.connection.info | SDP connection info | keyword |
sip.sdp.owner.ip | SDP owner IP | ip |
sip.sdp.owner.session_id | SDP owner session ID | keyword |
sip.sdp.owner.username | SDP owner user name | keyword |
sip.sdp.owner.version | SDP owner version | keyword |
sip.sdp.session.name | SDP session name | keyword |
sip.sdp.version | SDP version | keyword |
sip.status | Response status phrase. | keyword |
sip.supported | Supported methods. | keyword |
sip.to.display_info | To display info | keyword |
sip.to.tag | To tag | keyword |
sip.to.uri.host | To URI host | keyword |
sip.to.uri.original | To original URI | keyword |
sip.to.uri.original.text | Multi-field of sip.to.uri.original. | text |
sip.to.uri.port | To URI port | long |
sip.to.uri.scheme | To URI scheme | keyword |
sip.to.uri.username | To URI user name | keyword |
sip.type | Either request or response. | keyword |
sip.uri.host | The URI host. | keyword |
sip.uri.original | The original URI. | keyword |
sip.uri.original.text | Multi-field of sip.uri.original. | text |
sip.uri.port | The URI port. | long |
sip.uri.scheme | The URI scheme. | keyword |
sip.uri.username | The URI user name. | keyword |
sip.user_agent.original | keyword | |
sip.user_agent.original.text | Multi-field of sip.user_agent.original. | text |
sip.version | SIP protocol version. | keyword |
sip.via.original | The original Via value. | keyword |
sip.via.original.text | Multi-field of sip.via.original. | text |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
user.name | Short name or login of the user. | keyword |
user.name.text | Multi-field of user.name. | match_only_text |
An example event for sip looks as following:
{
"@timestamp": "2023-11-13T21:54:31.038Z",
"agent": {
"ephemeral_id": "7f204077-dee0-4442-b500-1b2f6d84d15a",
"id": "4f93724a-6328-4803-8108-b682e5d62ad4",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"ip": "10.0.2.20",
"port": 5060
},
"data_stream": {
"dataset": "network_traffic.sip",
"namespace": "ep",
"type": "logs"
},
"destination": {
"ip": "10.0.2.15",
"port": 5060
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "4f93724a-6328-4803-8108-b682e5d62ad4",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"action": "sip-invite",
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.sip",
"duration": 0,
"end": "2023-11-13T21:54:31.038Z",
"ingested": "2023-11-13T21:54:32Z",
"kind": "event",
"original": "INVITE sip:test@10.0.2.15:5060 SIP/2.0\r\nVia: SIP/2.0/UDP 10.0.2.20:5060;branch=z9hG4bK-2187-1-0\r\nFrom: \"DVI4/8000\" <sip:sipp@10.0.2.20:5060>;tag=1\r\nTo: test <sip:test@10.0.2.15:5060>\r\nCall-ID: 1-2187@10.0.2.20\r\nCSeq: 1 INVITE\r\nContact: sip:sipp@10.0.2.20:5060\r\nMax-Forwards: 70\r\nContent-Type: application/sdp\r\nContent-Length: 123\r\n\r\nv=0\r\no=- 42 42 IN IP4 10.0.2.20\r\ns=-\r\nc=IN IP4 10.0.2.20\r\nt=0 0\r\nm=audio 6000 RTP/AVP 5\r\na=rtpmap:5 DVI4/8000\r\na=recvonly\r\n",
"sequence": 1,
"start": "2023-11-13T21:54:31.038Z",
"type": [
"info",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.22.0.7"
],
"mac": [
"02-42-AC-16-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"network": {
"application": "sip",
"community_id": "1:xDRQZvk3ErEhBDslXv1c6EKI804=",
"direction": "unknown",
"iana_number": "17",
"protocol": "sip",
"transport": "udp",
"type": "ipv4"
},
"related": {
"hosts": [
"10.0.2.15",
"10.0.2.20"
],
"ip": [
"10.0.2.20",
"10.0.2.15"
],
"user": [
"test",
"sipp"
]
},
"server": {
"ip": "10.0.2.15",
"port": 5060
},
"sip": {
"call_id": "1-2187@10.0.2.20",
"contact": {
"display_info": "test",
"uri": {
"host": "10.0.2.15",
"original": "sip:test@10.0.2.15:5060",
"port": 5060,
"scheme": "sip",
"username": "test"
}
},
"content_length": 123,
"content_type": "application/sdp",
"cseq": {
"code": 1,
"method": "INVITE"
},
"from": {
"display_info": "DVI4/8000",
"tag": "1",
"uri": {
"host": "10.0.2.20",
"original": "sip:sipp@10.0.2.20:5060",
"port": 5060,
"scheme": "sip",
"username": "sipp"
}
},
"max_forwards": 70,
"method": "INVITE",
"sdp": {
"body": {
"original": "v=0\r\no=- 42 42 IN IP4 10.0.2.20\r\ns=-\r\nc=IN IP4 10.0.2.20\r\nt=0 0\r\nm=audio 6000 RTP/AVP 5\r\na=rtpmap:5 DVI4/8000\r\na=recvonly\r\n"
},
"connection": {
"address": "10.0.2.20",
"info": "IN IP4 10.0.2.20"
},
"owner": {
"ip": "10.0.2.20",
"session_id": "42",
"version": "42"
},
"version": "0"
},
"to": {
"display_info": "test",
"uri": {
"host": "10.0.2.15",
"original": "sip:test@10.0.2.15:5060",
"port": 5060,
"scheme": "sip",
"username": "test"
}
},
"type": "request",
"uri": {
"host": "10.0.2.15",
"original": "sip:test@10.0.2.15:5060",
"port": 5060,
"scheme": "sip",
"username": "test"
},
"version": "2.0",
"via": {
"original": [
"SIP/2.0/UDP 10.0.2.20:5060;branch=z9hG4bK-2187-1-0"
]
}
},
"source": {
"ip": "10.0.2.20",
"port": 5060
},
"status": "OK",
"type": "sip"
}Thrift
Apache Thrift is a communication protocol and RPC framework initially created at Facebook. It is sometimes used in microservices architectures because it provides better performance when compared to the more obvious HTTP/RESTful API choice, while still supporting a wide range of programming languages and frameworks.
Network Packet Capture works based on a copy of the traffic, which means that you get performance management features without having to modify your services in any way and without any latency overhead. Network Packet Capture captures the transactions from the network and indexes them in Elasticsearch so that they can be analyzed and searched.
Network Packet Capture indexes the method, parameters, return value, and exceptions
of each Thrift-RPC call. You can search by and create statistics based
on any of these fields. Network Packet Capture automatically fills in the status
column with either OK or Error, so it’s easy to find the problematic
RPC calls. A transaction is put into the Error state if it returned an
exception.
Network Packet Capture also indexes the event.duration field so you can get
performance analytics and find the slow RPC calls.
Thrift supports multiple transport and protocol
types. Currently Network Packet Capture
supports the default TSocket transport as well as the TFramed
transport. From the protocol point of view, Network Packet Capture currently
supports only the default TBinary protocol.
Network Packet Capture also has several configuration options that allow you to get the right balance between visibility, disk usage, and data protection. You can, for example, choose to obfuscate all strings or to store the requests but not the responses, while still capturing the response time for each of the RPC calls. You can also choose to limit the size of strings and lists to a given number of elements, so you can fine tune how much data you want to have stored in Elasticsearch.
The Thrift protocol has several specific configuration options.
Providing the Thrift IDL files to Network Packet Capture is optional. The binary Thrift messages include the called method name and enough structural information to decode the messages without needing the IDL files. However, if you provide the IDL files, Network Packet Capture can also resolve the service name, arguments, and exception names.
Configuration options
Also see Common protocol options.
transport_type
The Thrift transport type. Currently this option accepts the values
socket for TSocket, which is the default Thrift transport, and
framed for the TFramed Thrift transport. The default is socket.
protocol_type
The Thrift protocol type. Currently the only accepted value is binary
for the TBinary protocol, which is the default Thrift protocol.
idl_files
The Thrift interface description language (IDL) files for the service that Network Packet Capture is monitoring. Providing the IDL files is optional, because the Thrift messages contain enough information to decode them without having the IDL files. However, providing the IDL enables Network Packet Capture to include parameter and exception names.
string_max_size
The maximum length for strings in parameters or return values. If a string is longer than this value, the string is automatically truncated to this length. Network Packet Capture adds dots at the end of the string to mark that it was truncated. The default is 200.
collection_max_size
The maximum number of elements in a Thrift list, set, map, or structure.
If a collection has more elements than this value, Network Packet Capture captures
only the specified number of elements. Network Packet Capture adds a fictive last
element ... to the end of the collection to mark that it was
truncated. The default is 15.
capture_reply
If this option is set to false, Network Packet Capture decodes the method name from the reply and simply skips the rest of the response message. This setting can be useful for performance, disk usage, or data retention reasons. The default is true.
obfuscate_strings
If this option is set to true, Network Packet Capture replaces all strings found in
method parameters, return codes, or exception structures with the "*"
string.
drop_after_n_struct_fields
The maximum number of fields that a structure can have before Network Packet Capture ignores the whole transaction. This is a memory protection mechanism (so that Network Packet Capture’s memory doesn’t grow indefinitely), so you would typically set this to a relatively high value. The default is 500.
Fields published for Thrift packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
network_traffic.thrift.exceptions | If the call resulted in exceptions, this field contains the exceptions in a human readable format. | keyword |
network_traffic.thrift.method | The command/verb/method of the transaction. | keyword |
network_traffic.thrift.params | The RPC method call parameters in a human readable format. If the IDL files are available, the parameters use names whenever possible. Otherwise, the IDs from the message are used. | keyword |
network_traffic.thrift.path | The path the transaction refers to. | keyword |
network_traffic.thrift.query | The query in a human readable format. | keyword |
network_traffic.thrift.return_value | The value returned by the Thrift-RPC call. This is encoded in a human readable format. | keyword |
network_traffic.thrift.service | The name of the Thrift-RPC service as defined in the IDL files. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
thrift.exceptions | If the call resulted in exceptions, this field contains the exceptions in a human readable format. | keyword |
thrift.params | The RPC method call parameters in a human readable format. If the IDL files are available, the parameters use names whenever possible. Otherwise, the IDs from the message are used. | keyword |
thrift.return_value | The value returned by the Thrift-RPC call. This is encoded in a human readable format. | keyword |
thrift.service | The name of the Thrift-RPC service as defined in the IDL files. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for thrift looks as following:
{
"@timestamp": "2023-10-16T23:26:46.507Z",
"agent": {
"ephemeral_id": "69d13820-6026-4f1f-8829-05ce967ab5b7",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"bytes": 25,
"ip": "127.0.0.1",
"port": 50919
},
"data_stream": {
"dataset": "network_traffic.thrift",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 25,
"ip": "127.0.0.1",
"port": 9090
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.thrift",
"duration": 1354815,
"end": "2023-10-16T23:26:46.508Z",
"ingested": "2023-10-16T23:26:50Z",
"kind": "event",
"start": "2023-10-16T23:26:46.507Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"method": "testByte",
"network": {
"bytes": 50,
"community_id": "1:fs+HuhTN3hqKiWHtoK/DsQ0ni5Y=",
"direction": "ingress",
"protocol": "thrift",
"transport": "tcp",
"type": "ipv4"
},
"path": "",
"query": "testByte(1: 63)",
"related": {
"ip": [
"127.0.0.1"
]
},
"server": {
"bytes": 25,
"ip": "127.0.0.1",
"port": 9090
},
"source": {
"bytes": 25,
"ip": "127.0.0.1",
"port": 50919
},
"status": "OK",
"thrift": {
"params": "(1: 63)",
"return_value": "63"
},
"type": "thrift"
}
TLS
TLS is a cryptographic protocol that provides secure communications on top of an existing application protocol, like HTTP or MySQL.
Network Packet Capture intercepts the initial handshake in a TLS connection and extracts useful information that helps operators diagnose problems and strengthen the security of their network and systems. It does not decrypt any information from the encapsulated protocol, nor does it reveal any sensitive information such as cryptographic keys. TLS versions 1.0 to 1.3 are supported.
It works by intercepting the client and server "hello" messages, which contain the negotiated parameters for the connection such as cryptographic ciphers and protocol versions. It can also intercept TLS alerts, which are sent by one of the parties to signal a problem with the negotiation, such as an expired certificate or a cryptographic error.
Detailed information that is not defined in ECS is added under the
tls.detailed key. The include_detailed_fields configuration flag
is used to control whether this information is exported.
The fields under tls.detailed.client_hello contain the algorithms and
extensions supported by the client, as well as the maximum TLS version
it supports.
Fields under tls.detailed.server_hello contain the final settings for
the TLS session: The selected cipher, compression method, TLS version to
use and other extensions such as application layer protocol negotiation
(ALPN).
Configuration options
The send_certificates and include_detailed_fields settings are
useful for limiting the amount of data Network Packet Capture indexes, as multiple
certificates are usually exchanged in a single transaction, and those
can take a considerable amount of storage.
Also see Common protocol options.
send_certificates
This setting causes information about the certificates presented by the
client and server to be included in the detailed fields. The server’s
certificate is indexed under tls.detailed.server_certificate and its
certification chain under tls.detailed.server_certificate_chain. For
the client, the client_certificate and client_certificate_chain
fields are used. The default is true.
include_raw_certificates
You can set include_raw_certificates to include the raw certificate
chains encoded in PEM format, under the tls.server.certificate_chain
and tls.client.certificate_chain fields. The default is false.
include_detailed_fields
Controls whether the TLS fields are added to exported documents. When
set to false, only ECS TLS fields are included.
exported are included. The default is true.
fingerprints
Defines a list of hash algorithms to calculate the certificate’s
fingerprints. Valid values are sha1, sha256 and md5.
The default is to output SHA-1 fingerprints.
Fields published for TLS packets.
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
client.bytes | Bytes sent from the client to the server. | long |
client.geo.city_name | City name. | keyword |
client.geo.continent_name | Name of the continent. | keyword |
client.geo.country_iso_code | Country ISO code. | keyword |
client.geo.country_name | Country name. | keyword |
client.geo.location | Longitude and latitude. | geo_point |
client.geo.region_iso_code | Region ISO code. | keyword |
client.geo.region_name | Region name. | keyword |
client.ip | IP address of the client (IPv4 or IPv6). | ip |
client.port | Port of the client. | long |
client.process.args | The command-line of the process that initiated the transaction. | keyword |
client.process.executable | Absolute path to the client process executable. | keyword |
client.process.name | The name of the process that initiated the transaction. | keyword |
client.process.start | The time the client process started. | date |
client.process.working_directory | The working directory of the client process. | keyword |
cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
cloud.availability_zone | Availability zone in which this host is running. | keyword |
cloud.image.id | Image ID for the cloud instance. | keyword |
cloud.instance.id | Instance ID of the host machine. | keyword |
cloud.instance.name | Instance name of the host machine. | keyword |
cloud.machine.type | Machine type of the host machine. | keyword |
cloud.project.id | Name of the project in Google Cloud. | keyword |
cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
cloud.region | Region in which this host is running. | keyword |
container.id | Unique container id. | keyword |
container.image.name | Name of the image the container was built on. | keyword |
container.labels | Image labels. | object |
container.name | Container name. | keyword |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
destination.bytes | Bytes sent from the destination to the source. | long |
destination.domain | The domain name of the destination system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
destination.geo.city_name | City name. | keyword |
destination.geo.continent_name | Name of the continent. | keyword |
destination.geo.country_iso_code | Country ISO code. | keyword |
destination.geo.country_name | Country name. | keyword |
destination.geo.location | Longitude and latitude. | geo_point |
destination.geo.region_iso_code | Region ISO code. | keyword |
destination.geo.region_name | Region name. | keyword |
destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
destination.port | Port of the destination. | long |
ecs.version | ECS version this event conforms to. ecs.version is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword |
event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword |
event.duration | Duration of the event in nanoseconds. If event.start and event.end are known this value should be the difference between the end and start time. | long |
event.end | event.end contains the date when the event ended or when the activity was last observed. | date |
event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword |
event.start | event.start contains the date when the event started or when the activity was first observed. | date |
event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
flow.final | Indicates if event is last event in flow. If final is false, the event reports an intermediate flow state only. | boolean |
flow.id | Internal flow ID based on connection meta data and address. | keyword |
flow.vlan | VLAN identifier from the 802.1q frame. In case of a multi-tagged frame this field will be an array with the outer tag's VLAN identifier listed first. | long |
host.architecture | Operating system architecture. | keyword |
host.containerized | If the host is a container. | boolean |
host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
host.hostname | Hostname of the host. It normally contains what the hostname command returns on the host machine. | keyword |
host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of beat.name. | keyword |
host.ip | Host ip addresses. | ip |
host.mac | Host mac addresses. | keyword |
host.name | Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
host.os.build | OS build information. | keyword |
host.os.codename | OS codename, if any. | keyword |
host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
host.os.kernel | Operating system kernel version as a raw string. | keyword |
host.os.name | Operating system name, without the version. | keyword |
host.os.name.text | Multi-field of host.os.name. | text |
host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
host.os.version | Operating system version as a raw string. | keyword |
host.type | Type of host. For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
method | The command/verb/method of the transaction. For HTTP, this is the method name (GET, POST, PUT, and so on), for SQL this is the verb (SELECT, UPDATE, DELETE, and so on). | keyword |
network.bytes | Total bytes transferred in both directions. If source.bytes and destination.bytes are known, network.bytes is their sum. | long |
network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |
network.direction | Direction of the network traffic. When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. | keyword |
network.forwarded_ip | Host IP address when the source IP address is the proxy. | ip |
network.protocol | In the OSI Model this would be the Application Layer protocol. For example, http, dns, or ssh. The field value must be normalized to lowercase for querying. | keyword |
network.transport | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) The field value must be normalized to lowercase for querying. | keyword |
network.type | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc The field value must be normalized to lowercase for querying. | keyword |
network_traffic.status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
observer.hostname | Hostname of the observer. | keyword |
observer.ip | IP addresses of the observer. | ip |
observer.mac | MAC addresses of the observer. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
observer.name | Custom name of the observer. This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. If no custom name is needed, the field can be left empty. | keyword |
params | The request parameters. For HTTP, these are the POST or GET parameters. For Thrift-RPC, these are the parameters from the request. | text |
path | The path the transaction refers to. For HTTP, this is the URL. For SQL databases, this is the table name. For key-value stores, this is the key. | keyword |
process.args | Array of process arguments, starting with the absolute path to the executable. May be filtered to protect sensitive information. | keyword |
process.executable | Absolute path to the process executable. | keyword |
process.executable.text | Multi-field of process.executable. | match_only_text |
process.name | Process name. Sometimes called program name or similar. | keyword |
process.name.text | Multi-field of process.name. | match_only_text |
process.start | The time the process started. | date |
process.working_directory | The working directory of the process. | keyword |
process.working_directory.text | Multi-field of process.working_directory. | match_only_text |
query | The query in a human readable format. For HTTP, it will typically be something like GET /users/_search?name=test. For MySQL, it is something like SELECT id from users where name=test. | keyword |
related.hash | All the hashes seen on your event. Populating this field, then using it to search for hashes can help in situations where you're unsure what the hash algorithm is (and therefore which key name to search). | keyword |
related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword |
related.ip | All of the IPs seen on your event. | ip |
request | For text protocols, this is the request as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
resource | The logical resource that this transaction refers to. For HTTP, this is the URL path up to the last slash (/). For example, if the URL is /users/1, the resource is /users. For databases, the resource is typically the table name. The field is not filled for all transaction types. | keyword |
response | For text protocols, this is the response as seen on the wire (application layer only). For binary protocols this is our representation of the request. | text |
server.bytes | Bytes sent from the server to the client. | long |
server.domain | The domain name of the server system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
server.geo.city_name | City name. | keyword |
server.geo.continent_name | Name of the continent. | keyword |
server.geo.country_iso_code | Country ISO code. | keyword |
server.geo.country_name | Country name. | keyword |
server.geo.location | Longitude and latitude. | geo_point |
server.geo.region_iso_code | Region ISO code. | keyword |
server.geo.region_name | Region name. | keyword |
server.ip | IP address of the server (IPv4 or IPv6). | ip |
server.port | Port of the server. | long |
server.process.args | The command-line of the process that served the transaction. | keyword |
server.process.executable | Absolute path to the server process executable. | keyword |
server.process.name | The name of the process that served the transaction. | keyword |
server.process.start | The time the server process started. | date |
server.process.working_directory | The working directory of the server process. | keyword |
source.bytes | Bytes sent from the source to the destination. | long |
source.geo.city_name | City name. | keyword |
source.geo.continent_name | Name of the continent. | keyword |
source.geo.country_iso_code | Country ISO code. | keyword |
source.geo.country_name | Country name. | keyword |
source.geo.location | Longitude and latitude. | geo_point |
source.geo.region_iso_code | Region ISO code. | keyword |
source.geo.region_name | Region name. | keyword |
source.ip | IP address of the source (IPv4 or IPv6). | ip |
source.port | Port of the source. | long |
status | The high level status of the transaction. The way to compute this value depends on the protocol, but the result has a meaning independent of the protocol. | keyword |
tags | List of keywords used to tag each event. | keyword |
tls.cipher | String indicating the cipher used during the current connection. | keyword |
tls.client.certificate | PEM-encoded stand-alone certificate offered by the client. This is usually mutually-exclusive of client.certificate_chain since this value also exists in that list. | keyword |
tls.client.certificate_chain | Array of PEM-encoded certificates that make up the certificate chain offered by the client. This is usually mutually-exclusive of client.certificate since that value should be the first certificate in the chain. | keyword |
tls.client.hash.md5 | Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash. | keyword |
tls.client.hash.sha1 | Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash. | keyword |
tls.client.hash.sha256 | Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash. | keyword |
tls.client.issuer | Distinguished name of subject of the issuer of the x.509 certificate presented by the client. | keyword |
tls.client.ja3 | A hash that identifies clients based on how they perform an SSL/TLS handshake. | keyword |
tls.client.not_after | Date/Time indicating when client certificate is no longer considered valid. | date |
tls.client.not_before | Date/Time indicating when client certificate is first considered valid. | date |
tls.client.server_name | Also called an SNI, this tells the server which hostname to which the client is attempting to connect to. When this value is available, it should get copied to destination.domain. | keyword |
tls.client.subject | Distinguished name of subject of the x.509 certificate presented by the client. | keyword |
tls.client.supported_ciphers | Array of ciphers offered by the client during the client hello. | keyword |
tls.client.x509.alternative_names | List of subject alternative names (SAN). Name types vary by certificate authority and certificate type but commonly contain IP addresses, DNS names (and wildcards), and email addresses. | keyword |
tls.client.x509.issuer.common_name | List of common name (CN) of issuing certificate authority. | keyword |
tls.client.x509.issuer.country | List of country (C) codes | keyword |
tls.client.x509.issuer.distinguished_name | Distinguished name (DN) of issuing certificate authority. | keyword |
tls.client.x509.issuer.locality | List of locality names (L) | keyword |
tls.client.x509.issuer.organization | List of organizations (O) of issuing certificate authority. | keyword |
tls.client.x509.issuer.organizational_unit | List of organizational units (OU) of issuing certificate authority. | keyword |
tls.client.x509.issuer.state_or_province | List of state or province names (ST, S, or P) | keyword |
tls.client.x509.not_after | Time at which the certificate is no longer considered valid. | date |
tls.client.x509.not_before | Time at which the certificate is first considered valid. | date |
tls.client.x509.public_key_algorithm | Algorithm used to generate the public key. | keyword |
tls.client.x509.public_key_curve | The curve used by the elliptic curve public key algorithm. This is algorithm specific. | keyword |
tls.client.x509.public_key_exponent | Exponent used to derive the public key. This is algorithm specific. | long |
tls.client.x509.public_key_size | The size of the public key space in bits. | long |
tls.client.x509.serial_number | Unique serial number issued by the certificate authority. For consistency, if this value is alphanumeric, it should be formatted without colons and uppercase characters. | keyword |
tls.client.x509.signature_algorithm | Identifier for certificate signature algorithm. We recommend using names found in Go Lang Crypto library. See https://github.com/golang/go/blob/go1.14/src/crypto/x509/x509.go#L337-L353. | keyword |
tls.client.x509.subject.common_name | List of common names (CN) of subject. | keyword |
tls.client.x509.subject.country | List of country (C) code | keyword |
tls.client.x509.subject.distinguished_name | Distinguished name (DN) of the certificate subject entity. | keyword |
tls.client.x509.subject.locality | List of locality names (L) | keyword |
tls.client.x509.subject.organization | List of organizations (O) of subject. | keyword |
tls.client.x509.subject.organizational_unit | List of organizational units (OU) of subject. | keyword |
tls.client.x509.subject.state_or_province | List of state or province names (ST, S, or P) | keyword |
tls.client.x509.version_number | Version of x509 format. | keyword |
tls.curve | String indicating the curve used for the given cipher, when applicable. | keyword |
tls.detailed.alert_types | An array containing the TLS alert type for every alert received. | keyword |
tls.detailed.client_certificate_chain.alternative_names | Subject alternative names (SANs) in the certificate. | keyword |
tls.detailed.client_certificate_chain.issuer.common_name | keyword | |
tls.detailed.client_certificate_chain.issuer.country | keyword | |
tls.detailed.client_certificate_chain.issuer.distinguished_name | keyword | |
tls.detailed.client_certificate_chain.issuer.locality | keyword | |
tls.detailed.client_certificate_chain.issuer.organization | keyword | |
tls.detailed.client_certificate_chain.issuer.organizational_unit | keyword | |
tls.detailed.client_certificate_chain.issuer.postal_code | keyword | |
tls.detailed.client_certificate_chain.issuer.serial_number | keyword | |
tls.detailed.client_certificate_chain.issuer.state_or_province | keyword | |
tls.detailed.client_certificate_chain.issuer.street_address | keyword | |
tls.detailed.client_certificate_chain.not_after | End of the validity period (inclusive). | date |
tls.detailed.client_certificate_chain.not_before | Start of the validity period (inclusive). | date |
tls.detailed.client_certificate_chain.public_key_algorithm | Public key algorithm (e.g. RSA, DSA, ECDSA, Ed25519). | keyword |
tls.detailed.client_certificate_chain.public_key_size | Number of bits in the public key. | long |
tls.detailed.client_certificate_chain.serial_number | Base 10 representation of the certificate serial number. | keyword |
tls.detailed.client_certificate_chain.signature_algorithm | Signature algorithm (e.g. SHA256-RSA). | keyword |
tls.detailed.client_certificate_chain.subject.common_name | keyword | |
tls.detailed.client_certificate_chain.subject.country | keyword | |
tls.detailed.client_certificate_chain.subject.distinguished_name | keyword | |
tls.detailed.client_certificate_chain.subject.locality | keyword | |
tls.detailed.client_certificate_chain.subject.organization | keyword | |
tls.detailed.client_certificate_chain.subject.organizational_unit | keyword | |
tls.detailed.client_certificate_chain.subject.postal_code | keyword | |
tls.detailed.client_certificate_chain.subject.serial_number | keyword | |
tls.detailed.client_certificate_chain.subject.state_or_province | keyword | |
tls.detailed.client_certificate_chain.subject.street_address | keyword | |
tls.detailed.client_certificate_chain.version_number | The x509 certificate version. Version 3 is the latest and most common. | keyword |
tls.detailed.client_certificate_requested | Whether the server has requested the client to authenticate itself using a client certificate. | boolean |
tls.detailed.client_hello.extensions.unparsed | List of extensions that were left unparsed by Packetbeat. | keyword |
tls.detailed.client_hello.extensions.application_layer_protocol_negotiation | List of application-layer protocols the client is willing to use. | keyword |
tls.detailed.client_hello.extensions.ec_points_formats | List of Elliptic Curve (EC) point formats. Indicates the set of point formats that the client can parse. | keyword |
tls.detailed.client_hello.extensions.server_name_indication | List of hostnames | keyword |
tls.detailed.client_hello.extensions.session_ticket | Length of the session ticket, if provided, or an empty string to advertise support for tickets. | keyword |
tls.detailed.client_hello.extensions.signature_algorithms | List of signature algorithms that may be use in digital signatures. | keyword |
tls.detailed.client_hello.extensions.status_request.request_extensions | The number of certificate extensions for the request. | short |
tls.detailed.client_hello.extensions.status_request.responder_id_list_length | The length of the list of trusted responders. | short |
tls.detailed.client_hello.extensions.status_request.type | The type of the status request. Always "ocsp" if present. | keyword |
tls.detailed.client_hello.extensions.supported_groups | List of Elliptic Curve Cryptography (ECC) curve groups supported by the client. | keyword |
tls.detailed.client_hello.extensions.supported_versions | List of TLS versions that the client is willing to use. | keyword |
tls.detailed.client_hello.random | Random data used by the TLS protocol to generate the encryption key. | keyword |
tls.detailed.client_hello.session_id | Unique number to identify the session for the corresponding connection with the client. | keyword |
tls.detailed.client_hello.supported_compression_methods | The list of compression methods the client supports. See https://www.iana.org/assignments/comp-meth-ids/comp-meth-ids.xhtml | keyword |
tls.detailed.client_hello.version | The version of the TLS protocol by which the client wishes to communicate during this session. | keyword |
tls.detailed.ocsp_response | The result of an OCSP request. | keyword |
tls.detailed.resumption_method | If the session has been resumed, the underlying method used. One of "id" for TLS session ID or "ticket" for TLS ticket extension. | keyword |
tls.detailed.server_certificate_chain.alternative_names | Subject alternative names (SANs) in the certificate. | keyword |
tls.detailed.server_certificate_chain.issuer.common_name | keyword | |
tls.detailed.server_certificate_chain.issuer.country | keyword | |
tls.detailed.server_certificate_chain.issuer.distinguished_name | keyword | |
tls.detailed.server_certificate_chain.issuer.locality | keyword | |
tls.detailed.server_certificate_chain.issuer.organization | keyword | |
tls.detailed.server_certificate_chain.issuer.organizational_unit | keyword | |
tls.detailed.server_certificate_chain.issuer.postal_code | keyword | |
tls.detailed.server_certificate_chain.issuer.serial_number | keyword | |
tls.detailed.server_certificate_chain.issuer.state_or_province | keyword | |
tls.detailed.server_certificate_chain.issuer.street_address | keyword | |
tls.detailed.server_certificate_chain.not_after | End of the validity period (inclusive). | date |
tls.detailed.server_certificate_chain.not_before | Start of the validity period (inclusive). | date |
tls.detailed.server_certificate_chain.public_key_algorithm | Public key algorithm (e.g. RSA, DSA, ECDSA, Ed25519). | keyword |
tls.detailed.server_certificate_chain.public_key_size | Number of bits in the public key. | long |
tls.detailed.server_certificate_chain.serial_number | Base 10 representation of the certificate serial number. | keyword |
tls.detailed.server_certificate_chain.signature_algorithm | Signature algorithm (e.g. SHA256-RSA). | keyword |
tls.detailed.server_certificate_chain.subject.common_name | keyword | |
tls.detailed.server_certificate_chain.subject.country | keyword | |
tls.detailed.server_certificate_chain.subject.distinguished_name | keyword | |
tls.detailed.server_certificate_chain.subject.locality | keyword | |
tls.detailed.server_certificate_chain.subject.organization | keyword | |
tls.detailed.server_certificate_chain.subject.organizational_unit | keyword | |
tls.detailed.server_certificate_chain.subject.postal_code | keyword | |
tls.detailed.server_certificate_chain.subject.serial_number | keyword | |
tls.detailed.server_certificate_chain.subject.state_or_province | keyword | |
tls.detailed.server_certificate_chain.subject.street_address | keyword | |
tls.detailed.server_certificate_chain.version_number | The x509 certificate version. Version 3 is the latest and most common. | keyword |
tls.detailed.server_hello.extensions.unparsed | List of extensions that were left unparsed by Packetbeat. | keyword |
tls.detailed.server_hello.extensions.application_layer_protocol_negotiation | Negotiated application layer protocol | keyword |
tls.detailed.server_hello.extensions.ec_points_formats | List of Elliptic Curve (EC) point formats. Indicates the set of point formats that the server can parse. | keyword |
tls.detailed.server_hello.extensions.session_ticket | Used to announce that a session ticket will be provided by the server. Always an empty string. | keyword |
tls.detailed.server_hello.extensions.status_request.response | Whether a certificate status request response was made. | boolean |
tls.detailed.server_hello.extensions.supported_versions | Negotiated TLS version to be used. | keyword |
tls.detailed.server_hello.random | Random data used by the TLS protocol to generate the encryption key. | keyword |
tls.detailed.server_hello.selected_compression_method | The compression method selected by the server from the list provided in the client hello. | keyword |
tls.detailed.server_hello.session_id | Unique number to identify the session for the corresponding connection with the client. | keyword |
tls.detailed.server_hello.version | The version of the TLS protocol that is used for this session. It is the highest version supported by the server not exceeding the version requested in the client hello. | keyword |
tls.detailed.version | The version of the TLS protocol used. | keyword |
tls.established | Boolean flag indicating if the TLS negotiation was successful and transitioned to an encrypted tunnel. | boolean |
tls.next_protocol | String indicating the protocol being tunneled. Per the values in the IANA registry (https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids), this string should be lower case. | keyword |
tls.resumed | Boolean flag indicating if this TLS connection was resumed from an existing TLS negotiation. | boolean |
tls.server.certificate | PEM-encoded stand-alone certificate offered by the server. This is usually mutually-exclusive of server.certificate_chain since this value also exists in that list. | keyword |
tls.server.certificate_chain | Array of PEM-encoded certificates that make up the certificate chain offered by the server. This is usually mutually-exclusive of server.certificate since that value should be the first certificate in the chain. | keyword |
tls.server.hash.md5 | Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash. | keyword |
tls.server.hash.sha1 | Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash. | keyword |
tls.server.hash.sha256 | Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash. | keyword |
tls.server.issuer | Subject of the issuer of the x.509 certificate presented by the server. | keyword |
tls.server.ja3s | A hash that identifies servers based on how they perform an SSL/TLS handshake. | keyword |
tls.server.not_after | Timestamp indicating when server certificate is no longer considered valid. | date |
tls.server.not_before | Timestamp indicating when server certificate is first considered valid. | date |
tls.server.subject | Subject of the x.509 certificate presented by the server. | keyword |
tls.server.x509.alternative_names | List of subject alternative names (SAN). Name types vary by certificate authority and certificate type but commonly contain IP addresses, DNS names (and wildcards), and email addresses. | keyword |
tls.server.x509.issuer.common_name | List of common name (CN) of issuing certificate authority. | keyword |
tls.server.x509.issuer.country | List of country (C) codes | keyword |
tls.server.x509.issuer.distinguished_name | Distinguished name (DN) of issuing certificate authority. | keyword |
tls.server.x509.issuer.locality | List of locality names (L) | keyword |
tls.server.x509.issuer.organization | List of organizations (O) of issuing certificate authority. | keyword |
tls.server.x509.issuer.organizational_unit | List of organizational units (OU) of issuing certificate authority. | keyword |
tls.server.x509.issuer.state_or_province | List of state or province names (ST, S, or P) | keyword |
tls.server.x509.not_after | Time at which the certificate is no longer considered valid. | date |
tls.server.x509.not_before | Time at which the certificate is first considered valid. | date |
tls.server.x509.public_key_algorithm | Algorithm used to generate the public key. | keyword |
tls.server.x509.public_key_curve | The curve used by the elliptic curve public key algorithm. This is algorithm specific. | keyword |
tls.server.x509.public_key_exponent | Exponent used to derive the public key. This is algorithm specific. | long |
tls.server.x509.public_key_size | The size of the public key space in bits. | long |
tls.server.x509.serial_number | Unique serial number issued by the certificate authority. For consistency, if this value is alphanumeric, it should be formatted without colons and uppercase characters. | keyword |
tls.server.x509.signature_algorithm | Identifier for certificate signature algorithm. We recommend using names found in Go Lang Crypto library. See https://github.com/golang/go/blob/go1.14/src/crypto/x509/x509.go#L337-L353. | keyword |
tls.server.x509.subject.common_name | List of common names (CN) of subject. | keyword |
tls.server.x509.subject.country | List of country (C) code | keyword |
tls.server.x509.subject.distinguished_name | Distinguished name (DN) of the certificate subject entity. | keyword |
tls.server.x509.subject.locality | List of locality names (L) | keyword |
tls.server.x509.subject.organization | List of organizations (O) of subject. | keyword |
tls.server.x509.subject.organizational_unit | List of organizational units (OU) of subject. | keyword |
tls.server.x509.subject.state_or_province | List of state or province names (ST, S, or P) | keyword |
tls.server.x509.version_number | Version of x509 format. | keyword |
tls.version | Numeric part of the version parsed from the original string. | keyword |
tls.version_protocol | Normalized lowercase protocol name parsed from original string. | keyword |
type | The type of the transaction (for example, HTTP, MySQL, Redis, or RUM) or "flow" in case of flows. | keyword |
An example event for tls looks as following:
{
"@timestamp": "2023-10-16T23:27:36.939Z",
"agent": {
"ephemeral_id": "5041812f-2c64-48f2-b040-7814b7a8398f",
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"name": "docker-fleet-agent",
"type": "packetbeat",
"version": "8.6.2"
},
"client": {
"ip": "192.168.1.36",
"port": 60946
},
"data_stream": {
"dataset": "network_traffic.tls",
"namespace": "ep",
"type": "logs"
},
"destination": {
"domain": "play.google.com",
"ip": "216.58.201.174",
"port": 443
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "f923dfe0-3acb-4f62-9ab4-1fabb8e8e112",
"snapshot": false,
"version": "8.6.2"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "network_traffic.tls",
"duration": 15311303,
"end": "2023-10-16T23:27:36.954Z",
"ingested": "2023-10-16T23:27:37Z",
"kind": "event",
"start": "2023-10-16T23:27:36.939Z",
"type": [
"connection",
"protocol"
]
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "docker-fleet-agent",
"id": "f91b175388d443fca5c155815dfc2279",
"ip": [
"172.19.0.7"
],
"mac": [
"02-42-AC-13-00-07"
],
"name": "docker-fleet-agent",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.49-linuxkit",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.5 LTS (Focal Fossa)"
}
},
"network": {
"community_id": "1:hfsK5r0tJm7av4j7BtSxA6oH9xA=",
"direction": "unknown",
"protocol": "tls",
"transport": "tcp",
"type": "ipv4"
},
"related": {
"hash": [
"d470a3fa301d80227bc5650c75567d25"
],
"ip": [
"192.168.1.36",
"216.58.201.174"
]
},
"server": {
"domain": "play.google.com",
"ip": "216.58.201.174",
"port": 443
},
"source": {
"ip": "192.168.1.36",
"port": 60946
},
"status": "OK",
"tls": {
"cipher": "TLS_AES_128_GCM_SHA256",
"client": {
"ja3": "d470a3fa301d80227bc5650c75567d25",
"server_name": "play.google.com",
"supported_ciphers": [
"TLS_AES_128_GCM_SHA256",
"TLS_CHACHA20_POLY1305_SHA256",
"TLS_AES_256_GCM_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA",
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA",
"TLS_RSA_WITH_AES_128_CBC_SHA",
"TLS_RSA_WITH_AES_256_CBC_SHA",
"TLS_RSA_WITH_3DES_EDE_CBC_SHA"
]
},
"detailed": {
"client_certificate_requested": false,
"client_hello": {
"extensions": {
"_unparsed_": [
"23",
"renegotiation_info",
"51",
"45",
"28",
"41"
],
"application_layer_protocol_negotiation": [
"h2",
"http/1.1"
],
"ec_points_formats": [
"uncompressed"
],
"server_name_indication": [
"play.google.com"
],
"signature_algorithms": [
"ecdsa_secp256r1_sha256",
"ecdsa_secp384r1_sha384",
"ecdsa_secp521r1_sha512",
"rsa_pss_sha256",
"rsa_pss_sha384",
"rsa_pss_sha512",
"rsa_pkcs1_sha256",
"rsa_pkcs1_sha384",
"rsa_pkcs1_sha512",
"ecdsa_sha1",
"rsa_pkcs1_sha1"
],
"status_request": {
"request_extensions": 0,
"responder_id_list_length": 0,
"type": "ocsp"
},
"supported_groups": [
"x25519",
"secp256r1",
"secp384r1",
"secp521r1",
"ffdhe2048",
"ffdhe3072"
],
"supported_versions": [
"TLS 1.3",
"TLS 1.2",
"TLS 1.1",
"TLS 1.0"
]
},
"random": "03ce74e1536e0272c1d55b0c8cdf324e82f80a276e478645572324ce25910c00",
"session_id": "5d2b9f80d34143b5764ba6b23e1d4f9d1f172148b6fd83c81f42663459eaf6f6",
"supported_compression_methods": [
"NULL"
],
"version": "3.3"
},
"resumption_method": "id",
"server_hello": {
"extensions": {
"_unparsed_": [
"41",
"51"
],
"supported_versions": "TLS 1.3"
},
"random": "ebd86864767a7782922db6712f487c22c6cb65e54a895fefd3f60bc851591f19",
"selected_compression_method": "NULL",
"session_id": "5d2b9f80d34143b5764ba6b23e1d4f9d1f172148b6fd83c81f42663459eaf6f6",
"version": "3.3"
},
"version": "TLS 1.3"
},
"established": true,
"resumed": true,
"version": "1.3",
"version_protocol": "tls"
},
"type": "tls"
}
Licensing for Windows Systems
The Network Packet Capture Integration incorporates a bundled Npcap installation on Windows hosts. The installation is provided under an OEM license from Insecure.Com LLC ("The Nmap Project").
Changelog
| Version | Details | Kibana version(s) |
|---|---|---|
1.30.0 | Enhancement View pull request | 8.6.2 or higher |
1.29.1 | Enhancement View pull request | 8.6.2 or higher |
1.29.0 | Enhancement View pull request | 8.6.2 or higher |
1.28.0 | Enhancement View pull request | 8.6.2 or higher |
1.27.0 | Enhancement View pull request | 8.6.2 or higher |
1.26.0 | Enhancement View pull request | 8.6.2 or higher |
1.25.1 | Bug fix View pull request | 8.6.2 or higher |
1.25.0 | Enhancement View pull request | 8.6.2 or higher |
1.24.0 | Enhancement View pull request | 8.6.2 or higher |
1.23.0 | Enhancement View pull request | 8.6.2 or higher |
1.22.0 | Enhancement View pull request | 8.6.2 or higher |
1.21.0 | Enhancement View pull request | 8.6.2 or higher |
1.20.0 | Enhancement View pull request | 8.6.2 or higher |
1.19.3 | Bug fix View pull request | 8.6.2 or higher |
1.19.2 | Bug fix View pull request | 8.6.2 or higher |
1.19.1 | Bug fix View pull request | 8.6.2 or higher |
1.19.0 | Enhancement View pull request | 8.6.2 or higher |
1.18.0 | Enhancement View pull request | 8.6.2 or higher |
1.17.0 | Enhancement View pull request | 8.6.2 or higher |
1.16.0 | Enhancement View pull request | 8.6.2 or higher |
1.15.1 | Bug fix View pull request | 8.6.2 or higher |
1.15.0 | Enhancement View pull request | 8.6.2 or higher |
1.14.0 | Enhancement View pull request | 8.5.3 or higher |
1.13.0 | Enhancement View pull request | 8.5.3 or higher |
1.12.0 | Bug fix View pull request | 8.5.3 or higher |
1.11.1 | Bug fix View pull request | 8.4.0 or higher |
1.11.0 | Enhancement View pull request | 8.4.0 or higher |
1.10.1 | Bug fix View pull request | 8.4.0 or higher |
1.10.0 | Enhancement View pull request | — |
1.9.3 | Enhancement View pull request | 8.4.0 or higher |
1.9.2 | Bug fix View pull request | 8.4.0 or higher |
1.9.1 | Bug fix View pull request | 8.4.0 or higher |
1.9.0 | Enhancement View pull request | 8.4.0 or higher |
1.8.0 | Enhancement View pull request | 8.4.0 or higher |
1.7.1 | Enhancement View pull request | 8.4.0 or higher |
1.7.0 | Enhancement View pull request | 8.4.0 or higher |
1.6.1 | Enhancement View pull request | 8.4.0 or higher |
1.6.0 | Enhancement View pull request | 8.4.0 or higher |
1.5.0 | Enhancement View pull request | 8.4.0 or higher |
1.4.1 | Bug fix View pull request | 7.17.0 or higher |
1.4.0 | Enhancement View pull request | — |
1.3.1 | Enhancement View pull request | 7.17.0 or higher |
1.3.0 | Enhancement View pull request | — |
1.2.0 | Enhancement View pull request | 7.17.0 or higher |
1.1.0 | Enhancement View pull request | 7.17.0 or higher |
1.0.2 | Bug fix View pull request | — |
1.0.1 | Bug fix View pull request | — |
1.0.0 | Enhancement View pull request | — |
0.10.1 | Bug fix View pull request | — |
0.10.0 | Enhancement View pull request | — |
0.9.0 | Enhancement View pull request | — |
0.8.2 | Bug fix View pull request | — |
0.8.1 | Enhancement View pull request | — |
0.8.0 | Enhancement View pull request | — |
0.7.1 | Bug fix View pull request | — |
0.7.0 | Enhancement View pull request | — |
0.6.3 | Bug fix View pull request | — |
0.6.2 | Enhancement View pull request | — |
0.6.1 | Enhancement View pull request | — |
0.6.0 | Enhancement View pull request | — |
0.5.1 | Bug fix View pull request | — |
0.5.0 | Enhancement View pull request | — |
0.4.2 | Enhancement View pull request | — |
0.4.1 | Enhancement View pull request | — |
0.4.0 | Enhancement View pull request | — |
0.3.0 | Enhancement View pull request | — |
0.2.2 | Bug fix View pull request | — |
0.2.1 | Enhancement View pull request | — |
0.2.0 | Enhancement View pull request | — |
0.1.0 | Enhancement View pull request | — |
0.0.1 | Enhancement View pull request | — |