Log

Log

By using ServiceWatch logs, you can monitor, store, and access log files collected from the resources of the service that provides the logs.

Log Group

A log group is a container for log streams that share the same retention policy settings. Each log stream must belong to a single log group. For example, if there are separate log streams for the logs of each Kubernetes Engine cluster, you can group the log streams into a single log group called /scp/ske/{cluster name}.

Log Retention Policy

Log retention policy can set the period for storing log events in ServiceWatch. Log events whose period has expired are automatically deleted. Log The retention period assigned to the group applies to the log streams and log events belonging to the log group.

The retention period can be selected from the following and is set in days.

Log Stream

A log stream is a collection of log events sorted in the order they occurred from the same source. For example, all log events generated in a particular Kubernetes Engine cluster can constitute a single log stream.

Log Event

Log events are individual records that record logs generated from resources. A log event record includes a timestamp of when the event occurred, a log message, and two attributes. Each message must be encoded in UTF-8.

Log Pattern

You can create a log pattern to filter log data that matches the pattern. A log pattern defines the words or patterns to search for in the log data collected by ServiceWatch, allows you to view the status of log occurrences in a graph, and creates metrics that can be used to generate alert policies.

Log patterns are not applied retroactively to data. They are applied to log events collected after the log pattern is created.

Log Pattern Namespace

A namespace is a logical separation for distinguishing and grouping metrics. In ServiceWatch, it is divided into namespaces associated with services, namespaces for custom metrics, and namespaces for log patterns.

• Namespace associated with services such as Virtual Server
• Namespace composed of custom metrics, the namespace of metrics collected via the custom metrics API or ServiceWatch Agent
• Namespace of the metric created by the log pattern

When creating metrics for a log pattern, you can either create a new namespace for the log pattern or choose from existing log pattern namespaces.

Indicator Name

The monitored log information is the name of the metric generated by ServiceWatch. You must set it so that the metric name does not duplicate within the namespace where the metric will exist.

Indicator value

It is the numeric value posted to the metric each time a log matching the pattern is found. For example, when counting occurrences of a specific word (e.g., Error), this value becomes 1 for each occurrence. When calculating transmitted bytes, it can be incremented by the actual number of bytes found in the log event.

Default

It is the value recorded in the log pattern during periods when no matching logs can be found while collecting logs. Setting the default to 0 can prevent the metric from becoming irregular due to periods with no matching data in all such intervals.

If you set a dimension for a metric generated by a log pattern, you cannot set a default value for that metric.

Dimension

A dimension is a key-value pair that defines a metric additionally. You can add dimensions to metrics generated from log patterns. Since a dimension is part of a unique identifier for a metric, each time a unique name/value pair is extracted from the log, a new variant of that metric is created.

When selecting the log pattern format as either a space-separated pattern or a JSON format pattern, you can set the dimension, and it can be configured as one of the parameters set in the pattern. You can assign up to three dimensions to an indicator. If a default value is set, you cannot set dimensions. To set dimensions, you must configure it not to use the default value.

Pattern Format

This explains how ServiceWatch interprets data in each log event. The pattern format can be selected from three options as shown below.

• String pattern: Log containing a specific string
• Space-separated pattern: logs separated by spaces such as timestamps, IP addresses, strings, etc.
• JSON format pattern: logs containing specific JSON fields

Available regular expression syntax

When using regular expressions to search and filter log data, you must enclose the expression with %.

Only the following can be included in patterns that contain regular expressions.

• Alphanumeric - Alphanumeric refers to characters that are letters (A~Z or a~z) or numbers (0~9).
  • A-Z, a-z, 0-9 can be used as.
• The supported symbol characters are as follows.
  • :, _, #, =, @, /, ;, ,, -
  • For example, %servicewatch!% cannot be used because ! is not supported.
• The supported operators are as follows.
  • This includes ^, $, ?, [, ], {, }, |, \, *, +, ..
  • (, ) The operator is not supported.

​

String Pattern

String pattern using regular expressions

You can search for matching patterns in log events using a regex string pattern wrapped with %(percentage) at the beginning and end of the regex. Below is an example of a pattern that searches all log events composed of the ERROR keyword. Please refer to the Available Regex Syntax.

%ERROR%

  The above pattern matches log event messages like the following.
* <code>[2026-02-13 14:22:01] ERROR 500 POST /api/v1/checkout (192.168.1.10) - NullPointerException at com.app.controller.CheckoutController.java:55</code>
* <code>[ERROR] Configuration file not found: /etc/app/config.yaml</code>

##### String pattern in log events without format
String pattern for searching strings in log events that are not in formats like JSON.
Below is an example of a log event message, and you can see the log events that match according to various string pattern classifications.

ERROR CODE 400 BAD REQUEST ERROR CODE 401 UNAUTHORIZED REQUEST ERROR CODE 419 MISSING ARGUMENTS ERROR CODE 420 INVALID ARGUMENTS

  



Category
Pattern
Matching log event message




single string
ERROR CODE
log event containing ERROR
ERROR CODE 400 BAD REQUEST
ERROR CODE 401 UNAUTHORIZED REQUEST
ERROR CODE 419 MISSING ARGUMENTS
ERROR CODE 420 INVALID ARGUMENTS


Multiple strings (And condition)
ERROR REQUEST
log events containing the strings ERROR and REQUEST
ERROR CODE 400 BAD REQUEST
ERROR CODE 401 UNAUTHORIZED REQUEST


Multiple strings (Or condition)
?ERROR ?400
log events containing the ERROR or 400 string
ERROR CODE 400 BAD REQUEST
ERROR CODE 401 UNAUTHORIZED REQUEST
ERROR CODE 419 MISSING ARGUMENTS
ERROR CODE 420 INVALID ARGUMENTS


Exact matching string
“BAD REQUEST”
log event containing the exact phrase “BAD REQUEST”
ERROR CODE 400 BAD REQUEST


Exclude specific string
ERROR -400
A pattern where some terms are included and other terms are excluded. Enter - before the string you want to exclude. The following are log events that include the string ERROR and exclude the string 400
ERROR CODE 401 UNAUTHORIZED REQUEST
ERROR CODE 419 MISSING ARGUMENTS
ERROR CODE 420 INVALID ARGUMENTS




  Table. String pattern in log events without format





#### Space-separated pattern

Create a pattern to search for matching strings in log events separated by spaces.

##### Space-separated pattern example(1)
The following is an example of log events separated by spaces.

2023-10-27T10:00:01Z [INFO] 1234 login success 192.168.1.1

The log event above is a space-separated log event that includes <code>timestamp</code>, <code>logLevel</code>, <code>user_id</code>, <code>action</code>, <code>status</code>, <code>ip</code>. Text between brackets (<code>[]</code>) and double quotes (<code>""</code>) is considered a single field.

To create a pattern that searches for matching strings in space-separated log events, enclose the pattern in brackets (<code>[]</code>) and specify fields with names separated by commas (<code>,</code>). The following pattern parses six fields.

<code>[timestamp, logLevel, user_id, action, status = success, ip]</code> can find log events where the 5th field, <code>status</code>, is <code>success</code>.

##### Space-separated pattern example (2)

abc xxx.log james 2023-10-27T10:00:01Z POST 400 1024 abc xxx.log name 2023-10-27T10:00:02Z POST 410 512

The above log event is a space-separated log event that includes host, logName, user, timestamp, request, statusCode, and size.

A pattern like [host, logName, user, timestamp, request, statusCode=4*, size] can find log events where the 6th field, statusCode, starts with 4.

If you do not know the exact number of fields in a space-separated log event, you can use an ellipsis (…). A pattern like […, statusCode=4*, size] is a pattern that represents the first five fields with an ellipsis.

You can also create composite expressions using the AND (&&) operator and the OR (||) operator. A pattern such as […, statusCode=400 || statusCode=410, size] can find log events where the 6th field, statusCode, is 400 or 410.

You can use regular expressions to provide conditions for a pattern. A pattern such as [host, logName, user, timestamp, request, statusCode=%4[0-9]{2}%, size] can find log events where the sixth field, statusCode, is a number starting with 4.

JSON format pattern

You can create a pattern to search for matching strings or numeric values in JSON log events. Patterns are enclosed in curly braces ({}).

String-based JSON format pattern

• Use $. to represent JSON fields.
• The operator can use = or !=.
• The string to compare with the field can be enclosed in double quotes (""). Strings containing non-alphanumeric characters and underscore symbols must be enclosed in double quotes. Use an asterisk (*) as a wildcard to match text.

{ $.resourceType = "trail" }
{ $.resourceType = "%trail%" }
{ $.arrayKey[0] = "value" }

JSON format pattern that searches for numeric values

• Represent JSON fields using $..
• You can use numeric operators.
  • greater than(>), less than(<), equal(=), not equal(!=), greater than or equal to(>=), less than or equal to(<=)
• You can include the addition (+) or subtraction (-) symbols. You can use the asterisk (*) as a wildcard.

{ $.errorCode = 400} { $.errorCode >= 400} { $.errorCode != 500 } { $.sourceIPAddress != 123.123.* }

Export Log Group

From the log group, you can export log data to Object Storage for log retention and log analysis. You can export log groups for log data in the same Account.

To start exporting a log group, you need to create an Object Storage bucket to store the log data.

The log group export operation can take a long time depending on the amount of logs. When exporting a log group, you can reduce the export operation time by specifying a particular stream within the log group or by specifying a time range.

Log group export can only be executed one at a time on the same Account. To run another log group export, the current export task must be completed.

You can delete the log group export history after the export succeeds or after the export cancellation is completed. Canceling log group export does not delete the saved file of the exported log group. To delete the exported log group file, delete the stored file directly in Object Storage.