Agent Configuration

The Logship agent is configured through a JSON file called appsettings.json. The configuration controls where collected data is sent, what data sources are enabled, and how the agent behaves.

You can find an example configuration in the agent source on GitHub.

The configuration file has three top-level sections:

• Output — where to send collected data
• Sources — which data sources to enable and how to configure them
• Logging — log verbosity for the agent itself

Example

{
  "Output": {
    "endpoint": "https://localhost:5000",
    "account": "00000000-0000-0000-0000-000000000000",
    "interval": "00:00:02",
    "maximumBufferSize": 40000,
    "maximumFlushSize": 15000,
    "health": {
      "interval": "00:00:05"
    },
    "registration": {
      "registrationToken": ""
    }
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information"
    }
  },
  "Sources": {
    "DiskInformation": {
      "enabled": true,
      "interval": "00:00:05"
    },
    "SystemInformation": {
      "enabled": true,
      "interval": "01:00:00"
    },
    "NetworkInformation": {
      "enabled": true,
      "interval": "00:00:15"
    },
    "ProcessInformation": {
      "enabled": true,
      "interval": "00:00:30"
    },
    "UDPListener": {
      "enabled": true,
      "port": 49999
    }
  }
}

Output

The Output section controls where the agent sends collected data and how it batches uploads.

Field	Type	Required	Default	Description
endpoint	string	Yes	"console"	The HTTP endpoint to ship data to. Use "console" to print collected data to stdout instead of uploading.
account	guid	Yes	00000000-...	The account ID under which to upload collected data.
interval	timespan	No	00:00:10	How often data is pushed to the endpoint.
maximumBufferSize	int	No	10000	Maximum number of events to buffer. Events are dropped when this limit is reached. Range: 5,000–1,000,000.
maximumFlushSize	int	No	10000	Maximum number of events to send per upload batch. Range: 1,000–1,000,000.
dataPath	string	No	"./logship-agent-local-storage"	Path for local storage used by the agent.
health	object	No		Health reporting configuration.
health.interval	timespan	No	00:00:15	How often the agent reports its own health.
registration	object	No		Registration configuration for agent authentication.
registration.registrationToken	string	No	""	Token used to register the agent with the Logship backend.

{
  "Output": {
    "endpoint": "https://localhost:5000",
    "account": "00000000-0000-0000-0000-000000000000",
    "interval": "00:00:02",
    "maximumBufferSize": 40000,
    "maximumFlushSize": 15000,
    "dataPath": "./logship-agent-local-storage",
    "health": {
      "interval": "00:00:05"
    },
    "registration": {
      "registrationToken": "your-token-here"
    }
  }
}

Logging

Standard .NET logging configuration. See .NET Logging Configuration for more information.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "System.Net.Http.HttpClient.Default": "Warning"
    }
  }
}

Sources

The Sources section is an object where each key is a source name and each value is that source's configuration. Every source has an enabled field (default: true) that controls whether it is active. Sources that collect on an interval also have an interval field (default: 00:00:15).

DiskInformation

Collects disk and filesystem usage information.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "DiskInformation": {
    "enabled": true,
    "interval": "00:00:05"
  }
}

SystemInformation

Collects system-level information such as OS version and hardware details.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "SystemInformation": {
    "enabled": true,
    "interval": "01:00:00"
  }
}

NetworkInformation

Collects network interface statistics.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "NetworkInformation": {
    "enabled": true,
    "interval": "00:00:15"
  }
}

ProcessInformation

Collects cross-platform process information.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "ProcessInformation": {
    "enabled": true,
    "interval": "00:00:30"
  }
}

Proc (Linux)

Collects process information by reading /proc. Only works on Linux and requires access to /proc.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "Proc": {
    "enabled": true,
    "interval": "00:00:05"
  }
}

Proc.OpenFiles (Linux)

Collects open file descriptor information from /proc. Only works on Linux.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "Proc.OpenFiles": {
    "enabled": true,
    "interval": "00:05:00"
  }
}

Proc.Modules (Linux)

Collects loaded kernel module information from /proc. Only works on Linux.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.

{
  "Proc.Modules": {
    "enabled": true,
    "interval": "00:00:05"
  }
}

UDPListener

Listens on a UDP port for JSON-formatted events and forwards them to the Logship database.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
port	int	No		UDP port to listen on (1–65535).

{
  "UDPListener": {
    "enabled": true,
    "port": 49999
  }
}

UDP packets should be sent as JSON with the following format:

{
  "timestamp": "2024-01-01T00:00:00Z",
  "schema": "my_table",
  "data": { "key": "value" }
}

Field	Type	Required	Description
timestamp	datetime	No	Event timestamp. Defaults to current UTC time.
schema	string	Yes	The output table name in the Logship database.
data	object	Yes	Your data object, serialized and forwarded as-is.

HealthChecks

Periodically executes HTTP GET requests to configured endpoints and records the results.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
targets	array	Yes		Array of endpoint targets to check.
targets[].endpoint	string	Yes		The endpoint URI.
targets[].interval	timespan	No	00:00:15	The interval for this health check.
targets[].includeResponseHeaders	bool	No	false	Include response headers in output.
targets[].includeResponseBody	bool	No	false	Include response body in output.

{
  "HealthChecks": {
    "enabled": true,
    "targets": [
      {
        "endpoint": "https://example.com",
        "interval": "00:05:00",
        "includeResponseHeaders": true,
        "includeResponseBody": true
      }
    ]
  }
}

JournalCtl (Linux)

Collects logs from the systemd journal. Only works on Linux systems with journald.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
flags	int	No	0	Journalctl flags to use when collecting logs.
includeFields	array	No	[]	Additional journal fields to include in the output.
filters	array	No	[]	Array of filter groups to control which journal entries to collect.

Filters support matchAny (OR logic) and matchAll (AND logic) groupings. Each filter condition can be either a hasField check (field exists) or a fieldEquals check (field matches a specific value).

{
  "JournalCtl": {
    "enabled": true,
    "flags": 0,
    "includeFields": ["USERID"],
    "filters": [
      {
        "matchAny": [
          { "hasField": "CONTAINER_NAME" },
          {
            "fieldEquals": {
              "field": "SYSLOG_IDENTIFIER",
              "value": "systemd-resolved"
            }
          }
        ]
      },
      {
        "matchAll": [
          {
            "fieldEquals": {
              "field": "SYSLOG_IDENTIFIER",
              "value": "kernel"
            }
          }
        ]
      }
    ]
  }
}

SyslogTcp

Listens for syslog messages over TCP.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
endpoint	string	No	"127.0.0.1"	The address to listen on.
port	int	No	514	TCP port to listen on (1–65535).

{
  "SyslogTcp": {
    "enabled": true,
    "port": 5140
  }
}

LogFile

Tails log files using glob patterns. Supports multiline aggregation and JSON lines parsing.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
include	array	Yes	[]	Glob patterns for files to include (e.g. "*.log").
exclude	array	No	[]	Glob patterns for files to exclude (e.g. "*.tmp").
workingDirectory	string	No		Base directory for glob pattern matching.
encoding	string	No	"utf-8"	File encoding.
startAtBeginning	bool	No	false	Start reading from the beginning of the file instead of the tail.
ignoreCheckpoints	bool	No	false	Ignore saved file position checkpoints on startup.
ignoreOlderSecs	int	No		Ignore files not modified within this many seconds.
globMinimumCooldownMs	int	No	1000	Minimum interval between glob re-evaluations (ms). Min: 100.
readBufferSize	int	No	8192	File read buffer size in bytes. Min: 1024.
maxLineBytes	int	No	1048576	Maximum line length in bytes. Min: 1024.
multiline	object	No		Multiline aggregation settings.
jsonLines	object	No		JSON Lines parsing settings.

Multiline configuration:

Field	Type	Required	Default	Description
mode	string	No	"startPattern"	Multiline mode.
startPattern	string	No		Regex pattern that marks the start of a new log entry.
conditionPattern	string	No		Regex condition pattern for line continuation.
timeoutMs	int	No	1000	Timeout before flushing a multiline block (ms).

JSON Lines configuration:

Field	Type	Required	Default	Description
enabled	bool	No	false	Enable JSON Lines parsing.
skipInvalidLines	bool	No	true	Skip lines that aren't valid JSON.
timestampField	string	No		JSON field to use as the event timestamp.
messageField	string	No		JSON field to use as the log message.

{
  "LogFile": {
    "enabled": true,
    "include": ["*.log", "PMS Plugin Logs/*.log"],
    "exclude": ["*.tmp", "*.bak"],
    "workingDirectory": "/var/log/myapp/",
    "encoding": "utf-8",
    "startAtBeginning": false,
    "ignoreOlderSecs": 86400,
    "multiline": {
      "mode": "start_pattern",
      "startPattern": "^\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}",
      "timeoutMs": 5000
    }
  }
}

Otlp

Receives telemetry data via the OpenTelemetry Protocol (OTLP) over gRPC. Supports logs, metrics, and traces.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
port	int	No	4317	gRPC port to listen on (1–65535).

{
  "Otlp": {
    "enabled": true,
    "port": 4317
  }
}

MQTT

Subscribes to an MQTT broker and collects messages from topics.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
brokerAddress	string	No	"localhost"	MQTT broker hostname or IP address.
brokerPort	int	No	1883	MQTT broker port (1–65535).
clientId	string	No	"logship-agent"	MQTT client identifier.
username	string	No		Username for broker authentication.
password	string	No		Password for broker authentication.
topic	string	No	"#"	MQTT topic filter to subscribe to.
useTls	bool	No	false	Connect to the broker using TLS.
combined	bool	No	true	Combine all topic messages into a single output table.

{
  "MQTT": {
    "enabled": true,
    "brokerAddress": "mqtt.example.com",
    "brokerPort": 1883,
    "clientId": "logship-agent",
    "username": "user",
    "password": "secret",
    "topic": "sensors/#",
    "useTls": true,
    "combined": true
  }
}

NmapScanner

Runs periodic Nmap network scans against configured subnets.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The scan interval.
shellExec	bool	No	false	Use shell execution for Nmap.
subnets	array	Yes		Array of subnet configurations to scan.
subnets[].subnet	string	Yes		Subnet in CIDR notation (e.g. "192.168.0.0/24").
subnets[].nmapArgs	string	No	"-T4 -n"	Additional Nmap command line arguments.

{
  "NmapScanner": {
    "enabled": true,
    "interval": "00:01:00",
    "subnets": [
      {
        "subnet": "192.168.0.0/24"
      },
      {
        "subnet": "192.168.1.0/24",
        "nmapArgs": "-T4 -n -sV"
      }
    ]
  }
}

Windows.ETW (Windows)

Collects events from Windows Event Tracing for Windows (ETW) providers.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
sessionNamePrefix	string	No		Prefix for the ETW session name.
cleanupOldSessions	bool	No	true	Clean up old ETW sessions before starting a new one.
reuseExistingSession	bool	No	true	Reuse an existing ETW session with the same configuration.
providers	array	Yes		Array of ETW providers to collect from.
providers[].providerGuid	guid	No		GUID of the ETW provider.
providers[].providerName	string	No		Name of the ETW provider (alternative to GUID).
providers[].level	string	No	"Informational"	Trace event level: Critical, Error, Warning, Informational, Verbose.
providers[].keywords	long	No	All	Keyword bitmask to filter events.

{
  "Windows.ETW": {
    "enabled": true,
    "cleanupOldSessions": true,
    "reuseExistingSession": true,
    "providers": [
      {
        "ProviderGuid": "7f006a22-73fb-4c17-b1eb-0a3070f9f187"
      },
      {
        "ProviderGuid": "E6F378E8-21CE-49A9-8D98-1BAAF053AB51"
      }
    ]
  }
}

Windows.PerformanceCounters (Windows)

Collects Windows performance counter data.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.
counterRefreshInterval	timespan	No	00:01:00	How often to refresh the list of available counters.
counters	array	Yes		Array of performance counter path patterns to collect.

{
  "Windows.PerformanceCounters": {
    "enabled": true,
    "interval": "00:00:05",
    "counters": [
      "\\Process(*)\\*logship*",
      "\\Process(*)\\Logship.Agent.ConsoleHost"
    ]
  }
}

Internals

Collects internal agent metrics and tracing data for self-monitoring.

Field	Type	Required	Default	Description
enabled	bool	No	true	Enable this source.
interval	timespan	No	00:00:15	The collection interval.
enableMetrics	bool	No	true	Collect internal agent metrics.
enableTracing	bool	No	true	Collect internal agent trace data.

{
  "Internals": {
    "enabled": true,
    "interval": "00:00:15",
    "enableMetrics": true,
    "enableTracing": true
  }
}