Deployment · Cybersecurity
Deploy Central Logging with Rsyslog over TLS
UDP logging does not confirm delivery. TCP over TLS protects transport and authenticates the peer; it does not alone guarantee durable storage of the application event. Queueing, disk…
Technical review:
Architecture and operating model
UDP logging does not confirm delivery. TCP over TLS protects transport and authenticates the peer; it does not alone guarantee durable storage of the application event. Queueing, disk durability and collector retention require separate design.
Prepare a collector DNS name, CA and certificate chain. Verify the distribution GnuTLS driver package. Configure a TCP 6514 receiver, an omfwd destination and permitted certificate name; restrict the firewall to authorized senders.
Validate client configuration with rsyslogd -N1 before reloading. Send from one test host first. During collector downtime, observe queue accumulation, replay ordering and duplicates. Anonymous TLS must not be mistaken for authenticated TLS.
- 1Source event
- 2Disk-assisted queue
- 3Authenticated TLS
- 4Collector and retention
Design parameters
- Peer validation
- Validate the collector certificate name in x509/name mode; encryption alone is insufficient.
- Queue size
- Calculate event rate × average size × outage duration, allowing for queue and filesystem overhead.
- Retention
- Preserve source identity and timestamps; restrict sensitive or personal fields.
Worked example
At 200 events/s, 800 bytes/event and a 30-minute outage, raw data reaches about 288 MB. Measure actual disk usage and behavior at the queue limit before assuming a 400 MB queue is sufficient.
Example commands: replace lab values and confirm permissions and software versions before use.
rsyslogd -N1
logger -t doz-pilot "event-id=lab-001"
# Client omfwd fragment; CA/driver setup is required separately.
action(type="omfwd" target="logs.example.test" port="6514" protocol="tcp"
StreamDriver="gtls" StreamDriverMode="1" StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers="logs.example.test"
queue.type="LinkedList" queue.filename="forward_tls" queue.saveOnShutdown="on")Troubleshooting
| Observation | Likely cause / distinction | Verification |
|---|---|---|
| TLS fails | Driver package, CA or name mismatch. | Inspect rsyslog diagnostics and certificate SAN. |
| Events missing after outage | Queue overflow or inadequate shutdown persistence. | Compare counters, queue files and sent event IDs. |
Acceptance checks
- Validate the collector name against its certificate.
- Check configuration syntax.
- Trace one event end to end.
- Test collector downtime in a pilot.
- Alert on queue utilization.
- Verify retention and deletion policy.
Related concepts
TLS and certificate validation
TLS protects confidentiality and integrity in transit; certificate validation helps verify the peer’s identity. Evaluate names, chains, validity periods and trusted roots together. Encryption does not prove correct application authorization. If a reverse proxy or inspection device is used, show where TLS terminates. Disabling validation is not a permanent troubleshooting solution: investigate hostname mismatch, missing intermediate certificates and incorrect device clocks separately.
Telemetry and time correlation
Telemetry combines logs, metrics and events that explain system behaviour. A log describes an event, a metric shows behaviour over time, and a distributed trace follows a request across components. Clock differences can make one event appear to occur at several times. Use synchronized clocks, reliable source identifiers and consistent time-zone handling. Alarm design should consider duration and user impact alongside thresholds. Monitor gaps in collection separately: absence of logs must not be interpreted as absence of incidents.
Queues and concurrency
A queue holds work arriving faster than a resource can process it. More concurrency can improve utilization up to a point, then increase waiting time. In a stable system Little’s law relates L = λ × W: average work in the system equals throughput multiplied by average total time. Units must agree. At 2,000 operations/s and 5 ms total time, roughly 10 operations are present concurrently. This is a planning relationship; queue limits, bursts and highly variable service times still require measurement.
Capacity and usable headroom
Raw capacity is not the capacity available to applications. RAID or erasure coding, filesystems, reserved space, metadata, snapshots and growth headroom are separate deductions. TB and TiB representations also change the displayed number. Write calculations with units, establish protected usable capacity, then subtract operating reserves. Track growth rate as well as current utilization. The projected exhaustion date should leave enough time to procure and deploy additional capacity.
Trust boundary and failure domain
A trust boundary separates components governed by different access decisions; a failure domain groups resources that one event can affect together. Two VLANs do not create a strong trust boundary when routing between them is unrestricted. Backups in different folders still share a failure domain if one administrator can delete both. Assess physical location, identity provider, management account, network path and power source separately. Test which access paths and recovery options remain available when a component is lost.
Retention and capacity
Retention defines which recovery points are kept and for how long. Daily, weekly and monthly points do not represent identical change patterns; full-copy creation and chain dependencies affect physical capacity. Retention decisions combine business requirements, applicable obligations and technical capacity. Longer retention does not automatically provide better recovery: the right point must be discoverable and readable. When changing a policy, test whether existing points are deleted immediately or handled differently by the product.
Primary documentation
Prepared by the Doz Teknoloji technical team using the primary references below. Calculations and lab scenarios state their assumptions; validate the applicable product version before rollout.