Solutions

AWS Logging

Logit.io simplifies the process of running big data frameworks to analyse vast amounts of data efficiently

Start Free Trial View Pricing

log management

FilebeatLogstashFluentdSyslogWinlogbeat

Ship

Parse

Index

Alert

Live log stream

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

+ 12.4k events/s ingested

! 3 severity alerts fired

+ OpenSearch query 37ms

Trusted by engineering teams worldwide

At Logit.io, we prioritise efficient and productive log analysis solutions that is able to reduce manual tracing of log files and improve monitoring and analysis. One way you can stay on top of your system's logs is by thoroughly analysing Amazon log files.

By using the right tools suited for a wide range of AWS logging use cases, you'll be able to put error resolution and root-cause analysis first and control the accuracy of notifications and alerts.

What Are Amazon Logs?

AWS logs are created by a host of different Amazon Web Services including Cloudwatch, Cloudtrail, S3, ALB, VPC and EC2. Monitoring these logs is vital to ensure visibility of your operations, for example Cloudtrail should be monitored so that assets can be discovered and tracked proactively to maintain an accurate inventory.

With all these different types of services and their respective logs it can be difficult to keep track of the performance of all of your AWS environments, this is where a tool like Logit.io can be used to fully centralise your logs, metrics and traces into a simple easy to understand platform.

FilebeatLogstashFluentdSyslogWinlogbeat

Ship

Parse

Index

Alert

Live log stream

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

+ 12.4k events/s ingested

! 3 severity alerts fired

+ OpenSearch query 37ms

Faster Resolution and Log Aggregation

Since each independent Amazon Web Services process generates its own log files, its important to collect all logs into a centralised location for monitoring and analysis. With Amazon, the final destination of log files is determined when the cluster is created.

If you use more than one Amazon AWS service, you may quickly find that your logs grow out of control across a number of decentralised systems. A tool like Logit.io can process all of your AWS log files within a single centralised repository.

With all log files in a single, cloud-based centralised location, you can automatically parse the logs at once, monitor them and prepare them for further analysis, and visualisation. With Logit.io's AWS log aggregation, you'll be able to save time and resources on data processing — especially for critical situations such as when errors occur or for root-cause analysis.

shell

logit metrics scrape --target=k8s --interval=30s

→ Prometheus · Grafana dashboards synced

Compatible With Many Integration Options

At Logit.io, our service contain built-in support for sending data from countless different sources. Whether you're looking to be in full control or employ a degree of automation by using lightweight shippers, Logit.io is fully able to meet your data ingestion requirements.

Using Logit.io for log analysis grants you more flexibility when it comes to making the most of AWS products and services and also provides a hosted platform in which you can easily launch OpenSearch Stacks.

The Logit.io platform is also fully compatible with handling logs from Google Cloud Platform and Azure for complete cloud native monitoring.

FilebeatLogstashFluentdSyslogWinlogbeat

Ship

Parse

Index

Alert

Live log stream

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

+ 12.4k events/s ingested

! 3 severity alerts fired

+ OpenSearch query 37ms

Visualising Amazon Log Files

Create visualisations of your data with an interactive dashboard using your favourite visualisation tools, including Grafana®, Kibana and OpenSearch Dashboards & provide context to your data.

With synced dashboards, shareable reports, and advanced data filtering, you'll be able to make the most out of your system's Amazon log files.

With Logit.io, you'll also be able to analyse hybrid, public, and private cloud logs. Additionally, with Apache, another of many supported integrations you'll be able to easily create interactive visualisations of large datasets using Amazon.

alerts

Detect

Enrich

Route

Notify

! anomaly detected · checkout p95 > 500ms

→ context attached · service map · recent deploy

→ routed to #incidents · ack in 12s

Logit.io For OpenTelemetry

OpenTelemetry (OTel) combined with Logit.io makes end-to-end observability easy. Using OpenTelemetry, engineers can essentially standardize any data coming from any source.

A secure, compliant, and production-ready distribution of the OpenTelemetry project, Logit.io for OpenTelemetry provides unified analysis and complete centralization of any kind of telemetry data.

Since all major observability vendors are required to support the OpenTelemetry protocol, choosing an analysis service that is already OpenTelemetry-compliant is crucial to future-proofing your operations.

Find out more about OTel

FilebeatLogstashFluentdSyslogWinlogbeat

Ship

Parse

Index

Alert

Live log stream

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

--:--:-- INFO request.completed duration_ms=42 route="/api/v1/orders"

--:--:-- WARN latency.spike service=checkout p95=820ms threshold=500ms

--:--:-- INFO trace.exported spans=128 backend=jaeger status="ok"

--:--:-- INFO metric.scrape target=prometheus job=k8s-pods samples=8421

--:--:-- INFO log.shipped bytes=184032 index=logs-prod

--:--:-- WARN auth.failure ip=203.0.113.42 attempts=3 action=rate_limit

--:--:-- INFO alert.routed severity=high channel="#incidents" dedupe=on

--:--:-- INFO dashboard.refresh uid=ops-overview panels=14 cache=hit

--:--:-- ERROR disk.pressure node=worker-3 usage=92% reclaim=started

--:--:-- INFO pipeline.batch size=2048 lag_ms=18 status=healthy

--:--:-- WARN queue.backpressure topic=ingest depth=1200

--:--:-- INFO otel.export endpoint=collector.svc spans_ok=512

--:--:-- INFO search.query hits=1284 took_ms=37 index=logs-*

--:--:-- INFO retention.policy applied hot=14d warm=30d

--:--:-- WARN tls.cert.expiring host=ingest.logit.io days=12

--:--:-- INFO ha.failover check region=eu-west status=ready

+ 12.4k events/s ingested

! 3 severity alerts fired

+ OpenSearch query 37ms

Companies Feel The Difference When They Use Logit.io

“Internally, Logit.io has made it easier for us to provide better support for our customers, since finding individual messages based on various data in the payload has become easier.”
“At Youredi, pretty much everyone from our technical support teams through to our professional services teams uses Logit.io.”

Start your 14-day free trial

No credit card required. Managed OpenSearch, Prometheus, and Grafana from $25/mo.

Start Free Trial Read documentation