Metadata-Version: 2.1
Name: elb-log-tools
Version: 0.3.0
Summary: Tools for processing logs from AWS Elastic Load Balancer
Home-page: https://github.com/pymetrics/elb-log-tools
Author: Pymetrics, Inc.
Author-email: info@pymetrics.com
License: Apache-2.0
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Internet :: Log Analysis
Description-Content-Type: text/markdown
License-File: LICENSE.txt

# ELB Log Tools

Tools for processing logs from AWS Elastic Load Balancer

## Installation

```
$ pip install elb-log-tools
```

## Usage

### Specifying the log bucket name

There are two options:

1. Set the ``ELB_LOG_BUCKET`` environment variable, or
2. Pass the bucket name to ``elb-logs`` using the ``-b`` or ``--bucket`` argument.

   ```
   $ elb-logs -b my-log-bucket
   ```

### Print ELB logs to stdout
```
$ elb-logs | head -n 1
2019-11-06T23:00:47.799875Z production-pymcore-webserver 84.10.79.236:55801 172.16.11.238:80 0.000049 1.922708 0.000054 200 200 0 230454 "GET https://www.pymetrics.com:443/results/downloads/traits/ HTTP/1.1" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
```

### Filter ELB logs

Filtering currently only outputs the PATH component of the log file.

```
# Only output redirects
$ elb-logs | elb-filter --include-statuses 301 302 | head -n 1
/logout/
```

### Aggregate request counts based on pattern matching

``elb-patterncounts`` matches URL paths against a list of regex patterns, and
outputs the counts of requests matching the patterns in CSV format.

```
# patterns.txt has all the URL patterns we want included in the report

$ elb-logs | elb-filter -f 404 | elb-patterncounts --patternfile patterns.txt --all
Path,Hits
/api3/apps/applications/,66762
/api3/games/games/,42192
/api/game/submission/,35418
/api2/games/list/,33487
/api2/accounts/config/,19251
/health/,18891
```

## Examples

### Get most common requests for static files that 404:

```
$ elb-logs | elb-filter -i 404 | grep static | head -n 50 | uniq -c | sort -bgr
```
