apv- 🐍 advanced python logging 📔 |
| git clone git://git.acid.vegas/apv.git |
| Log | Files | Refs | Archive | README |
README.md (9618B)
1 # Advanced Python Logging (APV) 2 > Flexible & powerful logging solution for Python applications 3 4  5 6 ## Table of Contents 7 - [Introduction](#introduction) 8 - [Requirements](#requirements) 9 - [Features](#features) 10 - [Installation](#installation) 11 - [Configuration Options](#configuration-options) 12 - [Usage](#usage) 13 - [Basic Console Logging](#basic-console-logging) 14 - [Console Logging with Details](#console-logging-with-details) 15 - [File Logging with Rotation](#file-logging-with-rotation) 16 - [File Logging with Compression and JSON Format](#file-logging-with-compression-and-json-format) 17 - [Graylog Integration](#graylog-integration) 18 - [AWS CloudWatch Integration](#aws-cloudwatch-integration) 19 - [Mixing it all together](#mixing-it-all-together) 20 - [Testing](#testing) 21 22 ## Introduction 23 APV emerged from a simple observation: despite the abundance of logging solutions, there's a glaring lack of standardization in application logging. As a developer deeply entrenched in Elasticsearch, AWS, and Graylog ecosystems, I found myself repeatedly grappling with inconsistent log formats and cumbersome integrations. APV is my response to this challenge – a logging library that doesn't aim to revolutionize the field, but rather to streamline it. 24 25 This project is rooted in pragmatism. It offers a clean, efficient approach to generating logs that are both human-readable and machine-parseable. APV isn't about reinventing logging; it's about refining it. It provides a unified interface that plays well with various logging backends, from local files to cloud services, without sacrificing flexibility or performance. 26 27 While there's no shortage of logging libraries out there, APV represents a distillation of best practices I've encountered and challenges I've overcome. It's designed for developers who appreciate clean, consistent logs and seamless integration with modern logging infrastructures. If you're tired of wrestling with logging inconsistencies in production environments, APV might just be the solution you didn't know you were looking for. 28 29 ## Requirements 30 - [Python 3.10+](https://www.python.org/) 31 - [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html) *(Optional: for CloudWatch logging)* 32 - [ecs-logging](https://github.com/aws-observability/aws-ecs-logging) *(Optional: for ECS logging)* 33 34 ## Features 35 - **Console Logging with Colors**: Enhanced readability with colored log messages in the console. 36 - **File Logging**: Write logs to files with support for log rotation based on size and number of backups. 37 - **Log Compression**: Automatically compress old log files using gzip to save disk space. 38 - **JSON Logging**: Output logs in JSON format for better structure and integration with log management systems. 39 - **ECS Logging**: Output logs in ECS format for better integration with [Elasticsearch](https://www.elastic.co/elasticsearch/) 40 - **Detailed Log Messages**: Option to include module name, function name, and line number in log messages. 41 - **Graylog Integration**: Send logs to a [Graylog](https://www.graylog.org/) server using GELF over UDP. 42 - **AWS CloudWatch Integration**: Send logs to [AWS CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html). 43 - **Customizable Logging Levels**: Set the logging level to control verbosity. 44 45 ## Installation 46 To install APV, you can use the provided `setup.py` script: 47 48 1. **Clone the repository or download the source code.** 49 50 ```bash 51 git clone https://github.com/acidvegas/apv.git 52 ``` 53 54 2. **Navigate to the project directory.** 55 56 ```bash 57 cd apv 58 ``` 59 60 3. **Install the package using `setup.py`.** 61 62 ```bash 63 python setup.py install 64 ``` 65 66 - **With CloudWatch support:** 67 68 If you plan to use AWS CloudWatch logging, install with the `cloudwatch` extra: 69 70 ```bash 71 pip install .[cloudwatch] 72 ``` 73 74 - **Alternatively, install `boto3` separately:** 75 76 ```bash 77 pip install boto3 78 ``` 79 80 - **With ECS support:** 81 82 If you plan to use ECS logging, install with the `ecs` extra: 83 84 ```bash 85 pip install .[ecs] 86 ``` 87 88 - **Alternatively, install `ecs-logging` separately:** 89 90 ```bash 91 pip install ecs-logging 92 ``` 93 94 ## Configuration Options 95 96 The `setup_logging` function accepts the following keyword arguments to customize logging behavior: 97 98 | Name | Default | Description | 99 |--------------------------|--------------------------|--------------------------------------------------------------------------------------| 100 | `level` | `INFO` | The logging level. *(`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`)* | 101 | `date_format` | `'%Y-%m-%d %H:%M:%S'` | The date format for log messages. | 102 | `log_to_disk` | `False` | Whether to log to disk. | 103 | `max_log_size` | `10*1024*1024` *(10 MB)* | The maximum size of log files before rotation *(in bytes)*. | 104 | `max_backups` | `7` | The maximum number of backup log files to keep. | 105 | `log_file_name` | `'app'` | The base name of the log file. | 106 | `json_log` | `False` | Whether to log in JSON format. | 107 | `ecs_log` | `False` | Whether to log in ECS format. | 108 | `show_details` | `False` | Whether to include module name, function name, & line number in log messages. | 109 | `compress_backups` | `False` | Whether to compress old log files using gzip. | 110 | `enable_graylog` | `False` | Whether to enable logging to a Graylog server. | 111 | `graylog_host` | `None` | The Graylog server host. *(Required if `enable_graylog` is `True`)* | 112 | `graylog_port` | `None` | The Graylog server port. *(Required if `enable_graylog` is `True`)* | 113 | `enable_cloudwatch` | `False` | Whether to enable logging to AWS CloudWatch Logs. | 114 | `cloudwatch_group_name` | `None` | The name of the CloudWatch log group. *(Required if `enable_cloudwatch` is `True`)* | 115 | `cloudwatch_stream_name` | `None` | The name of the CloudWatch log stream. *(Required if `enable_cloudwatch` is `True`)* | 116 117 ## Usage 118 119 ### Basic Console Logging 120 121 ```python 122 import logging 123 import apv 124 125 # Set up basic console logging 126 apv.setup_logging(level='INFO') 127 128 logging.info('This is an info message.') 129 logging.error('This is an error message.') 130 ``` 131 132 ### Console Logging with Details 133 134 ```python 135 import logging 136 import apv 137 138 # Set up console logging with detailed information 139 apv.setup_logging(level='DEBUG', show_details=True) 140 141 logging.debug('Debug message with details.') 142 ``` 143 144 ### File Logging with Rotation 145 146 ```python 147 import logging 148 import apv 149 150 # Set up file logging with log rotation 151 apv.setup_logging( 152 level='INFO', 153 log_to_disk=True, 154 max_log_size=10*1024*1024, # 10 MB 155 max_backups=5, 156 log_file_name='application_log' 157 ) 158 159 logging.info('This message will be logged to a file.') 160 ``` 161 162 ### File Logging with Compression and JSON Format 163 164 ```python 165 import logging 166 import apv 167 168 # Set up file logging with compression and JSON format 169 apv.setup_logging( 170 level='DEBUG', 171 log_to_disk=True, 172 max_log_size=5*1024*1024, # 5 MB 173 max_backups=7, 174 log_file_name='json_log', 175 json_log=True, 176 compress_backups=True 177 ) 178 179 logging.debug('This is a debug message in JSON format.') 180 ``` 181 182 ### Graylog Integration 183 184 ```python 185 import logging 186 import apv 187 188 # Set up logging to Graylog server 189 apv.setup_logging( 190 level='INFO', 191 enable_graylog=True, 192 graylog_host='graylog.example.com', 193 graylog_port=12201 194 ) 195 196 logging.info('This message will be sent to Graylog.') 197 ``` 198 199 ### AWS CloudWatch Integration 200 201 ```python 202 import logging 203 import apv 204 205 # Set up logging to AWS CloudWatch Logs 206 apv.setup_logging( 207 level='INFO', 208 enable_cloudwatch=True, 209 cloudwatch_group_name='my_log_group', 210 cloudwatch_stream_name='my_log_stream' 211 ) 212 213 logging.info('This message will be sent to AWS CloudWatch.') 214 ``` 215 216 ### ECS Logging 217 218 ```python 219 import logging 220 import apv 221 222 # Set up logging to AWS CloudWatch Logs 223 apv.setup_logging( 224 level='INFO', 225 ecs_log=True 226 ) 227 ``` 228 229 ### Mixing it all together 230 231 ```python 232 import logging 233 import apv 234 235 # Set up logging to all handlers 236 apv.setup_logging( 237 level='DEBUG', 238 log_to_disk=True, 239 max_log_size=10*1024*1024, 240 max_backups=7, 241 log_file_name='app', 242 json_log=True, 243 compress_backups=True, 244 enable_graylog=True, 245 graylog_host='graylog.example.com', 246 graylog_port=12201, 247 enable_cloudwatch=True, 248 cloudwatch_group_name='my_log_group', 249 cloudwatch_stream_name='my_log_stream', 250 show_details=True 251 ) 252 ``` 253 254 ## Testing 255 256 To run the test suite, use the following command: 257 258 ```bash 259 python unittest.py 260 ``` 261 262 The test suite will run all the tests and provide output for each test.