Transcoding Job Monitoring Using AWS CloudWatch

These contain: FFmpeg encoder output Input/output file validation results DRM packaging status Error stack traces A Lambda function subscribed to these logs can extract critical errors using pattern matching: import reimport zlibdef lambda_handler(event, context): logs = zlib.decompress(event['awslogs']['data'], 16+zlib.MAX_WBITS) errors = re.findall(r'ERROR\s+#(\w+):\s(.+)', logs) if errors: publish_to_sns(errors) Custom Metric Filters CloudWatch Logs can be extended with metric filters to count specific log patterns as numeric metrics. This allows tracking of custom issues such as codec-specific encoding errors. For example, filtering logs for H.265 encoding failures enables the creation of a metric named H265ErrorCount. This metric can be visualized, alerted on, or used to identify trends in codec behavior. aws logs put-metric-filter \ --log-group-name '/aws/mediaconvert/jobs' \ --filter-name 'H265EncodeErrors' \ --filter-pattern '[codec=H265, level=ERROR]' \ --metric-transformations \ '[ { 'metricName': 'H265ErrorCount', 'metricNamespace': 'Custom/MediaConvert', 'metricValue': '1' } ]' Dashboard Configuration CloudWatch dashboards provide a unified view of transcoding activity by combining metric graphs and log queries into a single interface. These dashboards can display job counts over time, job errors per queue, and filterable log outputs for error messages. Widgets are defined in JSON and support custom queries, periods, and dimensions. This allows teams to monitor real-time system status and investigate job history efficiently. { 'widgets': [ { 'type': 'metric', 'x': 0, 'y': 0, 'width': 12, 'height': 6, 'properties': { 'metrics': [ ['AWS/MediaConvert', 'JobsCompleted', 'Queue', 'arn:aws:mediaconvert:us-east-1:123456789012:queues/Default'], ['AWS/MediaConvert', 'JobsErrored', 'Queue', 'arn:aws:mediaconvert:us-east-1:123456789012:queues/Default'], ['AWS/MediaConvert', 'JobsWarning', 'Queue', 'arn:aws:mediaconvert:us-east-1:123456789012:queues/Default'] ], 'period': 300, 'stat': 'Sum', 'region': 'us-east-1', 'title': 'Job Status Counts' } }, { 'type': 'log', 'x': 0, 'y': 6, 'width': 12, 'height': 6, 'properties': { 'query': 'SOURCE '/aws/mediaconvert/jobs' | filter @message like /ERROR/', 'region': 'us-east-1', 'title': 'Error Logs' } } ] } Cost Optimization Monitoring MediaConvert with CloudWatch incurs costs for metrics, logs, and dashboard queries. These costs can be managed by limiting log retention periods based on severity, reducing metric granularity after a certain retention window, and using metric math where applicable. Reducing costs involves: Setting log retention policies (e.g., 7 days for debug logs, 1 year for errors) Aggregating metrics at 5-minute resolution after 15 days Using metric math instead of separate custom metrics Integration with Incident Management CloudWatch alarms can trigger predefined remediation steps using AWS Systems Manager Automation. For instance, when consecutive job failures are detected, an automation document can invoke a Lambda function to restart failed jobs. --- documentType: Automation description: Restart failed MediaConvert jobs schemaVersion: '0.3' parameters: jobId: type: String mainSteps: - name: restartJob action: aws:invokeLambdaFunction inputs: FunctionName: restartMediaConvertJob Payload: '{'jobId': '{{jobId}}'}' This workflow executes when a CloudWatch alarm detects consecutive job failures, invoking a Lambda that restarts jobs using the MediaConvert API.

Boost Campaign Performance Through Video