Real-time Security Event Ingestion & Threat Analysis
An event-driven cloud security analytics pipeline that ingests AWS CloudTrail records, applies normalization and risk scoring heuristics, and fires alert integrations under 45 seconds to secure distributed multi-tenant AWS accounts.
Security teams were blind to privilege escalations, impossible travel access anomalies, and critical token abuses, with incident containment cycles taking hours due to fragmented SIEM reporting.
Reduced MTTR (Mean Time to Respond) from 3 hours to under 3 minutes, mitigated risk of credentials leakage, and established audit-ready continuous cloud monitoring compliance.
Serverless S3-triggered event pipelines normalizing raw JSON CloudTrail schemas.
Mathematical incident prioritization engine scoring API calls against historical baselines.
ChatOps alert integration formatting threat metadata with direct links to logs.
Distributed security architecture acquiring audit logs from organizational member accounts into a centralized security account using encrypted streams, then processing events asynchronously via concurrent serverless queues.
CloudTrail Logs (S3) ──> S3 Event Notification ──> Lambda Parser ──> EventBridge
│
┌──────────────────────────────────────────────────────────────────────┘
▼
Lambda Risk Scorer ──> DynamoDB (State Cache) ──> KMS Encrypted Evidence (S3)
│
└─[High Risk Risk >= 75]─> SNS ──> ChatOps (Slack) / PagerDuty Alertcloudtrail-detector/
├── src/
│ ├── handlers/
│ │ ├── parser.ts # Normalization logic
│ │ └── scorer.ts # Risk scoring engine
│ ├── rules/
│ │ └── privilege-escalation.ts # Signature checks
│ └── utils/
│ └── kms-helper.ts # Evidence encryption
└── terraform/
├── main.tf # Pipelines definitions
└── variables.tfRun active operations audits utilizing the custom sandbox telemetry receiver widget below.
Log streams arrive in bursts and processing must be real-time while ensuring zero dropouts.
Serverless S3 Event Notifications triggering Lambda
Accepted a 1-second cold start latency on initial batch wakeups to achieve 85% cost savings compared to maintaining a running MSK cluster.
Introduce an SQS buffer layer between S3 and Lambda to smooth out heavy burst workloads and handle retry logic.
Centralized security parsing functions must query DynamoDB and read logs across organizational accounts without global administrative permissions.
Intruders attempting to cover tracks might try to modify or delete logs and threat records in the evidence store.
Optim:Direct event routing, batch size optimizations, and memory tuning of parser Lambdas.
Optim:Configured concurrent execution limits and partitioned DynamoDB keys for risk caches.
High volumes of benign system activity (e.g. deployment roles creating access keys) triggered false positives, drowning security teams in warnings.
Root Cause:Heuristics were too simple, matching only on high-severity API calls without examining the calling entity context or history.
Traced alert history over 30 days and observed that 88% of alerts originated from trusted automated CI/CD roles executing predictable tasks.
Implemented context-aware suppression. Roles that matched a specific cryptographically signed identity configuration and ran inside known IP CIDR blocks were automatically suppressed or assigned a lower risk score.
Introduced some latency in evaluation to check the DynamoDB identity directory, adding ~40ms to processing time.
Static rules always decay; threat detection systems must adapt to environment contexts and identify normal automated patterns.
Clean log ingestion requires strong schema validation on entry, as API structures occasionally shift without notice.
Asynchronous event decoupling is essential; parsing failures must never block downstream incident processing.
Operational security metrics (false-positive ratios, analyst review times) are just as critical as architectural ones.
I would replace the DynamoDB caching layer with Redis to lower evaluation latency and enable easier cluster-wide scale.
Audit raw repository script configurations directly inside the active terminal workspace.
Keep exploring
A multi-cloud serverless deployment leveraging AWS Lambda, DynamoDB, and Azure Functions to build scalable pipelines that minimize operational overhead.
ArchitectureAPI Gateway → Lambda / Azure Functions → DynamoDB / CosmosDB
A collection of security assessment tools simulating attacks and identifying over-privileged credentials to verify enterprise infrastructure security.
ArchitectureIAM Config → Privilege Analyzer → Severity Dashboard
A thermal tracking platform deployed on harizeon.com that captures IoT sensor streams, plots container temperature metrics, and triggers instant alerts for cold-chain compliance.
ArchitectureIoT Sensors → AWS IoT Core → Express → DynamoDB → WebSockets → SNS