Skip to content

Fix AWS macie Timestamp and recipientAccountId fields#3608

Merged
crd1985 merged 1 commit into3.9from
fix-3604-aws-timestamps
Jul 4, 2019
Merged

Fix AWS macie Timestamp and recipientAccountId fields#3608
crd1985 merged 1 commit into3.9from
fix-3604-aws-timestamps

Conversation

@crd1985
Copy link
Contributor

@crd1985 crd1985 commented Jul 2, 2019

Related issue
#3604

Description

A new special processing has been implemented for 'macie' logs to transform recipientAccountId and Timestamps into lists. Now the format is homogeneous to prevent Elasticsearch errors when loading documents in the indexes.

Logs/Alerts example

{
   "timestamp":"2019-07-02T15:08:29.729+0000",
   "rule":{
      "level":12,
      "description":"AWS Macie CRITICAL: S3 Bucket IAM policy grants global read rights - S3 Bucket uses IAM policy to grant read rights to Everyone. Your IAM policy contains a clause that effectively grants read access to any user. Please audit this bucket, and data contained within and confirm that this is intentional. If intentional, please use the alert whitelist feature to prevent future alerts",
      "id":"12345",
      "firedtimes":201,
      "mail":true,
      "groups":[
         "amazon",
         "aws",
         "aws_macie"
      ]
   },
   "agent":{
      "id":"000",
      "name":"wazuh-manager"
   },
   "manager":{
      "name":"wazuh-manager"
   },
   "id":"123412341234.12341234",
   "full_log":"{\"integration\": \"aws\", \"aws\": {\"log_info\": {\"aws_account_alias\": \"\", \"log_file\": \"macie/2019/07/02/00/firehose_macie-1-2019-07-02-00-27-28-1234123\", \"s3bucket\": \"wazuh-bucket\"}, \"notification-type\": \"ALERT_UPDATED\", \"tags\": {\"value\": [\"Open Permissions\", \"Basic Alert\"]}, \"name\": \"S3 Bucket IAM policy grants global read rights\", \"severity\": \"CRITICAL\", \"url\": \"https://mt.us-east-1.macie.aws.amazon.com/posts/arnfake%data2\", \"alert-arn\": \"arn:aws:macie:us-east-1:fake:data/alert/12341234\", \"risk-score\": 9, \"updated-at\": \"2019-07-02T00:27:27.577585\", \"created-at\": \"2019-07-02T00:05:37.579000+00:00\", \"actor\": \"resources.wazuh.com\", \"summary\": {\"Description\": \"S3 Bucket uses IAM policy to grant read rights to Everyone. Your IAM policy contains a clause that effectively grants read access to any user. Please audit this bucket, and data contained within and confirm that this is intentional. If intentional, please use the alert whitelist feature to prevent future alerts\", \"Bucket\": {\"resources.wazuh.com\": 1}, \"Record Count\": 1, \"ACL\": {\"resources.wazuh.com\": {\"Owner\": {\"DisplayName\": \"wazuh\", \"ID\": \"1234123412341234123412341234\"}, \"Grants\": [{\"Grantee\": {\"Type\": \"CanonicalUser\", \"DisplayName\": \"wazuh\", \"ID\": \"1234123412341234123412341234\"}, \"Permission\": \"FULL_CONTROL\"}, {\"Grantee\": {\"Type\": \"Group\", \"URI\": \"http://acs.amazonaws.com/groups/global/AllUsers\"}, \"Permission\": \"READ\"}]}}, \"Event Count\": 1, \"Timestamps\": [\"2019-07-02T00:20:57.775900Z\"], \"recipientAccountId\": [\"12341234123412\"]}, \"source\": \"macie\"}}",
   "decoder":{
      "name":"json"
   },
   "data":{
      "integration":"aws",
      "aws":{
         "log_info":{
            "log_file":"macie/2019/07/02/00/firehose_macie-1-2019-07-02-00-27-28-asdfasdf",
            "s3bucket":"wazuh-bucket"
         },
         "notification-type":"ALERT_UPDATED",
         "tags":{
            "value":"Open Permissions,Basic Alert,"
         },
         "name":"S3 Bucket IAM policy grants global read rights",
         "severity":"CRITICAL",
         "url":"https://mt.us-east-1.macie.aws.amazon.com/posts/fake%data",
         "alert-arn":"arn:aws:macie:us-east-1:fake:data",
         "risk-score":"9",
         "updated-at":"2019-07-02T00:27:27.577585",
         "created-at":"2019-07-02T00:05:37.579000+00:00",
         "actor":"resources.wazuh.com",
         "summary":{
            "Description":"S3 Bucket uses IAM policy to grant read rights to Everyone. Your IAM policy contains a clause that effectively grants read access to any user. Please audit this bucket, and data contained within and confirm that this is intentional. If intentional, please use the alert whitelist feature to prevent future alerts",
            "Bucket":{
               "resources":{
                  "wazuh":{
                     "com":"1"
                  }
               }
            },
            "Record Count":"1",
            "ACL":{
               "resources":{
                  "wazuh":{
                     "com":{
                        "Owner":{
                           "DisplayName":"wazuh",
                           "ID":"1234123412341234"
                        }
                     }
                  }
               }
            },
            "Event Count":"1",
            "Timestamps":"2019-07-02T00:20:57.775900Z,",
            "recipientAccountId":"123412341234,"
         },
         "source":"macie"
      }
   },
   "location":"Wazuh-AWS"
}

Tests

  • No errors in Elasticsearch
  • No errors in ossec.log
  • Fields now are OK in Kibana App
    imagen

imagen

@crd1985 crd1985 self-assigned this Jul 2, 2019
@crd1985 crd1985 force-pushed the fix-3604-aws-timestamps branch from dfa994c to 8cdfa76 Compare July 4, 2019 08:41
@crd1985 crd1985 force-pushed the fix-3604-aws-timestamps branch from 8cdfa76 to 77b9ed2 Compare July 4, 2019 08:48
@crd1985
Copy link
Contributor Author

crd1985 commented Jul 4, 2019

As a result of the review of all fields under data.aws.summary, other ones have been added:

  • Bucket
  • DLP risk
  • IP
  • Location
  • Object
  • Owner
  • Themes
  • Events

In the case of Events, a bit more specific logic has been developed to preserve all subfields.

@crd1985 crd1985 requested a review from davidjiglesias July 4, 2019 10:03
@crd1985 crd1985 merged commit f3644a3 into 3.9 Jul 4, 2019
@crd1985 crd1985 deleted the fix-3604-aws-timestamps branch July 4, 2019 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants