Skip to content

Refactor cluster logs#8002

Merged
davidjiglesias merged 5 commits intomasterfrom
dev-7860-improve-cluster-logs
Mar 29, 2021
Merged

Refactor cluster logs#8002
davidjiglesias merged 5 commits intomasterfrom
dev-7860-improve-cluster-logs

Conversation

@Selutario
Copy link
Contributor

@Selutario Selutario commented Mar 24, 2021

Related issue
Closes #7860

Description

Hi team!

Cluster logs were difficult to understand. There was too much information in info level, so most of them have been moved to debug. We have divided the integrity process into two parts:

  • Integrity check: Worker files metadata are calculated and sent to master, where they are compared with local files metadata.
  • Integrity sync: If there are different files in master and worker, this process starts. This includes sending/synchronizing files so that both nodes have the same files.

Worker logs (info)

2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity check] Starting.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity check] Finished in 0.036s. Sync required.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity sync] Starting.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity sync] Files to create: 0 | Files to update: 0 | Files to delete: 1 | Files to send: 0
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity sync] Finished in 0.001s.
2021/03/24 13:26:18 INFO: [Worker worker1] [Keep Alive] Sucessful response from master: keepalive
2021/03/24 13:26:19 INFO: [Local Client] [Agent-info sync] Starting.
2021/03/24 13:26:19 INFO: [Local Client] [Agent-info sync] Finished in 0.008s (0 chunks sent).
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity check] Starting.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity check] Finished in 0.045s. Sync required.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity sync] Starting.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity sync] Files to create: 0 | Files to update: 0 | Files to delete: 0 | Files to send: 1
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity sync] Finished in 0.023s.
2021/03/24 13:26:29 INFO: [Local Client] [Agent-info sync] Starting.
2021/03/24 13:26:29 INFO: [Local Client] [Agent-info sync] Finished in 0.009s (0 chunks sent).
2021/03/24 13:26:36 INFO: [Worker worker1] [Integrity check] Starting.
2021/03/24 13:26:36 INFO: [Worker worker1] [Integrity check] Finished in 0.023s. Sync not required.

Master logs (info)

2021/03/24 13:26:18 INFO: [Master] [Local integrity] Starting.
2021/03/24 13:26:18 INFO: [Master] [Local integrity] Finished in 0.006s. Calculated metadata of 17 files.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity check] Starting. Received metadata of 18 files.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity check] Finished in 0.010s. Sync required.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity sync] Starting.
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity sync] Files to create in worker: 0 | Files to update in worker: 0 | Files to delete in worker: 1 | Files to receive: 0
2021/03/24 13:26:18 INFO: [Worker worker1] [Integrity sync] Finished in 0.008096s.
2021/03/24 13:26:19 INFO: [Worker worker1] [Agent-info sync] Starting.
2021/03/24 13:26:19 INFO: [Worker worker1] [Agent-info sync] Finished in 0.003s (0 chunks received).
2021/03/24 13:26:26 INFO: [Master] [Local integrity] Starting.
2021/03/24 13:26:26 INFO: [Master] [Local integrity] Finished in 0.008s. Calculated metadata of 16 files.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity check] Starting. Received metadata of 17 files.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity check] Finished in 0.018s. Sync required.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity sync] Starting.
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity sync] Files to create in worker: 0 | Files to update in worker: 0 | Files to delete in worker: 0 | Files to receive: 1
2021/03/24 13:26:27 INFO: [Worker worker1] [Integrity sync] Finished in 0.047881s.
2021/03/24 13:26:29 INFO: [Worker worker1] [Agent-info sync] Starting.
2021/03/24 13:26:29 INFO: [Worker worker1] [Agent-info sync] Finished in 0.004s (0 chunks received).
2021/03/24 13:26:34 INFO: [Master] [Local integrity] Starting.
2021/03/24 13:26:34 INFO: [Master] [Local integrity] Finished in 0.025s. Calculated metadata of 17 files.
2021/03/24 13:26:36 INFO: [Worker worker1] [Integrity check] Starting. Received metadata of 17 files.
2021/03/24 13:26:36 INFO: [Worker worker1] [Integrity check] Finished in 0.004s. Sync not required.

cluster_control -i more

# /var/ossec/bin/cluster_control -i more
Cluster name: wazuh

Connected nodes (1):

    master-node (wazuh-master)
        Version: 4.2.0
        Type: master
        Active agents: 1

    worker1 (172.21.0.5)
        Version: 4.2.0
        Type: worker
        Active agents: 0
        Status:
            Last keep Alive:
                Last received: 2021-03-24 13:25:18.645536.
            Integrity check:
                Last integrity check: 2021-03-24 13:25:15.224022 - 2021-03-24 13:25:15.228705.
                Permission to check integrity: True.
            Integrity sync:
                Last integrity synchronization: 2021-03-24 13:25:15.228839 - 2021-03-24 13:25:15.233563.
                Synchronized files: Shared: 0 | Missing: 0 | Extra: 1 | Extra valid: 0.
                Permission to sync extra valid files: True.
                Extra valid files correctly updated in master: 0.
            Agents-info:
                Last synchronization: 2021-03-24 13:25:19.297246 - 2021-03-24 13:25:19.298849.
                Number of synchronized chunks: 0.

Regards,
Selu.

@Selutario Selutario self-assigned this Mar 24, 2021
@Selutario Selutario force-pushed the dev-7860-improve-cluster-logs branch from 37450c2 to c172c7e Compare March 29, 2021 06:53
davidjiglesias
davidjiglesias requested changes Mar 29, 2021
View reviewed changes
davidjiglesias
davidjiglesias previously approved these changes Mar 29, 2021
View reviewed changes
@Selutario Selutario marked this pull request as ready for review March 29, 2021 10:50
@Selutario Selutario marked this pull request as draft March 29, 2021 10:50
@Selutario Selutario marked this pull request as ready for review March 29, 2021 11:11
@davidjiglesias davidjiglesias merged commit ff13cc1 into master Mar 29, 2021
@davidjiglesias davidjiglesias deleted the dev-7860-improve-cluster-logs branch March 29, 2021 15:53
@Selutario Selutario mentioned this pull request Apr 29, 2021
Merged
This was referenced May 14, 2021
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@davidjiglesias davidjiglesias davidjiglesias approved these changes

Assignees

@Selutario Selutario

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Improve cluster logs

2 participants

@Selutario @davidjiglesias