Skip to content

Commit 2b11c09

Browse files
ruromeroruromero
committed
added all-in-one logging-dump script and readme
1 parent 4e3d5e1 commit 2b11c09

File tree

3 files changed

+437
-7
lines changed

3 files changed

+437
-7
lines changed

‎docs/issues.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# Filing An Issue
2-
The Openshift Aggregated Logging integration is a multicomponent technology stack whose individual
3-
parts must all work together to gather and archive logs. It is important to include as much information
4-
as possible when issues arise in order to properly debug and diagnose. Run
5-
**ADD SCRIPT OR UTILITY HERE** which will gather the desired information. Alternatively, you can gather
6-
them manually.
2+
The Openshift Aggregated Logging integration is a multicomponent technology stack whose individual
3+
parts must all work together to gather and archive logs. It is important to include as much information
4+
as possible when issues arise in order to properly debug and diagnose. Run
5+
[logging gathering script](../hack/logging-dump.sh) which will gather the desired information. Alternatively, you can gather
6+
them manually.
77

88
**NOTE:** It is your responsiblity as the reporter of an issue to redact any sensitive information from
99
any and all artifacts that are included as part of the issue.
@@ -26,8 +26,7 @@ Please include as much of the following to help us expedite a resolution:
2626
4. Ansible installer logs
2727

2828
### Logging Component Details
29-
**MAYBE WE CAN GRAB SOME OF THESE BY LABEL**
30-
1. Elasticsearch logs: `SOME NON-OC LOGS COMMAND`
29+
1. Elasticsearch logs: `oc logs -l component=es $PODNAME`
3130
2. Other component logs: `oc logs $PODNAME`
3231
3. Fluentd spec: `oc get ds logging-fluentd -o yaml`
3332
4. Other component specs: `oc get dc $DCNAME -o yaml`

‎hack/README-dump.md

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Aggregated Logging Utilities
2+
3+
## logging-dump
4+
`logging-dump.sh` is a tool to gather as much information as possible from your logging cluster. In order to run the script, the user must be logged in and have a administrative permissions on project `logging`
5+
6+
Usage:
7+
```
8+
$ oc login -u admin https://openshift.example.com:8443
9+
$./logging-dump.sh [<component1> ... <componentN>]
10+
```
11+
12+
Components:
13+
* kibana
14+
* elasticsearch
15+
* curator
16+
* fluentd
17+
18+
Examples:
19+
* Get information from the nodes, project and all logging components
20+
21+
`$ ./logging-dump.sh`
22+
23+
* Get information only from elasticsearch and kibana componentes
24+
25+
`$ ./logging-dump.sh elasticsearch kibana`
26+
27+
* Resulting folder structure
28+
```
29+
└── logging-<yyyyMMdd_HHmmss>
30+
├── fluentd
31+
│ ├── logs
32+
│ │ ├── <logging-fluentd-pod-1>.log
33+
│ │ ├── ...
34+
│ │ └── <logging-fluentd-pod-N>.log
35+
│ ├── <logging-fluentd-pod-1>
36+
│ ├── ...
37+
│ └── <logging-fluentd-pod-N>
38+
├── kibana
39+
│ ├── logs
40+
│ │ ├── <logging-kibana-pod>-kibana.log
41+
│ │ ├── <logging-kibana-pod>-kibana-proxy.log
42+
│ │ ├── <logging-kibana-ops-pod>-kibana.log
43+
│ │ └── <logging-kibana-ops-pod>-kibana-proxy.log
44+
│ ├── <logging-kibana-pod>-kibana
45+
│ └── <logging-kibana-pod>-kibana-proxy
46+
│ ├── <logging-kibana-ops-pod>-kibana
47+
│ └── <logging-kibana-ops-pod>-kibana-proxy
48+
├── es
49+
│ ├── cluster-es
50+
| │ ├── aliases
51+
| │ ├── health
52+
| │ ├── indices
53+
| │ ├── nodes
54+
| │ ├── thread_pool
55+
| │ ├── pending_tasks
56+
| │ ├── recovery
57+
| │ ├── shards
58+
| │ └── unassigned_shards
59+
│ ├── cluster-es-ops
60+
| │ ├── aliases
61+
| │ ├── health
62+
| │ ├── indices
63+
| │ ├── nodes
64+
| │ ├── thread_pool
65+
| │ ├── pending_tasks
66+
| │ ├── recovery
67+
| │ ├── shards
68+
| │ └── unassigned_shards
69+
│ ├── logs
70+
│ │ ├── <logging-es-pod-1>.log
71+
│ │ ├── ...
72+
│ │ ├── <logging-es-ops-pod-1>.log
73+
│ │ └── ...
74+
│ ├── <logging-es-pod-1>
75+
│ └── ...
76+
│ ├── <logging-es-ops-pod-1>
77+
│ └── ...
78+
├── curator
79+
│ ├── logs
80+
│ │ ├── <logging-curator-pod>.log
81+
│ │ └── <logging-curator-ops-pod>.log
82+
│ ├── <logging-curator-pod>
83+
│ └── <logging-curator-ops-pod>
84+
└── project
85+
├── configmaps
86+
│ ├── ... # Configmaps
87+
├── daemonsets
88+
│ ├── ... # Daemonsets
89+
├── deploymentconfigs
90+
│ ├── ... # DeploymentConfigs
91+
├── pods
92+
│ ├── ... # Pods
93+
├── pvs
94+
│ ├── ... # Persistent Volumes
95+
├── pvcs
96+
│ ├── ... # Persistent Volume Claims
97+
├── routes
98+
│ ├── ... # Routes
99+
├── services
100+
│ ├── ... # Services
101+
├── serviceaccounts
102+
│ ├── ... # Service Accounts
103+
├── events
104+
├── logging-project
105+
├── nodes
106+
└── secrets
107+
```
108+
109+
110+
### Common
111+
* Nodes description `oc describe nodes`
112+
* Project `oc get project logging -o yaml`
113+
* Pod Logs
114+
* Docker image version `/root/buildinfo/Dockerfile-openshift3*`
115+
* Environment variables
116+
* Pod description
117+
* Deploymentconfigs
118+
* ServiceAccounts
119+
* Configmaps
120+
* Services
121+
* Routes
122+
* Persitent Volumes
123+
* Secrets (Only the name of the files included, not its content)
124+
125+
### Fluentd
126+
* Connectivity with Elasticsearch Servicewith Elasticsearch Service
127+
128+
### Curator
129+
* Connectivity with Elasticsearch Service
130+
131+
### Kibana
132+
* Connectivity with Elasticsearch Servicewith Elasticsearch Service
133+
* Kibana-Proxy oauth-secret `//TODO`
134+
135+
### Elasticsearch
136+
* Cluster health `/_cat/health?v`
137+
* Nodes and memory usage `/_cat/nodes?v`
138+
* Indices `/_cat/indices?v`
139+
* Aliases `/_cat/aliases?v`
140+
* Check 0/1 node folders
141+
* Conditional based on cluster health
142+
* Pending tasks `/_cluster/pending_tasks`
143+
* Recovery status `/_cat/recovery`
144+
* Indices health `/_cat/health?level=indices`
145+
* Unassigned Shards `/_cat/shards?h=index,shard,prirep,state,unassigned.reason,unassigned.description | grep UNASSIGNED`

0 commit comments

Comments
 (0)