-
Notifications
You must be signed in to change notification settings - Fork 0
Add Graylog Extended Log Format (GELF) for structured logging #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Hi @slissner, This looks great! Thanks for taking it on. Exception
This is because we can't hook into logback correctly with a Other Notes
Sending to GraylogSo far we've designed structured logging to assume that writing to the console or a file is enough to get the structured data to the backend.
I wonder if we can ask for an update to
We might be able to get it to work this way, but if feels a bit brittle. We'd be relying on a side-effect of the formatter.
I'm not sure how we could change the design and also make things work well with Log4j2. We should probably investigate Log4j2 options for Graylog. |
Adding support for GELF output to Spring's structured logging is a nice improvement. You may want to give https://github.com/osiegmar/logback-gelf/tree/external-encoder a try. I widened the encoder fields of A few hints regarding this PR and your log example:
|
Sounds good! I've just released logback-gelf version 6.1.0 which is identical to the branch you've tried. |
GELF for Spring Structured Logging (2024-09-05)After todays work, I reckon that this PR is ready for review. spring-boot-gelf example project
spring-boot changes
|
- Add `gelf` structured logging formatter with Logback - Add `gelf` structured logging formatter with Log4j2 - Describe GELF Formatter in the logging documentation - Add `logging.structured.gelf.service.*` properties to `additional-spring-configuration-metadata.json`
1c91275
to
4452595
Compare
I have now opened an official pull request against the Spring Boot repository: spring-projects#42158 I am closing this PR now. Thanks again for your contributions! |
Summary
As of Spring Boot v3.4, structured logging is now built-in into Spring Boot.
The following pull requests adds a Graylog Extended Log Format (GELF) formatter to the structured logging of Spring Boot.
Please review @phxql CC @philwebb
TODO
WARN : org.graylog2.inputs.codecs.GelfCodec - GELF message <6164add0-6ace-11ef-81af-0242ac150004> (received from <192.168.65.1:30756>) has invalid "timestamp": 1725462010.152 (type: STRING)
StructuredLogFormatter
for thelog4j2
integrationTo be discussed with the feature authors of Structured Logging
structured-console-appender.xml
inlogback.xml
instead ofconsole-appender.xml
LogbackException
(Log Output) when I start my example logger application. The reason for this is that upon the very first log message, no Spring Environment seems to be. This violates the precondition atorg.springframework.boot.logging.logback.StructuredLogEncoder.createFormatter(StructuredLogEncoder.java:71)
(Assert.state(environment != null, "Unable to find Spring Environment in logger context");
Some additional fields get swallowed when sending them to Graylog
In my example logger application, I am using the GELF Appender from https://github.com/osiegmar/logback-gelf from @osiegmar in order to send logs to Graylog.
My new GELF Formatter writes the following log output to my application console:
However, this is the message I have extracted from Graylog:
Note for example
_service_name
is present in the original console log statement, but not in the graylog message;userId
andtestkey_testmessage
are present because they were added to the MDC at the caller site;This is because Oliver's library uses its own GelfEncoder for logback. And this encoder encodes the ILoggingEvent instances. It does not simply pass on the JSON my new GELF Formatter generated in Structured Logging.
According to Oliver's implementation, only the fields from the MDC are passed on as additional fields. Fields such as those in
GraylogExtendedLogFormatStructuredLogFormatter
are NOT passed on.There are several ways of dealing with this:
GraylogExtendedLogFormatStructuredLogFormatter
.Screenshots
Spring Boot GELF Log Output:
Graylog:
See also