Poor performance of springdoc on large projects with many controllers and deeply nested types

[shitikanth]

Describe the bug

Springdoc takes a very large time to generate the API documentation on large spring boot projects with hundreds of controllers, especially with deeply nested classes.

The underlying issue is that springdoc calls ModelsConverters::resolveAsResolvedSchema for each annotated type from each controller method. This creates a new ModelConverterContextImpl object every single time, which means that we are not able to reuse the result of analyzing the same class from previous invocations. This results in, in some cases, the same class getting analyzed through reflection hundreds of times.

To Reproduce

I have created a reproducible test case in this branch. On my machine, generating the api docs for 200 generated controllers takes more than 30 seconds. Enabling the logs will reveal that the same annotated types are getting re-evaluated hundreds of times.

Expected behavior

• Springdoc should be able to generate the openapi spec even for large projects in less than 2 seconds

Suggested Solution

• We should reuse the ModelConverterContext across multiple invocations for the same api

As a proof of concept, I have posted an extremely dirty and hackish way to cache the context here. The tests stay green and it brings down the execution time for the aforementioned test case to less than 2 seconds.

[shitikanth]

The same issue has been previously reported as #2365, #2299, #2365

[mrBungalow]

Whoa, thank you @shitikanth !!!! We're bit by this as well. I couldn't create a reproducable set of classes as an example so I've given up on trying to fix this and just eat the 15-20 minute load time when we need to update our OpenApi. I used your fix and it loads in seconds.

[bnasslahsen]

@shitikanth,

To keep, the OpenApi spec consistent, new instance of ModelConverterContext is used. Ofc, there is a tradoff of performance.
I liked your fix, when started, but then i realized there is one test failing: SpringDocApp19Test in the following module: springdoc-openapi-kotlin-webmvc-tests.

Feel free to try again, if you find a better workaround.

[shitikanth]

@bnasslahsen I don't understand the incomplete tag or the resolution. I have given a reproducible test case. What more can I add to make the report more complete?

Also, I could not reproduce the test failure (for me, mvn verify completes successfully). I could not investigate further because there is no SpringDocApp19Test in module springdoc-openapi-kotlin-webmvc-tests.

[bnasslahsen]

@shitikanth,

Your proposal, is not complete to get merged, as not all the tests are failing.

This is the failing test: https://github.com/springdoc/springdoc-openapi/blob/main/springdoc-openapi-tests/springdoc-openapi-kotlin-webmvc-tests/src/test/kotlin/test/org/springdoc/api/v31/app19/SpringDocApp19Test.kt

If you uncover a workaround, please add a note, and we’ll reopen it.

[shitikanth]

This is a bug report, not a pull request. The bug report should still be valid irrespective of the feasibility of the proposed solution, no?

I obviously never intended the changes I posted to merged as is. It was a quick and dirty hack to illustrate the point that rescanning of the same classes over and over again is the reason behind the performance bug.

[bnasslahsen]

@shitikanth,

As mentioned earlier, alternative approaches like the one you suggested would conflict with the intended design and tests of the library.
For typical use cases, we haven’t received any reports of performance issues. Additionally, any initial loading cost is incurred only once and is cached thereafter.

We do acknowledge that there’s room for performance improvements, but this will not be considered a bug.

If you’d like to see enhancements that better match your specific needs, we warmly welcome contributions!
You’re encouraged to submit a PR directly, if you feel there are improvements.