Skip to content

feat(backend): Speed up graph create/update #10025

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

Pwuts
Copy link
Member

@Pwuts Pwuts commented May 22, 2025

Caching the repeated DB calls by the graph lifecycle hooks significantly speeds up graph update/create calls with many authenticated blocks (~300ms saved per authenticated block)

Changes 🏗️

  • Add and use IntegrationCredentialsManager.cached_getter(user_id) in lifecycle hooks
    • Split refresh_if_needed(..) method out of IntegrationCredentialsManager.get(..)
  • Simplify interface of lifecycle hooks: change get_credentials parameter to user_id

Checklist 📋

For code changes:

  • I have clearly listed my changes in the PR description
  • I have made a test plan
  • I have tested my changes according to the test plan:
    • Save a graph with nodes with credentials

@Pwuts Pwuts requested a review from a team as a code owner May 22, 2025 21:24
@Pwuts Pwuts requested review from kcze and aarushik93 and removed request for a team May 22, 2025 21:24
@github-project-automation github-project-automation bot moved this to 🆕 Needs initial review in AutoGPT development kanban May 22, 2025
Copy link

netlify bot commented May 22, 2025

Deploy Preview for auto-gpt-docs-dev canceled.

Name Link
🔨 Latest commit 676c268
🔍 Latest deploy log https://app.netlify.com/projects/auto-gpt-docs-dev/deploys/683358cbecc95f000803a6f6

@github-actions github-actions bot added platform/backend AutoGPT Platform - Back end size/m labels May 22, 2025
Copy link

netlify bot commented May 22, 2025

Deploy Preview for auto-gpt-docs canceled.

Name Link
🔨 Latest commit 676c268
🔍 Latest deploy log https://app.netlify.com/projects/auto-gpt-docs/deploys/683358cb4e0adf0008f3dc5f

Copy link

deepsource-io bot commented May 22, 2025

Here's the code health analysis summary for commits dc981b5..676c268. View details on DeepSource ↗.

Analysis Summary

AnalyzerStatusSummaryLink
DeepSource JavaScript LogoJavaScript✅ SuccessView Check ↗
DeepSource Python LogoPython✅ Success
❗ 7 occurences introduced
🎯 7 occurences resolved
View Check ↗

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

@Pwuts Pwuts changed the title refactor(backend): Reduce DB calls from graph create/update endpoints feat(backend): Reduce DB calls from graph create/update May 22, 2025
@AutoGPT-Agent
Copy link

The PR addresses reducing DB calls for graph operations as stated in the title. The changes include adding a credentials caching function and refactoring the credentials manager. The description clearly explains the changes and references issue #10024. The author has marked that they've made a test plan but has not completed testing according to that plan, as the 'tested my changes' checkbox is unchecked. All changes pass the user_id correctly in the credentials manager functions, maintaining security. The PR title follows conventional commit format with the correct type and scope.

@AutoGPT-Agent
Copy link

The PR meets most of the requirements, but the checklist is not completely filled out. In the test plan section, there are checkboxes that remain unchecked, indicating that the author has not yet tested their changes according to the plan they defined. The PR follows proper title formatting with 'feat(backend)' as the type and scope. The description clearly explains the purpose (speeding up graph operations with authenticated blocks) and lists the specific changes made. The code changes appear to properly handle user_id in credential management, which was an important requirement. All changes appear to be within scope of the PR title.

@AutoGPT-Agent
Copy link

The PR meets most of the requirements. The title follows conventional commit format with the type 'feat' and scope 'backend'. The description clearly references issue #10024 and explains the intent of the changes (speeding up graph update/create calls with authenticated blocks). The changes listed in the PR description match what's shown in the diff - extracting a refresh_if_needed method and implementing a _cached_credentials_getter function. However, the checklist in the PR description is incomplete - the author has marked that they have a test plan but has not actually performed the testing according to that plan (the testing checkbox is unchecked). The changes to data/*.py files appear to properly handle user_id checking through the credential manager.

@Pwuts
Copy link
Member Author

Pwuts commented May 22, 2025

If you read this, it works :)

@Pwuts Pwuts enabled auto-merge May 22, 2025 21:55
@Pwuts Pwuts requested a review from majdyz May 22, 2025 21:55
@Pwuts Pwuts changed the title feat(backend): Reduce DB calls from graph create/update feat(backend): Speed up graph create/update May 24, 2025
kcze
kcze previously approved these changes May 25, 2025
Copy link
Contributor

@kcze kcze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-project-automation github-project-automation bot moved this from 🆕 Needs initial review to 👍🏼 Mergeable in AutoGPT development kanban May 25, 2025
@Pwuts Pwuts requested a review from kcze May 25, 2025 17:52
@github-actions github-actions bot added size/l and removed size/m labels May 25, 2025
@Pwuts Pwuts removed the request for review from aarushik93 May 25, 2025 17:58
Copy link
Contributor

@kcze kcze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Pwuts Pwuts added this pull request to the merge queue May 26, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 26, 2025
@Pwuts Pwuts added this pull request to the merge queue May 26, 2025
@Pwuts Pwuts removed this pull request from the merge queue due to a manual request May 26, 2025
@Pwuts Pwuts added this pull request to the merge queue May 26, 2025
Merged via the queue into dev with commit 8e2fb2d May 26, 2025
22 checks passed
@Pwuts Pwuts deleted the pwuts/open-2506-fetch-user-credentials-only-once-for-on_graph_activate branch May 26, 2025 10:30
@github-project-automation github-project-automation bot moved this from 👍🏼 Mergeable to ✅ Done in AutoGPT development kanban May 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants