Basic/WIP implementation of Copliot into the AI System, as well as Support for Scratch projects. #8088

charlieduzstuf · 2025-12-22T19:29:45Z

Very basic/wip, but what this does:

Users can use Local Models or their own API keys to use the GDevelop AI Features more than the capped amount.
Parts of Vscode/Github Copilot are built into the AI now
GDevelop can open projects made in the Scratch Engine

Seemed like a cool PR to make tbh.

…PI key support Co-authored-by: charlieduzstuf <[email protected]>

Co-authored-by: charlieduzstuf <[email protected]>

…ebGPU acceleration Co-authored-by: charlieduzstuf <[email protected]>

Co-authored-by: charlieduzstuf <[email protected]>

…are assistance Co-authored-by: charlieduzstuf <[email protected]>

…and-api-keys Add local AI model support with unlimited requests, custom API keys, production-ready local inference with WebGPU acceleration, Scratch project import, and VS Code Copilot Chat integration

Copilot

Pull request overview

This PR adds three major features to GDevelop: local AI model support, custom API key integration, and Scratch project import functionality. The implementation provides unlimited AI requests through local models and user-provided API keys.

Key Changes:

Local AI model infrastructure with download scripts and management UI
Custom API key configuration for OpenAI, Anthropic, Google AI, and HuggingFace
Scratch (.sb3) project import with block-to-event conversion
GitHub Copilot-inspired chat interface with specialized agents

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
Local AI Models (18 files)	Infrastructure for downloading and running local LLMs with transformers.js
Scratch Importer (4 files)	Parser and converter for importing Scratch projects into GDevelop
Copilot Chat (3 files)	VS Code Copilot-inspired agent system for code assistance
UI Integration (3 files)	Dialog components and AI request routing with unlimited request support
Configuration (2 files)	AI preset configuration with local model detection
Tests & Docs (5 files)	Unit tests and comprehensive documentation
CI/CD (1 file)	GitHub Actions workflow for model downloads

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-22T19:39:52Z

newIDE/app/src/AiGeneration/Local/AiRequestWrapper.js

+  if (isUsingCustomKeys && !isLocal) {
+    // Use direct API call with custom keys
+    return createDirectAiRequest(params);
+  }
+
+  // Use GDevelop's backend (default)
+  return gdCreateAiRequest(getAuthorizationHeader, params);
+};
+


Duplicate code block detected. Lines 66-73 are identical to lines 54-63, creating unreachable code. The second block will never execute.

Suggested change

if (isUsingCustomKeys && !isLocal) {

// Use direct API call with custom keys

return createDirectAiRequest(params);

}

// Use GDevelop's backend (default)

return gdCreateAiRequest(getAuthorizationHeader, params);

};

Copilot · 2025-12-22T19:39:53Z

newIDE/app/src/AiGeneration/Local/index.js

+export {
+  saveApiKeys,
+  loadApiKeys,
+  getApiKeyForProvider,
+  clearApiKeys,
+  setActiveLocalModel,
+  getActiveLocalModel,
+  setUseLocalModel,
+  shouldUseLocalModel,
+  type ApiKeyConfig,
+} from './LocalStorage';


Missing type export. The type ApiKeyConfig is imported from './CustomApiKeysDialog' but the CustomApiKeysDialog file exports it. However, this export is then re-exported in LocalStorage.js line 7, but the index.js file should export it from LocalStorage, not from CustomApiKeysDialog to maintain consistency.

Copilot · 2025-12-22T19:39:53Z

newIDE/app/src/AiGeneration/Local/LocalStorage.js

+export const saveApiKeys = (apiKeys: Array<ApiKeyConfig>): void => {
+  try {
+    localStorage.setItem(API_KEYS_STORAGE_KEY, JSON.stringify(apiKeys));
+  } catch (error) {
+    console.error('Failed to save API keys:', error);
+  }
+};


Security concern: API keys are stored in plain text in localStorage. While mentioned as "encrypted by browser," localStorage does not encrypt data by default. Any script running on the page can access these keys. Consider using a more secure storage mechanism or warning users about this limitation.

Copilot · 2025-12-22T19:39:53Z

newIDE/app/src/AiGeneration/Local/CustomApiKeysDialog.js

+      .filter(([_, key]) => key && key.trim() !== '')
+      .map(([provider, apiKey]) => ({
+        provider,
+        // $FlowFixMe - We know apiKey is a string


The Flow type suppression comment "// $FlowFixMe - We know apiKey is a string" is unnecessary. The Object.entries() return type is correct, and the filter ensures only non-empty strings remain. Remove this suppression as the type is already correct.

Suggested change

// $FlowFixMe - We know apiKey is a string

Copilot · 2025-12-22T19:39:54Z

newIDE/app/src/AiGeneration/Local/index.js

@@ -0,0 +1,59 @@
+// @flow


The PR title and description contain a spelling error: "Copliot" should be "Copilot".

- Enable optional chaining in .flowconfig - Fix export syntax for type exports (separate type exports) - Fix orphaned code in AiRequestWrapper.js - Fix Partial type usage (use $Shape instead) - Fix exact type mismatches - Add type guards for transformersModule, navigator.gpu, performance.memory - Fix TextField and Column component prop usage - Fix default parameter exact type issues - Fix LocalStorage return type - Fix fs.rmSync property check with FlowFixMe Remaining: Test file errors and a few minor issues. Co-authored-by: charlieduzstuf <[email protected]>

- Fix test mock in AiConfiguration.spec.js (add missing properties) - Fix Object.entries type issue in CustomApiKeysDialog.js - Fix exponential spread warning in ConversationManager.js (use Object.assign) - Fix exact type issues in DirectApiClient.js (explicit object construction) Flow type checking now passing except for 1 pre-existing test module error unrelated to AI changes. Co-authored-by: charlieduzstuf <[email protected]>

- Auto-format all AI generation and Scratch importer files - Ensures code follows project formatting standards Co-authored-by: charlieduzstuf <[email protected]>

- Add conditionals to build-storybook workflow to skip AWS steps when secrets unavailable - Add conditionals to extract-translations workflow to skip Crowdin steps when secrets unavailable - This allows workflows to pass in forked repositories while still working normally in the main repo Co-authored-by: charlieduzstuf <[email protected]>

Fix Flow type errors and GitHub Actions workflows in AI generation modules

Copilot AI and others added 12 commits December 22, 2025 14:35

Initial plan

e8359fb

Add local AI models infrastructure with download scripts and custom A…

ef05438

…PI key support Co-authored-by: charlieduzstuf <[email protected]>

Integrate local model and API key dialogs into AI chat interface

ce5d596

Co-authored-by: charlieduzstuf <[email protected]>

Add local model inference stub and test files

32de913

Co-authored-by: charlieduzstuf <[email protected]>

Fix code review issues: import paths, compatibility, and deduplication

d00b480

Co-authored-by: charlieduzstuf <[email protected]>

Add comprehensive feature summary documentation

24d3910

Co-authored-by: charlieduzstuf <[email protected]>

Implement direct API client for custom API keys - bypass GDevelop quotas

4cc0351

Co-authored-by: charlieduzstuf <[email protected]>

Integrate custom API keys into AI request pipeline

1e729f6

Co-authored-by: charlieduzstuf <[email protected]>

Implement production-ready local inference with transformers.js and W…

ed70c9b

…ebGPU acceleration Co-authored-by: charlieduzstuf <[email protected]>

Add Scratch project import support based on Leopard.js concepts

efb9aae

Co-authored-by: charlieduzstuf <[email protected]>

Add VS Code Copilot Chat integration with agent system and context-aw…

0eaccb5

…are assistance Co-authored-by: charlieduzstuf <[email protected]>

Merge pull request #1 from charlieduzstuf/copilot/setup-local-models-…

bb2a26e

…and-api-keys Add local AI model support with unlimited requests, custom API keys, production-ready local inference with WebGPU acceleration, Scratch project import, and VS Code Copilot Chat integration

Copilot AI review requested due to automatic review settings December 22, 2025 19:29

charlieduzstuf requested a review from 4ian as a code owner December 22, 2025 19:29

Copilot started reviewing on behalf of charlieduzstuf December 22, 2025 19:30 View session

Copilot AI reviewed Dec 22, 2025

View reviewed changes

charlieduzstuf and others added 7 commits December 24, 2025 13:26

Merge branch '4ian:master' into master

3d94b9e

Initial plan

bfd8c6d

Format code with Prettier

7ff4661

- Auto-format all AI generation and Scratch importer files - Ensures code follows project formatting standards Co-authored-by: charlieduzstuf <[email protected]>

Merge pull request #2 from charlieduzstuf/copilot/fix-ci-job-errors

52d125d

Fix Flow type errors and GitHub Actions workflows in AI generation modules

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Basic/WIP implementation of Copliot into the AI System, as well as Support for Scratch projects. #8088

Basic/WIP implementation of Copliot into the AI System, as well as Support for Scratch projects. #8088

charlieduzstuf commented Dec 22, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Basic/WIP implementation of Copliot into the AI System, as well as Support for Scratch projects. #8088

Are you sure you want to change the base?

Basic/WIP implementation of Copliot into the AI System, as well as Support for Scratch projects. #8088

Conversation

charlieduzstuf commented Dec 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant