Skip to content

chore: cy prompt infrastructure #31748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: feat/cy-prompt
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions cli/types/cypress.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1826,6 +1826,11 @@ declare namespace Cypress {
*/
prevUntil<E extends Node = HTMLElement>(element: E | JQuery<E>, filter?: string, options?: Partial<Loggable & Timeoutable>): Chainable<JQuery<E>>

/**
* TODO: add docs
*/
prompt(message: string, options?: Partial<Loggable & Timeoutable>): Chainable<Subject>

/**
* Read a file and yield its contents.
*
Expand Down Expand Up @@ -3158,6 +3163,11 @@ declare namespace Cypress {
* @default false
*/
experimentalStudio: boolean
/**
* Enables the prompt command feature.
* @default false
*/
experimentalPromptCommand: boolean
/**
* Adds support for testing in the WebKit browser engine used by Safari. See https://on.cypress.io/webkit-experiment for more information.
* @default false
Expand Down
30 changes: 30 additions & 0 deletions guides/cy-prompt-development.md
Original file line number Diff line number Diff line change
@@ -1 +1,31 @@
# `cy.prompt` Development

In production, the code used to facilitate the prompt command will be retrieved from the Cloud.

To run against locally developed `cy.prompt`:

- Clone the `cypress-services` repo
- Run `yarn`
- Run `yarn watch` in `app/packages/cy-prompt`
- Set:
- `CYPRESS_INTERNAL_ENV=<environment>` (e.g. `staging` or `production` if you want to hit those deployments of `cypress-services` or `development` if you want to hit a locally running version of `cypress-services`)
- `CYPRESS_LOCAL_CY_PROMPT_PATH` to the path to the `cypress-services/app/packages/cy-prompt/dist/development` directory
- Clone the `cypress` repo
- Run `yarn`
- Run `yarn cypress:open`
- Log In to the Cloud via the App
- Open a project that has `experimentalPromptCommand: true` set in the `e2e` config of the `cypress.config.js|ts` file.

To run against a deployed version of `cy.prompt`:

- Set:
- `CYPRESS_INTERNAL_ENV=<environment>` (e.g. `staging` or `production` if you want to hit those deployments of `cypress-services` or `development` if you want to hit a locally running version of `cypress-services`)
```
Copy link
Preview

Copilot AI May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an unmatched code fence here that breaks the markdown list; remove or properly close the block so the document renders correctly.

Copilot uses AI. Check for mistakes.


## Testing

### Unit/Component Testing

The code that supports cloud `cy.prompt` and lives in the `cypress` monorepo is unit, integration, and e2e tested in a similar fashion to the rest of the code in the repo. See the [contributing guide](https://github.com/cypress-io/cypress/blob/ad353fcc0f7fdc51b8e624a2a1ef4e76ef9400a0/CONTRIBUTING.md?plain=1#L366) for more specifics.

The code that supports cloud `cy.prompt` and lives in the `cypress-services` monorepo has unit tests that live alongside the code in that monorepo.
3 changes: 3 additions & 0 deletions packages/config/__snapshots__/index.spec.ts.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ exports['config/src/index .getDefaultValues returns list of public config keys 1
'experimentalSourceRewriting': false,
'experimentalSingleTabRunMode': false,
'experimentalStudio': false,
'experimentalPromptCommand': false,
'experimentalWebKitSupport': false,
'fileServerFolder': '',
'fixturesFolder': 'cypress/fixtures',
Expand Down Expand Up @@ -137,6 +138,7 @@ exports['config/src/index .getDefaultValues returns list of public config keys f
'experimentalSourceRewriting': false,
'experimentalSingleTabRunMode': false,
'experimentalStudio': false,
'experimentalPromptCommand': false,
'experimentalWebKitSupport': false,
'fileServerFolder': '',
'fixturesFolder': 'cypress/fixtures',
Expand Down Expand Up @@ -224,6 +226,7 @@ exports['config/src/index .getPublicConfigKeys returns list of public config key
'experimentalSourceRewriting',
'experimentalSingleTabRunMode',
'experimentalStudio',
'experimentalPromptCommand',
'experimentalWebKitSupport',
'fileServerFolder',
'fixturesFolder',
Expand Down
6 changes: 6 additions & 0 deletions packages/config/src/options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,12 @@ const driverConfigOptions: Array<DriverConfigOption> = [
validation: validate.isBoolean,
isExperimental: true,
requireRestartOnChange: 'browser',
}, {
name: 'experimentalPromptCommand',
defaultValue: false,
validation: validate.isBoolean,
isExperimental: true,
requireRestartOnChange: 'server',
}, {
name: 'experimentalWebKitSupport',
defaultValue: false,
Expand Down
2 changes: 2 additions & 0 deletions packages/config/test/project/utils.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1078,6 +1078,7 @@ describe('config/src/project/utils', () => {
experimentalRunAllSpecs: { value: false, from: 'default' },
experimentalSingleTabRunMode: { value: false, from: 'default' },
experimentalStudio: { value: false, from: 'default' },
experimentalPromptCommand: { value: false, from: 'default' },
experimentalSourceRewriting: { value: false, from: 'default' },
experimentalWebKitSupport: { value: false, from: 'default' },
fileServerFolder: { value: '', from: 'default' },
Expand Down Expand Up @@ -1197,6 +1198,7 @@ describe('config/src/project/utils', () => {
experimentalRunAllSpecs: { value: false, from: 'default' },
experimentalSingleTabRunMode: { value: false, from: 'default' },
experimentalStudio: { value: false, from: 'default' },
experimentalPromptCommand: { value: false, from: 'default' },
experimentalSourceRewriting: { value: false, from: 'default' },
experimentalWebKitSupport: { value: false, from: 'default' },
fileServerFolder: { value: '', from: 'default' },
Expand Down
3 changes: 2 additions & 1 deletion packages/data-context/src/data/coreDataShape.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { FoundBrowser, Editor, AllowedState, AllModeOptions, TestingType, BrowserStatus, PACKAGE_MANAGERS, AuthStateName, MIGRATION_STEPS, MigrationStep, StudioLifecycleManagerShape } from '@packages/types'
import { FoundBrowser, Editor, AllowedState, AllModeOptions, TestingType, BrowserStatus, PACKAGE_MANAGERS, AuthStateName, MIGRATION_STEPS, MigrationStep, StudioLifecycleManagerShape, CyPromptLifecycleManagerShape } from '@packages/types'
import { WizardBundler, CT_FRAMEWORKS, resolveComponentFrameworkDefinition, ErroredFramework } from '@packages/scaffold-config'
import type { NexusGenObjects } from '@packages/graphql/src/gen/nxs.gen'
// tslint:disable-next-line no-implicit-dependencies - electron dep needs to be defined
Expand Down Expand Up @@ -165,6 +165,7 @@ export interface CoreDataShape {
eventCollectorSource: EventCollectorSource | null
didBrowserPreviouslyHaveUnexpectedExit: boolean
studioLifecycleManager?: StudioLifecycleManagerShape
cyPromptLifecycleManager?: CyPromptLifecycleManagerShape
}

/**
Expand Down
1 change: 1 addition & 0 deletions packages/driver/cypress.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export const baseConfig: Cypress.ConfigOptions = {
experimentalStudio: true,
experimentalMemoryManagement: true,
experimentalWebKitSupport: true,
experimentalPromptCommand: true,
hosts: {
'foobar.com': '127.0.0.1',
'*.foobar.com': '127.0.0.1',
Expand Down
9 changes: 9 additions & 0 deletions packages/driver/cypress/e2e/commands/prompt.cy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
describe('src/cy/commands/prompt', () => {
it('executes the prompt command', () => {
cy.visit('/fixtures/dom.html')

// TODO: add more tests when cy.prompt is built out, but for now this just
// verifies that the command executes without throwing an error
cy.prompt('Hello, world!')
})
})
2 changes: 1 addition & 1 deletion packages/driver/cypress/e2e/e2e/origin/commands/misc.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ it('verifies number of cy commands', () => {
'writeFile', 'fixture', 'clearLocalStorage', 'url', 'hash', 'location', 'end', 'noop', 'log', 'wrap', 'reload', 'go', 'visit',
'focused', 'get', 'contains', 'shadow', 'within', 'request', 'session', 'screenshot', 'task', 'find', 'filter', 'not',
'children', 'eq', 'closest', 'first', 'last', 'next', 'nextAll', 'nextUntil', 'parent', 'parents', 'parentsUntil', 'prev', 'press',
'prevAll', 'prevUntil', 'siblings', 'wait', 'title', 'window', 'document', 'viewport', 'server', 'route', 'intercept', 'origin',
'prevAll', 'prevUntil', 'prompt', 'siblings', 'wait', 'title', 'window', 'document', 'viewport', 'server', 'route', 'intercept', 'origin',
'mount', 'as', 'root', 'getAllLocalStorage', 'clearAllLocalStorage', 'getAllSessionStorage', 'clearAllSessionStorage',
'getAllCookies', 'clearAllCookies',
]
Expand Down
1 change: 1 addition & 0 deletions packages/driver/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
"@cypress/unique-selector": "0.0.5",
"@cypress/webpack-dev-server": "0.0.0-development",
"@cypress/webpack-preprocessor": "0.0.0-development",
"@module-federation/runtime": "^0.8.11",
"@packages/config": "0.0.0-development",
"@packages/errors": "0.0.0-development",
"@packages/net-stubbing": "0.0.0-development",
Expand Down
3 changes: 3 additions & 0 deletions packages/driver/src/cy/commands/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ import Window from './window'

import * as Xhr from './xhr'

import * as Prompt from './prompt'

export const allCommands = {
...Actions,
Agents,
Expand All @@ -70,6 +72,7 @@ export const allCommands = {
Misc,
Origin,
Popups,
Prompt,
Navigation,
...Querying,
Request,
Expand Down
59 changes: 59 additions & 0 deletions packages/driver/src/cy/commands/prompt/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import { init, loadRemote } from '@module-federation/runtime'
import type{ CyPromptDriverDefaultShape } from './prompt-driver-types'

interface CyPromptDriver { default: CyPromptDriverDefaultShape }

let initializedCyPrompt: CyPromptDriverDefaultShape | null = null
const initializeCloudCyPrompt = async (Cypress: Cypress.Cypress): Promise<CyPromptDriverDefaultShape> => {
const { success } = await Cypress.backend('wait:for:cy:prompt:ready')

if (!success) {
throw new Error('CyPromptDriver not found')
}

init({
remotes: [{
alias: 'cy-prompt',
type: 'module',
name: 'cy-prompt',
entryGlobalName: 'cy-prompt',
entry: '/__cypress-cy-prompt/cy-prompt.js',
shareScope: 'default',
}],
name: 'driver',
})

const module = await loadRemote<CyPromptDriver>('cy-prompt')

if (!module?.default) {
throw new Error('CyPromptDriver not found')
}

initializedCyPrompt = module.default

return module.default
}

export default (Commands, Cypress, cy) => {
Commands.addAll({
async prompt (message: string) {
if (!Cypress.config('experimentalPromptCommand')) {
// TODO: what do we want to do here?
throw new Error('cy.prompt() is not enabled. Please enable it by setting `experimentalPromptCommand: true` in your Cypress config.')
}

try {
let cloud = initializedCyPrompt

if (!cloud) {
cloud = await initializeCloudCyPrompt(Cypress)
}

return await cloud.cyPrompt(Cypress, message)
} catch (error) {
// TODO: handle this better
throw new Error('CyPromptDriver not found')
}
},
})
}
7 changes: 7 additions & 0 deletions packages/driver/src/cy/commands/prompt/prompt-driver-types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export interface CypressInternal extends Cypress.Cypress {
backend: (eventName: string, ...args: any[]) => Promise<any>
}

export interface CyPromptDriverDefaultShape {
cyPrompt: (Cypress: CypressInternal, text: string) => Promise<void>
}
2 changes: 1 addition & 1 deletion packages/driver/types/internal-types-lite.d.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
/// <reference path="./cy/logGroup.d.ts" />
/// <reference path="./cypress/log.d.ts" />

// All of the types needed by packages/app, without any of the additional APIs used in the driver only

declare namespace Cypress {
Expand Down Expand Up @@ -41,6 +40,7 @@ declare namespace Cypress {
(task: 'protocol:test:before:after:run:async', attributes: any, options: any): Promise<void>
(task: 'protocol:url:changed', input: any): Promise<void>
(task: 'protocol:page:loading', input: any): Promise<void>
(task: 'wait:for:cy:prompt:ready'): Promise<{ success: boolean }>
}

interface Devices {
Expand Down
4 changes: 4 additions & 0 deletions packages/frontend-shared/src/locales/en-US.json
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,10 @@
"name": "Studio",
"description": "Generate and save commands directly to your test suite by interacting with your app as an end user would."
},
"experimentalPromptCommand": {
"name": "Prompt command",
"description": "Enables support for the prompt command."
},
"experimentalWebKitSupport": {
"name": "WebKit Support",
"description": "Adds support for testing in the WebKit browser engine used by Safari. See https://on.cypress.io/webkit-experiment for more information."
Expand Down
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
import type { StudioManager } from './cloud/studio'
import { ProtocolManager } from './cloud/protocol'
import { getAndInitializeStudioManager } from './cloud/api/studio/get_and_initialize_studio_manager'
import type { StudioManager } from './studio'
import { ProtocolManager } from './protocol'
import { getAndInitializeStudioManager } from './api/studio/get_and_initialize_studio_manager'
import Debug from 'debug'
import type { CloudDataSource } from '@packages/data-context/src/sources'
import type { Cfg } from './project-base'
import type { Cfg } from '../project-base'
import _ from 'lodash'
import type { DataContext } from '@packages/data-context'
import api from './cloud/api'
import { reportStudioError } from './cloud/api/studio/report_studio_error'
import { CloudRequest } from './cloud/api/cloud_request'
import { isRetryableError } from './cloud/network/is_retryable_error'
import { asyncRetry } from './util/async_retry'
import { postStudioSession } from './cloud/api/studio/post_studio_session'
import api from './api'
import { reportStudioError } from './api/studio/report_studio_error'
import { CloudRequest } from './api/cloud_request'
import { isRetryableError } from './network/is_retryable_error'
import { asyncRetry } from '../util/async_retry'
import { postStudioSession } from './api/studio/post_studio_session'
import type { StudioStatus } from '@packages/types'

const debug = Debug('cypress:server:studio-lifecycle-manager')
const routes = require('./cloud/routes')
const routes = require('./routes')

export class StudioLifecycleManager {
private studioManagerPromise?: Promise<StudioManager | null>
Expand Down
64 changes: 64 additions & 0 deletions packages/server/lib/cloud/api/cy-prompt/get_cy_prompt_bundle.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import { asyncRetry, linearDelay } from '../../../util/async_retry'
import { isRetryableError } from '../../network/is_retryable_error'
import fetch from 'cross-fetch'
import os from 'os'
import { agent } from '@packages/network'
import { PUBLIC_KEY_VERSION } from '../../constants'
import { createWriteStream } from 'fs'
import { verifySignatureFromFile } from '../../encryption'

const pkg = require('@packages/root')
const _delay = linearDelay(500)

export const getCyPromptBundle = async ({ cyPromptUrl, projectId, bundlePath }: { cyPromptUrl: string, projectId: string, bundlePath: string }) => {
let responseSignature: string | null = null

await (asyncRetry(async () => {
const response = await fetch(cyPromptUrl, {
// @ts-expect-error - this is supported
agent,
method: 'GET',
headers: {
'x-route-version': '1',
'x-cypress-signature': PUBLIC_KEY_VERSION,
...(projectId ? { 'x-cypress-project-slug': projectId } : {}),
'x-cypress-cy-prompt-mount-version': '1',
'x-os-name': os.platform(),
'x-cypress-version': pkg.version,
},
encrypt: 'signed',
})

if (!response.ok) {
throw new Error(`Failed to download cy-prompt bundle: ${response.statusText}`)
}

responseSignature = response.headers.get('x-cypress-signature')

await new Promise<void>((resolve, reject) => {
const writeStream = createWriteStream(bundlePath)

writeStream.on('error', reject)
writeStream.on('finish', () => {
resolve()
})

// @ts-expect-error - this is supported
response.body?.pipe(writeStream)
})
}, {
maxAttempts: 3,
retryDelay: _delay,
shouldRetry: isRetryableError,
}))()

if (!responseSignature) {
throw new Error('Unable to get studio signature')
}

const verified = await verifySignatureFromFile(bundlePath, responseSignature)

if (!verified) {
throw new Error('Unable to verify studio signature')
Comment on lines +56 to +62
Copy link
Preview

Copilot AI May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message below refers to 'studio signature' but this is for cy-prompt; update it to reference 'cy-prompt signature' to avoid confusion.

Suggested change
throw new Error('Unable to get studio signature')
}
const verified = await verifySignatureFromFile(bundlePath, responseSignature)
if (!verified) {
throw new Error('Unable to verify studio signature')
throw new Error('Unable to get cy-prompt signature')
}
const verified = await verifySignatureFromFile(bundlePath, responseSignature)
if (!verified) {
throw new Error('Unable to verify cy-prompt signature')

Copilot uses AI. Check for mistakes.

Copy link
Preview

Copilot AI May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error thrown here mentions 'Unable to verify studio signature'—it should reference 'cy-prompt signature' for accuracy.

Suggested change
throw new Error('Unable to verify studio signature')
throw new Error('Unable to verify cy-prompt signature')

Copilot uses AI. Check for mistakes.

}
}
Loading