running locally works but not on an account

[drianhoxha]

running locally after some fixes works properly, but at the end of the review there is a 504 error that I think is caused by an NGINX 499 error.

[ibrahimcesar]

Can you provide some logs? Type of OS? I was not able to reproduce this error running locally.

[drianhoxha]

2025/10/10 07:16:44 [error] 36#36: *22 upstream timed out (110: Operation timed out) while reading response header from upstream, client: XXXXX, server: , request: "POST /api/analyzer/analyze HTTP/1.1", upstream: "http://XXXXX:3000/analyzer/analyze", host: "XXXXX", referrer: "XXXXX" // XXXXX because of privacy reasons

I received this in my aws instance after some debugging to see if it was nginx's fault. trying to review all pillars at once on MacOS runs until about 80ish % by which time there is a 504 error. reviewing pillars individually works fine. I tried adjusting nginx.conf and setting my load balancer's timeout to maximum as well without any success. deployment region is eu-central-1.

[ibrahimcesar]

If you are willing to experiment, if successful we can implement for every user:

1. Increase ECS Task CPU/Memory (Immediate Fix)

The backend task may need more resources for intensive Bedrock operations. Update the CDK stack:

# In wa_genai_stack.py, find the backend_task_definition and modify:
backend_task_definition = ecs.FargateTaskDefinition(
    self,
    "BackendTaskDef",
    runtime_platform=ecs.RuntimePlatform(
        operating_system_family=ecs.OperatingSystemFamily.LINUX,
        cpu_architecture=architecture["fargate_architecture"],
    ),
    task_role=app_execute_role,
    cpu=2048,  # Increase from default 256
    memory_limit_mib=4096,  # Increase from default 512
)

2. Add Backend Environment Variables

Add timeout configurations to the backend container:

# In the backend_container environment section, add:
environment={
    # ... existing vars ...
    "REQUEST_TIMEOUT": "7200000",  # 2 hours in milliseconds
    "BEDROCK_TIMEOUT": "300000",   # 5 minutes per Bedrock call
}

3. Update Nginx Configuration

Modify ecs_fargate_app/finch/nginx.conf:

# Increase all timeout values to 2 hours
proxy_connect_timeout 7200s;
proxy_send_timeout 7200s;
proxy_read_timeout 7200s;
fastcgi_send_timeout 7200s;
fastcgi_read_timeout 7200s;
keepalive_timeout 7200s;

# For WebSocket connections
location /socket.io/ {
    # ... existing config ...
    proxy_connect_timeout 2h;
    proxy_send_timeout 2h;
    proxy_read_timeout 2h;
}

# For API requests
location ~ ^/api/(.*) {
    # ... existing config ...
    proxy_connect_timeout 7200s;
    proxy_send_timeout 7200s;
    proxy_read_timeout 7200s;
}

4. Increase ALB Target Group Timeout

In the CDK stack, add after the frontend service creation:

# Configure longer health check and deregistration delay
frontend_service.target_group.configure_health_check(
    path="/healthz",
    timeout=Duration.seconds(30),
    interval=Duration.seconds(60),
    healthy_threshold_count=2,
    unhealthy_threshold_count=5
)

# Increase deregistration delay
frontend_service.target_group.set_attribute(
    "deregistration_delay.timeout_seconds", "300"
)

5. Backend NestJS Timeout (if needed)

If the backend uses NestJS with default timeouts, add to backend/src/main.ts:

// Increase server timeout
const server = await app.listen(3000);
server.setTimeout(7200000); // 2 hours

Deployment Order

1. Update CDK stack with CPU/memory and environment variables
2. Deploy: cdk deploy
3. The nginx changes will be included in the container rebuild

These changes will allow the analysis to run for up to 2 hours, which should be sufficient for analyzing all pillars simultaneously. If you got a success, please report back.