error hit both on Qwen3.5-35B-A3B/Qwen3.6-35B-A3B.
image:intel/llm-scaler-vllm:0.21.0-b3.1
MACHINE: xeon-w.
gpu: 2x B70.
OS: edgeai@edgeaihost15:$ uname -a
Linux edgeaihost15 6.17.0-1009-intel #9-Ubuntu SMP PREEMPT_DYNAMIC Mon Apr 6 17:05:13 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
edgeai@edgeaihost15:$
VLLM SERVER CMD:
root@edgeaihost15:/llm-scaler/vllm# TORCH_LLM_ALLREDUCE=1 VLLM_USE_V1=1 W_LONG_MAX_MODEL_LEN=1 VLLM_WORKER_MULTIPROC_METHOD=spawn python3 -m vllm.entrypoints.openai.api_server --model /llm/models/Qwen3.6-35B-A3B --dtype=float16 --enforce-eager --port 8000 --block-size 64 --gpu-memory-util 0.9 --trust-remote-code --max_num_batched_tokens=4096 --max_model_len 50000 --quantization sym_int4 -tp=2
REQUEST CMD:
root@edgeaihost15:/llm-scaler/vllm# curl http://127.0.01:8000/v1/completions -H "Content-Type: application/json" -d '{ "model": "/llm/models/Qwen3.5-35B-A3B", "prompt": "give the info about shanghai city and deliver one ppt FILE", "max_tokens": 2000, "temperature": 0}'
RESPONSE:
root@edgeaihost15:/llm-scaler/vllm#
root@edgeaihost15:/llm-scaler/vllm# curl http://127.0.01:8000/v1/completions -H "Content-Type: application/json" -d '{ "model": "/llm/models/Qwen3.6-35B-A3B", "prompt": "give the info about shanghai city and deliver one ppt FILE", "max_t okens": 2000, "temperature": 0}'
{"id":"cmpl-83e704bfaf30790a","object":"text_completion","created":1787221008,"prompt_routed_experts":null,"model":"/llm/mo dels/Qwen3.6-35B-A3B","choices":[{"index":0,"text":"\n\n\nHere's a thinking process:\n\n1. *Analyze User Request: *\n - Topic: Shanghai city\n - Output: Information about Shanghai + deliver one PPT file\n - Key Constraint s: I'm an AI text model. I cannot directly generate or send files (like .pptx). I need to clarify this limitation while p roviding a complete, ready-to-use PPT structure/content that the user can easily copy into PowerPoint. I should also offer to generate the file via code (Python/python-pptx) if they want, but the standard approach is to provide the slide-by-slide content in a format they can paste.\n\n2. Identify Core Requirements:\n - Provide comprehensive, well-structured in formation about Shanghai\n - Format it as a PowerPoint presentation (slide titles, bullet points, speaker notes if needed )\n - Acknowledge file delivery limitation and provide a practical workaround\n - Ensure accuracy, up-to-date info (as of 2024/2025), and professional tone\n\n3. Research/Verify Key Facts about Shanghai (Mental Knowledge Base):\n - Loc ation: Eastern China, Yangtze River Delta\n - Status: Direct-administered municipality, global financial/commercial hub\n - Population: ~24-25 million (city proper), ~29 million (metropolitan)\n - Area: ~6,340 km²\n - Economy: GDP ~$500B+ USD, major port (Shanghai Port is world's busiest container port)\n - History: Treaty port since 1842, rapid modernizati on post-1990 (Pudong development)\n - Culture: Blend of Chinese tradition & Western influence, "Paris of the East"\n - Landmarks: The Bund, Oriental Pearl Tower, Shanghai Tower, Yu Garden, Nanjing Road\n - Transportation: Metro (largest i n world by route length), Hongqiao & Pudong airports\n - Education/Innovation: Fudan, SJTU, AI & tech hubs\n - Challeng es: Pollution control, housing costs, aging population, climate resilience\n - Future: AI city, green transition, Yangtze River Delta integration\n\n4. Structure the PPT (10-12 slides standard):\n - Slide 1: Title Slide\n - Slide 2: Ov erview & Key Facts\n - Slide 3: Historical Evolution\n - Slide 4: Economic Powerhouse\n - Slide 5: Urban Landscape & Architecture\n - Slide 6: Culture & Lifestyle\n - Slide 7: Transportation & Infrastructure\n - Slide 8: Innovation & Education\n - Slide 9: Challenges & Sustainability\n - Slide 10: Future Outlook\n - Slide 11: Q&A / Thank You\n\n5. Draft Content (Slide by Slide - concise, presentation-ready):\n (I'll format it clearly so the user can copy-paste i nto PowerPoint)\n\n Slide 1: Title\n - Title: Shanghai: China’s Global Gateway\n - Subtitle: History, Economy, I nnovation & Urban Vision\n - Presenter/Date placeholders\n\n Slide 2: Overview & Key Facts\n - Population: ~24.8 million (city proper)\n - Area: 6,340 km²\n - Status: Direct-administered municipality, global financial hub\n - GDP: ~$530B USD (2023)\n - Location: Eastern China, Yangtze River Delta\n - Nickname: “Paris of the East”, “Magic City”\n\n Slide 3: Historical Evolution\n - 1842: Opened as treaty port after Opium War\n - 1920s-30s: “Paris of the East” – cultural & commercial boom\n - 1949-1978: Industrial base under planned economy\n - 1990: Pudong New Area developmen t – rapid modernization\n - 2010: Hosted World Expo\n - Today: Symbol of China’s reform & opening-up\n\n Slide 4: E conomic Powerhouse\n - World’s busiest container port (Shanghai Port)\n - Headquarters to 100+ Fortune Global 500 com panies\n - Key sectors: Finance, shipping, tech, manufacturing, retail\n - Free Trade Zone (2013): Pilot for financial & trade liberalization\n - GDP per capita: ~$38,000 USD (higher than national avg)\n\n Slide 5: Urban Landscape & Arc hitecture\n - The Bund: Historic colonial architecture\n - Pudong Skyline: Shanghai Tower (632m), Oriental Pearl Towe r, Jin Mao\n - Historic Districts: Yu Garden, Xintiandi, Tianzifang\n - Urban Planning: 15-minute city concept, green c orridors\n - Integration of heritage & hyper-modern design\n\n Slide 6: Culture & Lifestyle\n - Culinary: Shangha inese cuisine (xiaolongbao, scallion pancakes), global dining\n - Arts: Shanghai Museum, Power Station of Art, West Bund\ n - Events: Shanghai Fashion Week, International Film Festival\n - Daily Life: High digitalization (Alipay, WeChat), vi brant nightlife, café culture\n - Multicultural: Large expat community, international schools\n\n Slide 7: Transporta tion & Infrastructure\n - Metro: 20+ lines, ~800 km (world’s largest by route length)\n - Airports: Pudong (PVG), Hon gqiao (SHA) – 100M+ annual passengers\n - High-Speed Rail: Hongqiao hub connects to Beijing, Nanjing, Hangzhou\n - Smar t City: AI traffic management, contactless payments, EV infrastructure\n - Future: Maglev expansion, underground logistic s\n\n Slide 8: Innovation & Education\n - Universities: Fudan, Shanghai Jiao Tong University, Tongji\n - Tech Hub s: Zhangjiang Hi-Tech Park, AI & semiconductor focus\n - R&D Investment: ~4.5% of GDP\n - Startups: Strong in fintech, biotech, AI, new energy\n - Global Rankings: Top 10 in innovation indices (WIPO, QS)\n\n Slide 9: Challenges & Sustai nability\n - Environmental: Air/water quality improvements, carbon neutrality goals (2050)\n - Urban: Housing afforda bility, aging population, traffic management\n - Climate: Sea-level rise, flood resilience infrastructure\n - Policy: “ Dual circulation”, green finance, circular economy pilots\n - Balance: Growth vs. livability & ecological preservation\n\ n Slide 10: Future Outlook\n - AI & Digital Economy: “Shanghai AI City” initiative\n - Green Transition: Carbon t rading hub, renewable energy integration\n - Regional Integration: Yangtze River Delta economic zone\n - Global Hub: Fi nancial, shipping, trade, & innovation!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!","logprobs":null,"finish_reason":"length","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt _token_ids":null,"routed_experts":null}],"service_tier":null,"system_fingerprint":"vllm-0.21.1.dev0+gad7125a43.d20260812-tp 2-e64ab433","usage":{"prompt_tokens":12,"total_tokens":2012,"completion_tokens":2000,"prompt_tokens_details":null},"kv_tran root@edgeaihost15:/llm-scaler/vllm# exit
error hit both on Qwen3.5-35B-A3B/Qwen3.6-35B-A3B.
image:intel/llm-scaler-vllm:0.21.0-b3.1
MACHINE: xeon-w.
gpu: 2x B70.
OS: edgeai@edgeaihost15:
$ uname -a$Linux edgeaihost15 6.17.0-1009-intel #9-Ubuntu SMP PREEMPT_DYNAMIC Mon Apr 6 17:05:13 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
edgeai@edgeaihost15:
VLLM SERVER CMD:
root@edgeaihost15:/llm-scaler/vllm# TORCH_LLM_ALLREDUCE=1 VLLM_USE_V1=1 W_LONG_MAX_MODEL_LEN=1 VLLM_WORKER_MULTIPROC_METHOD=spawn python3 -m vllm.entrypoints.openai.api_server --model /llm/models/Qwen3.6-35B-A3B --dtype=float16 --enforce-eager --port 8000 --block-size 64 --gpu-memory-util 0.9 --trust-remote-code --max_num_batched_tokens=4096 --max_model_len 50000 --quantization sym_int4 -tp=2
REQUEST CMD:
root@edgeaihost15:/llm-scaler/vllm# curl http://127.0.01:8000/v1/completions -H "Content-Type: application/json" -d '{ "model": "/llm/models/Qwen3.5-35B-A3B", "prompt": "give the info about shanghai city and deliver one ppt FILE", "max_tokens": 2000, "temperature": 0}'
RESPONSE:
root@edgeaihost15:/llm-scaler/vllm#
root@edgeaihost15:/llm-scaler/vllm# curl http://127.0.01:8000/v1/completions -H "Content-Type: application/json" -d '{ "model": "/llm/models/Qwen3.6-35B-A3B", "prompt": "give the info about shanghai city and deliver one ppt FILE", "max_t okens": 2000, "temperature": 0}'
{"id":"cmpl-83e704bfaf30790a","object":"text_completion","created":1787221008,"prompt_routed_experts":null,"model":"/llm/mo dels/Qwen3.6-35B-A3B","choices":[{"index":0,"text":"\n\n\nHere's a thinking process:\n\n1. *Analyze User Request: *\n - Topic: Shanghai city\n - Output: Information about Shanghai + deliver one PPT file\n - Key Constraint s: I'm an AI text model. I cannot directly generate or send files (like .pptx). I need to clarify this limitation while p roviding a complete, ready-to-use PPT structure/content that the user can easily copy into PowerPoint. I should also offer to generate the file via code (Python/python-pptx) if they want, but the standard approach is to provide the slide-by-slide content in a format they can paste.\n\n2. Identify Core Requirements:\n - Provide comprehensive, well-structured in formation about Shanghai\n - Format it as a PowerPoint presentation (slide titles, bullet points, speaker notes if needed )\n - Acknowledge file delivery limitation and provide a practical workaround\n - Ensure accuracy, up-to-date info (as of 2024/2025), and professional tone\n\n3. Research/Verify Key Facts about Shanghai (Mental Knowledge Base):\n - Loc ation: Eastern China, Yangtze River Delta\n - Status: Direct-administered municipality, global financial/commercial hub\n - Population: ~24-25 million (city proper), ~29 million (metropolitan)\n - Area: ~6,340 km²\n - Economy: GDP ~$500B+ USD, major port (Shanghai Port is world's busiest container port)\n - History: Treaty port since 1842, rapid modernizati on post-1990 (Pudong development)\n - Culture: Blend of Chinese tradition & Western influence, "Paris of the East"\n - Landmarks: The Bund, Oriental Pearl Tower, Shanghai Tower, Yu Garden, Nanjing Road\n - Transportation: Metro (largest i n world by route length), Hongqiao & Pudong airports\n - Education/Innovation: Fudan, SJTU, AI & tech hubs\n - Challeng es: Pollution control, housing costs, aging population, climate resilience\n - Future: AI city, green transition, Yangtze River Delta integration\n\n4. Structure the PPT (10-12 slides standard):\n - Slide 1: Title Slide\n - Slide 2: Ov erview & Key Facts\n - Slide 3: Historical Evolution\n - Slide 4: Economic Powerhouse\n - Slide 5: Urban Landscape & Architecture\n - Slide 6: Culture & Lifestyle\n - Slide 7: Transportation & Infrastructure\n - Slide 8: Innovation & Education\n - Slide 9: Challenges & Sustainability\n - Slide 10: Future Outlook\n - Slide 11: Q&A / Thank You\n\n5. Draft Content (Slide by Slide - concise, presentation-ready):\n (I'll format it clearly so the user can copy-paste i nto PowerPoint)\n\n Slide 1: Title\n - Title: Shanghai: China’s Global Gateway\n - Subtitle: History, Economy, I nnovation & Urban Vision\n - Presenter/Date placeholders\n\n Slide 2: Overview & Key Facts\n - Population: ~24.8 million (city proper)\n - Area: 6,340 km²\n - Status: Direct-administered municipality, global financial hub\n - GDP: ~$530B USD (2023)\n - Location: Eastern China, Yangtze River Delta\n - Nickname: “Paris of the East”, “Magic City”\n\n Slide 3: Historical Evolution\n - 1842: Opened as treaty port after Opium War\n - 1920s-30s: “Paris of the East” – cultural & commercial boom\n - 1949-1978: Industrial base under planned economy\n - 1990: Pudong New Area developmen t – rapid modernization\n - 2010: Hosted World Expo\n - Today: Symbol of China’s reform & opening-up\n\n Slide 4: E conomic Powerhouse\n - World’s busiest container port (Shanghai Port)\n - Headquarters to 100+ Fortune Global 500 com panies\n - Key sectors: Finance, shipping, tech, manufacturing, retail\n - Free Trade Zone (2013): Pilot for financial & trade liberalization\n - GDP per capita: ~$38,000 USD (higher than national avg)\n\n Slide 5: Urban Landscape & Arc hitecture\n - The Bund: Historic colonial architecture\n - Pudong Skyline: Shanghai Tower (632m), Oriental Pearl Towe r, Jin Mao\n - Historic Districts: Yu Garden, Xintiandi, Tianzifang\n - Urban Planning: 15-minute city concept, green c orridors\n - Integration of heritage & hyper-modern design\n\n Slide 6: Culture & Lifestyle\n - Culinary: Shangha inese cuisine (xiaolongbao, scallion pancakes), global dining\n - Arts: Shanghai Museum, Power Station of Art, West Bund\ n - Events: Shanghai Fashion Week, International Film Festival\n - Daily Life: High digitalization (Alipay, WeChat), vi brant nightlife, café culture\n - Multicultural: Large expat community, international schools\n\n Slide 7: Transporta tion & Infrastructure\n - Metro: 20+ lines, ~800 km (world’s largest by route length)\n - Airports: Pudong (PVG), Hon gqiao (SHA) – 100M+ annual passengers\n - High-Speed Rail: Hongqiao hub connects to Beijing, Nanjing, Hangzhou\n - Smar t City: AI traffic management, contactless payments, EV infrastructure\n - Future: Maglev expansion, underground logistic s\n\n Slide 8: Innovation & Education\n - Universities: Fudan, Shanghai Jiao Tong University, Tongji\n - Tech Hub s: Zhangjiang Hi-Tech Park, AI & semiconductor focus\n - R&D Investment: ~4.5% of GDP\n - Startups: Strong in fintech, biotech, AI, new energy\n - Global Rankings: Top 10 in innovation indices (WIPO, QS)\n\n Slide 9: Challenges & Sustai nability\n - Environmental: Air/water quality improvements, carbon neutrality goals (2050)\n - Urban: Housing afforda bility, aging population, traffic management\n - Climate: Sea-level rise, flood resilience infrastructure\n - Policy: “ Dual circulation”, green finance, circular economy pilots\n - Balance: Growth vs. livability & ecological preservation\n\ n Slide 10: Future Outlook\n - AI & Digital Economy: “Shanghai AI City” initiative\n - Green Transition: Carbon t rading hub, renewable energy integration\n - Regional Integration: Yangtze River Delta economic zone\n - Global Hub: Fi nancial, shipping, trade, & innovation!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!","logprobs":null,"finish_reason":"length","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt _token_ids":null,"routed_experts":null}],"service_tier":null,"system_fingerprint":"vllm-0.21.1.dev0+gad7125a43.d20260812-tp 2-e64ab433","usage":{"prompt_tokens":12,"total_tokens":2012,"completion_tokens":2000,"prompt_tokens_details":null},"kv_tran root@edgeaihost15:/llm-scaler/vllm# exit