Multiple model selection with remote service #1049
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ort multiple models chatqna with wrapper is required
…ong with remote tgi/tei service with instructions
…ital Assistant, Code Generator, Content Summarizer respectively
…ntent exceeds the window height
…and Faqgen response
… adjust to different screen sizes
…n the chatqna prompt section
…different models from dropdown
for more information, see https://pre-commit.ci
…docker compose files Signed-off-by: sgurunat <[email protected]>
for more information, see https://pre-commit.ci
letonghan
reviewed
Oct 31, 2024
ProductivitySuite/docker_compose/intel/hpu/gaudi/set_env_remote.sh
Outdated
Show resolved
Hide resolved
ftian1
approved these changes
Nov 8, 2024
amberjain1
reviewed
Nov 8, 2024
amberjain1
reviewed
Nov 8, 2024
amberjain1
reviewed
Nov 11, 2024
amberjain1
reviewed
Nov 11, 2024
amberjain1
reviewed
Nov 11, 2024
amberjain1
reviewed
Nov 11, 2024
ProductivitySuite/docker_compose/intel/hpu/gaudi/compose_remote.yaml
Outdated
Show resolved
Hide resolved
amberjain1
reviewed
Nov 11, 2024
ProductivitySuite/docker_compose/intel/hpu/gaudi/compose_tgi_remote.yaml
Outdated
Show resolved
Hide resolved
…d of copying it from local
…om/sgurunat/GenAIExamples into multiple-model-with-remote-service
…s with vLLM Signed-off-by: sgurunat <[email protected]>
…om/sgurunat/GenAIExamples into multiple-model-with-remote-service
Signed-off-by: sgurunat <[email protected]>
…with set_env.sh Signed-off-by: sgurunat <[email protected]>
…M based instructions Signed-off-by: sgurunat <[email protected]>
…ose_remote.yaml file under gaudi folder Signed-off-by: sgurunat <[email protected]>
…t required Signed-off-by: sgurunat <[email protected]>
lvliang-intel
approved these changes
Nov 12, 2024
Collaborator
|
@sgurunat, |
Signed-off-by: sgurunat <[email protected]>
…om/sgurunat/GenAIExamples into multiple-model-with-remote-service
Contributor
Author
|
@lvliang-intel - Fixed it |
Collaborator
|
Please check the failed CI, https://github.com/opea-project/GenAIExamples/actions/runs/11795779662/job/32856256339?pr=1049 |
chensuyue
reviewed
Nov 13, 2024
chensuyue
reviewed
Nov 13, 2024
for more information, see https://pre-commit.ci
Signed-off-by: sgurunat <[email protected]>
…om/sgurunat/GenAIExamples into multiple-model-with-remote-service
chensuyue
reviewed
Nov 14, 2024
| git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../ | ||
|
|
||
| echo "Build all the images with --no-cache, check docker_image_build.log for details..." | ||
| docker compose -f build_vllm.yaml build --no-cache > ${LOG_PATH}/docker_image_build.log |
Collaborator
There was a problem hiding this comment.
build_vllm.yaml -> build.yaml and just build the images required by this test.
Contributor
Author
There was a problem hiding this comment.
Oh yeah, missed to change it. Updated it now. Thanks
chensuyue
reviewed
Nov 14, 2024
| docker compose -f build_vllm.yaml build --no-cache > ${LOG_PATH}/docker_image_build.log | ||
|
|
||
| docker pull ghcr.io/huggingface/tei-gaudi:latest | ||
| docker pull opea/vllm-hpu:latest |
Collaborator
There was a problem hiding this comment.
You don't need pull this, this will be build in the CI test.
Contributor
Author
There was a problem hiding this comment.
ok commented it out.
Signed-off-by: sgurunat <[email protected]>
Signed-off-by: sgurunat <[email protected]>
Collaborator
|
This PR should be closed right? |
Collaborator
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR contains changes related to multiple models selection in ProductivitySuite ChatQnA along with some minor enhancements to the UI. Also it contains docker compose files and instructions to run ProductivitySuite on Intel Gaudi server with remote TGI/TEI services.
Type of change
New feature (non-breaking change which adds new functionality)
Others (enhancement, documentation, validation, etc.)
New Features:
Add chatqna_wrapper.py along with updated Dockerfile.wrapper. To support multiple models chatqna with wrapper is required
ProductivitySuite: Add docker compose files for Intel Gaudi server along with remote tgi/tei service with instructions
ProductivitySuite UI: Add multiple models support. Choose different models from dropdown
Enhancements:
ProductivitySuite UI: Update names of ChatQnA, CodeGen, DocSum to Digital Assistant, Code Generator, Content Summarizer respectively
ProductivitySuite UI: Update Docsum to have vertical scroll bar if content exceeds the window height
ProductivitySuite UI: Remove <|eot_id|> string from the Chat, Docsum and Faqgen response
ProductivitySuite UI: Update contextWrapper and contextTitle width to adjust to different screen sizes
ProductivitySuite UI: Show system prompt input field always to edit in the chatqna prompt section
ProductivitySuite UI: Update max_new_tokens into max_tokens