Updates to the chatbot rag app (#118)

Author: miguelgrinberg

example-apps/chatbot-rag-app/.flaskenv

@@ -0,0 +1,2 @@
+FLASK_APP=api/app.py
+FLASK_RUN_PORT=3001

example-apps/chatbot-rag-app/Dockerfile

@@ -14,6 +14,7 @@ WORKDIR /app
 RUN mkdir -p ./frontend/build
 COPY --from=build-step ./app/frontend/build ./frontend/build 
 RUN mkdir ./api
+RUN mkdir ./data
 
 RUN apt-get update && apt-get install -y \
     build-essential \
@@ -24,10 +25,11 @@ RUN apt-get update && apt-get install -y \
 
 
 COPY api ./api
+COPY data ./data
 COPY requirements.txt ./requirements.txt
 RUN pip3 install -r ./requirements.txt
 ENV FLASK_ENV production
 
 EXPOSE 4000
 WORKDIR /app/api
-CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0", "--port=4000" ]
+CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0", "--port=4000" ]

example-apps/chatbot-rag-app/README.md

@@ -4,7 +4,7 @@ This is a sample app that combines Elasticsearch, Langchain and a number of diff
 
 ![Screenshot of the sample app](./app-demo.gif)
 
-## 1. Download the Project
+## Download the Project
 
 Download the project from Github and extract the `chatbot-rag-app` folder.
 
@@ -13,7 +13,7 @@ curl https://codeload.github.com/elastic/elasticsearch-labs/tar.gz/main | \
 tar -xz --strip=2 elasticsearch-labs-main/example-apps/chatbot-rag-app
 ```
 
-## 2. Installing and connecting to Elasticsearch
+## Installing and connecting to Elasticsearch
 
 ### Install Elasticsearch
 
@@ -29,6 +29,8 @@ export ELASTIC_USERNAME=...
 export ELASTIC_PASSWORD=...
 ```
 
+You can add these to a *.env* file for convenience. See the *env.example* file for a .env file template.
+
 ### Change the Elasticsearch index and chat_history index
 
 By default, the app will use the `workplace-app-docs` index and the chat history index will be `workplace-app-docs-chat-history`. If you want to change these, you can set the following environment variables:
@@ -38,7 +40,7 @@ ES_INDEX=workplace-app-docs
 ES_INDEX_CHAT_HISTORY=workplace-app-docs-chat-history
 ```
 
-## 3. Connecting to LLM
+## Connecting to LLM
 
 We support three LLM providers: Azure, OpenAI and Bedrock.
 
@@ -100,30 +102,12 @@ region=...
 To use Vertex AI you need to set the following environment variables. More infos [here](https://python.langchain.com/docs/integrations/llms/google_vertex_ai_palm).
 
 ```sh
-    export LLM_TYPE=vertex
-    export VERTEX_PROJECT_ID=<gcp-project-id>
-    export VERTEX_REGION=<gcp-region> # Default is us-central1
-    export GOOGLE_APPLICATION_CREDENTIALS=<path-json-service-account>
+export LLM_TYPE=vertex
+export VERTEX_PROJECT_ID=<gcp-project-id>
+export VERTEX_REGION=<gcp-region> # Default is us-central1
+export GOOGLE_APPLICATION_CREDENTIALS=<path-json-service-account>
 ```
 
-## 3. Ingest Data
-
-You can index the sample data from the provided .json files in the `data` folder:
-
-```sh
-python data/index-data.py
-```
-
-by default, this will index the data into the `workplace-app-docs` index. You can change this by setting the `ES_INDEX` environment variable.
-
-### Indexing your own data
-
-`index-data.py` is a simple script that uses Langchain to index data into Elasticsearch, using the `JSONLoader` and `CharacterTextSplitter` to split the large documents into passages. Modify this script to index your own data.
-
-Langchain offers many different ways to index data, if you cant just load it via JSONLoader. See the [Langchain documentation](https://python.langchain.com/docs/modules/data_connection/document_loaders)
-
-Remember to keep the `ES_INDEX` environment variable set to the index you want to index into and to query from.
-
 ## Running the App
 
 Once you have indexed data into the Elasticsearch index, there are two ways to run the app: via Docker or locally. Docker is advised for testing & production use. Locally is advised for development.
@@ -136,18 +120,22 @@ Build the Docker image and run it with the following environment variables.
 docker build -f Dockerfile -t chatbot-rag-app .
 ```
 
-Then run it with the following environment variables. In the example below, we are using OpenAI LLM.
+#### Ingest data
+
+Make sure you have a *.env* file with all your variables, then run:
+
+```sh
+docker run --rm --env-file .env chatbot-rag-app flask create-index
+```
+
+See "Ingest data" section under Running Locally for more details about the `flask create-index` command.
+
+#### Run API and frontend
 
-If you're using one of the other LLMs, you will need to set the appropriate environment variables via `-e` flag.
+You will need to set the appropriate environment variables in your *.env* file. See the *env.example* file for instructions.
 
 ```sh
-docker run -p 4000:4000 \
-  -e "ELASTIC_CLOUD_ID=<cloud_id>" \
-  -e "ELASTIC_USERNAME=elastic" \
-  -e "ELASTIC_PASSWORD=<password>" \
-  -e "LLM_TYPE=openai" \
-  -e "OPENAI_API_KEY=<openai_key>" \
-  -d chatbot-rag-app
+docker run --rm -p 4000:4000 --env-file .env -d chatbot-rag-app
 ```
 
 ### Locally (for development)
@@ -171,21 +159,37 @@ python -m venv .venv
 
 # Activate the virtual environment
 source .venv/bin/activate
-```
 
-```sh
 # Install Python dependencies
-pip install -r requirements.txt
+pip install -r api/requirements.txt
 
 # Install Node dependencies
-cd frontend && yarn
+cd frontend && yarn && cd ..
+```
+
+#### Ingest data
+
+You can index the sample data from the provided .json files in the `data` folder:
+
+```sh
+flask create-index
 ```
 
+By default, this will index the data into the `workplace-app-docs` index. You can change this by setting the `ES_INDEX` environment variable.
+
+##### Indexing your own data
+
+The ingesting logic is stored in `data/index-data.py`. This is a simple script that uses Langchain to index data into Elasticsearch, using the `JSONLoader` and `CharacterTextSplitter` to split the large documents into passages. Modify this script to index your own data.
+
+Langchain offers many different ways to index data, if you cant just load it via JSONLoader. See the [Langchain documentation](https://python.langchain.com/docs/modules/data_connection/document_loaders)
+
+Remember to keep the `ES_INDEX` environment variable set to the index you want to index into and to query from.
+
 #### Run API and frontend
 
 ```sh
 # Launch API app
-python api/app.py
+flask run
 
 # In a separate terminal launch frontend app
 cd frontend && yarn start

example-apps/chatbot-rag-app/api/app.py

@@ -4,6 +4,8 @@
 from uuid import uuid4
 from chat import chat, ask_question, parse_stream_message
 import threading
+import os
+import sys
 
 app = Flask(__name__, static_folder="../frontend/build", static_url_path="/")
 CORS(app)
@@ -35,5 +37,15 @@ def api_chat():
     )
 
 
+@app.cli.command()
+def create_index():
+    """Create or re-create the Elasticsearch index."""
+    basedir = os.path.abspath(os.path.dirname(__file__))
+    sys.path.append(f'{basedir}/../')
+
+    from data import index_data
+    index_data.main()
+
+
 if __name__ == "__main__":
     app.run(port=3001, debug=True)

example-apps/chatbot-rag-app/api/chat.py

@@ -20,6 +20,7 @@
 INDEX_CHAT_HISTORY = os.getenv(
     "ES_INDEX_CHAT_HISTORY", "workplace-app-docs-chat-history"
 )
+ELSER_MODEL = os.getenv("ELSER_MODEL", ".elser_model_2")
 POISON_MESSAGE = "~~~END~~~"
 SESSION_ID_TAG = "[SESSION_ID]"
 SOURCE_TAG = "[SOURCE]"
@@ -71,7 +72,7 @@ def on_llm_end(self, response, *, run_id, parent_run_id=None, **kwargs):
 store = ElasticsearchStore(
     es_connection=elasticsearch_client,
     index_name=INDEX,
-    strategy=ElasticsearchStore.SparseVectorRetrievalStrategy(),
+    strategy=ElasticsearchStore.SparseVectorRetrievalStrategy(model_id=ELSER_MODEL),
 )
 
 general_system_template = """

example-apps/chatbot-rag-app/data/index-data.py renamed to example-apps/chatbot-rag-app/data/index_data.py

@@ -1,9 +1,10 @@
-from elasticsearch import Elasticsearch
+from elasticsearch import Elasticsearch, NotFoundError
 from langchain.vectorstores import ElasticsearchStore
 from langchain.document_loaders import JSONLoader
 from langchain.text_splitter import CharacterTextSplitter
 from dotenv import load_dotenv
 import os
+import time
 
 load_dotenv()
 
@@ -14,12 +15,34 @@
 ELASTIC_CLOUD_ID = os.getenv("ELASTIC_CLOUD_ID")
 ELASTIC_USERNAME = os.getenv("ELASTIC_USERNAME")
 ELASTIC_PASSWORD = os.getenv("ELASTIC_PASSWORD")
+ELSER_MODEL = os.getenv("ELSER_MODEL", ".elser_model_2")
 
 elasticsearch_client = Elasticsearch(
     cloud_id=ELASTIC_CLOUD_ID, basic_auth=(ELASTIC_USERNAME, ELASTIC_PASSWORD)
 )
 
 
+def install_elser():
+    try:
+        elasticsearch_client.ml.get_trained_models(model_id=ELSER_MODEL)
+        print(f"\"{ELSER_MODEL}\" model is available")
+    except NotFoundError:
+        print(f"\"{ELSER_MODEL}\" model not available, downloading it now")
+        elasticsearch_client.ml.put_trained_model(model_id=ELSER_MODEL,
+                                                  input={"field_names": ["text_field"]})
+        while True:
+            status = elasticsearch_client.ml.get_trained_models(model_id=ELSER_MODEL,
+                                                                include="definition_status")
+            if status["trained_model_configs"][0]["fully_defined"]:
+                # model is ready
+                break
+            time.sleep(1)
+
+        print("Model downloaded, starting deployment")
+        elasticsearch_client.ml.start_trained_model_deployment(model_id=ELSER_MODEL,
+                                                               wait_for="fully_allocated")
+
+
 # Metadata extraction function
 def metadata_func(record: dict, metadata: dict) -> dict:
     metadata["name"] = record.get("name")
@@ -31,7 +54,9 @@ def metadata_func(record: dict, metadata: dict) -> dict:
     return metadata
 
 
-if __name__ == "__main__":
+def main():
+    install_elser()
+
     print(f"Loading data from ${FILE}")
 
     loader = JSONLoader(
@@ -54,9 +79,15 @@ def metadata_func(record: dict, metadata: dict) -> dict:
         f"Creating Elasticsearch sparse vector store in Elastic Cloud: {ELASTIC_CLOUD_ID}"
     )
 
+    elasticsearch_client.indices.delete(index=INDEX, ignore_unavailable=True)
+
     ElasticsearchStore.from_documents(
         docs,
         es_connection=elasticsearch_client,
         index_name=INDEX,
-        strategy=ElasticsearchStore.SparseVectorRetrievalStrategy(),
+        strategy=ElasticsearchStore.SparseVectorRetrievalStrategy(model_id=ELSER_MODEL),
     )
+
+
+if __name__ == "__main__":
+    main()

example-apps/chatbot-rag-app/env.example

@@ -0,0 +1,34 @@
+# Make a copy of this file with the name .env and assign values to variables
+
+# Your Elastic Cloud credentials
+ELASTIC_USERNAME=elastic
+ELASTIC_CLOUD_ID=
+ELASTIC_PASSWORD=
+
+# The name of the Elasticsearch indexes
+ES_INDEX=workplace-app-docs
+ES_INDEX_CHAT_HISTORY=workplace-app-docs-chat-history
+
+# Uncomment and complete if you want to use OpenAI
+# LLM_TYPE=openai
+# OPENAI_API_KEY=
+
+# Uncomment and complete if you want to use Azure OpenAI
+# LLM_TYPE=azure
+# OPENAI_VERSION=
+# OPENAI_BASE_URL=
+# OPENAI_API_KEY=
+# OPENAI_ENGINE=
+
+# Uncomment and complete if you want to use Bedrock LLM
+# LLM_TYPE=bedrock
+# AWS_ACCESS_KEY=
+# AWS_SECRET_KEY=
+# AWS_REGION=
+# AWS_MODEL_ID=
+
+# Uncomment and complete if you want to use Vertex AI
+# LLM_TYPE=vertex
+# VERTEX_PROJECT_ID=
+# VERTEX_REGION=
+# GOOGLE_APPLICATION_CREDENTIALS=

example-apps/chatbot-rag-app/requirements.txt

@@ -18,7 +18,19 @@ exceptiongroup==1.1.3
 Flask==2.3.3
 Flask-Cors==4.0.0
 frozenlist==1.4.0
+google-api-core==2.14.0
 google-auth==2.23.2
+google-cloud-aiplatform==1.35.0
+google-cloud-bigquery==3.13.0
+google-cloud-core==2.3.3
+google-cloud-resource-manager==1.10.4
+google-cloud-storage==2.11.0
+google-crc32c==1.5.0
+google-resumable-media==2.6.0
+googleapis-common-protos==1.61.0
+grpc-google-iam-v1==0.12.7
+grpcio==1.59.3
+grpcio-status==1.59.3
 idna==3.4
 importlib-metadata==6.8.0
 itsdangerous==2.1.2
@@ -28,6 +40,7 @@ jq==1.4.1
 jsonpatch==1.33
 jsonpointer==2.4
 langchain==0.0.315
+langsmith==0.0.65
 MarkupSafe==2.1.3
 marshmallow==3.20.1
 multidict==6.0.4
@@ -36,15 +49,19 @@ numexpr==2.8.5
 numpy==1.25.2
 openai==0.27.9
 packaging==23.1
+proto-plus==1.22.3
+protobuf==4.25.1
 pyasn1==0.5.0
 pyasn1-modules==0.3.0
 pydantic==2.3.0
 pydantic_core==2.6.3
 python-dateutil==2.8.2
+python-dotenv==1.0.0
 PyYAML==6.0.1
 requests==2.31.0
 rsa==4.9
 s3transfer==0.7.0
+shapely==2.0.2
 six==1.16.0
 sniffio==1.3.0
 SQLAlchemy==2.0.20
@@ -56,4 +73,3 @@ urllib3==1.26.16
 Werkzeug==2.3.7
 yarl==1.9.2
 zipp==3.17.0
-google-cloud-aiplatform==1.35.0