diff --git a/README.md b/README.md index 127cd7d..af79b40 100644 --- a/README.md +++ b/README.md @@ -14,12 +14,12 @@ Check out [demo here](https://youtu.be/XXXXXXX) The demo shows a Retrieval-Augmented Generation using the following modules: - * Spring AI API - * Oracle DB 23ai - * OpenAI Embeddings - * OpenAI Chat - * OLLAMA local LLM embeddings model - * OLLAMA local LLM LLama2 model for chat +* Spring AI API +* Oracle DB 23ai +* OpenAI Embeddings +* OpenAI Chat +* OLLAMA local LLM embeddings model +* OLLAMA local LLM LLama2 model for chat This demo is based on a early draft example of **Spring AI API**'s implementation for the **Oracle 23ai** as vector store, according to the specifications reported here: **[Vector DBs](https://docs.spring.io/spring-ai/reference/api/vectordbs.html)**. @@ -30,8 +30,8 @@ There are two different types of files that contribute to the Retrieval-Augmente The interface, that uses Oracle Database 23ai as a Vector Store in a Spring AI pipeline, is the following: - ```java - public interface VectorStore { +``` +public interface VectorStore { void add(List documents); @@ -40,41 +40,41 @@ The interface, that uses Oracle Database 23ai as a Vector Store in a Spring AI p List similaritySearch(SearchRequest request); List similaritySearch(String query); - } - ``` +} +``` These operations allow uploading documents into a vector database, searching for similar documents using the specific vector distance algorithm chosen (you can change this in the `.properties` files). - ```java - default List similaritySearch(String query) { - return this.similaritySearch(SearchRequest.query(query)); - } - ``` +``` +default List similaritySearch(String query) { + return this.similaritySearch(SearchRequest.query(query)); +} +``` The file `src/main/java/com/example/demoai/OracleDBVectorStore.java` holds this implementation. The Vector Store saves the data in this **VECTORTABLE**: - ```sql - CREATE TABLE VECTORTAB ( - id NUMBER GENERATED AS IDENTITY, - text CLOB, - embeddings VECTOR, - metadata JSON, - PRIMARY KEY (id) - ); - ``` +``` +CREATE TABLE VECTORTAB ( + id NUMBER GENERATED AS IDENTITY, + text CLOB, + embeddings VECTOR, + metadata JSON, + PRIMARY KEY (id) +); +``` The **id** will be based on an generated **Identity** Column key, but this can be changed if you prefer. The metadata content depends on what's coming from Document object, and in this case it will hold the following data: - ```json - { - "page_number":"xxx", - "file_name":"xxx", - } - ``` +``` +{ + "page_number":"xxx", + "file_name":"xxx", +} +``` This table is created at each application startup by default but, by configuring the `config.dropDb` parameter to `false` in `application-dev.properties`, you can accumulate data every time you start up the application startup, in the same vector tab, and these documents will increase the vector database's knowledge base. @@ -115,30 +115,41 @@ The following tests have also been implemented, to debug and play with the solut ### JDBC driver for Oracle DB 23ai + + This demo works with the latest `ojdbc11.jar` driver related to the Oracle DBMS (23.4). To run this project, download this driver from Oracle site or directly from your DB server, looking in the directory: `$ORACLE_HOME/jdbc/lib/ojdbc11.jar`. After downloading in your local home dir, import it as a local Maven artifact with this command: - ```bash - mvn install:install-file -Dfile=/ojdbc11.jar -DgroupId=com.oracle.database.jdbc -DartifactId=ojdbc11 -Dversion=23.4.0.0 -Dpackaging=jar -DgeneratePom=true - ``` +``` +mvn install:install-file -Dfile=/ojdbc11.jar -DgroupId=com.oracle.database.jdbc -DartifactId=ojdbc11 -Dversion=23.4.0.0 -Dpackaging=jar -DgeneratePom=true +``` +or including in the `pom.xml` the following dependency: + +```xml + + com.oracle.database.jdbc + ojdbc11 + 23.4.0.24.05 + +``` ### Environment variables Set the correct environment variables in a `env.sh` (or put these directly into `/home/$USER/.bashrc`) file with this content, according your server IPs (if you're planning on deploying with oLLaMA): - ```bash - export OPENAI_URL=https://api.openai.com - export OPENAI_MODEL=gpt-3.5-turbo - export OPENAI_EMBEDDING_MODEL=text-embedding-ada-002 - export VECTORDB=[VECTORDB_IP] - export DB_USER=vector - export DB_PASSWORD=vector - export OLLAMA_URL=http://[GPU_SERVER_IP]:11434 - export OLLAMA_EMBEDDINGS=NousResearch--llama-2-7b-chat-hf - export OLLAMA_MODEL=llama2:7b-chat-fp16 - export OPENAI_API_KEY=[YOUR_OPENAI_KEY] - #export OPENAI_URL=http://[GPU_SERVER_IP]:3000 - #export OPENAI_MODEL=NousResearch--llama-2-7b-chat-hf - ``` +``` +export OPENAI_URL=https://api.openai.com +export OPENAI_MODEL=gpt-3.5-turbo +export OPENAI_EMBEDDING_MODEL=text-embedding-ada-002 +export VECTORDB=[VECTORDB_IP] +export DB_USER=vector +export DB_PASSWORD=vector +export OLLAMA_URL=http://[GPU_SERVER_IP]:11434 +export OLLAMA_EMBEDDINGS=NousResearch--llama-2-7b-chat-hf +export OLLAMA_MODEL=llama2:7b-chat-fp16 +export OPENAI_API_KEY=[YOUR_OPENAI_KEY] +#export OPENAI_URL=http://[GPU_SERVER_IP]:3000 +#export OPENAI_MODEL=NousResearch--llama-2-7b-chat-hf +``` To invoke both OpenAI `gpt-3.5-turbo` and `text-embedding-ada-002`, you'll also need your `YOUR_OPENAI_KEY`, which must be obtained directly from the [Open AI developer platform](https://platform.openai.com/). @@ -148,9 +159,9 @@ As you can see, you can configure also the `OPENAI_URL`, which helps to invoke O Set env with command in a shell: - ```bash - source ./env.sh - ``` +``` +source ./env.sh +``` ## 1. Setup @@ -158,26 +169,26 @@ Set env with command in a shell: 1. Download and install from [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/) site an **Oracle Database 23ai Free**, for example, as a docker container in this way: - ```bash - docker run -d -p 1521:1521 --name db23ai container-registry.oracle.com/database/free:latest - docker exec db23ai ./setPassword.sh manager - ``` +``` +docker run -d -p 1521:1521 --name db23ai container-registry.oracle.com/database/free:latest +docker exec db23ai ./setPassword.sh manager +``` 2. After startup, download and install an Oracle Instant Client from the same [site](https://www.oracle.com/database/free/get-started/), and connect to the instance as shown here: - ```bash - sqlplus sys/manager@"${VECTORDB}:1521/FREEPDB1" as sysdba - ``` +``` +sqlplus sys/manager@"${VECTORDB}:1521/FREEPDB1" as sysdba +``` 3. If running locally: - ```bash - sqlplus sys/manager@"localhost:1521/FREEPDB1" as sysdba - ``` +``` +sqlplus sys/manager@"localhost:1521/FREEPDB1" as sysdba +``` to create a **vector** user to run the example: -```bash +``` create user vector identified by "vector"; grant connect to vector; grant resource to vector; @@ -189,46 +200,46 @@ Once we've created the user, we'll be able to use it in our Spring AI applicatio If running locally: -```bash +``` sqlplus vector/vector@"localhost:1521/FREEPDB1" as sysdba ``` We can check the content by connecting to the Oracle DB: - ```bash - sqlplus vector/vector@"${VECTORDB}:1521/ORCLPDB1" - ``` +``` +sqlplus vector/vector@"${VECTORDB}:1521/FREEPDB1" +``` ### Application In the `application-dev.properties` files will be used the environment variables set at the step before: - ```properties - spring.ai.openai.api-key=${OPENAI_API_KEY} - spring.ai.openai.base-url=${OPENAI_URL} - spring.ai.openai.chat.options.model=${OPENAI_MODEL} - spring.ai.openai.embedding.options.model=${OPENAI_EMBEDDING_MODEL} - spring.ai.openai.chat.options.temperature=0.3 - spring.datasource.url=jdbc:oracle:thin:@${VECTORDB}:1521/ORCLPDB1 - spring.datasource.username=${DB_USER} - spring.datasource.password=${DB_PASSWORD} - spring.datasource.driver-class-name=oracle.jdbc.OracleDriver - config.tempDir=tempDir - config.dropDb=true - config.vectorDB=vectortable - config.distance=EUCLIDEAN - spring.servlet.multipart.max-file-size=10MB - spring.servlet.multipart.max-request-size=20MB - spring.ai.ollama.base-url=${OLLAMA_URL} - spring.ai.ollama.embedding.options.model=${OLLAMA_EMBEDDINGS} - spring.ai.ollama.chat.options.model=${OLLAMA_MODEL} - ``` +``` +spring.ai.openai.api-key=${OPENAI_API_KEY} +spring.ai.openai.base-url=${OPENAI_URL} +spring.ai.openai.chat.options.model=${OPENAI_MODEL} +spring.ai.openai.embedding.options.model=${OPENAI_EMBEDDING_MODEL} +spring.ai.openai.chat.options.temperature=0.3 +spring.datasource.url=jdbc:oracle:thin:@${VECTORDB}:1521/ORCLPDB1 +spring.datasource.username=${DB_USER} +spring.datasource.password=${DB_PASSWORD} +spring.datasource.driver-class-name=oracle.jdbc.OracleDriver +config.tempDir=tempDir +config.dropDb=true +config.vectorDB=vectortable +config.distance=EUCLIDEAN +spring.servlet.multipart.max-file-size=10MB +spring.servlet.multipart.max-request-size=20MB +spring.ai.ollama.base-url=${OLLAMA_URL} +spring.ai.ollama.embedding.options.model=${OLLAMA_EMBEDDINGS} +spring.ai.ollama.chat.options.model=${OLLAMA_MODEL} +``` In `application.properties`, check if the default env is set as `dev`: - ```properties - spring.profiles.active=dev - ``` +``` +spring.profiles.active=dev +``` Then build and run the application: @@ -244,29 +255,29 @@ Check code: pom.xml: - ```xml - - - ``` +``` + + +``` DemoaiController.java: - ```java +``` //CHANGE //import org.springframework.ai.ollama.OllamaEmbeddingClient; //import org.springframework.ai.ollama.OllamaChatClient; ... //CHANGE - private final EmbeddingClient embeddingClient; - //private final OllamaEmbeddingClient embeddingClient; + private final EmbeddingClient embeddingClient; + //private final OllamaEmbeddingClient embeddingClient; - //CHANGE + //CHANGE private final ChatClient chatClient; //private final OllamaChatClient chatClient; @@ -278,11 +289,11 @@ DemoaiController.java: //public DemoaiController(OllamaEmbeddingClient embeddingClient, @Qualifier("openAiChatClient") ChatClient chatClient, VectorService vectorService) { // Ollama Embeddings - OpenAI Completion //public DemoaiController(OllamaEmbeddingClient embeddingClient, OllamaChatClient chatClient, VectorService vectorService) { // Ollama full - ``` +``` VectorService.java: - ```java +``` //CHANGE //import org.springframework.ai.ollama.OllamaChatClient; ... @@ -293,11 +304,11 @@ VectorService.java: //CHANGE VectorService(@Qualifier("openAiChatClient") ChatClient aiClient) { //VectorService(OllamaChatClient aiClient) { - ``` +``` DemoaiApplication.java: - ```java +``` //CHANGE //import org.springframework.ai.ollama.OllamaEmbeddingClient; @@ -309,49 +320,49 @@ DemoaiApplication.java: return new OracleDBVectorStore(t, ec); } - ``` +``` ### Pre document store #### Generic chat - ```bash - curl -X POST http://localhost:8080/ai/generate \ - -H "Content-Type: application/json" \ - -d '{"message":"What is a Generative AI?"}' | jq -r .generation - ``` +```bash +curl -X POST http://localhost:8080/ai/generate \ + -H "Content-Type: application/json" \ + -d '{"message":"What is a Generative AI?"}' | jq -r .generation +``` Here's a sample output from the command: - ```text +``` Generative AI refers to artificial intelligence systems that are capable of creating new content, such as images, text, or music, based on patterns and examples provided to them. These systems use algorithms and machine learning techniques to generate realistic and original content that mimics human creativity. Generative AI can be used in a variety of applications, such as creating art, writing stories, or designing products. - ``` +``` #### RAG request without any data stored in the DB - ```bash - curl -X POST http://localhost:8080/ai/rag \ +``` +curl -X POST http://localhost:8080/ai/rag \ -H "Content-Type: application/json" \ -d '{"message":"Can I use any kind of development environment to run the example?"}' - ``` +``` Output from the command: - ```json +``` { "generation" : "Based on the provided documents, it is not specified whether any kind of development environment can be used to run the example. Therefore, I'm sorry but I haven't enough information to answer." } - ``` +``` ### Search on data coming from a PDF stored Store a PDF document in the DBMC 23c library: [**Oracle® Database: Get Started with Java Development**](https://docs.oracle.com/en/database/oracle/oracle-database/23/tdpjd/get-started-java-development.pdf) in the Oracle DB 23ai with embeddings coming from the OpenAI Embedding service. Dowload locally, and run in a shell: - ```bash - curl -X POST -F "file=@./docs/get-started-java-development.pdf" http://localhost:8080/ai/store - ``` +``` +curl -X POST -F "file=@./docs/get-started-java-development.pdf" http://localhost:8080/ai/store +``` ->> **Note**: this process usually takes time because document will be splitted in hundreds or thousands of chunks, and for each one it will asked for an embeddings vector to OpenAI API service. In this case has been choosen a small document to wait a few seconds. +**Note**: this process usually takes time because document will be splitted in hundreds or thousands of chunks, and for each one it will asked for an embeddings vector to OpenAI API service. In this case has been choosen a small document to wait a few seconds. #### Q&A Sample @@ -361,83 +372,83 @@ Let's look at some info in this document and try to query comparing the results ![dbtype](./img/dbtype.png) - ```bash - curl -X POST http://localhost:8080/ai/rag \ +``` +curl -X POST http://localhost:8080/ai/rag \ -H "Content-Type: application/json" \ -d '{"message":"Which kind of database you can use to run the Java Web example application) "}' | jq -r .generation - ``` +``` Response: - ```text +``` You can use either Oracle Autonomous Database or Oracle Database Free available on OTN to run the Java Web example application. - ``` +``` - **4.1.5 Integrated Development Environment** ![ide](./img/ide.png) - ```bash - curl -X POST http://localhost:8080/ai/rag \ +``` +curl -X POST http://localhost:8080/ai/rag \ -H "Content-Type: application/json" \ -d '{"message":"Can I use any kind of development environment to run the example?"}' | jq -r .generation - ``` +``` Response: - ```text +``` Based on the information provided in the documents, you can use an Integrated Development Environment (IDE) like IntelliJ Idea community version to develop the Java application that connects to the Oracle Database. The guide specifically mentions using IntelliJ Idea for creating and updating the files for the application. Therefore, it is recommended to use IntelliJ Idea as the development environment for running the example. - ``` +``` - **4.2 Verifying the Oracle Database Installation** ![dbverify](./img/dbverify.png) - ```bash - curl -X POST http://localhost:8080/ai/rag \ +``` +curl -X POST http://localhost:8080/ai/rag \ -H "Content-Type: application/json" \ -d '{"message":"To run the example, how can I check if the dbms it is working correctly?"}' | jq -r .generation - ``` +``` Response: - ```text +``` To check if the Oracle Database is working correctly, you can verify the installation by connecting to the database using the following commands: 1. Navigate to the Oracle Database bin directory: $ cd $ORACLE_HOME/bin 2. Connect to the database as sysdba: $ ./sqlplus / as sysdba If the connection is successful, you will see an output confirming that you are connected to the root container of the database. This indicates that the Oracle Database installation is working correctly. Additionally, you can download the Client Credentials for an ATP instance and verify the connection by following the steps provided in the documentation. - ``` +``` First, let's ask for a question not related to the document stored: - ```bash - curl -X POST http://localhost:8080/ai/rag \ +``` +curl -X POST http://localhost:8080/ai/rag \ -H "Content-Type: application/json" \ -d '{"message":"How is the weather tomorrow?"}' | jq -r .generation - ``` +``` Response: - ```json - { - "generation" : "I'm sorry but I haven't enough info to answer." - } - ``` +``` +{ + "generation" : "I'm sorry but I haven't enough info to answer." +} +``` Then, let's test similarity search for message **"To run the example, how can I check if the dbms it is working correctly?"** example. The `top_k` parameter determines how many nearest chunks to retrieve is set to **4** by default, and the result set is by default in reverse order. So, we need to execute the fololwing command: - ```bash - curl -X POST http://localhost:8080/ai/search-similar \ +``` +curl -X POST http://localhost:8080/ai/search-similar \ -H "Content-Type: application/json" \ -d '{"message":"To run the example, how can I check if the dbms it is working correctly?"}' | jq '.[3]' - ``` +``` Then, we test the deletion. Indexes begin counting at `1`, so let's execute the following command to delete occurrences 1, 4 and 5: - ```bash - curl "http://localhost:8080/ai/delete?id=1&id=5&id=4" - ``` +``` +curl "http://localhost:8080/ai/delete?id=1&id=5&id=4" +``` ## 2. Running generations and chat with private LLMs through OLLAMA @@ -477,13 +488,13 @@ The following shape and images are recommended for the server: (it will require 7. At the end of creation process, obtain the **Public IPv4 address**, and with your private key (the one you generated or uploaded during creation), connect to: - ```bash +``` ssh -i ./.key opc@[GPU_SERVER_IP] - ``` +``` 8. Install and configure docker to use GPUs: - ```bash +``` sudo /usr/libexec/oci-growfs curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo sudo dnf install -y dnf-utils zip unzip @@ -491,70 +502,70 @@ The following shape and images are recommended for the server: (it will require sudo dnf remove -y runc sudo dnf install -y docker-ce --nobest sudo useradd docker_user - ``` +``` 9. We need to make sure that your Operating System user has permissions to run Docker containers. To do this, we can run the following command: - ```bash - sudo visudo - ``` +``` +sudo visudo +``` - And add this line at the end: +And add this line at the end: - ```bash - docker_user ALL=(ALL) NOPASSWD: /usr/bin/docker - ``` +``` +docker_user ALL=(ALL) NOPASSWD: /usr/bin/docker +``` 10. For convenience, we need to switch to our new user. For this, run: - ```bash - sudo su - docker_user - ``` +``` +sudo su - docker_user +``` 11. Finally, let's add an alias to execute Docker with admin privileges every time we type `docker` in our shell. For this, we need to modify a file, depending on your OS (in `.bash_profile` (MacOS) / `.bashrc` (Linux)). Insert, at the end of the file, this command: - ```bash - alias docker="sudo /usr/bin/docker" - exit - ``` +``` +alias docker="sudo /usr/bin/docker" +exit +``` 12. We finalize our installation by executing: - ```bash - sudo yum install -y nvidia-container-toolkit - sudo nvidia-ctk runtime configure --runtime=docker - sudo systemctl restart docker - nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json - ``` +``` +sudo yum install -y nvidia-container-toolkit +sudo nvidia-ctk runtime configure --runtime=docker +sudo systemctl restart docker +nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json +``` 13. If you're on Ubuntu instead, run: - ```bash - sudo apt-get install nvidia-container-toolkit=1.14.3-1 \ +``` +sudo apt-get install nvidia-container-toolkit=1.14.3-1 \ nvidia-container-toolkit-base=1.14.3-1 \ libnvidia-container-tools=1.14.3-1 \ libnvidia-container1=1.14.3-1 - sudo apt-get install -y nvidia-docker2 - ``` +sudo apt-get install -y nvidia-docker2 +``` 13. Let's reboot and re-connect to the VM, and run again: - ```bash - sudo reboot now - # after restart, run: - sudo su - docker_user - ``` +``` +sudo reboot now +# after restart, run: +sudo su - docker_user +``` 14. Run `docker` to check if everything it's ok. 15. Let's run a Docker container with the `ollama/llama2` model for embeddings/completion: - ```bash - docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama serve - docker exec -it ollama ollama pull nomic-embed-text - docker exec -it ollama ollama pull llama2:13b-chat-fp16 - docker logs -f --tail 10 ollama - ``` +``` +docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama serve +docker exec -it ollama ollama pull nomic-embed-text +docker exec -it ollama ollama pull llama2:13b-chat-fp16 +docker logs -f --tail 10 ollama +``` Both the model, for embeddings/completion will run under the same server, and they will be addressed providing in the REST request for the specific model required. @@ -584,29 +595,29 @@ To handle the firewall, we need to open port `11434` on our Security List. For t 7. Update the `env.sh` file and run `source ./env.sh`: - ```bash - #export OPENAI_URL=http://[GPU_SERVER_IP]:3000 - export OPENAI_URL=https://api.openai.com - #export OPENAI_MODEL=NousResearch--llama-2-7b-chat-hf - export OPENAI_MODEL=gpt-3.5-turbo - export OPENAI_EMBEDDING_MODEL=text-embedding-ada-002 - export VECTORDB=[VECTORDB_IP] - export DB_USER=vector - export DB_PASSWORD=vector - export OLLAMA_URL=http://[GPU_SERVER_IP]:11434 - export OLLAMA_EMBEDDINGS=NousResearch--llama-2-7b-chat-hf - export OLLAMA_MODEL=llama2:7b-chat-fp16 - export OPENAI_API_KEY=[YOUR_OPENAI_KEY] - ``` +``` +#export OPENAI_URL=http://[GPU_SERVER_IP]:3000 +export OPENAI_URL=https://api.openai.com +#export OPENAI_MODEL=NousResearch--llama-2-7b-chat-hf +export OPENAI_MODEL=gpt-3.5-turbo +export OPENAI_EMBEDDING_MODEL=text-embedding-ada-002 +export VECTORDB=[VECTORDB_IP] +export DB_USER=vector +export DB_PASSWORD=vector +export OLLAMA_URL=http://[GPU_SERVER_IP]:11434 +export OLLAMA_EMBEDDINGS=NousResearch--llama-2-7b-chat-hf +export OLLAMA_MODEL=llama2:7b-chat-fp16 +export OPENAI_API_KEY=[YOUR_OPENAI_KEY] +``` 8. Test with a shell running: - ```xml - curl ${OLLAMA_URL}/api/generate -d '{ +``` +curl ${OLLAMA_URL}/api/generate -d '{ "model": "llama2:7b-chat-fp16", "prompt":"Why is the sky blue?" - }' - ``` +}' +``` You'll receive the response in continuous sequential responses, facilitating the delivery of the content little by little, instead of forcing users to wait for the whole response to be generated before it's desplayed to them. @@ -614,7 +625,7 @@ You'll receive the response in continuous sequential responses, facilitating the * pom.xml: uncomment the ollama dependency: -```xml +``` @@ -626,7 +637,7 @@ You'll receive the response in continuous sequential responses, facilitating the * DemoaiController.java - uncomment with final source code: -```java +``` //CHANGE import org.springframework.ai.ollama.OllamaEmbeddingClient; //import org.springframework.ai.ollama.OllamaChatClient; @@ -651,7 +662,7 @@ You'll receive the response in continuous sequential responses, facilitating the VectorService.java - check if it's like this: - ```java +``` //CHANGE //import org.springframework.ai.ollama.OllamaChatClient; @@ -665,7 +676,7 @@ VectorService.java - check if it's like this: VectorService(@Qualifier("openAiChatClient") ChatClient aiClient) { //VectorService(OllamaChatClient aiClient) { - ``` +``` Test as done before. In the gpu docker logs, you'll see the chunks coming to be embedded. @@ -673,7 +684,7 @@ Test as done before. In the gpu docker logs, you'll see the chunks coming to be DemoaiController.java - uncomment with final source code: - ```java +``` //CHANGE import org.springframework.ai.ollama.OllamaEmbeddingClient; import org.springframework.ai.ollama.OllamaChatClient; @@ -696,11 +707,11 @@ DemoaiController.java - uncomment with final source code: public DemoaiController(OllamaEmbeddingClient embeddingClient, OllamaChatClient chatClient, VectorService vectorService) { // Ollama full - ``` +``` VectorService.java - uncomment with final source code: - ```java +``` //CHANGE import org.springframework.ai.ollama.OllamaChatClient; @@ -714,21 +725,21 @@ VectorService.java - uncomment with final source code: //CHANGE //VectorService(@Qualifier("openAiChatClient") ChatClient aiClient) { VectorService(OllamaChatClient aiClient) { - ``` +``` Test as before. Now, you could see a tiny degradation of quality (since the model is quite small), mantaining the same embeddings and context retrieval. For example, with the question: - ```text +``` Q: Which kind of database you can use to run the Java Web example application: A: The document states that the application uses Oracle JDBC Thin driver, Universal Connection Pool (UCP), and Java in the Database (using embedded OJVM). Therefore, you can use an Oracle database to run the Java Web example application. - ```text +``` This first result doesn't return a very good, personalized result. However, if we good. But for: - ```text +``` Q: Can I use any kind of development environment to run the example? A: the user can use any development environment to run the example as long as it has a Java compiler and a database manager installed. The DOCUMENTS text does not specify any particular IDE that must be used, so any IDE with Java development capabilities should work. However, the guide does recommend using IntelliJ Idea Community Edition for ease of development, but this is not a requirement.\n\nTo answer the user's question, you could say: \"Yes, you can use any development environment to run the example as long as it has a Java compiler and a database manager installed. While the guide recommends using IntelliJ Idea Community Edition for ease of development, any IDE with Java development capabilities should work.\ - ``` +``` The result is more customized and acceptable. @@ -746,13 +757,13 @@ To setup this platform, follow the instruction included in **Lab1: Provision an 1. In the `application.properties` change the active env as `prod`: - ```properties - spring.profiles.active=prod - ``` +``` +spring.profiles.active=prod +``` 2. In the `application-prod.properties`, change the parameters in `< >` with the values set in `env.sh`: - ```properties +``` spring.ai.openai.api-key= spring.ai.openai.base-url= spring.ai.openai.chat.options.model=gpt-3.5-turbo @@ -770,39 +781,39 @@ To setup this platform, follow the instruction included in **Lab1: Provision an spring.ai.ollama.base-url= spring.ai.ollama.embedding.options.model=nomic-embed-text spring.ai.ollama.chat.options.model=llama2:7b-chat-fp16 - ``` +``` 3. Open a terminal, and using the **Kubernetes** admin command, open a port forward to the backend: - ```bash - kubectl -n obaas-admin port-forward svc/obaas-admin 8080:8080 - ``` +``` +kubectl -n obaas-admin port-forward svc/obaas-admin 8080:8080 +``` 4. Using the command-line tool `oractl`, deploy the application running the following commands: - ```bash - oractl:>connect - ? username obaas-admin - ? password ************** +``` +oractl:>connect +? username obaas-admin +? password ************** - oractl:>create --app-name rag - oractl:>deploy --app-name rag --service-name demoai --artifact-path /Users/cdebari/Documents/GitHub/spring-ai-demo/target/demoai-0.0.1-SNAPSHOT.jar --image-version 0.0.1 --service-profile prod +oractl:>create --app-name rag +oractl:>deploy --app-name rag --service-name demoai --artifact-path /Users/cdebari/Documents/GitHub/spring-ai-demo/target/demoai-0.0.1-SNAPSHOT.jar --image-version 0.0.1 --service-profile prod - ``` +``` 5. Let's test the application with port forwarding. First, we need to stop the current `demoai` instance running on the background, to free the previous port being used; and, in a different terminal, run a port forwarding on port 8080 to the remote service on the **Oracle Backend for Spring Boot and Microservices**: - ```bash - kubectl -n rag port-forward svc/demoai 8080:8080 - ``` +``` +kubectl -n rag port-forward svc/demoai 8080:8080 +``` 6. In a different terminal, test the service as done before, for example: - ```bash - curl -X POST http://localhost:8080/ai/rag \ +``` +curl -X POST http://localhost:8080/ai/rag \ -H "Content-Type: application/json" \ - -d '{"message":"How is computed the Interest? Give me as much details as you can"}' | jq -r .generation - ``` + -d '{"message":"Can I use any kind of development environment to run the example?"}' | jq -r .generation +``` ## Notes/Issues diff --git a/docs/bankofamerica.pdf b/docs/bankofamerica.pdf deleted file mode 100644 index 55896a3..0000000 Binary files a/docs/bankofamerica.pdf and /dev/null differ diff --git a/docs/citigroup.pdf b/docs/citigroup.pdf deleted file mode 100644 index 9e960d4..0000000 Binary files a/docs/citigroup.pdf and /dev/null differ diff --git a/target/classes/application-dev.properties b/target/classes/application-dev.properties new file mode 100644 index 0000000..8ce1f6a --- /dev/null +++ b/target/classes/application-dev.properties @@ -0,0 +1,20 @@ +spring.ai.openai.api-key=${OPENAI_API_KEY} +spring.ai.openai.base-url=${OPENAI_URL} +spring.ai.openai.chat.options.model=${OPENAI_MODEL} +spring.ai.openai.embedding.options.model=${OPENAI_EMBEDDING_MODEL} +spring.ai.openai.chat.options.temperature=0.5 +#spring.datasource.url=jdbc:oracle:thin:@${VECTORDB}:1521/ORCLPDB1 +spring.datasource.url=jdbc:oracle:thin:@${VECTORDB}:1521/FREEPDB1 +spring.datasource.username=${DB_USER} +spring.datasource.password=${DB_PASSWORD} +spring.datasource.driver-class-name=oracle.jdbc.OracleDriver +config.tempDir=tempDir +config.dropDb=true +config.vectorDB=vectortable +config.distance=EUCLIDEAN +#config.distance=COSINE +spring.servlet.multipart.max-file-size=10MB +spring.servlet.multipart.max-request-size=20MB +spring.ai.ollama.base-url=${OLLAMA_URL} +spring.ai.ollama.embedding.options.model=${OLLAMA_EMBEDDINGS} +spring.ai.ollama.chat.options.model=${OLLAMA_MODEL} \ No newline at end of file diff --git a/target/classes/application-prod.properties b/target/classes/application-prod.properties new file mode 100644 index 0000000..4ad2d6a --- /dev/null +++ b/target/classes/application-prod.properties @@ -0,0 +1,17 @@ +spring.ai.openai.api-key=${OPENAI_API_KEY} +spring.ai.openai.base-url=${OPENAI_URL} +spring.ai.openai.chat.options.model=${OPENAI_MODEL} +spring.ai.openai.embedding.options.model=${OPENAI_EMBEDDING_MODEL} +spring.datasource.url=jdbc:oracle:thin:@${VECTORDB}:1521/FREEPDB1 +spring.datasource.username=${DB_USER} +spring.datasource.password=${DB_PASSWORD} +spring.datasource.driver-class-name=oracle.jdbc.OracleDriver +config.tempDir=tempDir +config.dropDb=true +config.vectorDB=vectortable +config.distance=EUCLIDEAN +spring.servlet.multipart.max-file-size=10MB +spring.servlet.multipart.max-request-size=20MB +spring.ai.ollama.base-url=${OLLAMA_URL} +spring.ai.ollama.embedding.options.model=${OLLAMA_EMBEDDINGS} +spring.ai.ollama.chat.options.model=${OLLAMA_MODEL} \ No newline at end of file diff --git a/target/classes/application.properties b/target/classes/application.properties new file mode 100644 index 0000000..257b306 --- /dev/null +++ b/target/classes/application.properties @@ -0,0 +1 @@ +spring.profiles.active=dev \ No newline at end of file diff --git a/target/classes/json_dual.json b/target/classes/json_dual.json new file mode 100644 index 0000000..9c2af6a --- /dev/null +++ b/target/classes/json_dual.json @@ -0,0 +1,2 @@ +[{"_id":10,"departmentName":"ACCOUNTING","location":"NEW YORK","employees":[{"employeeNumber":7782,"employeeName":"CLARK","job":"MANAGER","salary":2450},{"employeeNumber":7839,"employeeName":"KING","job":"PRESIDENT","salary":5000},{"employeeNumber":7934,"employeeName":"MILLER","job":"CLERK","salary":1300}],"_metadata":{"etag":"E546E2220E8F9620E36C2A7F8858D6F7","asof":"00000000004822A1"}}, +{"_id":20,"departmentName":"RESEARCH","location":"DALLAS","employees":[{"employeeNumber":7369,"employeeName":"SMITH","job":"CLERK","salary":800},{"employeeNumber":7566,"employeeName":"JONES","job":"MANAGER","salary":2975},{"employeeNumber":7902,"employeeName":"FORD","job":"ANALYST","salary":3000}],"_metadata":{"etag":"3709C26AFF2507248097F11BF906C723","asof":"00000000004822A1"}}] \ No newline at end of file diff --git a/target/classes/prompt-template.txt b/target/classes/prompt-template.txt new file mode 100644 index 0000000..295f11f --- /dev/null +++ b/target/classes/prompt-template.txt @@ -0,0 +1,11 @@ +DOCUMENTS: +{documents} + +QUESTION: +{question} + +INSTRUCTIONS: +Answer the users question using the DOCUMENTS text above. +Keep your answer ground in the facts of the DOCUMENTS. +If the DOCUMENTS doesn’t contain the facts to answer the QUESTION, return: +I'm sorry but I haven't enough information to answer. diff --git a/tempDir/uploads_10524787376583635372/get-started-java-development.pdf b/tempDir/uploads_10524787376583635372/get-started-java-development.pdf deleted file mode 100644 index e4613e0..0000000 Binary files a/tempDir/uploads_10524787376583635372/get-started-java-development.pdf and /dev/null differ diff --git a/tempDir/uploads_16194215709716612925/get-started-java-development.pdf b/tempDir/uploads_16194215709716612925/get-started-java-development.pdf deleted file mode 100644 index e4613e0..0000000 Binary files a/tempDir/uploads_16194215709716612925/get-started-java-development.pdf and /dev/null differ