Merge pull request #33 from leemthompo/notebooks-readme-updates

Add readmes to notebook folders

Author: leemthompo

notebooks/generative-ai/README.md

@@ -0,0 +1,26 @@
+# Generative AI notebooks
+
+This folder contains notebooks that demonstrate various use cases for Elasticsearch as the retrieval engine and vector store for LLM-powered applications.
+
+The following notebooks are available:
+
+- [Question answering](#question-answering)
+<!-- - [Chatbot](#chatbot) -->
+
+## Notebooks
+
+### Question answering
+
+In the [`question-answering.ipynb`](./question-answering.ipynb) notebook you'll learn how to:
+
+- Retrieve sample workplace documents from a given URL.
+- Set up an Elasticsearch client.
+- Chunk documents into 800-token passages with an overlap of 400 tokens using the `CharacterTextSplitter` from `langchain`.
+- Use `OpenAIEmbeddings` from `langchain` to create embeddings for the content.
+- Retrieve embeddings for the chunked passages using OpenAI.
+- Persist the passage documents along with their embeddings into Elasticsearch.
+- Set up a question-answering system using `OpenAI` and `ElasticKnnSearch` from `langchain` to retrieve answers along with their source documents.
+
+<!-- ### Chatbot
+
+In the [`chatbot.ipynb`](./chatbot.ipynb) notebook you'll learn how to: -->

notebooks/generative-ai/question-answering.ipynb

@@ -42,7 +42,7 @@
       },
       "outputs": [],
       "source": [
-        "!pip install -qU langchain jq openai elasticsearch tiktoken"
+        "!pip install -qU langchain jq openai elasticsearch"
       ]
     },
     {

notebooks/integrations/README.md

@@ -0,0 +1,7 @@
+# Integrations notebooks
+
+The following subfolders contain notebooks that demonstrate how to integrate popular services and projects with Elasticsearch:
+
+  - [OpenAI](./openai/README.md)
+  - [Hugging Face](./hugging-face/README.md)
+  - [LlamaIndex](./llama-index/README.md)

notebooks/integrations/hugging-face/README.md

@@ -0,0 +1,18 @@
+# Hugging Face integrations notebooks
+
+This folder contains notebooks that demonstrate how to work with Hugging Face hosted models in Elasticsearch.
+
+The following notebooks are available:
+
+- [Loading an NLP model from Hugging Face](#loading-an-nlp-model-from-hugging-face)
+
+## Notebooks
+
+### Loading an NLP model from Hugging Face
+
+In the [`loading-model-from-hugging-face.ipynb`](./loading-model-from-hugging-face.ipynb) notebook you'll learn how to:
+
+- Use the `eland` tool to import the [`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model from Hugging Face.
+- Index documents into Elasticsearch and generate text embeddings using an ingest pipeline.
+- Transform search queries into vectors using the Hugging Face model and perform vector searches over the dataset.
+

notebooks/integrations/hugging-face/loading-model-from-hugging-face.ipynb

@@ -0,0 +1,17 @@
+# LlamaIndex notebooks
+
+This folder contains notebooks that demonstrate how to work with the LlamaIndex data framework using Elasticsearch.
+
+<!-- The following notebooks are available:
+The following notebooks are available:
+
+0. [Quick start](#0-quick-start)
+1. [Keyword, querying, filtering](#1-keyword-querying-filtering)
+2. [Hybrid search](#2-hybrid-search)
+3. [Semantic search with ELSER](#3-semantic-search-with-elser)
+4. [Multilingual semantic search](#4-multilingual-semantic-search)
+
+## Notebooks
+
+### <NOTEBOOK_NAME>
+--->

notebooks/integrations/llama-index/README.md

@@ -0,0 +1,21 @@
+# OpenAI integrations notebooks
+
+This folder contains notebooks that demonstrate how to integrate popular OpenAI services with Elasticsearch.
+
+The following notebooks are available:
+
+- [OpenAI embeddings and retrieval augmented generation (RAG)](#openai-embeddings-and-retrieval-augmented-generation-rag)
+
+## Notebooks
+
+### OpenAI embeddings and retrieval augmented generation (RAG)
+
+In the [`openai-KNN-RAG.ipynb`](./openai-KNN-RAG.ipynb) notebook you'll learn how to:
+
+- Index the OpenAI Wikipedia vector dataset into Elasticsearch.
+- Embed a question with the OpenAI embeddings endpoint.
+- Perform semantic search on the Elasticsearch index using the encoded question.
+- Send the top search results to the OpenAI Chat Completions API endpoint for retrieval augmented generation (RAG).
+- Use OpenAI's model to generate a response for a given conversation.
+- Set a system message which defines the assistant's role and how user messages are processed.
+

notebooks/integrations/openai/README.md

@@ -0,0 +1,26 @@
+# LangChain notebooks
+
+This folder contains notebooks that demonstrate how to use Elasticsearch with the LangChain framework for building applications powered by language models.
+
+<!--
+The following notebooks are available:
+
+- [Embeddings](#embeddings)
+- [Vector store](#vector-store)
+
+## Notebooks
+
+### Embeddings
+
+In the [`langchain-elasticsearch-embeddings.ipynb`](./langchain-elasticsearch-embeddings.ipynb) notebook you'll learn how to:
+
+-
+-
+
+### Vector store
+
+In the [`langchain-vector-store.ipynb`](./langchain-vector-store.ipynb) notebook you'll learn how to:
+
+-
+-
+-->