Moved mlx into sub-directory, added Markdown export

.githooks/pre-commit

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+cd $(git rev-parse --show-toplevel)
+pwd
+
+# For all commits of mlx files, create corresponding Markdown (md) files.
+# If the mlx files are in .../mlx-scripts/*.mlx, the corresponding
+# md files will go into .../*.md.
+#
+# This script assumes that the mlx files as currently in the file system
+# are what is being committed, instead of doing a lot of extra work to
+# get them from the stage area.
+#
+# Note that this script will not remove media files. If an mlx has
+# fewer plots at some point in the future, there will be file system
+# cruft. Which doesn't hurt the md display in GitHub or elswehere.
+changedMlxFiles=`git diff --cached --name-only --diff-filter=d '*.mlx'`
+
+if [ -n "$changedMlxFiles" ]; then
+    # Keep the line break here, we replace end-of-line with "' '" to get the quotes right
+    matlab -batch "for file = {'${changedMlxFiles//
+/' '}'}, export(file{1},replace(erase(file{1},'mlx-scripts'),'.mlx','.md')); end"
+    tmp=${changedMlxFiles//mlx-scripts\//}
+    mdFiles=${tmp//.mlx/.md}
+    for file in $mdFiles; do
+        if [ -d ${file%.md}_media ]; then
+            git add ${file%.md}_media/
+        fi
+        perl -pi -e "\$cnt++ if /^#/; " \
+            -e "\$_ .= \"\nTo run the code shown on this page, open the MLX file in MATLAB: [mlx-scripts/$(basename $file .md).mlx](mlx-scripts/$(basename $file .md).mlx) \n\" if /^#/ && \$cnt==1;" \
+            $file
+    done
+    git add $mdFiles
+fi

DEVELOPMENT.md

@@ -0,0 +1,11 @@
+# Notes for Developers
+
+Nothing in this file should be required knowledge to use the repository. These are notes for people actually making changes that are going to be submitted and incorporated into the main branch.
+
+## Git Hooks
+
+After checkout, link or (on Windows) copy the files from `.githooks` into the local `.git/hooks` folder:
+
+```
+(cd .git/hooks/; ln -s ../../.githooks/pre-commit .)
+```

README.md

@@ -52,16 +52,16 @@ To use this repository with a local installation of MATLAB, first clone the repo
 ## Examples
 To learn how to use this in your workflows, see [Examples](/examples/).
 
-- [ProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode.mlx](/examples/ProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode.mlx): Learn to implement a simple chat that stream the response. 
-- [SummarizeLargeDocumentsUsingChatGPTandMATLAB.mlx](/examples/SummarizeLargeDocumentsUsingChatGPTandMATLAB.mlx): Learn to create concise summaries of long texts with ChatGPT. (Requires Text Analytics Toolbox™)
-- [CreateSimpleChatBot.mlx](/examples/CreateSimpleChatBot.mlx): Build a conversational chatbot capable of handling various dialogue scenarios using ChatGPT. (Requires Text Analytics Toolbox)
-- [AnalyzeScientificPapersUsingFunctionCalls.mlx](/examples/AnalyzeScientificPapersUsingFunctionCalls.mlx): Learn how to create agents capable of executing MATLAB functions. 
-- [AnalyzeTextDataUsingParallelFunctionCallwithChatGPT.mlx](/examples/AnalyzeTextDataUsingParallelFunctionCallwithChatGPT.mlx): Learn how to take advantage of parallel function calling. 
-- [RetrievalAugmentedGenerationUsingChatGPTandMATLAB.mlx](/examples/RetrievalAugmentedGenerationUsingChatGPTandMATLAB.mlx): Learn about retrieval augmented generation with a simple use case. (Requires Text Analytics Toolbox™)
-- [DescribeImagesUsingChatGPT.mlx](/examples/DescribeImagesUsingChatGPT.mlx): Learn how to use GPT-4 Turbo with Vision to understand the content of an image. 
-- [AnalyzeSentimentinTextUsingChatGPTinJSONMode.mlx](/examples/AnalyzeSentimentinTextUsingChatGPTinJSONMode.mlx): Learn how to use JSON mode in chat completions
-- [UsingDALLEToEditImages.mlx](/examples/UsingDALLEToEditImages.mlx): Learn how to generate images
-- [UsingDALLEToGenerateImages.mlx](/examples/UsingDALLEToGenerateImages.mlx): Create variations of images and editimages. 
+- [ProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode.md](/examples/ProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode.md): Learn to implement a simple chat that stream the response. 
+- [SummarizeLargeDocumentsUsingChatGPTandMATLAB.md](/examples/SummarizeLargeDocumentsUsingChatGPTandMATLAB.md): Learn to create concise summaries of long texts with ChatGPT. (Requires Text Analytics Toolbox™)
+- [CreateSimpleChatBot.md](/examples/CreateSimpleChatBot.md): Build a conversational chatbot capable of handling various dialogue scenarios using ChatGPT. (Requires Text Analytics Toolbox)
+- [AnalyzeScientificPapersUsingFunctionCalls.md](/examples/AnalyzeScientificPapersUsingFunctionCalls.md): Learn how to create agents capable of executing MATLAB functions. 
+- [AnalyzeTextDataUsingParallelFunctionCallwithChatGPT.md](/examples/AnalyzeTextDataUsingParallelFunctionCallwithChatGPT.md): Learn how to take advantage of parallel function calling. 
+- [RetrievalAugmentedGenerationUsingChatGPTandMATLAB.md](/examples/RetrievalAugmentedGenerationUsingChatGPTandMATLAB.md): Learn about retrieval augmented generation with a simple use case. (Requires Text Analytics Toolbox™)
+- [DescribeImagesUsingChatGPT.md](/examples/DescribeImagesUsingChatGPT.md): Learn how to use GPT-4 Turbo with Vision to understand the content of an image. 
+- [AnalyzeSentimentinTextUsingChatGPTinJSONMode.md](/examples/AnalyzeSentimentinTextUsingChatGPTinJSONMode.md): Learn how to use JSON mode in chat completions
+- [UsingDALLEToEditImages.md](/examples/UsingDALLEToEditImages.md): Learn how to generate images
+- [UsingDALLEToGenerateImages.md](/examples/UsingDALLEToGenerateImages.md): Create variations of images and editimages. 
 
 ## License
 

examples/AnalyzeScientificPapersUsingFunctionCalls.md

@@ -0,0 +1,158 @@
+
+# Analyze Scientific Papers Using ChatGPT™ Function Calls
+
+To run the code shown on this page, open the MLX file in MATLAB: [mlx-scripts/AnalyzeScientificPapersUsingFunctionCalls.mlx](mlx-scripts/AnalyzeScientificPapersUsingFunctionCalls.mlx) 
+
+This example shows how to extract recent scientific papers from ArXiv, summarize them using ChatGPT, and write the results to a CSV file using the `openAIFunction` function.
+
+-  The example contains three steps: 
+-  Define a custom function for ChatGPT to use to process its input and output. 
+-  Extract papers from ArXiv. 
+-  Use ChatGPT to assess whether a paper is relevant to your query, and to add an entry to the results table if so. 
+
+To run this example, you need a valid API key from a paid OpenAI™ API account.
+
+```matlab
+loadenv(".env")
+addpath('../..')
+```
+# Initialize OpenAI API Function and Chat
+
+Use `openAIFunction` to define functions that the model will be able to requests calls. 
+
+
+Set up the function to store paper details and initiate a chat with the OpenAI API with a defined role as a scientific paper expert.
+
+
+Define the function that you want the model to have access to. In this example the used function is `writePaperDetails`.
+
+```matlab
+f = openAIFunction("writePaperDetails", "Function to write paper details to a table.");
+f = addParameter(f, "name", type="string", description="Name of the paper.");
+f = addParameter(f, "url", type="string", description="URL containing the paper.");
+f = addParameter(f, "explanation", type="string", description="Explanation on why the paper is related to the given topic.");
+
+paperVerifier = openAIChat("You are an expert in filtering scientific papers. " + ...
+    "Given a certain topic, you are able to decide if the paper" + ...
+    " fits the given topic or not.");
+
+paperExtractor = openAIChat("You are an expert in extracting information from a paper.", Tools=f);
+
+function writePaperDetails(name, url, desc)
+filename = "papers_to_read.csv";
+T = table(name, url, desc, VariableNames=["Name", "URL", "Description"]);
+writetable(T, filename, WriteMode="append");
+end
+```
+# Extract Papers From ArXiv
+
+Specify the category of interest, the date range for the query, and the maximum number of results to retrieve from the ArXiv API.
+
+```matlab
+category = "cs.CL";
+endDate = datetime("today", "Format","uuuuMMdd");
+startDate = datetime("today", "Format","uuuuMMdd") - 5;
+maxResults = 40;
+urlQuery = "https://export.arxiv.org/api/query?search_query=" + ...
+    "cat:" + category + ...
+    "&submittedDate=["+string(startDate)+"+TO+"+string(endDate)+"]"+...
+    "&max_results=" + maxResults + ...
+    "&sortBy=submittedDate&sortOrder=descending";
+
+options = weboptions('Timeout',160);
+code = webread(urlQuery,options);
+```
+
+Extract individual paper entries from the API response and use ChatGPT to determine whether each paper is related to the specified topic.
+
+
+ChatGPT will parse the XML file, so we only need to extract the relevant entries.
+
+```matlab
+entries = extractBetween(code, '<entry>', '</entry>');
+```
+# Write Relevant Information to Table
+
+Create empty file and determine the topic of interest.
+
+```matlab
+filename = "papers_to_read.csv";
+T = table([], [], [], VariableNames=["Name", "URL", "Description"]);
+writetable(T, filename);
+
+topic = "Large Language Models";
+```
+
+Loop over the entries and see if they are relevant to the topic of interest.
+
+```matlab
+for i = 1:length(entries)
+    prompt =  "Given the following paper:" + newline +...
+        string(entries{i})+ newline +...
+        "Is it related to the topic: "+ topic +"?" + ...
+        " Answer 'yes' or 'no'.";
+    [text, response] = generate(paperVerifier, prompt);
+
+```
+
+If the model classifies this entry as relevant, then it tries to request a function call.
+
+```matlab
+    if contains("yes", text, IgnoreCase=true)
+        prompt =  "Given the following paper:" + newline + string(entries{i})+ newline +...
+            "Given the topic: "+ topic + newline + "Write the details to a table.";
+        [text, response] = generate(paperExtractor, prompt);
+```
+
+If `function_call` if part of the response, it means the model is requesting a function call. The function call request should contain the needed arguments to call the function specified at the end of this example and defined with `openAIFunctions`.
+
+```matlab
+        if isfield(response, "tool_calls")
+            funCall = response.tool_calls;
+            functionCallAttempt(funCall);
+        end
+    end
+end
+```
+
+Read the generated file. 
+
+```matlab
+data = readtable("papers_to_read.csv", Delimiter=",")
+```
+# Helper Function
+
+This function handles function call attempts from the model, checking the function name and arguments before calling the appropriate function to store the paper details.
+
+```matlab
+function functionCallAttempt(funCall)
+```
+
+The model can sometimes hallucinate function names, so you need to ensure that it's suggesting the correct name.
+
+```matlab
+if funCall.function.name == "writePaperDetails"
+    try
+```
+
+The model can sometimes return improperly formed JSON, which needs to be handled.
+
+```matlab
+        funArgs = jsondecode(funCall.function.arguments);
+    catch ME
+        error("Model returned improperly formed JSON.");
+    end
+```
+
+The model can hallucinate arguments. The code needs to ensure the arguments have been defined before calling the function.
+
+```matlab
+    if isfield(funArgs, "name") && isfield(funArgs, "url") && isfield(funArgs,"explanation")
+        writePaperDetails(string(funArgs.name), string(funArgs.url), string(funArgs.explanation));
+    end
+end
+end
+```
+
+*Copyright 2023\-2024 The MathWorks, Inc.*
+

examples/AnalyzeSentimentinTextUsingChatGPTinJSONMode.md

@@ -0,0 +1,88 @@
+
+# Analyze Sentiment in Text Using ChatGPT™ in JSON Mode
+
+To run the code shown on this page, open the MLX file in MATLAB: [mlx-scripts/AnalyzeSentimentinTextUsingChatGPTinJSONMode.mlx](mlx-scripts/AnalyzeSentimentinTextUsingChatGPTinJSONMode.mlx) 
+
+This example shows how to use ChatGPT for sentiment analysis and output the results in JSON format. 
+
+
+To run this example, you need a valid API key from a paid OpenAI™ API account.
+
+```matlab
+loadenv(".env")
+addpath('../..')
+```
+
+Define some text to analyze the sentiment.
+
+```matlab
+inputText = ["I can't stand homework.";
+    "This sucks. I'm bored.";
+    "I can't wait for Halloween!!!";
+    "I am neigher for or against the idea.";
+    "My cat is adorable ❤️❤️";
+    "I hate chocolate"];
+```
+
+Define the expected output JSON Schema.
+
+```matlab
+jsonSchema = '{"sentiment": "string (positive, negative, neutral)","confidence_score": "number (0-1)"}';
+```
+
+Define the the system prompt, combining your instructions and the JSON Schema. In order for the model to output JSON, you need to specify that in the prompt, for example, adding *"designed to output to JSON"*  to the prompt.
+
+```matlab
+systemPrompt = "You are an AI designed to output to JSON. You analyze the sentiment of the provided text and  " + ...
+    "Determine whether the sentiment is positive, negative, or neutral and provide a confidence score using " + ...
+    "the schema: " + jsonSchema;
+prompt = "Analyze the sentiment of the provided text. " + ...
+    "Determine whether the sentiment is positive, negative," + ...
+    " or neutral and provide a confidence score";
+```
+
+Create a chat object with `ModelName gpt-3.5-turbo` and specify `ResponseFormat` as `"json".`
+
+```matlab
+model = "gpt-3.5-turbo";
+chat = openAIChat(systemPrompt, ModelName=model, ResponseFormat="json");
+```
+
+```matlabTextOutput
+Warning: When using JSON mode, you must also prompt the model to produce JSON yourself via a system or user message.
+```
+
+Concatenate the prompt and input text and generate an answer with the model.
+
+```matlab
+scores = cell(1,numel(inputText));
+for i = 1:numel(inputText)
+```
+
+Generate a response from the message. 
+
+```matlab
+    [json, message, response] = generate(chat,prompt + newline + newline + inputText(i));
+    scores{i} = jsondecode(json);
+end
+```
+
+Extract the data from the JSON ouput. 
+
+```matlab
+T = struct2table([scores{:}]);
+T.text = inputText;
+T = movevars(T,"text","Before","sentiment")
+```
+| |text|sentiment|confidence_score|
+|:--:|:--:|:--:|:--:|
+|1|"I can't stand homework."|'negative'|0.9500|
+|2|"This sucks. I'm bored."|'negative'|0.9000|
+|3|"I can't wait for Halloween!!!"|'positive'|0.9500|
+|4|"I am neigher for or against the idea."|'neutral'|1|
+|5|"My cat is adorable ❤️❤️"|'positive'|0.9500|
+|6|"I hate chocolate"|'negative'|0.9000|
+
+
+*Copyright 2024 The MathWorks, Inc.*
+