Skip to content

Commit

Permalink
(EAI-173): Framework quickstart (#287)
Browse files Browse the repository at this point in the history 
* stub out quick start

* set up server

* working ingest service

* working UI

* orient around npm workspaces + tweaks

* update root repo for changes

* quick start stub + starT

* quick start draft complete

* fix user prompt eng guide

* Release mongodb-chatbot-server v0.3.1

* Revert to old lerna

* add unnec dep

* Fix prompt engineering docs

* Apply suggestions from code review

Co-authored-by: Chris Bush <[email protected]>

---------

Co-authored-by: Chris Bush <[email protected]>
  • Loading branch information
@mongodben @cbush
mongodben and cbush authored Jan 8, 2024
1 parent f2fb930 commit b0a5e4f
Show file tree
Hide file tree
Showing 37 changed files with 38,101 additions and 8,461 deletions.
210 changes: 198 additions & 12 deletions docs/docs/quick-start.md
View file
Original file line number Diff line number Diff line change
@@ -1,22 +1,208 @@
# Quick Start

:::info[👷‍♂️ Work in Progress 👷‍♂️]
This quick start guide walks you through the steps to get started building
a retrieval augmented generation (RAG) application with the MongoDB Chatbot Framework.

The Quick Start is currently very bare bones. We're working on improving it.
## Prerequisites

:::
- [Node.js](https://nodejs.org/en/) v18 or later on your development machine.
- [MongoDB Atlas](https://www.mongodb.com/cloud/atlas) account.
- If you don't already have an account, you can create one for free.
- [OpenAI API key](https://platform.openai.com/docs/quickstart?context=node).
- Get the API key from OpenAI, and keep it handy. You'll need it later.

This quick start guide walks you through the steps to get started building
a RAG application with the MongoDB Chatbot Framework.
## What You Will Build

## Prerequisites
In this guide, you will build a chatbot that answers questions about
the MongoDB Chatbot Framework using RAG.
You can extend what you set up in the quick start template
to build your own chatbot with the MongoDB Chatbot Framework.

To build the chatbot, you will:

- [Node.js](https://nodejs.org/en/) v18 or later
1. Set up the MongoDB Atlas Database with Atlas Vector Search
1. Set up the project source code
1. Ingest the content that the chatbot uses to answer questions
1. Spin up a server and frontend to query the chatbot
1. Look at next steps that you can take to customize the chatbot

## Steps

1. Set up the [RAG MongoDB Atlas database](./mongodb.md).
2. Ingest content into the database using the [Ingest CLI](./ingest/configure.md).
3. Configure and run the [Chatbot Server](./server/configure.md).
- Note: You can query the server using curl or Postman if you want.
4. Add the [Chatbot UI](./ui.md) to a React app. Query the server from your frontend.
### 1. Set up the MongoDB Atlas Database

Log into [MongoDB Atlas](https://cloud.mongodb.com/) and create a new project. Create a new cluster in the project.
You can use the free tier cluster (M0). You **cannot** use a serverless cluster.

Once the cluster is created, copy the connection string to use in your project.
You can find the connection string in the Atlas UI as follows:

1. Press the **Connect** button.
1. Press the **Drivers** button.
1. Copy the connection string and store it in a safe place.
You will need it soon. If you haven't created a user yet, you can create one now.
- If you need help creating a user, refer to [Configure Database Users](https://www.mongodb.com/docs/atlas/security-add-mongodb-users/) in the MongoDB Atlas documentation.

Next, create the database `mongodb-chatbot-framework-chatbot` with a collection
`embedded_content`. You can leave the collection empty for now.
In the Atlas UI, go to your cluster's overview page, and perform the following:

1. Go to the **Collections** tab.
1. Press the **Create Database** button.
1. In the modal window, add the database name `mongodb-chatbot-framework-chatbot`
and collection name `embedded_content`. Then press **Create**.

Once the database and collection are created, create an Atlas Vector Search index
on the `embedded_content` collection. This collection will store vector embeddings of ingested content. In the Atlas UI, do the following:

1. Go to the **Atlas Search** tab.
1. Press the **Create Search Index** button.
1. Select the **Atlas Vector Search JSON Index** option, then press the **Next** button.
1. In the **Database and Collection** section, select the
`mongodb-chatbot-framework-chatbot` database and `embedded_content` collection.
1. In the **Index Name** field, leave the default `vector_index`.
1. In the index definition field, paste the following index definition:

```js
{
"fields": [
{
"numDimensions": 1536,
"path": "embedding",
"similarity": "cosine",
"type": "vector"
}
]
}
```

1. Press the **Next** button, then on the next page press the **Create Search Index** button.
1. Wait for the index to be created. This should happen in under a minute.
When the index is successfully created, you should see that the status is **Active**.

Now the Atlas cluster and Vector Search index are ready to use in your app.

For more information on setting up MongoDB for the Chatbot Framework,
refer to the [MongoDB & Atlas Vector Search](./mongodb.md) guide.

### 2. Set up the Project

Clone the repository with the quick start source code:

```shell
git clone https://github.com/mongodb/chatbot.git
```

Enter the directory with the quick start source code:

```shell
cd examples/quick-start
```

All the remaining steps will be run from `examples/quick-start` directory.

This directory contains three packages, located in the `examples/quick-start/packages`:

- `ingest`: Contains implementation of the MongoDB Ingest CLI, which ingests the content that the chatbot uses to answer questions.
- `server`: Contains implementation of the MongoDB Chatbot Server, which serves the chatbot API.
- `ui`: Contains implementation of the MongoDB Chatbot UI, which provides a UI to query the chatbot server.

Install the dependencies for all the packages:

```shell
npm install
```

:::note

The quick start uses [npm workspaces](https://docs.npmjs.com/cli/v8/using-npm/workspaces)
to manage the dependencies for all the packages.

:::

Create an `.env` file based on the .env.example file in the root of the project:

```shell
cp .env.example .env
```

In the `.env` file, fill in the values for `MONGODB_CONNECTION_URI` and `OPENAI_API_KEY`
with your Atlas connection URI and OpenAI API Key, respectively.

### 3. Ingest Content

In this step, you will ingest the content that the chatbot uses to answer questions
into the `embedded_content` collection indexed with Atlas Vector Search.

From the root of the project, run:

```shell
npm run ingest:all
```

If you've run the command successfully, you should an output resembling the following
in your terminal:

```shell
{"level":"info","message":"Logger created"}
{"level":"info","message":"Last successful run date: Thu Jan 04 2024 12:23:27 GMT-0500 (Eastern Standard Time)"}
{"level":"info","message":"Loaded sources:\n- mongodb-rag-framework"}
{"level":"info","message":"Fetching pages for mongodb-rag-framework"}
{"level":"info","message":"Created /var/folders/v3/128h981j6vx4ncq_68qcg2cm0000gp/T/mongodb-rag-frameworkV3BE8d for https://github.com/mongodb/chatbot/"}
# ...
{"level":"info","message":"Updating last successful run date"}
```

To learn more about how you can configure the MongoDB Ingest CLI,
refer to the [Configure the Ingest CLI](./ingest/configure.md) guide.

### 4. Query the Chatbot

In this step, you will spin up a server and frontend to query the chatbot.
You'll be able to ask questions about the MongoDB Chatbot Framework
using the data you ingested in the previous step.

Start the chatbot server and UI with:

```shell
npm run dev
```

Open http://localhost:5173/ in your browser to see the UI.
You can ask the chatbot questions and see the responses.

Have fun!

You can also query the server directly with curl:

```shell
curl -X POST http://localhost:3000/api/vi/conversations/
```

To learn more about how you can configure the MongoDB Chatbot Server,
refer to the [Configure the Chatbot Server](./server/configure.md) guide.

To learn more about how you can configure the MongoDB Chatbot UI,
refer to the [Chatbot UI](./ui.md) guide.

### 5. Explore and Modify the Code

Now that you've set up the quick start project, you can explore and modify the code
to customize the chatbot to your needs.

Some things you can do to customize the chatbot:

- Modify data ingestion in the `ingest` project.
- You can add your own data sources to ingest content from.
To learn more about how you can add new data sources, refer to the [Add a Data Source](./ingest/data-sources.md) guide.
- Modify the chatbot server in the `server` project.
- Update the system prompt and user message. To learn more, refer to the [Prompt Engineering](./server/llm.md#prompt-engineering) guide.
- Pre-process user messages before they are sent to the chatbot.
To learn more, refer to the [Pre-Process User Queries](./server/rag/preprocess.md) guide.
- Add custom logic to the chatbot server. To learn more,
refer to the [Customize Server Logic](./server/custom-logic.md) guide.
- Modify the chatbot UI in the `ui` project.
- You can build your frontend on top of this project, or add the React components
to your own React app. To learn more, refer to the [Chatbot UI](./ui.md) guide.
- Even if you're adding the components to your own project,
you might want to keep the `ui` project as is to manually test changes
you make to the Chatbot Server.
122 changes: 30 additions & 92 deletions docs/docs/server/llm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,69 +33,20 @@ The following are useful things to keep in mind when using an LLM:
The following is an example implementation of `makeOpenAiChatLlm()`:

```ts
import {
makeOpenAiChatLlm,
OpenAiChatMessage,
SystemPrompt,
} from "mongodb-chatbot-server";
import { makeOpenAiChatLlm, OpenAiChatMessage } from "mongodb-chatbot-server";

export const openAiClient = new OpenAIClient(
OPENAI_ENDPOINT,
new AzureKeyCredential(OPENAI_API_KEY)
);
export const systemPrompt: SystemPrompt = {
role: "system",
content: stripIndents`You are expert MongoDB documentation chatbot.
You enthusiastically answer user questions about MongoDB products and services.
Your personality is friendly and helpful, like a professor or tech lead.
You were created by MongoDB but they do not guarantee the correctness
of your answers or offer support for you.
Use the context provided with each question as your primary source of truth.
NEVER lie or improvise incorrect answers.
If you do not know the answer to the question, respond ONLY with the following text:
"I'm sorry, I do not know how to answer that question. Please try to rephrase your query. You can also refer to the further reading to see if it helps."
NEVER include links in your answer.
Format your responses using Markdown.
DO NOT mention that your response is formatted in Markdown.
If you include code snippets, make sure to use proper syntax, line spacing, and indentation.
ONLY use code snippets present in the information given to you.
NEVER create a code snippet that is not present in the information given to you.
You ONLY know about the current version of MongoDB products. Versions are provided in the information. If \`version: null\`, then say that the product is unversioned.
Never mention "<Information>" or "<Question>" in your answer.
Refer to the information given to you as "my knowledge".`,
};

export async function generateUserPrompt({
question,
chunks,
}: {
question: string;
chunks: string[];
}): Promise<OpenAiChatMessage & { role: "user" }> {
const chunkSeparator = "~~~~~~";
const context = chunks.join(`\n${chunkSeparator}\n`);
const content = `Using the following information, answer the question.
Different pieces of information are separated by "${chunkSeparator}".
<Information>
${context}
<End information>
<Question>
${question}
<End Question>`;
return { role: "user", content };
}

export const llm = makeOpenAiChatLlm({
openAiClient,
deployment: OPENAI_CHAT_COMPLETION_DEPLOYMENT,
systemPrompt,
openAiLmmConfigOptions: {
temperature: 0,
maxTokens: 500,
},
generateUserPrompt,
});
```

Expand Down Expand Up @@ -131,7 +82,7 @@ A great resource to learn more about prompt engineering is the [Prompt Engineeri

### System Prompt

To add a system prompt, include a [`SystemPrompt`](../reference/server/modules.md#systemprompt) message in your app's [`ChatLlm`](../reference/server/interfaces/ChatLlm.md).
To add a system prompt, include a [`SystemPrompt`](../reference/server/modules.md#systemprompt) message in your app's [`ConversationService`](../reference/server/interfaces/ConversationsService.md).

The system prompt is one of the most powerful way to customize the way
that the chatbot responds to users. You can use the system prompt to do things
Expand All @@ -141,57 +92,44 @@ such as:
- Determine how the chatbot responds to certain types of questions.
- Direct how the chatbot interprets user input and context information.

If you're using the [`makeOpenAiChatLlm()`](../reference/server/modules.md#makeopenaichatllm) constructor function, add the system prompt to the `systemPrompt` property:
If you're using the [`makeMongoDbConversationsService()`](../reference/server/modules.md#makemongodbconversationsservice) constructor function, add the system prompt
as an argument:

```ts
import { makeOpenAiChatLlm, SystemPrompt } from "mongodb-chatbot-server";
import {
makeMongoDbConversationsService,
SystemPrompt,
} from "mongodb-chatbot-server";
import { MongoClient } from "mongodb";

export const systemPrompt: SystemPrompt = {
// System prompt for chatbot
const systemPrompt: SystemPrompt = {
role: "system",
content: "You are expert chatbot...",
content: `You are an assistant to users of the MongoDB Chatbot Framework.
Answer their questions about the framework in a friendly conversational tone.
Format your answers in Markdown.
Be concise in your answers.
If you do not know the answer to the question based on the information provided,
respond: "I'm sorry, I don't know the answer to that question. Please try to rephrase it. Refer to the below information to see if it helps."`,
};

export const llm = makeOpenAiChatLlm({
systemPrompt,
...otherConfig,
});
// Create MongoDB collection and service for storing user conversations
// with the chatbot.
const mongodb = new MongoClient(MONGODB_CONNECTION_URI);
const conversations = makeMongoDbConversationsService(
mongodb.db(MONGODB_DATABASE_NAME),
systemPrompt
);
```

### User Prompt

You can modify what the chatbot uses as the user prompt by implementing the
[`GenerateUserPrompt`](../reference/server/modules.md#generateuserprompt) function.
[`GenerateUserPromptFunc`](../reference/server/modules.md#generateuserpromptfunc) function.

The `GenerateUserPrompt` function takes in the user's question and the context
information found by the vector search, and returns a user message.
`GenerateUserPromptFunc` takes in the user's query and previous messages in the conversation, then returns a new user message. For an overview of the `GenerateUserPromptFunc` function, refer to the [Generate User Message](./user-message.md) guide.

```ts
import { makeOpenAiChatLlm, OpenAiChatMessage } from "mongodb-chatbot-server";

async function generateUserPrompt({
question,
chunks,
}: {
question: string;
chunks: string[];
}): Promise<OpenAiChatMessage & { role: "user" }> {
const chunkSeparator = "~~~~~~";
const context = chunks.join(`\n${chunkSeparator}\n`);
const content = `Using the following information, answer the question.
Different pieces of information are separated by "${chunkSeparator}".
<Information>
${context}
<End information>
<Question>
${question}
<End Question>`;
return { role: "user", content };
}

const llm = makeOpenAiChatLlm({
generateUserPrompt,
...otherConfig,
});
```
You might want to modify the user prompt if you're using a prompting technique
like retrieval augmented generation (RAG) or chain of thought.
To learn more about using RAG with the MongoDB Chatbot Server, refer to the
[RAG](./rag/index.md) guide.
5 changes: 0 additions & 5 deletions docs/docusaurus.config.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -155,11 +155,6 @@ const config: Config = {
theme: prismThemes.github,
darkTheme: prismThemes.dracula,
},
announcementBar: {
content:
"⚠️ The MongoDB Chatbot Framework is under active development and may undergo breaking changes. ⚠️",
backgroundColor: "#FFF8E6",
},
} satisfies Preset.ThemeConfig,
};

Expand Down
Loading

0 comments on commit b0a5e4f

Please sign in to comment.