Fix Postgres records ordering + Docs update (#224)

* Fix Postgres search by similarity SQL to sort records by similarity (descending) * Update docs, add links to nuget packages
microsoft · Dec 29, 2023 · eed4b78 · eed4b78
1 parent 164f1f6
commit eed4b78
Show file tree

Hide file tree

Showing 16 changed files with 167 additions and 47 deletions.
diff --git a/COMMUNITY.md b/COMMUNITY.md
@@ -25,5 +25,4 @@ We have a growing and active channel on Discord where you can get help, engage
 in lively discussion, and share what you've built with Kernel Memory and
 Semantic Kernel!
 
-Join our Discord:
-[https://aka.ms/SKDiscord](https://aka.ms/SKDiscord)
+[Join our Discord](https://aka.ms/KMdiscord)
diff --git a/README.md b/README.md
@@ -1,13 +1,16 @@
 # Kernel Memory
 
 [![License: MIT](https://img.shields.io/github/license/microsoft/kernel-memory)](https://github.com/microsoft/kernel-memory/blob/main/LICENSE)
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
-**Kernel Memory** (KM) is an open-source [service](service/Service/README.md) and
-[plugin](https://www.microsoft.com/en-us/microsoft-365/blog/2023/05/23/empowering-every-developer-with-plugins-for-microsoft-365-copilot/)
+**Kernel Memory** (KM) is a **multi-modal [AI Service](service/Service/README.md)**
 specialized in the efficient indexing of datasets through custom continuous data
-hybrid pipelines. For some scenarios KM is also available as a library, and soon
-as a Docker container.
+hybrid pipelines, with support for
+**[Retrieval Augmented Generation](https://en.wikipedia.org/wiki/Prompt_engineering#Retrieval-augmented_generation)** (RAG),
+synthetic memory, prompt engineering, and custom semantic memory processing.
+
+KM includes a GPT **[Plugin](https://www.microsoft.com/en-us/microsoft-365/blog/2023/05/23/empowering-every-developer-with-plugins-for-microsoft-365-copilot/)**,
+**web clients**, a .NET library for embedded applications, and soon as a Docker container.
 
 ![image](https://github.com/microsoft/kernel-memory/assets/371009/31894afa-d19e-4e9b-8d0f-cb889bf5c77f)
 
@@ -22,23 +25,6 @@ Designed for seamless integration as a Plugin with
 Copilot and ChatGPT, Kernel Memory enhances data-driven features in applications
 built for most popular AI platforms.
 
-## Repository Guidance
-
-This repository is a public resource designed to showcase best practices and efficient architecture
-for specific programming scenarios. Although bug fixes and secure, scalable enhancements are part
-of our focus, rigorous reviews and strategic considerations of the code are recommended before
-production use. Similar to the nature of AI development, the project will rapidly evolve. We warmly
-welcome you to participate in the development of Kernel Memory. Feel free to contribute opening
-GitHub Issues, sending us PRs, and joining our Discord community. Thank you and happy coding!
-
-### Packages for Python, Java and other languages
-
-A python package with a Web Client and Semantic Kernel plugin will soon be available.
-We also welcome PR contributions to support more languages.
-
-In the meantime you can easily connect to Kernel Memory web service following the
-existing **OpenAPI** swagger at http://127.0.0.1:9001/swagger/index.html.
-
 ### .NET packages
 
 * **Microsoft.KernelMemory.WebClient:** The web client library, can be used to call
@@ -53,17 +39,74 @@ existing **OpenAPI** swagger at http://127.0.0.1:9001/swagger/index.html.
   [![Nuget package](https://img.shields.io/nuget/vpre/Microsoft.KernelMemory.SemanticKernelPlugin)](https://www.nuget.org/packages/Microsoft.KernelMemory.SemanticKernelPlugin/)
   [![Example code](https://img.shields.io/badge/example-code-blue)](examples/011-dotnet-using-MemoryPlugin)
 
+* **Microsoft.KernelMemory.Abstractions:** The internal interfaces and models
+  shared by all packages, used to extend KM to support third party services.
+  .NET Standard 2.0 compatible.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.Abstractions)](https://www.nuget.org/packages/Microsoft.KernelMemory.Abstractions/)
+
+* **Microsoft.KernelMemory.MemoryDb.AzureAISearch:** Memory storage using
+  **[Azure AI Search](extensions/AzureAISearch)**.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.MemoryDb.AzureAISearch)](https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.AzureAISearch/)
+
+* **Microsoft.KernelMemory.MemoryDb.Postgres:** Memory storage using
+  **[PostgreSQL](extensions/Postgres)**.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.MemoryDb.Postgres)](https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.Postgres/)
+
+* **Microsoft.KernelMemory.MemoryDb.Qdrant:** Memory storage using
+  **[Qdrant](extensions/Qdrant)**.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.MemoryDb.Qdrant)](https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.Qdrant/)
+
+* **Microsoft.KernelMemory.AI.AzureOpenAI:** Integration with **[Azure OpenAI](extensions/OpenAI)** LLMs.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.AI.AzureOpenAI)](https://www.nuget.org/packages/Microsoft.KernelMemory.AI.AzureOpenAI/)
+
+* **Microsoft.KernelMemory.AI.LlamaSharp:** Integration with **[LLama](extensions/LlamaSharp)** LLMs.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.AI.LlamaSharp)](https://www.nuget.org/packages/Microsoft.KernelMemory.AI.LlamaSharp/)
+
+* **Microsoft.KernelMemory.AI.OpenAI:** Integration with **[OpenAI](extensions/OpenAI)** LLMs.
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.AI.OpenAI)](https://www.nuget.org/packages/Microsoft.KernelMemory.AI.OpenAI/)
+
+* **Microsoft.KernelMemory.DataFormats.AzureAIDocIntel:** Integration with
+  [Azure AI Document Intelligence](extensions/AzureAIDocIntel).
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.DataFormats.AzureAIDocIntel)](https://www.nuget.org/packages/Microsoft.KernelMemory.DataFormats.AzureAIDocIntel/)
+
+* **Microsoft.KernelMemory.Orchestration.AzureQueues:** Ingestion and synthetic memory
+  pipelines via [Azure Queue Storage](extensions/AzureQueues).
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.Orchestration.AzureQueues)](https://www.nuget.org/packages/Microsoft.KernelMemory.Orchestration.AzureQueues/)
+
+* **Microsoft.KernelMemory.Orchestration.RabbitMQ:** Ingestion and synthetic memory
+  pipelines via [RabbitMQ](extensions/RabbitMQ).
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.Orchestration.RabbitMQ)](https://www.nuget.org/packages/Microsoft.KernelMemory.Orchestration.RabbitMQ/)
+
+* **Microsoft.KernelMemory.ContentStorage.AzureBlobs:** Used to store content on
+  [Azure Storage Blobs](extensions/AzureBlobs).
+
+  [![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.ContentStorage.AzureBlobs)](https://www.nuget.org/packages/Microsoft.KernelMemory.ContentStorage.AzureBlobs/)
+
 * **Microsoft.KernelMemory.Core:** The core library, can be used to build custom
   pipelines and handlers, and contains a serverless client to use memory in a
   synchronous way, without the web service. .NET 6+.
 
   [![Nuget package](https://img.shields.io/nuget/vpre/Microsoft.KernelMemory.Core)](https://www.nuget.org/packages/Microsoft.KernelMemory.Core/)
   [![Example code](https://img.shields.io/badge/example-code-blue)](examples/001-dotnet-Serverless)
 
-* **Microsoft.KernelMemory.Abstractions:** The internal interfaces and models
-  shared by all packages. .NET Standard 2.0 compatible.
+### Packages for Python, Java and other languages
+
+Kernel Memory service offers a **Web API** out of the box, including the **OpenAPI
+swagger** documentation that you can leverage to test the API and create custom
+web clients. For instance, after starting the service locally, see http://127.0.0.1:9001/swagger/index.html.
 
-  [![Nuget package](https://img.shields.io/nuget/vpre/Microsoft.KernelMemory.Abstractions)](https://www.nuget.org/packages/Microsoft.KernelMemory.Abstractions/)
+A python package with a Web Client and Semantic Kernel plugin will soon be available.
+We also welcome PR contributions to support more languages.
 
 ### Data formats
 
@@ -355,3 +398,12 @@ running the service locally with OpenAPI enabled.
 5. [Script to start RabbitMQ for development tasks](tools/run-rabbitmq.sh)
 6. [.NET appsettings.json generator](tools/InteractiveSetup)
 
+## Repository Guidance
+
+This repository is a public resource designed to showcase best practices and efficient architecture
+for specific programming scenarios. Although bug fixes and secure, scalable enhancements are part
+of our focus, rigorous reviews and strategic considerations of the code are recommended before
+production use. Similar to the nature of AI development, the project will rapidly evolve. We warmly
+welcome you to participate in the development of Kernel Memory. Feel free to contribute opening
+GitHub Issues, sending us PRs, and joining our Discord community. Thank you and happy coding!
+
diff --git a/extensions/AzureAIDocIntel/README.md b/extensions/AzureAIDocIntel/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with Azure AI Document Intelligence
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.DataFormats.AzureAIDocIntel)](https://www.nuget.org/packages/Microsoft.KernelMemory.DataFormats.AzureAIDocIntel/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Azure AI Document Intelligence](https://azure.microsoft.com/products/ai-services/ai-document-intelligence)
 adapter allowing to use the Azure service in Kernel Memory to extract content

diff --git a/extensions/AzureAISearch/README.md b/extensions/AzureAISearch/README.md
@@ -1,7 +1,7 @@
 # Kernel Memory with Azure AI Search
 
-[//]: # ([![Nuget package]&#40;https://img.shields.io/nuget/Microsoft.KernelMemory.MemoryDb.AzureAISearch&#41;]&#40;https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.AzureAISearch/&#41;)
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.MemoryDb.AzureAISearch)](https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.AzureAISearch/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Azure AI Search](https://azure.microsoft.com/products/ai-services/ai-search)
 adapter allowing to use Kernel Memory with Azure AI Search.
diff --git a/extensions/AzureBlobs/README.md b/extensions/AzureBlobs/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with Azure Blob Storage
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.ContentStorage.AzureBlobs)](https://www.nuget.org/packages/Microsoft.KernelMemory.ContentStorage.AzureBlobs/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Azure Blob Storage](https://learn.microsoft.com/azure/storage/blobs)
 adapter allowing Kernel Memory to upload documents and maintain their state in Azure Blobs.
diff --git a/extensions/AzureOpenAI/README.md b/extensions/AzureOpenAI/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with Azure OpenAI
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.AI.AzureOpenAI)](https://www.nuget.org/packages/Microsoft.KernelMemory.AI.AzureOpenAI/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service)
 LLM connector to access to Azure OpenAI LLM models and generate embeddings and text.
diff --git a/extensions/AzureQueues/README.md b/extensions/AzureQueues/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with Azure Queue Storage
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.Orchestration.AzureQueues)](https://www.nuget.org/packages/Microsoft.KernelMemory.Orchestration.AzureQueues/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Azure Queue Storage](https://learn.microsoft.com/azure/storage/queues)
 adapter allowing to orchestrate Kernel Memory pipelines with Azure Queues.
diff --git a/extensions/LlamaSharp/README.md b/extensions/LlamaSharp/README.md
@@ -1,6 +1,12 @@
 # Kernel Memory with LLama
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.AI.LlamaSharp)](https://www.nuget.org/packages/Microsoft.KernelMemory.AI.LlamaSharp/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
+
+This project contains the
+[LLama](https://ai.meta.com/blog/large-language-model-llama-meta-ai/)
+LLM connector to access to LLama LLM models and generate text,
+leveraging the
+[LLamaSharp]((https://scisharp.github.io/LLamaSharp)) project.
+
 
-This project contains the [LLama](https://scisharp.github.io/LLamaSharp)
-LLM connector to access to LLama LLM models and generate text.
diff --git a/extensions/OpenAI/README.md b/extensions/OpenAI/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with OpenAI
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.AI.OpenAI)](https://www.nuget.org/packages/Microsoft.KernelMemory.AI.OpenAI/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [OpenAI](https://openai.com)
 LLM connector to access to OpenAI LLM models and generate embeddings and text.
diff --git a/extensions/Postgres/Postgres/Db/PostgresDbClient.cs b/extensions/Postgres/Postgres/Db/PostgresDbClient.cs
@@ -397,6 +397,7 @@ DO UPDATE SET
                 SELECT {columns}, 1 - ({this._colEmbedding} <=> @embedding) AS {similarityActualValue}
                 FROM {tableName}
                 WHERE {filterSql}
+                ORDER BY {similarityActualValue} DESC
                 LIMIT @limit
                 OFFSET @offset
             ";

diff --git a/extensions/Postgres/README.md b/extensions/Postgres/README.md
@@ -1,7 +1,7 @@
 # Kernel Memory with Postgres + pgvector
 
-[![Nuget package](https://img.shields.io/nuget/vpre/Microsoft.KernelMemory.Postgres)](https://www.nuget.org/packages/Microsoft.KernelMemory.Postgres/)
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.MemoryDb.Postgres)](https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.Postgres/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Postgres](https://www.postgresql.org) adapter allowing
 to use Kernel Memory with [Postgres+pgvector](https://github.com/pgvector/pgvector).

diff --git a/extensions/Qdrant/README.md b/extensions/Qdrant/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with Qdrant
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.MemoryDb.Qdrant)](https://www.nuget.org/packages/Microsoft.KernelMemory.MemoryDb.Qdrant/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [Qdrant](https://qdrant.tech) adapter allowing to use Kernel Memory with Qdrant.
 

diff --git a/extensions/RabbitMQ/README.md b/extensions/RabbitMQ/README.md
@@ -1,6 +1,7 @@
 # Kernel Memory with RabbitMQ
 
-[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/SKDiscord)
+[![Nuget package](https://img.shields.io/nuget/v/Microsoft.KernelMemory.Orchestration.RabbitMQ)](https://www.nuget.org/packages/Microsoft.KernelMemory.Orchestration.RabbitMQ/)
+[![Discord](https://img.shields.io/discord/1063152441819942922?label=Discord&logo=discord&logoColor=white&color=d82679)](https://aka.ms/KMdiscord)
 
 This project contains the [RabbitMQ](https://www.rabbitmq.com/) adapter allowing to orchestrate
 Kernel Memory pipelines with RabbitMQ queues.
diff --git a/service/tests/FunctionalTests/FunctionalTests.csproj b/service/tests/FunctionalTests/FunctionalTests.csproj
@@ -12,6 +12,7 @@
     </PropertyGroup>
 
     <ItemGroup>
+        <ProjectReference Include="..\..\..\extensions\Postgres\Postgres\Postgres.csproj" />
         <ProjectReference Include="..\..\Abstractions\Abstractions.csproj"/>
         <ProjectReference Include="..\..\Core\Core.csproj"/>
         <ProjectReference Include="..\..\..\clients\dotnet\SemanticKernelPlugin\SemanticKernelPlugin.csproj"/>