🤖️ halbot

Just another Gemini / ChatGPT / Claude / Mistral (by ollama) Telegram bob, which is simple design, easy to use, extendable and fun.

Live demo, click to watch on YouTube:

Screenshots: 👈 Click here to see screenshots

Features

• Telegram Bot (Telegram Bot token required)
• ChatGPT (OpenAI API key required)
• Gemini (Google Gemini API Key required)
• Claude (Anthropic API Key required)
• Mistral (Install Ollama and enable Mistral)
• Speech-to-Text (OpenAI or Google Cloud API key required, or your own engine)
• Text-to-Speech (OpenAI or Google Cloud API key required, or your own engine)
• Text-to-Image by DALL·E (OpenAI API key required, or your own engine)
• OCR/OBJECT_DETECT (Google Cloud API key required, or your own engine)
• Feeding webpage and YouTube to enhance your prompt
• Custom prompt and 🧠 Awesome ChatGPT Prompts at your fingertips
• Support private and public mode, with multiple authenticate methods.
• Middleware style workflow, easy to extend.
• Built-in support parsing webpages, YouTube videos, PDFs, images, Office documents, code files, text files...
• Realtime stream-style response, no more waiting.
• Multimodal support for all supported models.
• Automatically selects the optimal model for the task.
• Audio input and output support for supported models, not just TTS.
• Google Search as a tool support for Gemini 2.0.
• Markdown rendering
• Reference rendering
• Code block rendering, developers friendly.
• Threaded conversation support.
• ESM from the ground up

Basic usage

Configuration

Make the halbot json config file and put it in this path ~/.halbot.json.

Basic config demo:

{
    "telegramToken": "[[Telegram Bot API Token]]",
    "openaiApiKey": "[[OpenAI API Key]]"
}

All supported configuration fields:

{

    // REQUIRED, string.
    "telegramToken": "[[Telegram Bot API Token]]",

    // Set some of these fields if you need Google's Gemini, TTS, STT, OCR, OBJECT_DETECT, Embedding features.
    // OPTIONAL, string.
    "googleApiKey": "[[Google Cloud / Gemini API Key]]",
    // OPTIONAL, string, default: "gemini-pro-vision".
    "geminiModel": "[[Custom Gemini Model ID]]",
    // OPTIONAL, integer, default: 0.
    "geminiPriority": "[[Custom Gemini Priority]]",

    // Set some of these fields if you need OpenAI's ChatGPT, Whisper, Embedding features.
    // OPTIONAL, string.
    "openaiApiKey": "[[OpenAI API Key]]",
    // OPTIONAL, string.
    "openaiEndpoint": "[[Custom OpenAI API endpoint]]",
    // OPTIONAL, string, default: "gpt-3.5-turbo".
    "chatGptModel": "[[Custom ChatGPT Model ID]]",
    // OPTIONAL, integer, default: 1.
    "chatGptPriority": "[[Custom ChatGPT Priority]]",

    // Set some of these fields if you need to use custom ChatGPT API.
    // OPTIONAL, string.
    "chatGptApiKey": "[[Custom ChatGPT API Key]]",
    // OPTIONAL, string.
    "chatGptEndpoint": "[[Custom ChatGPT API endpoint]]",

    // Set some of these fields if you need Anthropic's Claude features.
    // OPTIONAL, string.
    "claudeApiKey": "[[Anthropic API Key]]",
    // OPTIONAL, string, default: "claude".
    "claudeModel": "[[Custom Claude Model ID]]",
    // OPTIONAL, integer, default: 2.
    "claudePriority": "[[Custom Claude Priority]]",

    // Set some of these fields if you need Mistral features.
    // OPTIONAL, boolean.
    "mistralEnabled": "[[Enable Mistral hosted by Ollama]]",
    // OPTIONAL, string.
    "mistralEndpoint": "[[Custom Mistral API endpoint]]",
    // OPTIONAL, string, default: "Mistral" (Mistral 7B).
    "mistralModel": "[[Custom Mistral Model ID]]",
    // OPTIONAL, integer, default: 3.
    "mistralPriority": "[[Custom Mistral Priority]]",

    // OPTIONAL, undefined || array of string.
    // To open the bot to PUBLIC, DO NOT set this field;
    // To restrict the bot to PRIVATE, set chat/group/channel ids in this array.
    "private": ["[[CHAT_ID]]", "[[GROUP_ID]]", "[[CHANNEL_ID]]", ...],

    // OPTIONAL, string.
    // Set some of these fields if you want to use a `magic word` to authenticate the bot.
    "magicWord": "[[Your Magic Word here]]",

    // OPTIONAL, string.
    // Use a HOME GROUP to authentication users.
    // Anyone in this group can access the bot.
    "homeGroup": "[[GROUP_ID]]",

    // OPTIONAL, array of enum string.
    // Enum: 'private', 'mention', 'group', 'channel'.
    // Defaule: ['private', 'mention'].
    // By default, it will only reply to `private` chats and group `mention`s.
    // Adding 'group' or 'channel' may cause too much disturbance.
    "chatType": ["mention", "private"],

    // OPTIONAL, string.
    "hello": "[[initial prompt]]",

    // OPTIONAL, string.
    "info": "[[bot description]]",

    // OPTIONAL, string.
    "help": "[[help information]]",

    // OPTIONAL, object.
    // Sessions/conversations storage.
    // support PostgreSQL, MariaDB/MySQL and Redis for now.
    // If omitted, the bot will use memory storage and sync to this file.
    // Example: (Compatibility: https://node-postgres.com/apis/pool)
    // PostgreSQL is recommended for vector storage.
    "storage": {
        "provider": "POSTGRESQL",
        "host": "[[DATABASE HOST]]",
        "database": "[[DATABASE NAME]]",
        "user": "[[DATABASE USER]]",
        "password": "[[DATABASE PASSWORD]]",
        "vector": true, // REQUIRED
        ...[[OTHER DATABASE OPTIONS]],
    },
    // OR: (Compatibility: https://github.com/sidorares/node-mysql2)
    "storage": {
        "provider": "[["MARIADB" || "MYSQL"]]",
        "host": "[[DATABASE HOST]]",
        "database": "[[DATABASE NAME]]",
        "user": "[[DATABASE USER]]",
        "password": "[[DATABASE PASSWORD]]",
        "charset": "utf8mb4", // REQUIRED
        ...[[OTHER DATABASE OPTIONS]],
    },
    // OR: (Compatibility: https://github.com/luin/ioredis)
    "storage": {
        "provider": "REDIS",
        "host": "[[REDIS HOST]]",
        "password": "[[REDIS PASSWORD]]",
        ...[[OTHER REDIS OPTIONS]],
    },

}

Run it

In peace-of-mind:

$ npx halbot

If you have multible AI engines configed, use '/chatgpt' or '/bing' to switch between them, or you can use '/*' to ask them all at the same time.

Integrate to your project

Install:

$ npm i halbot

Code:

import halbot from 'halbot';

const config = {

    // ...[[ALL THE CONFIG FIELDS SUPPORTED ABOVE]]],

    // OPTIONAL, function.
    // Your own authentication logic.
    // return true if the user is authenticated.
    // return false if the user is not authenticated.
    auth: async (ctx) => {
        // ctx is the `telegraf` context object: https://telegraf.js.org/#context-class
        // It has been extended: https://github.com/Leask/utilitas/blob/master/lib/bot.mjs
        return true;
    },

    // OPTIONAL, object (key renderd as name) or array (name ignored).
    ai: {
        [[aiNameA]]: [[aiConfigA]],
        [[aiNameB]]: [[aiConfigB]],
        // ...
    },

    // OPTIONAL, object.
    // Your own speech-to-text and text-to-speech engine.
    speech: {
        stt: [[sttApi]],
        tts: [[ttsApi]],
    },

    // OPTIONAL, object.
    // Your own computer-vision engine.
    vision: {
        see: [[ocrAndObjectDetectApi]],
        read: [[documentAnnotateApi]],
    },

    // OPTIONAL, object.
    // Your own image-generator engine.
    image: {
        generate: [[textToImageApi]],
    },

    // OPTIONAL, string.
    // Path to your own middlewares.
    // ./skills
    //  |- skill_a.mjs
    //    | const action = async (bot) => {
    //    |     bot.use(async (ctx, next) => {
    //    |         ctx.reply('42');
    //    |         await next();
    //    |     });
    //    | };
    //    |
    //    | export const { run, priority, func } = {
    //    |     run: true,
    //    |     priority: 100,
    //    |     func: action,
    //    | };
    skillPath: [[pathToYourMiddlewares]],

    // OPTIONAL, object.
    // Using customized storage engine.
    // `storage` should Should be compatible with the `Map` interface:
    // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
    storage: {
        provider: [[POSTGRESQL || MARIADB || MYSQL || REDIS]],
        get: async (key) => { /* Return session object by chatId. */ },
        set: async (key, session) => { /* Save session object by chatId. */ },
        client: { /* Customized database client / pool. */ },
            query: async (topic) => { /* Search history and session by topic. */ },
            upsert: async (event) => { /* Save event for history and session. */ },
        },
    },

    // OPTIONAL, function.
    // Using customized embedding engine for history and session search.
    embedding: async (text) => { /* Return vector embedding of the text. */ },

    // OPTIONAL, array of string.
    // Supported mime types of your vision-enabled AI models.
    // If omitted, bot will use standard OCR and Object Detect to handle images.
    supportedMimeTypes: [...[[mimeTypes]]],

    // OPTIONAL, object.
    // Adding extra commands.
    cmds: {
        [[commandA]]: [[descriptionA]],
        [[commandB]]: [[descriptionB]],
        ...[[OTHER COMMANDS]],
    },

    // OPTIONAL, object.
    // Adding extra configurations
    args: {
        [[argA]]: {
            type: 'string',
            short: [[shortCut]],
            default: [[defaultValue]],
            desc: [[description]],
        },
        [[argB]]: {
            type: 'binary',
            short: [[shortCut]],
            default: [[defaultValue]],
            desc: [[description]],
        },
        ...[[OTHER ARGS]],
    },

};

await halbot(config);

Foundations

• halbot uses my other project 🧰 utilitas as the basic framework to handle all the dirty work.
• halbot uses 🤖 utilitas.bot as a Telegram bot engine.
• halbot uses 🤖 utilitas.alan to communicate with the AI engines.

Contact me