lobe-chat

[Edit section]

[Copy link]

• • •

The Lobe Chat repository is a framework for building AI-powered chat applications, enabling engineers to integrate advanced AI functionalities such as natural language processing, text-to-speech, and speech-to-text to enhance user interactions and automate responses.

The most significant parts of the repo include:

• The …/store directory, which is central to the application's state management. It uses the Zustand library to create a modular and efficient state management system, with subdirectories for different aspects of the application like user preferences, chat, files, global state, and sessions. The state management system is designed to be clear and maintainable, with each subdirectory handling specific functionalities:

  • …/user manages user-related states, including preferences, settings, and synchronization.
  • …/chat handles chat-related states, such as messages and chat tools.
  • …/global manages the global state of the application. Middleware for persistent storage is implemented in …/createHyperStorage, which supports multiple storage backends. State Management

• The …/(backend) directory, which is the backbone of the Lobe Chat application's backend functionality. It contains modular and extensible integration with various services and features, including:

  • Authentication and webhook handling for services like Casdoor, Clerk, and Logto
  • Text-to-speech and speech-to-text capabilities using providers such as Microsoft, OpenAI, and Edge
  • Proxy handling for API requests
  • Tokenization and text processing The backend API supports a range of AI model providers and includes asynchronous processing capabilities, enhanced file management services, and knowledge base interactions. Backend API

• The …/Conversation directory, which contains the core functionality for the conversation feature, including message rendering, actions and tools management, error handling, and plugins content rendering. It is structured to provide a seamless chat experience, with components and utilities that manage chat interactions and initialization. Conversation Feature

• The …/database directory, which manages both local and server-side databases. It uses the Dexie.js library for IndexedDB database management on the client-side and includes schemas and models for various entities such as agents, files, knowledge bases, and sessions. Database Management

• The …/utils directory, which provides utility functions and classes that support various functionalities within the application, such as configuration management, error handling, data fetching, platform detection, and data comparison. Utility Functions

• The …/types directory, which provides the core data models, configurations, and types used throughout the application. This directory ensures type safety and consistency across the codebase. Type Definitions

The key algorithms and technologies the repo relies on include:

• Zustand for state management
• Dexie.js for IndexedDB database management
• tRPC for type-safe API communication
• NextAuth.js for authentication
• Various AI model providers (e.g., OpenAI, Google, Anthropic, Hunyuan, Wenxin) for natural language processing and generation
• Edge runtime for improved performance of certain API routes

Key design choices include:

• Modular architecture allowing for easy integration and extension of services
• Use of Zustand for efficient state management
• Implementation of local and server-side databases for persistent storage and synchronization capabilities
• Flexible authentication system supporting multiple providers (e.g., Casdoor, Clerk, Logto)
• Asynchronous processing for file handling and knowledge base operations
• Integration of multiple AI model providers to offer a wide range of language processing capabilities
• Implementation of text-to-speech and speech-to-text functionalities to enhance user interaction
• Use of Docker for containerization and deployment flexibility

State Management

[Edit section]

[Copy link]

References: src/store, src/features/Conversation, src/features/FileManager, src/features/KnowledgeBaseModal, src/features/FileSidePanel, src/store/file, src/store/knowledgeBase, src/types/files, src/types/knowledgeBase, src/types/chunk, src/types/eval

• • •

The …/store directory serves as the central hub for managing the state and actions of the Lobe Chat application. It is structured into several sub-directories, each dedicated to a distinct aspect of the application's functionality:

Agent Store State Management

[Edit section]

[Copy link]

References: src/store/agent/slices/chat/action.ts, src/store/agent/slices/chat/initialState.ts, src/store/agent/slices/chat/selectors.ts

• • •

In the Lobe Chat application, agent configurations are managed through a state management system that leverages the zustand library for state updates and the swr library for data fetching and caching. The …/action.ts file encapsulates the logic for initializing the agent store, updating agent configurations, and handling agent-related actions.

Chat Store State Management

[Edit section]

[Copy link]

References: src/store/chat, src/store/chat/slices/portal/action.test.ts, src/store/chat/slices/portal/action.ts, src/store/chat/slices/portal/initialState.ts, src/store/chat/slices/portal/selectors.test.ts, src/store/chat/slices/portal/selectors.ts

The Zustand state management library orchestrates the chat functionality within the Lobe Chat application, structured into slices within the …/chat directory. Each slice is dedicated to a specific aspect of chat functionality.

File Store State Management

[Edit section]

[Copy link]

References: src/store/file, src/store/file/slices/chat, src/store/file/slices/fileManager

• • •

The file store state management in Lobe Chat is implemented using Zustand, with the main store defined in …/store.ts. The FileStore type combines various sub-states and actions, including FilesStoreState, FileAction, TTSFileAction, FileManageAction, FileChunkAction, and FileUploadAction.

User Preference and Settings Management

[Edit section]

[Copy link]

References: src/store/user/slices/preference, src/store/user/slices/settings

• • •

User preferences and settings are managed through a combination of selectors and actions in the …/settings and …/preference directories.

Knowledge Base State Management

[Edit section]

[Copy link]

References: src/store/knowledgeBase, src/store/knowledgeBase/slices/ragEval

• • •

The useKnowledgeBaseStore hook manages the state for knowledge base functionality. It provides actions for creating, updating, and removing knowledge base entries, as well as selectors for accessing the current state.

Backend API

[Edit section]

[Copy link]

References: src/app/api, src/server, src/services, Dockerfile, src/app/api/chat/agentRuntime.ts, src/server/globalConfig, src/config/modelProviders/siliconcloud.ts, src/config/modelProviders/zhipu.ts, src/app/api/webhooks/casdoor, CHANGELOG.md, src/libs/agent-runtime/minimax, src/libs/agent-runtime/utils/streams, src/app/api/chat/wenxin, src/config/modelProviders/wenxin.ts, src/libs/agent-runtime/wenxin

• • •

The …/server directory contains the core backend functionality for the Lobe Chat application. It includes several sub-directories and files that handle various aspects of the application's server-side operations.

Asynchronous Processing and File Management

[Edit section]

[Copy link]

References: src/server/routers/async, src/server/services/chunk/index.ts, src/services/file

• • •

The ChunkService class in …/index.ts manages content chunking and embedding for files. Key functionalities include:

Environment Configuration and Global Settings

[Edit section]

[Copy link]

References: src/server/globalConfig/index.ts, src/config

The getServerGlobalConfig() function in …/index.ts manages global configuration settings for the Lobe Chat application's backend services. It integrates various configuration modules and environment variables to create a unified configuration object. Key aspects include:

Docker Configuration and Deployment

[Edit section]

[Copy link]

References: Dockerfile, scripts/serverLauncher/startServer.js

• • •

The Docker configuration for the Lobe Chat application is designed to create a secure and efficient production environment. The Dockerfile utilizes a multi-stage build process, starting with a node:20-slim base image and installing essential packages like ca-certificates and proxychains-ng. A significant focus is placed on security by preparing a distroless directory, which is a minimal runtime environment that contains only the application and its runtime dependencies. This approach minimizes the attack surface of the container by excluding unnecessary packages and files.

Database Management

[Edit section]

[Copy link]

References: src/database, src/database/server/schemas/lobechat, src/database/server/models, src/database/server/models/chunk.ts, src/server/modules/S3, src/types/files, src/types/knowledgeBase, src/types/chunk

• • •

The …/database directory contains the core functionality for managing the local database in the Lobe Chat application. This includes the implementation of the BrowserDB class, which extends the Dexie.js library to create and manage a versioned IndexedDB database, as well as the functionality for migrating user settings from previous versions of the application.

Database Schemas and Models

[Edit section]

[Copy link]

References: src/database/server/schemas/lobechat

• • •

The Lobe Chat application's database schema and models are structured to handle various entities such as user settings, session groups, files, messages, plugins, topics, and users. The schema definitions use the Drizzle ORM for PostgreSQL databases.

Server-Side Database Management

[Edit section]

[Copy link]

References: src/database/server/schemas/lobechat, src/database/server/models

• • •

The …/lobechat directory contains schema definitions for the Lobe Chat application using the Drizzle ORM. Key tables include:

Session Management

[Edit section]

[Copy link]

References: src/database/server/models/session.ts

• • •

In the Lobe Chat application, session management is centralized within the SessionModel class located at …/session.ts. This class is equipped with methods to handle various aspects of session data within the database, utilizing the drizzle-orm library for database interactions.

Chunk Model and File Management

[Edit section]

[Copy link]

References: src/database/server/models/chunk.ts

• • •

The ChunkModel class provides essential methods for managing text chunks associated with files in the Lobe Chat application's database. Located in …/chunk.ts, one of its key methods is deleteOrphanChunks, which plays a significant role in maintaining the integrity of the file management system. Here's how it functions:

RAG Evaluation Data Models

[Edit section]

[Copy link]

References: src/database/server/models/ragEval

• • •

The EvalDatasetModel, EvalDatasetRecordModel, EvalEvaluationModel, and EvaluationRecordModel classes in …/ragEval manage RAG evaluation data. These models interact with the serverDB instance using drizzle-orm for database operations.

Utility Functions

[Edit section]

[Copy link]

References: src/utils, src/utils/tokenizer, src/utils/imageToBase64.test.ts, src/utils/toolManifest.ts

The …/utils directory contains utility functions and classes that provide functionalities to the Lobe Chat application. These utilities cover tasks such as image handling, URL manipulation, trace handling, configuration, platform detection, unique identifier generation, data handling, and other common utility needs.

Configuration Management

[Edit section]

[Copy link]

References: src/utils/config.ts

• • •

The …/config.ts file in the …/utils directory provides utility functions for exporting and importing configuration files in the Lobe Chat application. The file defines the structure of the configuration file, including different types of configurations such as agents, sessions, settings, and a combined "all" configuration.

Error Handling and Data Fetching

[Edit section]

[Copy link]

References: src/utils/fetch

• • •

The …/fetch directory contains utilities for handling HTTP requests and responses, with a focus on fetching data using Server-Sent Events (SSE) and managing smooth animations for message display.

Platform and Environment Detection

[Edit section]

[Copy link]

References: src/utils/platform.ts, src/utils/platform.test.ts

• • •

In …/platform.ts, a set of utility functions and variables are provided to ascertain the platform, browser, and environment specifics of the user's device. The UAParser is utilized for parsing the user agent string to extract this information.

Data Comparison and Transformation

[Edit section]

[Copy link]

References: src/utils/difference.ts, src/utils/difference.test.ts

• • •

In …/difference.ts, the difference() function is responsible for identifying and returning the differences between two objects. It operates by:

Unique Identifier Generation

[Edit section]

[Copy link]

References: src/utils/uuid.ts

• • •

The …/uuid.ts file provides utilities for generating unique identifiers, which are essential for various operations within the Lobe Chat application that require distinct values, such as session identifiers or temporary file names. Two methods are available for generating these identifiers:

Model Parsing and Transformation

[Edit section]

[Copy link]

References: src/utils/parseModels.ts, src/utils/parseModels.test.ts

• • •

In …/parseModels.ts, the utility functions handle the conversion of model strings into structured data, which is essential for the dynamic configuration of chat models within the Lobe Chat application. The functions include:

Data Formatting Utilities

[Edit section]

[Copy link]

References: src/utils/format.ts, src/utils/format.test.ts

• • •

In …/format.ts, four utility functions are provided to convert numerical data into a user-friendly format. The formatSize() function translates byte values into a string representation using KB, MB, or GB units, depending on the size. Similarly, formatSpeed() converts bytes per second into a readable string with units appropriate for the speed, such as B/s, KB/s, MB/s, or GB/s. For time values, formatTime() turns seconds into a string that displays the duration in seconds, minutes, or hours. Additionally, formatTokenNumber() takes a number as input and returns a string representation of the number in a formatted way, using 'K' for thousands and 'M' for millions.

Local Storage Management

[Edit section]

[Copy link]

References: src/utils/localStorage.ts

• • •

The …/localStorage.ts file contains the AsyncLocalStorage class, which is designed to facilitate asynchronous interactions with the browser's local storage. This class provides an abstraction layer for data persistence in the web environment, offering a structured approach to local storage management.

Open Graph Tag Generation

[Edit section]

[Copy link]

References: src/utils/genOG.ts

• • •

In the …/genOG.ts file, two functions are provided to ensure Open Graph tags have appropriately sized titles and descriptions, which are crucial for sharing content on social media platforms with clear and concise metadata.

Tool Call Name Generation

[Edit section]

[Copy link]

References: src/utils/toolCall.ts

• • •

The …/toolCall.ts file includes the genToolCallingName() function, which generates unique names for tool calls within the Lobe Chat application. The uniqueness is achieved by hashing the tool's name when necessary to adhere to length constraints.

Enhanced State Subscription Management

[Edit section]

[Copy link]

References: src/utils/zustand.ts

• • •

The …/zustand.ts file introduces the StoreApiWithSelector interface, an enhancement to the Zustand library's StoreApi. This interface is pivotal for developers requiring precise control over state changes within their applications. It achieves this by providing a custom subscribe method, which allows for subscriptions to specific slices of the application state, rather than the entire state object.

JSON Parsing and Validation

[Edit section]

[Copy link]

References: src/utils/safeParseJSON.ts, src/utils/safeParseJSON.test.ts

• • •

The …/safeParseJSON.ts file introduces safeParseJSON, a utility function designed to parse JSON strings into JavaScript objects. The function ensures safe parsing by returning undefined when encountering invalid JSON strings or non-string inputs. This approach prevents runtime errors that would typically arise from using JSON.parse() directly on malformed JSON data.

Asynchronous String Tokenization

[Edit section]

[Copy link]

References: src/utils/tokenizer

• • •

In the Lobe Chat application, the directory …/tokenizer is dedicated to the task of encoding strings asynchronously. The process is designed to be efficient and responsive, utilizing web workers on the client side for strings up to 50,000 characters, and deferring to server-side processing for longer strings to avoid performance bottlenecks.

File Download Utility

[Edit section]

[Copy link]

References: src/utils/client/downloadFile.ts

• • •

The downloadFile function in …/downloadFile.ts facilitates file downloads from URLs:

Image Processing Utilities

[Edit section]

[Copy link]

References: src/utils/imageToBase64.ts, src/utils/imageToBase64.test.ts

• • •

The …/imageToBase64.ts file contains two key functions for image processing:

Type Definitions

[Edit section]

[Copy link]

References: src/types, src/libs/agent-runtime/types/type.ts, src/types/tool/builtin.ts, src/types/user/settings/keyVaults.ts, src/types/agent/index.ts, src/types/message/index.ts, src/types/asyncTask.ts, src/types/files, src/types/knowledgeBase, src/types/chunk, src/types/rag.ts

The …/types directory provides the core data models, configurations, and types used throughout the Lobe Chat application. This includes the definition of types for agents, messages, settings, tools, trace events, and various other aspects of the application.

Authentication and User Management

[Edit section]

[Copy link]

References: src/types/next-auth.d.ts, src/types/user/index.ts

• • •

Within the Lobe Chat application, user authentication and session management are handled through the Session and JWT types, as defined in …/next-auth.d.ts. The Session type includes a user property, which contains a firstName attribute, allowing the application to maintain the user's first name as part of the session data.

Agents

[Edit section]

[Copy link]

References: src/types/agent/index.ts

• • •

The …/agent directory contains the configuration and type definitions for the Lobe agent, a key component of the Lobe chat application. The main file, …/index.ts, defines the interfaces and types that describe the configuration options for the agent, including settings for text-to-speech (TTS) services, language model parameters, and plugin management.

Messages

[Edit section]

[Copy link]

References: src/types/message/index.ts, src/types/message/tools.ts

• • •

The …/message directory contains the core data models and types related to chat messages in the Lobe application. It defines the structure and metadata associated with chat messages, including information about errors, translations, and text-to-speech (TTS).

Tools

[Edit section]

[Copy link]

References: src/types/tool/builtin.ts, src/types/tool/index.ts

• • •

The …/tool directory contains the types and interfaces related to the tools used in the Lobe Chat application. This includes the definition of built-in tools, custom plugins, and general tool metadata.

Trace Events

[Edit section]

[Copy link]

References: src/types/trace

The …/action.ts file defines the types and payloads for various trace events used in the Lobe Chat application. These trace events are used to record and track changes made to messages, such as modifications, deletions, regenerations, and copying.

Error Handling and Validation

[Edit section]

[Copy link]

References: src/types/fetch.ts, src/types/message/tools.ts

• • •

Within the Lobe Chat application, error handling and validation are managed through a set of defined types and schemas, ensuring that errors are consistently handled and that data conforms to expected structures. The …/fetch.ts file introduces the ChatErrorType object, which enumerates possible error scenarios that the application might encounter. These errors are categorized into business, client-side HTTP, and server-side HTTP errors, each associated with specific conditions such as InvalidAccessCode, Unauthorized, or InternalServerError.

Large Language Models (LLM)

[Edit section]

[Copy link]

References: src/types/llm.ts

• • •

Within …/types, the file …/llm.ts is dedicated to defining the structures and interfaces that facilitate the integration and management of Large Language Models (LLM) within the Lobe Chat application. The file includes several key elements that serve to configure and represent the various aspects of LLMs:

Import and Export Functionality

[Edit section]

[Copy link]

References: src/types/importer.ts

• • •

The Lobe Chat application facilitates data interchange through a set of types and interfaces defined in …/importer.ts. The import functionality is structured around sessions and messages, with a focus on the integrity and organization of the imported data.

Server Configuration

[Edit section]

[Copy link]

References: src/types/serverConfig.ts

• • •

The …/serverConfig.ts file centralizes the server-side configuration for the Lobe Chat application, encapsulating settings for language model providers, server configurations, and system agent settings. It defines several key interfaces and types that shape how the server behaves and interacts with different components of the application.

Service Operations

[Edit section]

[Copy link]

References: src/types/service.ts

• • •

Within the Lobe Chat application, service operations are encapsulated by the BatchTaskResult interface, which is central to representing the outcomes of batch operations. Located in …/service.ts, this interface is pivotal for tracking the progress and results of tasks that process multiple items in a single operation.

Key Vault Configuration

[Edit section]

[Copy link]

References: src/types/user/settings/keyVaults.ts

Within the Lobe Chat application, key vault configurations are managed through a set of interfaces defined in …/keyVaults.ts. These interfaces are designed for storing and managing the credentials required to interact with various AI services. The key vault types supported include:

Model Provider Types

[Edit section]

[Copy link]

References: src/libs/agent-runtime/types/type.ts

• • •

In the Lobe Chat application, the handling of different AI model providers is facilitated through the ModelProvider enum defined in …/type.ts. This enum includes a comprehensive list of supported model providers, each represented as an enum value. The addition of the Taichu value to the ModelProvider enum signifies the integration of a new provider, expanding the application's capabilities in terms of the AI services it can utilize.

Sync Types

[Edit section]

[Copy link]

References: src/types/sync.ts

In the Lobe Chat application, synchronization is managed through a set of types and enums defined in …/sync.ts. The synchronization process is crucial for ensuring that users have a consistent experience across different devices and sessions.

Session Types

[Edit section]

[Copy link]

References: src/types/session/index.ts

• • •

In the Lobe Chat application, session management is facilitated through the types and interfaces defined in …/index.ts. The ChatSessionList interface is a key component that structures the session data, bundling both session groups and individual chat sessions. This organization allows for a coherent representation of sessions within the application, enabling efficient access and manipulation of session-related information.

File Management Types

[Edit section]

[Copy link]

References: src/types/files

• • •

The …/files directory defines key data structures for file management:

Knowledge Base Types

[Edit section]

[Copy link]

References: src/types/knowledgeBase

• • •

The …/index.ts file defines key types and enums for knowledge base management:

Chunk Types

[Edit section]

[Copy link]

References: src/types/chunk

• • •

The ChunkDocument interface defines the structure of text chunks within the application. Key properties include:

Asynchronous Task Types

[Edit section]

[Copy link]

References: src/types/asyncTask.ts

• • •

The …/asyncTask.ts file defines enumerations and interfaces for managing asynchronous tasks:

Retrieval-Augmented Generation (RAG) Types

[Edit section]

[Copy link]

References: src/types/rag.ts

• • •

The SemanticSearchSchema defines the structure for semantic search requests in the chat application. It includes fields for:

Wenxin AI Integration Types

[Edit section]

[Copy link]

References: src/libs/agent-runtime/wenxin/type.ts

• • •

The integration of Wenxin AI into the Lobe Chat application is facilitated through a set of TypeScript interfaces that define the structure of responses and token usage for the AI's interactions. The TokenUsage interface captures details about the number of tokens consumed during the generation of prompts and completions, providing insight into the cost efficiency of interactions with the Wenxin AI. This information is crucial for monitoring and managing the usage of AI resources, ensuring that the application remains within operational limits.

Reusable Components

[Edit section]

[Copy link]

References: src/features/FileViewer, src/features/Conversation, src/features/FileSidePanel, src/features/KnowledgeBaseModal, src/types/files, src/types/knowledgeBase, src/app/(main)/chat/(workspace)/@portal/FilePreview, src/app/(main)/settings/llm/components/ProviderConfig, src/app/(main)/settings/llm/components/ProviderModelList, src/components/ModelSelect, src/components/FileParsingStatus, src/app/(main)/chat/(workspace)/@portal/features, src/app/(main)/chat/(workspace)/@portal/Plugins, src/components/DragUpload, src/components/FunctionModal, src/components/GuideModal

• • •

The Lobe Chat application leverages a set of reusable React components to facilitate various functionalities such as file viewing, chat interactions, and knowledge base management. Central to these features is the FileViewer component located at …/FileViewer, which dynamically renders different file types including PDFs, text files, images, and Microsoft Office documents.

File Management Components

[Edit section]

[Copy link]

References: src/features/FileManager, src/features/FileViewer, src/components/DragUpload

• • •

The FileManager component in …/index.tsx orchestrates file management functionality, integrating subcomponents:

Artifact Rendering Components

[Edit section]

[Copy link]

References: src/app/(main)/chat/(workspace)/@portal/Artifacts/Body/Renderer/React

• • •

In the Lobe Chat application, the ReactRenderer component plays a crucial role in the visualization of code artifacts within the chat workspace. It is designed to provide an interactive sandbox environment where users can view and engage with code snippets directly. The component is optimized with memoization to avoid unnecessary re-renders, enhancing performance.

Knowledge Base Management Components

[Edit section]

[Copy link]

References: src/features/KnowledgeBaseModal

• • •

The KnowledgeBaseModal directory contains components for managing knowledge bases in Lobe Chat. Key components include:

Branding and Customization Components

[Edit section]

[Copy link]

References: src/components/Branding/OrgBrand, src/components/Branding/ProductLogo, src/components/Branding/WelcomeLogo

• • •

The Lobe Chat application's branding and customization components are pivotal in establishing the visual identity of the platform. The OrgBrand component, located at …/index.tsx, serves as a conditional renderer that either displays the organization's name or defaults to the Lobe Hub branding, depending on the isCustomORG flag. This flag is a crucial determinant for the component's rendering path, which is either a simple text span or the LobeHub component for the default branding.

Contributing Guidelines

[Edit section]

[Copy link]

References: contributing, contributing/Home.md, contributing/Basic/Feature-Development.md, contributing/Basic/Intro.zh-CN.md, contributing/Basic/Intro.md, contributing/Basic/Feature-Development.zh-CN.md

The Contributing Guidelines section provides documentation and guidelines for developers who want to contribute to the Lobe Chat project. This section offers information about the project's architecture, feature development, state management, and internationalization.

Contributing Process

[Edit section]

[Copy link]

References: contributing/Basic, contributing/Home.md

• • •

Contributors to the LobeChat project must adhere to established coding standards, which include consistent code formatting and style. The project employs tools such as ESLint, Prettier, remarklint, and stylelint to enforce these standards. These tools are configured with the @lobehub/lint package to ensure uniformity across JavaScript, Markdown, and CSS codebases.

Feature Development Process

[Edit section]

[Copy link]

References: contributing/Basic/Feature-Development.md, contributing/Basic/Feature-Development.zh-CN.md

• • •

The LobeChat application feature development, such as the sessionGroup, adheres to a structured process encompassing data model creation and UI integration.

Introduction to LobeChat

[Edit section]

[Copy link]

References: contributing/Basic/Intro.md, contributing/Basic/Intro.zh-CN.md

LobeChat leverages the Next.js framework, providing a scalable foundation for server-side rendering and static site generation, which enhances SEO and performance. The application's UI leverages Ant Design for a comprehensive suite of high-quality React components that ensure a consistent and professional user interface. Custom UI components are managed within the lobe-ui library, allowing for tailored user experiences.

Local Development Environment Setup

[Edit section]

[Copy link]

References: contributing/Basic/Intro.md

• • •

To set up the local development environment for the LobeChat application, developers should start by cloning the repository from its source. This is achieved by executing the git clone command with the appropriate repository URL. Once the repository is cloned, the next step involves installing all the necessary dependencies. This is done by running the command within the root directory of the project, which is lobe-chat.

Code Style and Contribution Guide

[Edit section]

[Copy link]

References: contributing/Basic/Intro.md

• • •

Maintaining a consistent code style and adhering to a defined contribution process is vital for collaborative development in the LobeChat project. The project leverages several tools to enforce coding standards and facilitate a smooth contribution workflow:

Internationalization Implementation

[Edit section]

[Copy link]

References: contributing/Basic/Intro.md

• • •

LobeChat leverages i18next and lobe-i18n for internationalization, enabling the application to support multiple languages. The implementation facilitates the dynamic loading of language resources, which allows for the addition of new languages without significant changes to the application's core functionality.

Resources and References

[Edit section]

[Copy link]

References: contributing/Basic/Intro.md

Developers can leverage a variety of resources to understand and utilize the LobeChat technology stack effectively. The …/Intro.md serves as a gateway to these resources, providing essential links and references.

File Management and Upload

[Edit section]

[Copy link]

References: src/services/file, src/services/upload.ts, src/services/__tests__/upload_legacy.test.ts, src/types/files, src/app/(main)/chat/(workspace)/@portal/FilePreview, src/app/(main)/files/(content)/@modal/(.)[id], src/const/file.ts, src/features/FileManager, src/features/FileViewer, src/server/utils/files.ts

• • •

The file management system in Lobe Chat provides a unified interface for handling files across client and server environments. The fileService object, defined in …/index.ts, abstracts the differences between client-side and server-side implementations, allowing seamless file operations regardless of the runtime environment.

Client-Side File Management

[Edit section]

[Copy link]

References: src/services/file/client.ts

• • •

The ClientService class manages file operations on the client-side using local storage. Key functionalities include:

Server-Side File Management

[Edit section]

[Copy link]

References: src/services/file/server.ts

• • •

The ServerService class in …/server.ts implements the IFileService interface, providing server-side file operations using a tRPC API and S3 storage. Key functionalities include:

Unified File Service Interface

[Edit section]

[Copy link]

References: src/services/file/index.ts, src/services/file/type.ts

• • •

The fileService object provides a unified interface for file operations, abstracting the differences between client and server implementations. This abstraction is achieved through the IFileService interface, which defines a common set of methods for file management:

File Upload Functionality

[Edit section]

[Copy link]

References: src/services/upload.ts

• • •

The UploadService class in …/upload.ts provides two primary methods for file uploads:

File Preview and Detail Display

[Edit section]

[Copy link]

References: src/app/(main)/chat/(workspace)/@portal/FilePreview, src/app/(main)/files/(content)/@modal/(.)[id]

• • •

The FilePreview component renders file previews within the chat workspace portal. It uses useChatStore to access the previewFileId and useFetchFileItem from useFileStore to fetch file data. When data is available, it renders a Flexbox containing a FileViewer component with the file data.

Chat Input File Handling

[Edit section]

[Copy link]

References: src/features/ChatInput

• • •

The FileUpload component in …/ClientMode.tsx and …/ServerMode.tsx handle file uploads within the chat input. Key aspects include:

File Manager Components

[Edit section]

[Copy link]

References: src/features/FileManager

• • •

The FileManager component orchestrates file-related functionality, incorporating:

File Viewer Components

[Edit section]

[Copy link]

References: src/features/FileViewer

• • •

The FileViewer component in …/index.tsx renders different file viewers based on file type. It uses the DocViewer from @cyntler/react-doc-viewer for various file types.

Server-Side File Utilities

[Edit section]

[Copy link]

References: src/server/utils/files.ts

• • •

The getFullFileUrl function in …/files.ts generates complete URLs for files stored in S3-compatible storage. It takes into account the configuration settings specified in the fileEnv object:

File Type Definitions

[Edit section]

[Copy link]

References: src/types/files

• • •

The …/files directory defines key data structures for file management:

File Constants

[Edit section]

[Copy link]

References: src/const/file.ts

• • •

The FILE_UPLOAD_BLACKLIST constant in …/file.ts defines a list of file extensions that are not allowed to be uploaded. Currently, it only includes .DS_Store, a system file used by macOS to store custom attributes of its containing folder.

Database Migration Scripts

[Edit section]

[Copy link]

References: scripts/migrateServerDB

• • •

The database migration scripts for the Lobe Chat application are located in the …/migrateServerDB directory. These scripts handle the process of updating the server-side database schema as the application evolves.

RAG Evaluation Tables

[Edit section]

[Copy link]

References: src/database/server/migrations/0008_add_rag_evals.sql

• • •

The SQL migration script creates four tables to store RAG evaluation data:

Migration Error Handling

[Edit section]

[Copy link]

References: scripts/migrateServerDB

• • •

Error handling during database migrations is implemented in the …/migrateServerDB directory. The primary focus is on detecting and reporting issues related to the pgvector extension in PostgreSQL.

Text Processing and Query Chains

[Edit section]

[Copy link]

References: src/chains

• • •

The …/chains directory contains a collection of functions that generate ChatStreamPayload objects for various text processing tasks. These functions are designed to work with language models to perform tasks such as language detection, emoji selection, and content summarization.

Query Rewriting and Context-Based Answering

[Edit section]

[Copy link]

References: src/chains/rewriteQuery.ts, src/chains/answerWithContext.ts

The chainRewriteQuery function in …/rewriteQuery.ts generates a ChatStreamPayload object for rewriting queries based on conversation context. It takes a follow-up question and previous messages as input, constructing two messages:

Text Abstraction and Summarization

[Edit section]

[Copy link]

References: src/chains/abstractChunk.ts, src/chains/summaryDescription.ts, src/chains/summaryTitle.ts, src/chains/summaryTags.ts

The chainAbstractChunkText function in …/abstractChunk.ts generates a ChatStreamPayload for text summarization. It constructs messages for a language model, including system instructions and the input text to be summarized.

Language Processing and Translation

[Edit section]

[Copy link]

References: src/chains/langDetect.ts, src/chains/translate.ts, src/chains/pickEmoji.ts

• • •

The chainLangDetect function in …/langDetect.ts detects the language of input text. It returns a ChatStreamPayload object containing:

Agent Name Generation

[Edit section]

[Copy link]

References: src/chains/summaryAgentName.ts

• • •

The chainSummaryAgentName function generates a chat stream payload for creating concise and poetic names for design or artistic works. It constructs a series of messages that define the agent's behavior:

Drag and Drop File Upload

[Edit section]

[Copy link]

References: src/components/DragUpload, src/features/ChatInput

• • •

The drag-and-drop file upload functionality in Lobe Chat is implemented using the useDragUpload hook and the DragUpload component. This feature enhances user experience by allowing direct file uploads through drag-and-drop interactions.

DragUpload Component

[Edit section]

[Copy link]

References: src/components/DragUpload

• • •

The DragUpload component, defined in …/index.tsx, renders a custom UI overlay when a file is being dragged over the application. It utilizes the useDragUpload hook from …/useDragUpload.tsx to manage the drag-and-drop state and handle file uploads.

File Upload Handling

[Edit section]

[Copy link]

References: src/services/upload.ts

The UploadService class in …/upload.ts provides two primary methods for file uploading:

Chat Input File Integration

[Edit section]

[Copy link]

References: src/features/ChatInput/ActionBar/Upload

• • •

The FileUpload component in …/ClientMode.tsx integrates file upload functionality within the chat input interface. It uses the Upload component from Ant Design to handle file uploads and determines the user's upload capabilities based on the current agent model.

File Type Restrictions

[Edit section]

[Copy link]

References: src/const/file.ts

• • •

The FILE_UPLOAD_BLACKLIST constant in …/file.ts defines a list of file extensions that are prohibited from being uploaded. Currently, the blacklist only includes .DS_Store files, which are system files created by macOS to store custom attributes of a folder.

Asynchronous Server Operations

[Edit section]

[Copy link]

References: src/libs/trpc, src/server/routers/async, src/app/(backend)/trpc

• • •

The Lobe Chat application implements asynchronous server operations using tRPC (TypeScript-RPC) for efficient and type-safe communication between the client and server. The core functionality is defined in the …/trpc directory, with additional implementation in …/async.

Asynchronous Router Setup

[Edit section]

[Copy link]

References: src/server/routers/async

• • •

The asynchronous router is set up in …/index.ts using the trpc library. It includes:

JWT Authentication Middleware

[Edit section]

[Copy link]

References: src/libs/trpc/middleware/jwtPayload.ts

• • •

The jwtPayloadChecker middleware serves as a security layer for the Lobe Chat application's server operations, particularly for those that are asynchronous. It operates within the tRPC framework, utilizing the trpc.middleware() function to intercept incoming requests and validate the presence and correctness of JWT tokens. Here's how it functions:

File Processing

[Edit section]

[Copy link]

References: src/server/routers/async/file.ts

• • •

The fileRouter in …/file.ts handles two primary asynchronous file operations:

Knowledge Base Management

[Edit section]

[Copy link]

References: src/server/routers/async

• • •

The asyncRouter in …/index.ts handles asynchronous operations for knowledge base management. It incorporates the fileRouter from …/file.ts, which provides two key mutations:

RAG Evaluation

[Edit section]

[Copy link]

References: src/server/routers/async/ragEval.ts

• • •

The ragEvalRouter handles asynchronous operations for RAG evaluations. Key functionalities include:

tRPC Client Implementation

[Edit section]

[Copy link]

References: src/libs/trpc/client/async.ts

• • •

The asyncClient is created using createTRPCClient from the tRPC library, configured to interact with the AsyncRouter interface. This client enables making asynchronous requests to the server-side tRPC endpoints.

Knowledge Base Management

[Edit section]

[Copy link]

References: src/store/knowledgeBase, src/types/knowledgeBase, src/app/(main)/discover

• • •

The Knowledge Base functionality in Lobe Chat is implemented using a combination of state management, data structures, and asynchronous operations. The core of this functionality is defined in the …/knowledgeBase directory, which contains the store implementation, actions, and selectors for managing knowledge base items.

Knowledge Base Store Management

[Edit section]

[Copy link]

References: src/store/knowledgeBase

The Knowledge Base store is implemented using Zustand, with the main store creation logic in …/store.ts. The store combines state and actions from multiple slices:

Knowledge Base Integration with Chat

[Edit section]

[Copy link]

References: src/features/Conversation, src/features/ChatInput

• • •

Knowledge bases are integrated into the chat interface through components in …/Conversation. The ChatItem component renders individual messages, including those related to knowledge base interactions. File uploads and semantic search functionality are handled within the chat input and message rendering components.

Knowledge Base Data Structures

[Edit section]

[Copy link]

References: src/types/knowledgeBase

• • •

The …/index.ts file defines key data structures for representing knowledge base items and their metadata:

Asynchronous Knowledge Base Operations

[Edit section]

[Copy link]

References: src/server/routers/async, src/server/routers/async/ragEval.ts

• • •

The asyncRouter in …/index.ts handles asynchronous operations for knowledge base management. It incorporates the following routers:

RAG Evaluation Management

[Edit section]

[Copy link]

References: src/app/(main)/repos/[id]/evals, src/services/ragEval.ts, src/database/server/models/ragEval

• • •

The RAG evaluation system allows users to create, manage, and execute evaluations for knowledge bases. Key components include:

AI Model Provider Configuration

[Edit section]

[Copy link]

References: src/config/modelProviders, src/libs/agent-runtime, src/app/(main)/settings/llm/ProviderList, src/app/api/chat/agentRuntime.ts

• • •

The AI Model Provider Configuration in Lobe Chat is implemented through a series of TypeScript files in the …/modelProviders directory. Each file defines a ModelProviderCard object for a specific AI model provider, containing detailed information about the available models and their capabilities.

SiliconCloud Models

[Edit section]

[Copy link]

References: src/config/modelProviders/siliconcloud.ts

• • •

SiliconCloud offers a suite of language models designed to accelerate the development of Artificial General Intelligence (AGI). Each model within the SiliconCloud platform is characterized by a unique set of capabilities, catering to a variety of tasks such as instruction-following, chat, and specialized domains like mathematics and coding. The SiliconCloud object, an instance of ModelProviderCard, serves as a reference for accessing these models, providing essential metadata for integration and user interaction.

Zhipu Models

[Edit section]

[Copy link]

References: src/config/modelProviders/zhipu.ts

• • •

The ZhiPu AI platform is integrated into the Lobe Chat application through the ZhiPu object, which serves as a model provider card detailing the offerings of the platform. The card includes a collection of chat models, each with a unique identifier and display name, along with a description that outlines the model's capabilities. These models are flagged for their availability (enabled), and their ability to support function calls (functionCall) and vision tasks (vision).

Google Gemini Models

[Edit section]

[Copy link]

References: src/config/modelProviders/google.ts

• • •

The Lobe Chat application integrates the Google Gemini series of AI models, offering advanced multi-modal capabilities. The configuration for these models is defined within the Google object in the …/google.ts file. This object includes a chatModels array, which lists various Gemini models, each with a set of properties that describe their features and usage constraints.

OpenRouter Models

[Edit section]

[Copy link]

References: src/config/modelProviders/openrouter.ts

• • •

The OpenRouter service platform serves as a gateway to a diverse range of large language models, offering a centralized access point for models from providers like OpenAI, Google, Anthropic, and others. Each model within the OpenRouter platform is meticulously described, highlighting its unique capabilities, token limits, and whether it supports vision tasks or function calls. The OpenRouter object, defined in …/openrouter.ts, is structured as a ModelProviderCard, which is instrumental in the Lobe Chat application for configuring and displaying model options to users.

Taichu Models

[Edit section]

[Copy link]

References: src/config/modelProviders/taichu.ts

• • •

The Taichu platform introduces the language model designed to excel in language understanding and text generation tasks. This model, identified by the unique identifier taichu_llm, is configured to handle a significant token limit of 32,768, accommodating extensive language inputs and outputs. The object in …/taichu.ts encapsulates the model's capabilities and serves as the central configuration point within the Lobe Chat application.

Spark Models

[Edit section]

[Copy link]

References: src/config/modelProviders/spark.ts

The Spark object in …/spark.ts configures a suite of Spark language models, each tailored for distinct use cases and performance needs within the Lobe Chat application. The models range from Spark Lite, designed for efficiency on low-power devices, to Spark 4.0 Ultra, which boasts advanced text understanding and summarization capabilities. These models are characterized by their varying token processing limits, from the streamlined Spark Lite to the expansive Spark Pro 128K, capable of handling long-form content up to 131,072 tokens.

OpenAI Models

[Edit section]

[Copy link]

References: src/config/modelProviders/openai.ts

• • •

The OpenAI object in …/openai.ts defines configurations for various OpenAI models, including GPT-3.5 and GPT-4 variants. Each model is represented by an object in the chatModels array with the following key properties:

MiniCPM-V Models

[Edit section]

[Copy link]

References: src/config/modelProviders/ollama.ts

• • •

MiniCPM-V models, specifically the MiniCPM-V 8B, are integrated into the Lobe Chat application, offering advanced capabilities such as OCR recognition and multimodal understanding. These models are part of the array of chatModels provided by Ollama, each designed for specific applications ranging from code generation to conversational interactions.

Fireworks AI Models

[Edit section]

[Copy link]

References: src/config/modelProviders/fireworksai.ts

• • •

The FireworksAI object in …/fireworksai.ts configures a suite of AI chat models specific to the Fireworks AI platform. Each model within the chatModels array is characterized by a unique identifier (id), a descriptive name (displayName), and a detailed explanation of its capabilities (description). The models are designed to cater to various functionalities, with some offering vision-language tasks (vision) and others supporting function calling (functionCall). The token limits (tokens) for each model are specified, indicating the maximum input length the model can process.

Bedrock Models

[Edit section]

[Copy link]

References: src/config/modelProviders/bedrock.ts

• • •

The Lobe Chat application integrates AWS Bedrock models through the Bedrock object, which is defined in …/bedrock.ts. This object includes an array of chatModels, each representing a different language model provided by Bedrock. These models are characterized by their unique id, descriptive displayName, and a detailed description that outlines the model's capabilities and performance benchmarks. The Bedrock object also specifies the pricing for each model, which is essential for users to understand the cost associated with the model's usage.

Mistral Models

[Edit section]

[Copy link]

References: src/config/modelProviders/mistral.ts

The Mistral AI platform offers a suite of language models, each tailored with specific capabilities to enhance the Lobe Chat application's conversational AI features. The configuration for Mistral models is encapsulated within the Mistral object, which is part of the ModelProviderCard type, indicating its role as a configuration entity for language model providers within the application. The Mistral object includes a collection of models such as Mistral 7B, Mixtral 8x7B, Mistral Nemo, Mistral Small, Mistral Large, Codestral, Codestral Mamba, and Pixtral 12B. Each model is described with a display name, providing users with the necessary information to select the most appropriate model for their needs.

AI360 Models

[Edit section]

[Copy link]

References: src/config/modelProviders/ai360.ts

• • •

The ModelProviderCard for AI360 models, defined in …/ai360.ts, encapsulates the configurations for chat models offered by AI360. Key aspects of this configuration include:

Anthropic Models

[Edit section]

[Copy link]

References: src/config/modelProviders/anthropic.ts

• • •

The Anthropic object in …/anthropic.ts defines a set of Claude models with their specific capabilities:

Azure AI Models

[Edit section]

[Copy link]

References: src/config/modelProviders/azure.ts

• • •

The Azure AI models within the Lobe Chat application are configured through the Azure object, which is a type of ModelProviderCard. This object contains a collection of chatModels, each representing a distinct AI model offered by Azure. The models include gpt-35-turbo, gpt-35-turbo-16k, gpt-4-turbo, gpt-4-vision, gpt-4o-mini, and gpt-4o. Each model is equipped with a set of properties that define its capabilities and limits, such as maxOutput and tokens, which dictate the maximum length of generated text and the token consumption per request, respectively.