C++ LLM Chat System Design: Model Management, Session Management, and SQLite Persistence

[AI Readability Summary]

This article focuses on three core pillars of a C++ LLM chat SDK: model management, session management, and SQLite persistence. It addresses common challenges such as integrating multiple models through one interface, preserving multi-turn context, and preventing chat history loss after shutdown. It also presents practical data structures and interface designs you can apply directly. Keywords: C++, Session Management, SQLite.

Technical Item	Specification
Primary Language	C++
Data Storage Protocol	SQLite / SQL
Model Integration Pattern	Provider-based polymorphic abstraction
Persistence Method	Local single-file database
Thread Safety Concerns	Session creation, update, and deletion
GitHub Stars	Not provided in the source
Core Dependencies	sqlite3, optionally cpp-httplib and spdlog

The system reduces multi-model integration complexity through a unified abstraction

The original project already integrates deepseek-chat, gpt-4o-mini, and gemini-2.0-flash. Each model maps to its own Provider class, while LLMManager orchestrates them through a unified interface. The core value of this design is not simply that it can call different models, but that it pushes model-specific differences outside the boundary of the main SDK.

When developers add a new model later, they only need to implement a new Provider and register it with the manager. They do not need to modify upper-layer business logic. This approach keeps routing, authentication, and request format differences inside the Provider layer and preserves a stable SDK interface.

AI Visual Insight: The image shows the interface layout of the LLMManager header file, typically including capabilities such as model registration, Provider lookup by name, and default model configuration. This highlights that its role is to serve as the model access entry point rather than to contain low-level inference logic.

A simplified model manager interface can be designed like this

class LLMManager {
public:
    void registerProvider(const std::string& name, std::shared_ptr
<IProvider> provider); // Register a model provider
    std::shared_ptr
<IProvider> getProvider(const std::string& name); // Get a model by name

private:
    std::unordered_map<std::string, std::shared_ptr<IProvider>> providers_; // Store model mappings
};

This code uses a unified mapping table to hide implementation differences across LLM Providers.

Session management is essential infrastructure for preserving context in multi-turn conversations

Once chat enters the multi-turn stage, message storage, session ownership, history replay, and chronological ordering all become mandatory requirements. The model itself does not maintain application-side context, so the business layer must explicitly introduce a SessionManager.

The original article breaks the requirements down comprehensively: create sessions, generate unique session IDs, generate message IDs, append messages, query sessions, list sessions, delete sessions, update timestamps, clear sessions, and count totals. Together, these capabilities form the state layer of the chat system.

Session data structures should cover both metadata and message collections

At a minimum, a session should contain the session ID, model name, creation time, update time, and message list. If the system supports title renaming, archiving, or summarization, you should also reserve extension fields.

struct Message {
    std::string message_id;   // Unique message ID
    std::string role;         // user / assistant / system
    std::string content;      // Message body
    int64_t timestamp;        // Timestamp
};

struct Session {
    std::string session_id;           // Unique session ID
    std::string model_name;           // Model bound to the current session
    int64_t created_at;               // Creation time
    int64_t updated_at;               // Last update time
    std::vector
<Message> messages;    // Historical messages
};

This code defines the core in-memory model required to support multi-turn conversational context.

AI Visual Insight: The diagram should show the composition relationship between Session and Message, along with the mapping structure from session_id to session objects. It emphasizes that the session management module is fundamentally an in-memory repository built from indexes, metadata, and message collections.

Returning session IDs instead of full objects is a more robust interface design

The original article specifically emphasizes that getSessionLists() returns a list of session IDs rather than a full collection of session objects. This is a strong engineering decision.

First, returning full objects introduces extra copy overhead. Second, if external modules manipulate session objects directly, coupling increases. Third, SQLite queries, deletes, and updates naturally revolve around primary keys, so returning IDs aligns better with CRUD semantics.

Session lists should be sorted by most recent update time

This is a common interaction pattern in chat products. Displaying the most recently used sessions first matches user expectations and makes context recovery easier.

std::vector<std::string> SessionManager::getSessionLists() {
    std::vector<std::pair<std::string, int64_t>> items;
    // Collect session_id and update time
    for (const auto& [id, session] : sessions_) {
        items.push_back({id, session.updated_at});
    }
    // Sort by update time in descending order
    std::sort(items.begin(), items.end(), [](auto& a, auto& b) {
        return a.second > b.second;
    });

    std::vector<std::string> result;
    for (const auto& item : items) {
        result.push_back(item.first); // Return only the session ID
    }
    return result;
}

This code outputs a session ID list prioritized by recent activity while reducing coupling between modules.

SQLite provides a low-cost and sufficiently stable persistence layer for local chat systems

Sessions stored only in memory disappear entirely after the program exits, so persistence is required. Compared with standalone services such as MySQL, SQLite is a better fit for SDK and desktop scenarios: zero configuration, a single file, serverless operation, cross-platform support, and transactional guarantees.

For chat systems, SQLite offers very practical benefits: low deployment cost, no need for users to install a separate database service, and a natural fit for storing sessions and messages locally. Choosing SQLite as the storage backend for a Chat SDK is a classic engineering-first decision.

Installing and linking SQLite on Ubuntu is straightforward

sudo apt install sqlite3          # Install the command-line tool
sudo apt install libsqlite3-dev   # Install the development library

This code installs the SQLite command-line tool and the C/C++ development headers and libraries.

target_link_libraries(${SDK_NAME} sqlite3) # Link the sqlite3 shared library

This code links the SQLite library to the target program in a CMake project.

SQLite prepared statements are well suited for high-frequency message writes

The original article highlights sqlite3_prepare_v2, sqlite3_bind_text, sqlite3_step, and sqlite3_finalize. This API set separates SQL compilation from parameter binding, which reduces string concatenation and improves security.

In a chat system, inserting messages, updating session timestamps, and querying history by session are all typical prepared-statement scenarios. Message content may contain quotes, line breaks, and special characters, so parameter binding is much safer than handwritten SQL.

sqlite3_stmt* stmt = nullptr;
const char* sql = "insert into message(session_id, role, content, ts) values(?, ?, ?, ?)";

sqlite3_prepare_v2(db, sql, -1, &stmt, nullptr);                 // Precompile the SQL statement
sqlite3_bind_text(stmt, 1, sessionId.c_str(), -1, SQLITE_TRANSIENT); // Bind the session ID
sqlite3_bind_text(stmt, 2, role.c_str(), -1, SQLITE_TRANSIENT);      // Bind the role
sqlite3_bind_text(stmt, 3, content.c_str(), -1, SQLITE_TRANSIENT);   // Bind the message content
sqlite3_bind_int64(stmt, 4, timestamp);                              // Bind the timestamp
sqlite3_step(stmt);                                                  // Execute the write
sqlite3_finalize(stmt);                                              // Release resources

This code safely writes one chat message into the SQLite database.

The data management module should be the only bridge between the session layer and the database layer

In the project, DataManager handles database initialization, table creation, insertion, deletion, querying, and updates. SessionManager owns a DataManager object and synchronizes changes to disk whenever a session changes. This creates a two-layer structure made up of in-memory state and persistent state.

The key advantage of this design is clear responsibility separation: SessionManager manages business state, while DataManager manages database operations. If you later replace SQLite with MySQL or PostgreSQL, in theory you only need to replace the data-layer implementation instead of rewriting session logic.

AI Visual Insight: The image should describe the interface boundary of DataManager, such as database connection handling, table creation, session insertion, message writing, history queries, and session deletion. It illustrates its role as a persistence adapter layer.

The recommended database design should contain at least two tables

create table if not exists session (
    session_id text primary key,
    model_name text not null,
    created_at integer not null,
    updated_at integer not null
);

create table if not exists message (
    message_id text primary key,
    session_id text not null,
    role text not null,
    content text not null,
    ts integer not null
);

This schema stores session metadata and message records in separate tables, which improves queryability and maintainability.

FAQ

Why should the model management layer use a Provider + Manager two-layer structure?

Because different models expose different APIs, authentication methods, and response formats. The Provider encapsulates those differences, while the Manager provides unified orchestration. This significantly reduces the cost of adapting upper-layer business logic.

Why is it better for the session list to return session_id instead of the full Session?

Returning IDs is lighter and avoids unnecessary object copies. It also ensures that other modules continue to access data through SessionManager, which prevents broader coupling caused by direct manipulation of session objects.

Why does this project prioritize SQLite over MySQL?

SQLite requires zero configuration, uses a single file, and is easy to distribute. It is especially suitable for SDKs, local desktop applications, and lightweight chat systems. For low- to medium-scale session history storage, it provides a better cost-to-value ratio.

Core Summary: This article reconstructs the core design of a C++ AI chat SDK: it uses LLMManager to unify multi-model Providers, relies on SessionManager to preserve multi-turn conversational context, and uses SQLite to persist sessions and messages. The resulting architecture is well suited for building lightweight, locally deployable, multi-model chat applications.