Turning a Discord Server into Searchable Memory, Without Losing a Single Deletion

31.10.2026•

What I learned building a Discord connector for cognee: day documents, invisible edits, and the zero-width characters that almost corrupted emoji.

*Messages are grouped by channel and day, synced incrementally, and forgotten when they are deleted.*

Most teams have made an important decision in a Discord channel and then lost it a week later under a few hundred messages. Discord search helps if you remember the exact words. It does not help if you want to ask “What did we decide about the release?”

For Mergetober, cognee’s integration hackathon, I built a Discord connector for issue #4729. It syncs a server’s channels, threads and forum posts into cognee, so you can ask questions across the whole server. The pull request is cognee-community #357.

Here is how it works, and the problems that only showed up once a reviewer ran it hard.

The basics in one paragraph

cognee is an open-source memory layer for AI agents: cognee.remember(...) turns documents into a knowledge graph, and cognee.recall(...) answers questions from it. dlt is a Python data loading library. The connector is a dlt resource, a generator that yields documents, plus "tombstone" rows that tell cognee to forget something. cognee takes that resource directly.

import os
import cognee
from cognee_community_connector_discord import discord_source
await cognee.remember(
    discord_source(
        guild_id=os.environ["DISCORD_GUILD_ID"],
        token=os.environ["DISCORD_BOT_TOKEN"],
    ),
    dataset_name="discord",
    primary_key="id",
    write_disposition="merge",  # REQUIRED, the add pipeline defaults to "replace"
    max_rows_per_table=0,
    self_improvement=False,
)
answer = await cognee.recall("What did we decide about the release?", datasets=["discord"])

Why one document per channel per day

A single Discord message is usually too small to mean anything on its own. “Agreed, ship it” needs the messages around it. A whole channel as one document is the opposite problem: every new message would make cognee re-process months of history.

So the connector groups messages into one document per channel or thread per UTC day, with an id like discord:<channel id>:2026-10-21. A conversation stays together, and a new message only re-processes that one day.

Inside a day, the text is cleaned up so it reads well for a language model:

User, role and channel mentions like <@123> become real names.
Custom emoji become :name:.
Attachments are kept as file names only, because Discord’s CDN links are signed and expire.
Replies are marked, thread starters show the original message, and system messages like “pinned a message” are skipped.
Rich embeds, stickers and polls are kept as text. This came from review: a channel fed only by a GitHub or CI webhook posts embeds with no plain text, and the first version ingested nothing from it.

A Discord channel from your test server, next to the day document the connector produced for it.

The edit you cannot see

Discord’s API pages through messages with a cursor: “give me messages after this id”. That is perfect for new messages. It is blind to everything behind the cursor. If someone edits or deletes a message from yesterday, a forward cursor will never notice, and Discord has no “what changed since” endpoint.

*Every run re-reads the last few days, so edits and deletions in that window are caught.*

The connector handles this with a rescan window. Each run re-reads from the start of the day of min(cursor, now - rescan_days), seven days by default. Edits and deletions in that window update their day's document, and a day whose messages were all deleted is removed from memory.

This is a trade-off, and I wanted to be honest about it in the README. An edit or deletion older than the window goes unnoticed unless you run with full_resync=True. A channel with nothing new and nothing in the window costs zero requests, so the trade keeps quiet servers cheap.

The first review: four things a live server would show

The first review opened with: “The first four are the ones I’d want before merge, they are all things a live server would show.” Every one was about failure modes I had not hit with my test data.

1. Missing permission is not deletion. I had treated Discord error codes 50001 and 50013 (missing access, missing permissions) the same as 10003 (unknown channel). The result: if the bot lost access to one channel, the connector decided the channel was gone and deleted every day ever ingested from it. Now a refused channel is left exactly as it is and counted as unreadable. Only a channel that disappears from the server's channel list is forgotten.

2. One locked channel stopped everything. The archived thread listing was not wrapped like the message reads, so one channel without Read Message History made the whole run fail before syncing anything.

3. Rate limits can be huge. On the global limit, Discord can send retry_after values in the hundreds of seconds. My client slept 60 seconds four times and then crashed. Now a wait over 60 seconds stops the run cleanly, keeps everything finished, and reports failed_rate_limit. 401 and 403 are never retried, because Discord bans IPs that send too many invalid requests.

4. Embeds rendered to nothing, as described above.

The Message Content Intent trap

This one is worth knowing if you ever build a Discord bot. Without the Message Content Intent enabled, Discord does not return an error. It returns messages with an empty content field. A sync would happily ingest a server full of empty days.

So before reading anything, the connector checks GET /applications/@me and raises a clear error unless the intent is enabled. Failing loudly is much better than succeeding quietly with no data.

Zero-width characters, and why the first fix was wrong

Bots and webhooks often fill required embed fields with a zero-width space (U+200B), because Discord rejects empty values. Python’s str.strip() does not remove it, so an "empty" field still looked like content. The reviewer caught that, and I added a small cleaning step that removed zero-width characters.

Then the cleaning step started running on message text too, and the reviewer came back with a better point. My list also included U+200C and U+200D, the zero-width non-joiner and joiner. Those are not filler. They are real parts of text:

The family emoji 👨‍👩‍👧 is three emoji joined with U+200D. Strip it and you get three separate people.
The rainbow flag 🏳️‍🌈 is joined the same way.
Persian words like می‌خواهم use U+200C between letters.

So my “cleanup” was corrupting what we stored. The fix removes only the true filler characters (U+200B, U+2060, U+FEFF) and keeps the joiners inside real text. A value that is nothing but filler and joiners still counts as empty, so a bot’s placeholder field is still dropped. A test now checks the family emoji, the flag and the Persian word in messages, embed fields and polls.

It is a good reminder that Unicode handling is never “just strip it”.

Big servers and the request budget

A big server cannot be synced in one run without hitting rate limits, so every run has a request budget (2,000 by default). The connector syncs channels in rotation, remembers where it stopped, and continues on the next run.

The reviewer found two problems with that. Deletions were only decided after a run walked every channel, so on a big server they might never happen. And a channel that answered 403 counted as “finished”, so a run could make zero real progress while reporting a normal pause. The fixes: deleted and deselected channels are now decided right after discovery, which always completes, and a refused channel no longer counts as progress.

The latest round also made the default sync name per server, discord_<guild id>, so two servers synced into one dataset can never forget each other's channels.

How it was tested

There are 80 tests, and they run on Python 3.10, 3.11 and 3.13 without a Discord account. The fake API follows Discord’s documented behaviour: newest-first pages, archived threads paged by archive_timestamp, the real error codes, and empty history when Read Message History is missing.

I also checked it on a live test server: a first sync, a run with no changes, deletions, a channel losing and regaining Read Message History, and a channel the bot cannot see.

What I would do the same way next time

Treat “access denied” and “deleted” as completely different answers. Mixing them up is the fastest way to destroy someone’s data.
Look for what your cursor cannot see. Forward cursors miss edits and deletions; decide on purpose how you will catch them.
Fail loudly when the data looks impossible. Empty content from a missing intent should be an error, not a silent sync.
Be careful with “cleaning” text. Characters that look invisible can carry meaning.