Architectural Refactor for Project Structure

[kzndotsh]

Architectural Refactor for Project Structure

The Problem

As Tux has grown, our project structure within the tux/ directory has evolved organically. While functional, it has reached a point where related components are spread out, and the separation between core logic, user-facing features, and utilities is not as clear as it could be. This can increase the cognitive load for new contributors and make maintenance and scaling more difficult in the long run.

The Goal

The purpose of this discussion is to collaboratively decide on a new, more intentional architecture for the project. The ideal structure should:

• Improve maintainability and readability.
• Provide a clear separation of concerns (e.g., backend vs. Discord features).
• Be scalable for future growth (such as adding a web dashboard or API).
• Enhance the overall developer experience.

The Proposals

Below are nine various proposals for restructuring the tux/ directory. They range from simple refactors to patterns inspired by community standards, modern web apps, and formal software architecture. These proposals do not imply automatic consideration because of their existence, they are simply a means of inspiration/brainstorm/example to get the discussion flowing.

Please review each one. They are collapsed for readability.

---

Proposal 1: The "core" and "features" Split

A straightforward approach that divides the application into two main packages: core for backend logic and features for user-facing components.

Structure (Proposal 1)

tux/
├── core/                 # Backend, services, and infrastructure
│   ├── app.py            # (from tux/app.py)
│   ├── bot.py            # (from tux/bot.py)
│   ├── database/         # (from tux/database/*)
│   ├── handlers/         # (from tux/handlers/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   └── utils/            # (from tux/utils/*)
│
├── features/             # User-facing Discord cogs and UI
│   ├── cogs/             # (from tux/cogs/*)
│   ├── ui/               # (from tux/ui/*)
│   └── help.py           # (from tux/help.py)
│
├── cli/                  # (from tux/cli/*)
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 1)

• Simplicity: Very easy to understand the high-level separation.
• Clear Division: Creates a strong boundary between what is "internal" and "external."

Cons (Proposal 1)

• Generic Naming: features is a bit vague and could become a miscellaneous dumping ground.
• Awkward Dependencies: Shared utilities might be difficult to place without creating circular dependencies or duplication.

---

Proposal 2: The "bot" and "services" Split

This option frames the structure around the two main parts of the application: the bot itself (all Discord-related code) and the backend services that support it.

Structure (Proposal 2)

tux/
├── bot/                  # All Discord-facing logic
│   ├── cogs/             # (from tux/cogs/*)
│   ├── ui/               # (from tux/ui/*)
│   ├── core.py           # (from tux/bot.py)
│   └── help.py           # (from tux/help.py)
│
├── services/             # Backend systems and external API wrappers
│   ├── database/         # (from tux/database/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   ├── sentry/           # (from tux/utils/sentry_manager.py)
│   └── task_manager/     # (from tux/utils/task_manager.py)
│
├── cli/                  # (from tux/cli/*)
├── utils/                # Shared utilities (from tux/utils/constants.py, exceptions.py, etc.)
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 2)

• Intuitive: The separation between bot and services is conceptually clean and easy to grasp.
• Scalable: New services or bot features have a clear home.
• Shared utils: A dedicated, top-level utils directory solves the problem of common, shared code.

Cons (Proposal 2)

• Slightly More Complex: Introduces one more top-level directory than Proposal 1, which slightly increases complexity.

---

Proposal 3: The Hyper-Specific Functional Split

This is the most granular approach, creating specific top-level directories for each major function of the application. It follows a more rigorous domain-driven design philosophy.

Structure (Proposal 3)

tux/
├── core/                 # The absolute essential components (Bot class, app setup)
│   ├── bot.py            # (from tux/bot.py)
│   └── app.py            # (from tux/app.py)
│
├── discord/              # All direct Discord API interactions and features
│   ├── cogs/             # (from tux/cogs/*)
│   ├── ui/               # (from tux/ui/*)
│   └── handlers/         # (from tux/handlers/*)
│
├── systems/              # Backend logic and external service integrations
│   ├── database/         # (from tux/database/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   └── tasks/            # (from tux/utils/task_manager.py)
│
├── cli/                  # (from tux/cli/*)
├── utils/                # Shared utilities (from tux/utils/constants.py, etc.)
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 3)

• Extremely Organized: Every component has a very specific, well-named home.
• Highly Scalable: This pattern is used in very large codebases and scales well as complexity grows.

Cons (Proposal 3)

• Over-Structuring: Might be too much for the current size of the project. Can make it harder for new contributors to find things.
• Increased Cognitive Load: More directories to remember and navigate.

---

Proposal 4: The "app" Monolith with "ext"

This model keeps most of the application together in a main app directory but moves all external-facing components (like Discord cogs and the CLI) into an ext (extensions) directory. This is similar to the pattern used by frameworks like Flask or Django.

Structure (Proposal 4)

tux/
├── app/                  # The core application monolith
│   ├── bot.py            # (from tux/bot.py)
│   ├── database/         # (from tux/database/*)
│   ├── handlers/         # (from tux/handlers/*)
│   ├── services/         # (from tux/wrappers/*, tux/utils/task_manager.py)
│   └── utils/            # (from tux/utils/*)
│
├── ext/                  # Extensions that plug into the app
│   ├── cli/              # (from tux/cli/*)
│   ├── cogs/             # (from tux/cogs/*)
│   └── ui/               # (from tux/ui/*)
│
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 4)

• Framework-like: Follows a pattern common in popular web frameworks.
• Clear Entry Points: ext contains all the ways a user or developer "enters" the application.

Cons (Proposal 4)

• Monolithic app: The app directory itself can become large and less organized over time.
• Less Intuitive: The distinction between app and ext may not be as immediately clear as bot vs. services.

---

Proposal 5: The "src" Layout with Domain Grouping

This approach uses a standard "src" layout and groups code by its domain or high-level feature, rather than its technical function.

Structure (Proposal 5)

src/
└── tux/
    ├── __main__.py
    ├── bot.py              # Core bot setup (from tux/bot.py)
    ├── cli.py              # Core CLI setup (from tux/cli/core.py)
    │
    ├── common/             # Shared utilities, database, wrappers
    │   ├── database/       # (from tux/database/*)
    │   └── utils/          # (from tux/utils/*)
    │
    ├── moderation/         # All moderation-related code (cogs, logic, etc.)
    │   ├── cog.py          # (combines files from tux/cogs/moderation/*)
    │   └── services.py     # (logic that might be in the cog today)
    │
    ├── fun/                # Fun commands and features
    │   ├── cog.py          # (combines files from tux/cogs/fun/*)
    │   └── services.py
    │
    └── guild/              # Guild-specific features
        ├── cog.py          # (combines files from tux/cogs/guild/*)
        └── config_view.py  # (from tux/ui/views/config.py)

Pros (Proposal 5)

• Feature-Oriented: Organizes code around what it does for the user, making it easy to work on a single feature.
• Encapsulation: Keeps all code related to a feature (Discord commands, backend logic, UI) in one place.

Cons (Proposal 5)

• Unconventional: This is a less common pattern and might be confusing for developers used to technical layering (/cogs, /services).
• Potential for Duplication: Common logic might be duplicated across feature packages if not carefully managed in common.

---

Proposal 6: The "Web App" / Next.js Inspired Structure

This model borrows concepts from modern web frameworks like Next.js, organizing the project by features ("routes") and co-locating related code. It emphasizes a strong separation between feature logic, reusable UI components, and shared libraries.

Structure (Proposal 6)

tux/
├── app/                  # "Routes" or features, like Next.js's app router
│   ├── moderation/       # Each feature is a self-contained unit
│   │   ├── commands.py   # (from tux/cogs/moderation/*)
│   │   ├── views.py      # (specific views, e.g. from tux/ui/views/tldr.py if it were a command)
│   │   └── actions.py    # (logic currently inside the command functions)
│   │
│   └── ... (other cogs as feature folders)
│
├── components/           # Reusable UI components (like React components)
│   ├── embeds.py         # (from tux/ui/embeds.py)
│   ├── views.py          # (generic views, e.g. tux/ui/views/confirmation.py)
│   └── modals.py         # (from tux/ui/modals/*)
│
├── lib/                  # Shared libraries and utilities
│   ├── database/         # (from tux/database/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   └── utils/            # (from tux/utils/*)
│
├── core/                 # Core application setup (the "root layout")
│   ├── bot.py            # (from tux/bot.py)
│   ├── app.py            # (from tux/app.py)
│   └── handlers/         # (from tux/handlers/*)
│
├── cli/                  # (from tux/cli/*)
├── main.py
└── ...

Pros (Proposal 6)

• Modern & Scalable: This pattern is popular in web development for a reason; it scales very well and is intuitive for developers familiar with these frameworks.
• Co-location: Keeping feature-specific logic, commands, and UI together makes development on a single feature much faster and easier to reason about.
• Clear Separation: There's a very strong distinction between features (app), reusable UI (components), shared logic (lib), and application setup (core).

Cons (Proposal 6)

• Unconventional for Bots: This is not a typical structure for a Discord bot project. It might require more initial setup and explanation for contributors unfamiliar with web development patterns.
• Potential for Boilerplate: Each new feature in app/ might require creating several files (commands.py, views.py, etc.), which could feel repetitive.

---

Proposal 7: The Hybrid "Monorepo-Ready" Structure (Community Inspired)

This proposal is a synthesis of the most common and scalable patterns observed across dozens of other major Discord bots, combined with modern architectural practices. It is designed to be immediately familiar, highly organized, and ready to scale into a full-stack application (e.g., with a website or API).

Structure (Proposal 7)

tux/
├── bot/                  # Contains all code for the Discord bot application.
│   │                     # This makes it easy to add `website/` or `api/` later.
│   ├── core/             # Essential bot setup, client, global checks, and handlers.
│   │   ├── client.py     # (from tux/bot.py)
│   │   ├── checks.py     # (from tux/utils/checks.py)
│   │   └── handlers.py   # (from tux/handlers/event.py, error.py)
│   │
│   ├── features/         # Replaces 'cogs'. Holds all user-facing commands, grouped by feature.
│   │   ├── moderation/   # (from tux/cogs/moderation/ban.py, kick.py, etc.)
│   │   ├── fun/          # (from tux/cogs/fun/fact.py, etc.)
│   │   └── utility/      # (from tux/cogs/utility/ping.py, afk.py, etc.)
│   │
│   ├── components/       # Reusable UI components (embeds, views, modals).
│   │   ├── embeds.py     # (from tux/ui/embeds.py)
│   │   └── views.py      # (from tux/ui/views/*)
│
├── services/             # Backend logic, external API wrappers, database clients.
│   │                     # These are kept separate as they could be used by other apps (website, etc).
│   ├── database/         # (from tux/database/*)
│   ├── sentry/           # (from tux/utils/sentry_manager.py)
│   └── wrappers/         # (from tux/wrappers/*)
│
├── shared/               # Python code shared by ALL applications (bot, cli, future web).
│   ├── constants.py      # (from tux/utils/constants.py)
│   ├── exceptions.py     # (from tux/utils/exceptions.py)
│   └── types.py          # (new file for shared data types)
│
├── cli/                  # (from tux/cli/*)
└── main.py

Pros (Proposal 7)

• Community Standard: Aligns closely with the successful patterns seen in other large bots, making it easier for new contributors to understand.
• Monorepo-Ready: The top-level bot/ directory provides a clean namespace, making it trivial to add a website/ or api/ directory at the same level in the future.
• Extremely Clear Separation: The roles of bot/, services/, and shared/ are unambiguous.
• Clean Feature Development: Working on a feature in bot/features/ is self-contained, but it has easy access to reusable UI from bot/components/ and logic from services/.

Cons (Proposal 7)

• Slightly More Verbose Imports: Imports might be slightly longer (e.g., from bot.features import moderation), but this explicitness is generally considered a good thing for clarity.

---

Proposal 8: The "Clean/Hexagonal" Architecture

This is a highly disciplined, academic approach based on patterns like Clean Architecture. The goal is maximum separation of concerns, making the core application logic completely independent of external frameworks (like Discord.py) and services (like the database). The "core" of the application knows nothing about the "outside world."

Structure (Proposal 8)

tux/
├── domain/                 # The absolute core. Pure Python objects and business rules.
│   │                       # NO discord.py, database, or API logic here.
│   ├── moderation/
│   │   ├── entities.py       # e.g., A `Case` dataclass (replaces prisma model)
│   │   └── repository.py     # Defines an *interface* for how to save a `Case`
│   │
│   └── ...
│
├── application/            # The "use cases." Orchestrates the domain logic.
│   ├── services/
│   │   └── moderation.py   # `ModerationService` with `ban_member()` logic
│   │
│   └── dto.py              # e.g., `BanMemberDTO` for carrying data from discord to service
│
├── infrastructure/         # The "outside world." Frameworks, drivers, and tools.
│   ├── discord/            # Adapter for Discord
│   │   ├── cogs/           # e.g., `moderation.py` (from tux/cogs/moderation/ban.py)
│   │   ├── bot.py          # The bot instance (from tux/bot.py)
│   │   └── ui/             # (from tux/ui/*)
│   │
│   ├── database/           # Adapter for the database
│   │   └── repositories/   # *Implements* the domain repository with prisma-client-py
│   │
│   └── cli/                # Adapter for the CLI (from tux/cli/*)
│
└── main.py                   # Wires everything together (Dependency Injection).

How it Works (Proposal 8)

1. A command in infrastructure/discord/cogs/moderation.py is invoked.
2. It calls a method in the application/services/moderation.py service.
3. The service uses pure domain/moderation/entities.py objects and calls methods on a repository interface defined in domain/moderation/repository.py.
4. The actual database work is done by the concrete implementation in infrastructure/database/repositories/, which was "injected" into the service when the bot started.

Pros (Proposal 8)

• Maximum Decoupling: The core logic can be reused if you ever switch frameworks or add a gRPC API.
• Supreme Testability: Unit test the application layer without mocking Discord or database objects.
• Enforced Boundaries: Impossible to accidentally mix database code with command logic.

Cons (Proposal 8)

• Very High Complexity & Boilerplate: This is a heavyweight pattern for large, enterprise-scale applications.
• Unconventional for Bots: Steep learning curve for new contributors.
• Potential Over-Engineering: Benefits may not be realized without concrete plans to reuse the core logic on multiple platforms.

---

Proposal 9: The "Self-Contained Packages" (Modular Plugins) Structure

This approach treats every feature of the bot as a complete, self-contained Python package. The main application is merely a lightweight shell responsible for discovering and loading these "plugins."

Structure (Proposal 9)

tux/
├── packages/             # A directory containing all the installable feature packages
│   ├── moderation/
│   │   ├── __init__.py   # Defines the package's public API and a `setup` function
│   │   ├── commands.py   # (from tux/cogs/moderation/*)
│   │   ├── services.py   # (new file for logic currently in cogs)
│   │   └── models.py     # (conceptual mapping from prisma/schema/moderation.prisma)
│   │
│   └── ...
│
├── core/                 # The application shell and shared infrastructure
│   ├── bot.py            # The bot instance; discovers and runs `setup` from each package
│   ├── database.py       # (from tux/database/client.py)
│   ├── ui/               # (generic UI like tux/ui/views/confirmation.py)
│   └── utils/            # (generic utils from tux/utils/*)
│
├── cli.py                # (from tux/cli/core.py)
└── main.py

How it Works (Proposal 9)

1. On startup, the core/bot.py scans the packages/ directory.
2. For each package (e.g., moderation), it imports it and calls a well-defined entry point function, like setup(bot).
3. The moderation package's setup function is responsible for adding its own cogs, views, and listeners to the bot instance. It gets any dependencies like a database session from the bot.

Pros (Proposal 9)

• Ultimate Modularity & Reusability: Features are truly independent and could be shared between bots.
• Clear Ownership & Boundaries: Perfect for larger teams to own entire features.
• Extensibility: Adding a new feature is as simple as dropping a new, compliant package into the packages directory.

Cons (Proposal 9)

• High Initial Overhead: Requires a more complex package loading and dependency injection mechanism.
• Discovery Challenges: Finding where functionality lives means first identifying the responsible package.
• Cross-Package Communication: Direct communication between packages can be complex.

---

Request for Feedback

We invite all maintainers and contributors to review these proposals and provide your thoughts. To help guide the discussion, please consider the following:

1. Which proposal(s) do you prefer, and why?
2. Which proposal(s) do you dislike, and why?
3. Are there any modifications you would suggest to one of the existing proposals?
4. What are the most important factors for you in a project structure (e.g., ease of finding files, scalability, low complexity, contributor friendliness)?

Your feedback is crucial for setting a clear direction for the future of Tux. Thank you!

[arutonee1]

I think in general, more structure is better.* I personally find it easier to find things in a project when I don't have to guess as to where it is, and one of the easiest ways to achieve this is with a highly structured hierarchical setup.

If we are optimizing for approachability from a new contributor perspective, I think it would make the most sense to decouple technical details from the structuring as much as possible, but the way described in the "src" method is way too prone to duplication and general confusion, especially since there's already a wealth of unused or duplicated code in the current codebase. This method would also make it a bit harder to make "drop-in" features- I personally think one should be able to derive a single class or apply a single decorator and have the feature just work, without any extra poking around.

A combination of the self-contained packages (9) and hyper-specific split (3) is what I would lean towards, though there are definitely examples of the other structures working out if designed properly. The core/features split (1) also looks appealing, but may need some care when implementing.

* Of course, to a certain degree- it's very easy to get caught up in restructuring and forget to write any meaningful code. I do still think that being as specific as possible is definitely worth it though ^w^

[HikariNee]

I personally really like Proposal 3, albeit with a few changes.

For one, discord should be named something like exts for clarity (as cogs are like extensions for the bot).

I also feel that discord/ui and discord/handlers should be moved to a common directory (or utils) at the top level, as they're shared between different cogs.

I don't think we should shoehorn everything into the tux directory though, for example the tux CLI isn't a part of the bot, but rather an interface for it. It'd be much cleaner to have in its own directory at the top level in the parent of tux.

In essence, what I am saying is,

cli/
tux/
tux/core
tux/common
tux/exts
tux/utils
tux/main.py

I do think the core naming is a bit of a misnomer though, as usually main.py is associated with being the core program.

[RainzDev]

For proposal 2, I do like that idea. When it comes to organizing, I believe that moving files around to the appropriate directories would look great and understandable for new maintainers. Same goes for proposal 1. Even though it can be quite vague in some cases, I don't think it's a bad idea.

[ela-codes]

I find the structure in #7 - The Hybrid "Monorepo-Ready" structure provides the most value for the bot's current status and makes room for long-term goals.

#1 and #2 are simpler, great for contribution and is easier to get acquainted with but it lacks the clarity and flexibility that #7 offers.

For example,

dropping "everything" (for a lack of a better word) into a general-purpose folders like features or services might be okay in the short term but it will become tech debt as we grow. It's easy to start treating them as catch-all folders, iykwim (which makes navigation and maintenance harder over time).

what I like about the hybrid structure is its explicitness and it's intentional. So there is a top-level folder that separates between core components such as bot/, services/, and shared/. The subfolders inside them make it easy to locate what we're looking for. If we need to ever add another top-level component like api/ or web/, this structure allows for that growth without needing to restructure again.

i also agree with the concern about verbose imports. That said, we can manage this with consistent naming conventions or even import aliases to avoid excessive verbosity while keeping things... readable.

that's my 2 canadian cents. great discussion opener btw!!

[anemoijereja-eden]

I love 9 for the clearly encapsulated code and division of responsibility. making the project more loosely integrated and having every bit of code related to a module right there inside it does, in my opinion, make it far easier for contributors to understand and work with the modules. the discovery step would add some overhead, though not enough to be that inconvenient, check https://github.com/anemoijereja-eden/chandragen to see some examples (the formatter system loads every single bit of formatting code as modules and still runs pretty fast) That overhead also only applies on initialization, as soon as the bot is running all the code is in cache.

I do, however, understand not wanting to go for such a drastic architecture shift. for a simple organization change rather that a full design paradigm shift I personally find 7 the most attractive. it cleanly divides responsibilities for sections of the code and leaves plenty of room for expansion. getting some things out of the tux/ dir is definitely a huge plus.

[Bxelz]

Proposal 2 would be my go-to. In my opinion, splitting the two components apart really helps take the pressure off when you're just focused on a specific part of the bot (like a command). You don’t have to worry about all the backend you might not be touching, and I think it'll just make everything feel more organized and easier to get around.

Then again, structure 9 basically does the same thing, with a more detailed and descriptive base layout. And honestly, I do sort of prefer the way 9's command layout is handled, although a little contradictory to what I just said for 2.

[midischwarz12]

Preface

First, I want to start by saying I'm not a super-active contributor to this project. So I don't have a large intuition on what is tightly coupled. Maybe y'all (@kzndotsh and @electron271) can chime in on that part and help answer questions. I don't want to "rollplay" as some active contributor as I've only sent in a few PRs and have a shallow view of the codebase. I want to give a more conditional response because of my absense of experience and understanding. Especially since there's really two major contributors to this project, I really believe that the others commenting on this issue (especially those that have made 0 contributions) should do the same.

From all of the projects that I've maintained, shipped, etc, there's been one common theme. Coupling and abstraction go hand and hand. As you abstract two or more components, they become tightly coupled together. And vice versa, components that are less abstracted or generalized together will have a looser coupling.

If you want a visual explanation of this idea, I deeply recommend watching CodeAesthetic's video on "Abstraction".

My idea here is that things that naturally end up very tightly coupled should be organized closely together. And things that are less coupled should naturally be more separate.

---

Now whether it's best to make these "less coupled" components into packages, sub-directories, entirely different repos, etc, I can chime a lot more on. That said, it's conditional.

Proposal

In reality, no single proposal alone will be decided on. It will most likely be a little mix of a few proposals. So I think it is important to address each one.

Features

If your "features" mentioned in Proposal 1 are less coupled together, I think it would be best to have them as each as individual sub-directories under the parent directory bot, app, core, or whatever it gets named. The structure would more like this:

bot/                 # Backend, services, and infrastructure
├── app.py            # (from tux/app.py)
├── bot.py            # (from tux/bot.py)
├── database/         # (from tux/database/*)
├── handlers/         # (from tux/handlers/*)
├── wrappers/         # (from tux/wrappers/*)
├── utils/            # (from tux/utils/*)
├── cogs/             # (from tux/cogs/*)
├── ui/               # (from tux/ui/*)
└── help.py           # (from tux/help.py)

I do not believe it is beneficial to have it under a separate features directory unless there is some sort of abstraction or generalization connecting them together. Regardless, even if there needed to be a features directory to hold them together, I think it's still best under the bot / app / core directory since these features are (from my understanding) being directly consumed by the rest of the app. That key part of being directly consumed means these are really nothing more than modules in my head.

Moreover, if there is some shared logic and abstraction, I would create the features directory and try to reuse that amongst all of the features.

Services

The "services" mentioned in Proposal 2 seem to be like wrappers of external processes, systems, etc or just API handling for external services. That said, I'm not sure if there's anything coupling them together. They mostly seem like independent parts. I can imagine that if there is some coupling, it isn't even between all the services. Related services that show some coupling should be organized together.

Hyper-Specific

I don't think there should be a "core" directory. Just put it under tux directory which I really think should just be src or app.

I would really go and question if "utils" is even utilities.

Again, if cli functionality is somewhat coupled, I'd go and organize that together which it seems so.

Extensions

I don't have much to say here besides there's no runtime plugin system so I don't think this matters. All pros found here can be more or less found in the other proposals. If you are actually wanting to design a "framework" design, I'd actually go and make a discord bot framework and have that as a separate thing. I would never go and half-bake that idea. And I really only see benefit to that if you're reusing that framework across multiple bots. Maybe you guys have a long term goal of doing that. But if that's the case, that framework, per se, is a separate project that should be in its own repo. It would be best if it was not a part of a single specifc bot. It would also be awkward for other bots to have to use a framework that is part of some monolith deeply integrated into some discord bot repository. So go 100% or don't at all.

Domains

Again, I would go with what's naturally coupled here instead of enforcing arbitrary organization.

My intuition is that fun is just a directly of miscellaneous fun command that don't actually share a lot of logic. If that's the case, I wouldn't have this directory at all and just let each module live on its own.

I would also assume that guild and `moderations would have some level of shared logic thus coupling. If so, I'd actually have these sub directories exist.

See "Web App" for my thoughts on common which I really see as lib.

"Web App" like

See my comments on "Extensions". If it's a framework, make it a separate, external, generalized framework. If not, don't lean into it at all.

However, I do like the idea of a lib directory for shared logic that is directly consumed. But I would keep this only to functions, types, etc that you're using all throughout your code base at a general level. This is not for logic specific to any coupling of components. Keep those components coupled and that code near those components. If you're interacting with the database a lot all through your codebase and you have your own specific functions or interface for interacting with it, this is a perfect spot for it. But again, this is based on how coupled this functionality is to other components, how generalized, and how directly consumed it is.

Monorepo-ready

See my concern for "Services" on services. And shared is really lib, so see my concerns for "Web App" on lib.

Clean

I think this aims to create separation of concerns, but in reality, it separates coupled ideas and separates things with the same concern. If anything, I think this is separating more arbitrarily and less about "separation of concerns". This enforced de-abstracting things which naturally are related and would force you to repeat yourself instead of promoting code reuse. You'll also have longer tree traversals to access similar things.

Side note: I think it's funny that this is a "Clean Code" architecture b/c it goes against some of the "Clean Code" principles like DRY.

Self-Contained Packages

If you have packages, actually make separate python packages and import them. They should be version tracked separately in separate repositories. If that is too much separation b/c of how little the functionality is or because of how coupled it is, I wouldn't make separate packages at all.

Conclusion

Please ping / tag me and respond with the missing information that my response requires. I think the next step would be to take all of the ideas from these proposals we like (not necessarily the proposals as a whole) and plan next on how to mix and merge these ideas into a cohesive mass.

[anemoijereja-eden]

I'm thinking that a hybrid between 7 and 9 would actually be great, as a "tightly integrated monorepo with support for fully modular extensions"

• all of the core commands and services would be well integrated with each other, providing base systems that act as a backbone
• a module loader would be written to handle extensions to the bot, with support for loading self-contained modules per #9
• minor cogs, like the fun cogs, would be moved out of the core and into the extension system
• extensions can provide database table models, but they would be handled separately from the main database tables, and therefore not considered in migrations.
• core services like afk, moderation cases, automod backing systems, etc would provide APIs for extension modules to use

effectively, it's #9 but with the scope of the "core" bot system expanded out. we could ship a few modules with the base bot, but then leave the rest of it up to the end user. we could maintain a few official modules, while keeping the actual scope of what the modules do fairly small. the use of a unified API for it makes contributing easier, because a contributor would not need to have an understanding of the full scope of the bot itself, just the module layout and extension APIs.

[anemoijereja-eden]

the module loading system would also ideally have some configuration options per-module that could be changed at runtime for things like channel restrictions, role level restrictions, etc.

[meatsnails]

I'm most in favor for proposal 9 i think having a modular system would benefit any self-hosters who wants only some features and not others while also allowing for custom features that might not fit into the main bot/repo for example something like a "fishboard" separate from the starboard. and to add onto this a module skeleton or example repo (using git submodule) possibly to show how one could be made

[electron271]

yes i think proposal 9 would be best, may also let us expand on #704

i also do like the idea from @anemoijereja-eden about a hybrid of proposal 9 and 7

[pawbtism]

A mix of 7 and 9 looks best to me. A monorepo but instead of "features" to replace cogs there are installable "packages".

[kzndotsh]

Tux Architecture: Modular, Scalable, and Minimal

Issue: #924 Architectural Refactor for Project Structure

---

Introduction & Goals

Tux is designed for clarity, extensibility, and ease of contribution. The architecture separates core logic, infrastructure, features, and user add-ons, while keeping the root directory minimal and clean.

---

Finalized Directory Structure

tux/
├── core/
│   ├── __init__.py
│   ├── app.py
│   ├── bot.py
│   ├── cog_loader.py
│   ├── config.py
│   ├── env.py
│   ├── ui/
│   │   ├── __init__.py
│   │   ├── embeds.py
│   │   ├── buttons.py
│   │   ├── help_components.py
│   │   ├── views/
│   │   └── modals/
│   └── utils/
│       ├── __init__.py
│       ├── functions.py
│       ├── constants.py
│       ├── exceptions.py
│       ├── ascii.py
│       ├── regex.py
│       ├── emoji.py
│       ├── converters.py
│       ├── checks.py
│       ├── flags.py
│       ├── help_utils.py
│       └── banner.py
├── infra/
│   ├── __init__.py
│   ├── database/
│   │   ├── __init__.py
│   │   ├── client.py
│   │   └── controllers/
│   ├── logger.py
│   ├── sentry.py
│   ├── wrappers/
│   │   ├── __init__.py
│   │   ├── godbolt.py
│   │   ├── wandbox.py
│   │   ├── github.py
│   │   ├── xkcd.py
│   │   └── tldr.py
│   ├── hot_reload.py
│   └── handlers/
│       ├── __init__.py
│       ├── error.py
│       ├── sentry.py
│       ├── event.py
│       └── activity.py
├── modules/
│   ├── __init__.py
│   ├── moderation/
│   ├── fun/
│   ├── info/
│   ├── admin/
│   ├── snippets/
│   ├── levels/
│   ├── guild/
│   ├── services/
│   ├── tools/
│   ├── utility/
│   └── ...
├── custom_modules/
│   └── ...
├── cli/
│   ├── __init__.py
│   └── ...
├── assets/
│   ├── emojis/
│   ├── branding/
│   ├── embeds/
│   └── ...
├── main.py
└── tests/
    ├── core/
    ├── infra/
    ├── modules/
    ├── cli/
    └── ...

---

Rationale for the Structure

• Minimal root-level clutter: Only essential directories at the top level.
• Clear separation:
  • core/: Orchestration, startup, shared UI, helpers.
  • infra/: Infrastructure (DB, logging, wrappers, hot reload, infra handlers).
  • modules/: All official, loadable feature modules/cogs/extensions.
  • custom_modules/: For self-hosters' add-ons (local or as submodules).
  • cli/: CLI tools, dev commands.
  • assets/: Static files, images, emojis, etc.
  • main.py: Entrypoint.
  • tests/: Mirrors the main structure for clarity and maintainability.
• Extensible: Easy to add new features, infra, or custom modules.
• Contributor-friendly: Logical, discoverable, and ready for growth.

---

Proposals Reviewed

As Tux has grown, the need for a more intentional, scalable, and contributor-friendly project structure has become clear. Multiple proposals were presented, ranging from simple splits to modular plugin architectures. This document summarizes the discussion and feedback from contributors and presents a final, consensus-driven proposal.

---

Extensibility Model: Modules & Custom Modules

Tux supports two types of extensions:

1. Modules (Official/Core Features)

• Location: modules/
• Maintained by: Tux team (public, tracked in git)
• Loaded by default: Yes
• Purpose: Core features, public bot functionality, main way to extend Tux
• How to contribute: Add a new folder in modules/ and submit a PR

2. Custom Modules (User/Server-Specific Add-ons)

• Location: custom_modules/ (at project root, .gitignored)
• Maintained by: Self-hoster/forker (private, not tracked in git)
• Loaded by default: Only if present
• Purpose: Server-specific, private, or experimental features; allows self-hosters to add features without touching the main codebase
• How to use: Drop a .py file or package in custom_modules/ and restart the bot
• Submodules: You can add custom modules as git submodules for versioned, shared, or organization-managed add-ons.

---

How Modules Work (Official Features)

• Each module is a Python package (folder with __init__.py) in modules/.
• Modules are implemented as discord.py extensions and typically register one or more cogs.
• Modules are always loaded by the bot at startup.
• Example module layout:

modules/
  moderation/
    __init__.py
    commands.py
    services.py
    tasks.py
    models.py
    README.md

---

How Custom Modules Work (Self-Hoster Add-ons)

• Place .py files or extension packages in custom_modules/.
• These are loaded automatically if present, but are not tracked in git (unless you use a submodule).
• Submodules: You can add a custom module repo as a submodule in custom_modules/ for versioned, shared, or private extensions.
• .gitignore: custom_modules/ is .gitignored by default, so your local or submodule code is never committed to the main repo.

Loader Logic Example:

import os
import glob
from discord.ext import commands

bot = commands.Bot(command_prefix="!")

# Load official modules
for mod in ["modules.moderation", "modules.fun", "modules.starboard"]:
    bot.load_extension(mod)

# Load all custom modules (if any)
if os.path.isdir("custom_modules"):
    # Load .py files
    for file in glob.glob("custom_modules/*.py"):
        modname = f"custom_modules.{os.path.splitext(os.path.basename(file))[0]}"
        bot.load_extension(modname)
    # Load packages (folders with __init__.py)
    for folder in os.listdir("custom_modules"):
        folder_path = os.path.join("custom_modules", folder)
        if os.path.isdir(folder_path) and os.path.isfile(os.path.join(folder_path, "__init__.py")):
            modname = f"custom_modules.{folder}"
            bot.load_extension(modname)

# IMPORTANT: Also load all cogs/extensions in infra/handlers/ just like modules and custom_modules
if os.path.isdir("infra/handlers"):
    for file in glob.glob("infra/handlers/*.py"):
        modname = f"infra.handlers.{os.path.splitext(os.path.basename(file))[0]}"
        bot.load_extension(modname)
    for folder in os.listdir("infra/handlers"):
        folder_path = os.path.join("infra/handlers", folder)
        if os.path.isdir(folder_path) and os.path.isfile(os.path.join(folder_path, "__init__.py")):
            modname = f"infra.handlers.{folder}"
            bot.load_extension(modname)

How to add a custom module as a submodule:

git submodule add https://github.com/yourorg/tux-my-cool-module custom_modules/my_cool_module
git submodule update --init --recursive

After updating or cloning, run:

git submodule update --init --recursive

---

Unified Summary Table

Feature	modules/ (Module)	custom_modules/ (Custom Module)	custom_modules/ (Submodule)	infra/handlers/ (Infra Cog)
Public/Official	✅	❌	❌	✅
Always loaded	✅	❌ (only if present)	❌ (only if present)	✅ (if present)
Git-tracked	✅	❌ (.gitignored)	✅ (as submodule ref)	✅
For self-hosters	❌	✅	✅	❌ (unless forked)
For PRs	✅	❌	❌	✅
Versioned	Main repo	Local only	Separate repo	Main repo

---

Handlers and Cog Loader

• Cog-based infra handlers (e.g., error, event, sentry, activity) live in infra/handlers/ and are dynamically loaded by the cog loader (core/cog_loader.py), just like modules and custom_modules. This is a key part of the architecture: infra/handlers/ is treated as a source of loadable cogs/extensions, not just static infra.
• Feature modules live in modules/ and are also loaded by the cog loader.
• Custom modules in custom_modules/ are supported for self-hosters and loaded the same way.

---

Database Considerations

• Core-only: Most modules and custom modules should use the core's database API/models for common needs.
• Advanced: Advanced users can define models in their modules or custom modules. The bot can be designed to discover and register these models at startup, but migrations require careful management.
• Isolated: For small, isolated data, modules can use local files (e.g., SQLite, JSON).

---

Tests

• All tests live in tests/, mirroring the main structure for discoverability and maintainability.
• Example: tests/core/, tests/infra/, tests/modules/, etc.

---

Conclusion

This structure provides a clean, scalable, and contributor-friendly foundation for Tux, supporting both official features and self-hosted extensions, with clear separation of concerns and minimal root-level clutter.

---

References:

• Issue #924: Architectural Refactor for Project Structure
• discord.py Cogs and Extensions
• Jishaku: Drop-in Extension Example
• discord.py-ext-prometheus: Drop-in Monitoring Example