406 lines
12 KiB
Markdown
406 lines
12 KiB
Markdown
# Feature-Oriented Event-Driven Application Skeleton
|
|
|
|
A Python-based asynchronous application framework designed for building scalable, event-driven systems using a feature-oriented architecture. This skeleton provides a solid foundation for building complex applications with clear separation of concerns, dependency injection, and event-driven communication patterns.
|
|
|
|
## Overview
|
|
|
|
This project implements a feature-oriented architecture where functionality is organized into independent, self-contained features that communicate through an event-driven model. Each feature is responsible for its own initialization, configuration, and bootstrap logic, making the system highly modular and testable.
|
|
|
|
### Key Concepts
|
|
|
|
- **Feature-Oriented**: Application logic is divided into independent features, each with its own provider for configuration and lifecycle management
|
|
- **Event-Driven**: Features communicate asynchronously through an event emitter
|
|
- **Async-First**: Built on Python's `asyncio` for high-performance concurrent operations
|
|
- **Dependency Injection**: Uses the `punq` container for loose coupling and testability
|
|
- **Kernel-Centric**: Core infrastructure (application lifecycle, messaging, events) lives in the `kernel` namespace
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
FeatureOrientedSkeleton/
|
|
├── kernel/ # Core framework and infrastructure
|
|
│ ├── application/ # Application lifecycle management
|
|
│ │ ├── application.py # Abstract base application class
|
|
│ │ ├── kernel.py # Abstract kernel (initialization point)
|
|
│ │ ├── stop_event.py # Graceful shutdown signaling
|
|
│ │ ├── enum/ # Core enumerations
|
|
│ │ └── manager/ # Event managers
|
|
│ ├── event/ # Event handling infrastructure
|
|
│ │ ├── listener.py # Base listener interface
|
|
│ │ └── registrar.py # Event listener registration mixin
|
|
│ ├── feature/ # Feature provider interface
|
|
│ │ └── provider.py # Abstract feature provider
|
|
│ └── rabbitmq/ # RabbitMQ client and connection
|
|
│ ├── client.py # RabbitMQ messaging client
|
|
│ ├── enum/ # RabbitMQ-specific types
|
|
│ └── exception/ # RabbitMQ exceptions
|
|
│
|
|
├── shared/ # Shared utilities and implementations
|
|
│ ├── application/ # Concrete application implementations
|
|
│ │ ├── rabbitmq_application.py # RabbitMQ-based application
|
|
│ │ ├── rabbitmq_kernel.py # RabbitMQ kernel with DI setup
|
|
│ │ └── factory/ # Factory patterns for common objects
|
|
│ ├── event/ # Shared event types and structures
|
|
│ ├── service/ # Cross-cutting services
|
|
│ ├── manager/ # Utilities (argument parsing, etc)
|
|
│ └── view/ # View models and configurations
|
|
│
|
|
├── feature/ # Application features (domain logic)
|
|
│ ├── listen_rabbitmq/ # Feature: Listen to RabbitMQ messages
|
|
│ │ ├── listen_rabbitmq.py # Feature provider
|
|
│ │ ├── consumer/ # Message consumption logic
|
|
│ │ ├── service/ # Business logic and services
|
|
│ │ └── type/ # Domain types and models
|
|
│ │
|
|
│ ├── log_bus_message/ # Feature: Log received messages
|
|
│ │ ├── log_bus_message.py # Feature provider
|
|
│ │ └── listener/ # Event listeners
|
|
│ │
|
|
│ ├── prepare_error_notification/ # Feature: Prepare error notifications
|
|
│ ├── send_telegram_notification/ # Feature: Send Telegram notifications
|
|
│ └── [other-features]/ # Additional features
|
|
│
|
|
├── entry/ # Application entry points
|
|
│ └── log_user_messages.py # CLI entry script
|
|
│
|
|
├── typings/ # Type stubs for third-party libraries
|
|
│ ├── aioredis/
|
|
│ ├── aioredlock/
|
|
│ └── punq/
|
|
│
|
|
├── deploy/ # Deployment configurations
|
|
│ ├── local/ # Local development (Docker Compose)
|
|
│ │ ├── compose.yaml
|
|
│ │ └── python.Dockerfile
|
|
│ └── main/ # Production deployment
|
|
│ ├── compose.yaml
|
|
│ └── python.Dockerfile
|
|
│
|
|
├── examples/ # Example payloads and usage
|
|
├── Makefile # Development commands
|
|
├── pyrightconfig.json # Type checking configuration
|
|
├── requirements.txt # Python dependencies
|
|
├── .env.example # Environment variables template
|
|
└── README.md # This file
|
|
```
|
|
|
|
## Architecture Patterns
|
|
|
|
### 1. Feature Provider Pattern
|
|
|
|
Each feature must implement a **Feature Provider** - a class that inherits from `Provider` (and optionally `Registrar` mixin):
|
|
|
|
```python
|
|
from typing import override
|
|
from kernel.feature.provider import Provider
|
|
from kernel.event.registrar import Registrar
|
|
from shared.event.type import Type as EventType
|
|
|
|
class MyFeature(Provider, Registrar):
|
|
@override
|
|
def register(self) -> None:
|
|
"""
|
|
Called during DI container initialization.
|
|
- Register classes in the container
|
|
- Load configurations
|
|
- Set up initial state
|
|
"""
|
|
self._container.register(MyService)
|
|
|
|
@override
|
|
def bootstrap(self) -> None:
|
|
"""
|
|
Called after all features are registered.
|
|
- Start background tasks
|
|
- Attach event listeners
|
|
- Initialize runtime state
|
|
"""
|
|
self.add_event_listener(EventType.MESSAGE_RECEIVED, MyEventListener)
|
|
```
|
|
|
|
### 2. Event Listener Pattern
|
|
|
|
Create an event listener that inherits from `Listener[T]`:
|
|
|
|
```python
|
|
from kernel.event.listener import Listener
|
|
from shared.event.type import Type as EventType
|
|
|
|
class MyEventListener(Listener[MyEventType]):
|
|
async def handle(self, event: str, payload: MyEventType) -> None:
|
|
# Process the event
|
|
pass
|
|
```
|
|
|
|
Attach it in your feature's `bootstrap()` method using the `Registrar` mixin.
|
|
|
|
### 3. Dependency Injection
|
|
|
|
All dependencies are resolved through the `punq` Container:
|
|
|
|
```python
|
|
from punq import Container
|
|
|
|
class MyService:
|
|
def __init__(self, container: Container) -> None:
|
|
self._container = container
|
|
|
|
def do_something(self) -> None:
|
|
dependency = self._container.resolve(SomeDependency)
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### 1. Clone and Setup
|
|
|
|
```bash
|
|
git clone <repository>
|
|
cd FeatureOrientedSkeleton
|
|
cp .env.example .env
|
|
```
|
|
|
|
### 2. Using Makefile (Recommended)
|
|
|
|
```bash
|
|
# Build and start all services
|
|
make up
|
|
|
|
# View application logs
|
|
make logs
|
|
|
|
# Stop services
|
|
make stop
|
|
|
|
# Clean up containers and volumes
|
|
make clean
|
|
```
|
|
|
|
### 3. Manual Docker Compose
|
|
|
|
```bash
|
|
# Start services
|
|
docker-compose -f deploy/local/compose.yaml up
|
|
|
|
# Stop services
|
|
docker-compose -f deploy/local/compose.yaml down
|
|
```
|
|
|
|
## Running the Example
|
|
|
|
The project includes an example application that listens to RabbitMQ messages and logs them:
|
|
|
|
```bash
|
|
make up
|
|
# Application starts and listens on queue 'application.skeleton'
|
|
# Send messages and view logs
|
|
make logs
|
|
```
|
|
|
|
## Creating a New Feature
|
|
|
|
### Step 1: Create Feature Directory
|
|
|
|
```bash
|
|
mkdir -p feature/my_feature/{service,listener,type,consumer}
|
|
touch feature/my_feature/__init__.py
|
|
```
|
|
|
|
### Step 2: Create Feature Provider
|
|
|
|
```python
|
|
# feature/my_feature/my_feature.py
|
|
from typing import override
|
|
from kernel.feature.provider import Provider
|
|
from kernel.event.registrar import Registrar
|
|
|
|
class MyFeature(Provider, Registrar):
|
|
@override
|
|
def register(self) -> None:
|
|
# Register dependencies and services
|
|
pass
|
|
|
|
@override
|
|
def bootstrap(self) -> None:
|
|
# Attach event listeners if needed
|
|
pass
|
|
```
|
|
|
|
### Step 3: Create Event Listener (if needed)
|
|
|
|
```python
|
|
# feature/my_feature/listener/my_listener.py
|
|
from kernel.event.listener import Listener
|
|
|
|
class MyListener(Listener[MyEventType]):
|
|
async def handle(self, event: str, payload: MyEventType) -> None:
|
|
# Handle the event
|
|
pass
|
|
```
|
|
|
|
### Step 4: Use in Entry Script
|
|
|
|
```python
|
|
# entry/my_listener.py
|
|
import asyncio
|
|
from shared.application.factory.application import Application as ApplicationFactory
|
|
from feature.my_feature import MyFeature
|
|
|
|
if __name__ == "__main__":
|
|
factory = ApplicationFactory()
|
|
|
|
application = factory.create_rabbitmq([
|
|
MyFeature,
|
|
])
|
|
|
|
asyncio.run(application.launch(main))
|
|
```
|
|
|
|
## Message Format
|
|
|
|
Messages must conform to a specific JSON format:
|
|
|
|
```json
|
|
{
|
|
"service": "service_name",
|
|
"environment": "production",
|
|
"exception": "ErrorType",
|
|
"message": "Error message text",
|
|
"type": "error",
|
|
"datetime": "2024-01-01T12:00:00"
|
|
}
|
|
```
|
|
|
|
See `examples/` directory for sample messages.
|
|
|
|
## Environment Configuration
|
|
|
|
Copy `.env.example` to `.env` and configure:
|
|
|
|
```env
|
|
# Application
|
|
APP_NAME=application-skeleton
|
|
APP_ENV=local
|
|
|
|
# RabbitMQ
|
|
RABBITMQ_HOST=rabbitmq
|
|
RABBITMQ_PORT=5672
|
|
RABBITMQ_USER=guest
|
|
RABBITMQ_PASS=guest
|
|
RABBITMQ_VHOST=/
|
|
RABBITMQ_TIMEOUT=10
|
|
|
|
# Redis
|
|
REDIS_HOST=redis
|
|
REDIS_PORT=6379
|
|
REDIS_DB=0
|
|
```
|
|
|
|
## Development
|
|
|
|
### Makefile Commands
|
|
|
|
```bash
|
|
make build # Build Docker images
|
|
make up # Start services (builds if needed)
|
|
make down # Stop services
|
|
make restart # Restart services
|
|
make logs # View application logs
|
|
make stop # Stop and remove containers
|
|
make clean # Remove all containers and volumes
|
|
```
|
|
|
|
### Type Checking
|
|
|
|
```bash
|
|
pyright
|
|
```
|
|
|
|
### Code Style
|
|
|
|
Follow PEP 8 conventions. Configuration in `.editorconfig`.
|
|
|
|
## Key Design Principles
|
|
|
|
1. **Kernel is Core**: Infrastructure lives in `kernel/`, never import from `feature/` into `kernel/`
|
|
2. **Shared for Common Code**: Cross-cutting concerns and shared implementations belong in `shared/`
|
|
3. **Features are Independent**: Features should not depend on other features directly
|
|
4. **DI Over Coupling**: Use the DI container to manage dependencies
|
|
5. **Async-First**: Embrace async/await throughout the codebase
|
|
6. **Events for Communication**: Features communicate through events, not direct calls
|
|
7. **Provider Pattern**: Each feature must implement the Provider pattern with `register()` and `bootstrap()` methods
|
|
|
|
## Performance Considerations
|
|
|
|
- **Graceful Shutdown**: Application responds to SIGTERM and SIGINT signals
|
|
- **Resource Cleanup**: Set `stop_grace_period: 3s` in Docker Compose for clean shutdown
|
|
- **Connection Pooling**: RabbitMQ and Redis connections are pooled for efficiency
|
|
- **Event-Driven**: Non-blocking event handling allows processing many messages concurrently
|
|
|
|
## Troubleshooting
|
|
|
|
### Application doesn't stop gracefully
|
|
|
|
Ensure `stop_grace_period` is set in `docker-compose.yaml`:
|
|
|
|
```yaml
|
|
services:
|
|
application:
|
|
stop_grace_period: 3s
|
|
```
|
|
|
|
### Messages not being processed
|
|
|
|
1. Check RabbitMQ is healthy: `docker ps`
|
|
2. Verify queue and routing key match the listeners
|
|
3. View logs: `make logs`
|
|
4. Check environment variables in `.env`
|
|
|
|
### Type checking errors
|
|
|
|
Ensure typings are installed:
|
|
|
|
```bash
|
|
ls typings/
|
|
# Should contain stubs for aioredis, aioredlock, punq
|
|
```
|
|
|
|
### ConnectionError to RabbitMQ
|
|
|
|
1. Verify `depends_on` conditions in compose.yaml:
|
|
```yaml
|
|
depends_on:
|
|
rabbitmq:
|
|
condition: service_healthy
|
|
```
|
|
2. Check healthcheck passes: `docker ps`
|
|
3. Verify credentials in `.env`
|
|
|
|
## Dependencies
|
|
|
|
Core dependencies include:
|
|
|
|
- `punq` - Dependency injection container
|
|
- `aio-pika` - RabbitMQ client
|
|
- `redis` - Redis client
|
|
- `aioredlock` - Distributed locking
|
|
- `pyee` - Event emitter
|
|
|
|
See `requirements.txt` for complete list and versions.
|
|
|
|
## Contributing
|
|
|
|
1. Follow the feature-oriented structure
|
|
2. Each feature should be self-contained
|
|
3. Use dependency injection for all dependencies
|
|
4. Add type annotations to all functions
|
|
5. Write docstrings for public APIs
|
|
6. Test features in isolation using the DI container
|
|
|
|
## License
|
|
|
|
[Add your license here]
|
|
|
|
---
|
|
|
|
**For more information on feature-oriented architecture and event-driven systems, see the examples in the `examples/` directory.**
|