Skeleton ready
This commit is contained in:
405
README.md
Normal file
405
README.md
Normal file
@@ -0,0 +1,405 @@
|
||||
# Feature-Oriented Event-Driven Application Skeleton
|
||||
|
||||
A Python-based asynchronous application framework designed for building scalable, event-driven systems using a feature-oriented architecture. This skeleton provides a solid foundation for building complex applications with clear separation of concerns, dependency injection, and event-driven communication patterns.
|
||||
|
||||
## Overview
|
||||
|
||||
This project implements a feature-oriented architecture where functionality is organized into independent, self-contained features that communicate through an event-driven model. Each feature is responsible for its own initialization, configuration, and bootstrap logic, making the system highly modular and testable.
|
||||
|
||||
### Key Concepts
|
||||
|
||||
- **Feature-Oriented**: Application logic is divided into independent features, each with its own provider for configuration and lifecycle management
|
||||
- **Event-Driven**: Features communicate asynchronously through an event emitter
|
||||
- **Async-First**: Built on Python's `asyncio` for high-performance concurrent operations
|
||||
- **Dependency Injection**: Uses the `punq` container for loose coupling and testability
|
||||
- **Kernel-Centric**: Core infrastructure (application lifecycle, messaging, events) lives in the `kernel` namespace
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
FeatureOrientedSkeleton/
|
||||
├── kernel/ # Core framework and infrastructure
|
||||
│ ├── application/ # Application lifecycle management
|
||||
│ │ ├── application.py # Abstract base application class
|
||||
│ │ ├── kernel.py # Abstract kernel (initialization point)
|
||||
│ │ ├── stop_event.py # Graceful shutdown signaling
|
||||
│ │ ├── enum/ # Core enumerations
|
||||
│ │ └── manager/ # Event managers
|
||||
│ ├── event/ # Event handling infrastructure
|
||||
│ │ ├── listener.py # Base listener interface
|
||||
│ │ └── registrar.py # Event listener registration mixin
|
||||
│ ├── feature/ # Feature provider interface
|
||||
│ │ └── provider.py # Abstract feature provider
|
||||
│ └── rabbitmq/ # RabbitMQ client and connection
|
||||
│ ├── client.py # RabbitMQ messaging client
|
||||
│ ├── enum/ # RabbitMQ-specific types
|
||||
│ └── exception/ # RabbitMQ exceptions
|
||||
│
|
||||
├── shared/ # Shared utilities and implementations
|
||||
│ ├── application/ # Concrete application implementations
|
||||
│ │ ├── rabbitmq_application.py # RabbitMQ-based application
|
||||
│ │ ├── rabbitmq_kernel.py # RabbitMQ kernel with DI setup
|
||||
│ │ └── factory/ # Factory patterns for common objects
|
||||
│ ├── event/ # Shared event types and structures
|
||||
│ ├── service/ # Cross-cutting services
|
||||
│ ├── manager/ # Utilities (argument parsing, etc)
|
||||
│ └── view/ # View models and configurations
|
||||
│
|
||||
├── feature/ # Application features (domain logic)
|
||||
│ ├── listen_rabbitmq/ # Feature: Listen to RabbitMQ messages
|
||||
│ │ ├── listen_rabbitmq.py # Feature provider
|
||||
│ │ ├── consumer/ # Message consumption logic
|
||||
│ │ ├── service/ # Business logic and services
|
||||
│ │ └── type/ # Domain types and models
|
||||
│ │
|
||||
│ ├── log_bus_message/ # Feature: Log received messages
|
||||
│ │ ├── log_bus_message.py # Feature provider
|
||||
│ │ └── listener/ # Event listeners
|
||||
│ │
|
||||
│ ├── prepare_error_notification/ # Feature: Prepare error notifications
|
||||
│ ├── send_telegram_notification/ # Feature: Send Telegram notifications
|
||||
│ └── [other-features]/ # Additional features
|
||||
│
|
||||
├── entry/ # Application entry points
|
||||
│ └── log_user_messages.py # CLI entry script
|
||||
│
|
||||
├── typings/ # Type stubs for third-party libraries
|
||||
│ ├── aioredis/
|
||||
│ ├── aioredlock/
|
||||
│ └── punq/
|
||||
│
|
||||
├── deploy/ # Deployment configurations
|
||||
│ ├── local/ # Local development (Docker Compose)
|
||||
│ │ ├── compose.yaml
|
||||
│ │ └── python.Dockerfile
|
||||
│ └── main/ # Production deployment
|
||||
│ ├── compose.yaml
|
||||
│ └── python.Dockerfile
|
||||
│
|
||||
├── examples/ # Example payloads and usage
|
||||
├── Makefile # Development commands
|
||||
├── pyrightconfig.json # Type checking configuration
|
||||
├── requirements.txt # Python dependencies
|
||||
├── .env.example # Environment variables template
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### 1. Feature Provider Pattern
|
||||
|
||||
Each feature must implement a **Feature Provider** - a class that inherits from `Provider` (and optionally `Registrar` mixin):
|
||||
|
||||
```python
|
||||
from typing import override
|
||||
from kernel.feature.provider import Provider
|
||||
from kernel.event.registrar import Registrar
|
||||
from shared.event.type import Type as EventType
|
||||
|
||||
class MyFeature(Provider, Registrar):
|
||||
@override
|
||||
def register(self) -> None:
|
||||
"""
|
||||
Called during DI container initialization.
|
||||
- Register classes in the container
|
||||
- Load configurations
|
||||
- Set up initial state
|
||||
"""
|
||||
self._container.register(MyService)
|
||||
|
||||
@override
|
||||
def bootstrap(self) -> None:
|
||||
"""
|
||||
Called after all features are registered.
|
||||
- Start background tasks
|
||||
- Attach event listeners
|
||||
- Initialize runtime state
|
||||
"""
|
||||
self.add_event_listener(EventType.MESSAGE_RECEIVED, MyEventListener)
|
||||
```
|
||||
|
||||
### 2. Event Listener Pattern
|
||||
|
||||
Create an event listener that inherits from `Listener[T]`:
|
||||
|
||||
```python
|
||||
from kernel.event.listener import Listener
|
||||
from shared.event.type import Type as EventType
|
||||
|
||||
class MyEventListener(Listener[MyEventType]):
|
||||
async def handle(self, event: str, payload: MyEventType) -> None:
|
||||
# Process the event
|
||||
pass
|
||||
```
|
||||
|
||||
Attach it in your feature's `bootstrap()` method using the `Registrar` mixin.
|
||||
|
||||
### 3. Dependency Injection
|
||||
|
||||
All dependencies are resolved through the `punq` Container:
|
||||
|
||||
```python
|
||||
from punq import Container
|
||||
|
||||
class MyService:
|
||||
def __init__(self, container: Container) -> None:
|
||||
self._container = container
|
||||
|
||||
def do_something(self) -> None:
|
||||
dependency = self._container.resolve(SomeDependency)
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Clone and Setup
|
||||
|
||||
```bash
|
||||
git clone <repository>
|
||||
cd FeatureOrientedSkeleton
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
### 2. Using Makefile (Recommended)
|
||||
|
||||
```bash
|
||||
# Build and start all services
|
||||
make up
|
||||
|
||||
# View application logs
|
||||
make logs
|
||||
|
||||
# Stop services
|
||||
make stop
|
||||
|
||||
# Clean up containers and volumes
|
||||
make clean
|
||||
```
|
||||
|
||||
### 3. Manual Docker Compose
|
||||
|
||||
```bash
|
||||
# Start services
|
||||
docker-compose -f deploy/local/compose.yaml up
|
||||
|
||||
# Stop services
|
||||
docker-compose -f deploy/local/compose.yaml down
|
||||
```
|
||||
|
||||
## Running the Example
|
||||
|
||||
The project includes an example application that listens to RabbitMQ messages and logs them:
|
||||
|
||||
```bash
|
||||
make up
|
||||
# Application starts and listens on queue 'application.skeleton'
|
||||
# Send messages and view logs
|
||||
make logs
|
||||
```
|
||||
|
||||
## Creating a New Feature
|
||||
|
||||
### Step 1: Create Feature Directory
|
||||
|
||||
```bash
|
||||
mkdir -p feature/my_feature/{service,listener,type,consumer}
|
||||
touch feature/my_feature/__init__.py
|
||||
```
|
||||
|
||||
### Step 2: Create Feature Provider
|
||||
|
||||
```python
|
||||
# feature/my_feature/my_feature.py
|
||||
from typing import override
|
||||
from kernel.feature.provider import Provider
|
||||
from kernel.event.registrar import Registrar
|
||||
|
||||
class MyFeature(Provider, Registrar):
|
||||
@override
|
||||
def register(self) -> None:
|
||||
# Register dependencies and services
|
||||
pass
|
||||
|
||||
@override
|
||||
def bootstrap(self) -> None:
|
||||
# Attach event listeners if needed
|
||||
pass
|
||||
```
|
||||
|
||||
### Step 3: Create Event Listener (if needed)
|
||||
|
||||
```python
|
||||
# feature/my_feature/listener/my_listener.py
|
||||
from kernel.event.listener import Listener
|
||||
|
||||
class MyListener(Listener[MyEventType]):
|
||||
async def handle(self, event: str, payload: MyEventType) -> None:
|
||||
# Handle the event
|
||||
pass
|
||||
```
|
||||
|
||||
### Step 4: Use in Entry Script
|
||||
|
||||
```python
|
||||
# entry/my_listener.py
|
||||
import asyncio
|
||||
from shared.application.factory.application import Application as ApplicationFactory
|
||||
from feature.my_feature import MyFeature
|
||||
|
||||
if __name__ == "__main__":
|
||||
factory = ApplicationFactory()
|
||||
|
||||
application = factory.create_rabbitmq([
|
||||
MyFeature,
|
||||
])
|
||||
|
||||
asyncio.run(application.launch(main))
|
||||
```
|
||||
|
||||
## Message Format
|
||||
|
||||
Messages must conform to a specific JSON format:
|
||||
|
||||
```json
|
||||
{
|
||||
"service": "service_name",
|
||||
"environment": "production",
|
||||
"exception": "ErrorType",
|
||||
"message": "Error message text",
|
||||
"type": "error",
|
||||
"datetime": "2024-01-01T12:00:00"
|
||||
}
|
||||
```
|
||||
|
||||
See `examples/` directory for sample messages.
|
||||
|
||||
## Environment Configuration
|
||||
|
||||
Copy `.env.example` to `.env` and configure:
|
||||
|
||||
```env
|
||||
# Application
|
||||
APP_NAME=application-skeleton
|
||||
APP_ENV=local
|
||||
|
||||
# RabbitMQ
|
||||
RABBITMQ_HOST=rabbitmq
|
||||
RABBITMQ_PORT=5672
|
||||
RABBITMQ_USER=guest
|
||||
RABBITMQ_PASS=guest
|
||||
RABBITMQ_VHOST=/
|
||||
RABBITMQ_TIMEOUT=10
|
||||
|
||||
# Redis
|
||||
REDIS_HOST=redis
|
||||
REDIS_PORT=6379
|
||||
REDIS_DB=0
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Makefile Commands
|
||||
|
||||
```bash
|
||||
make build # Build Docker images
|
||||
make up # Start services (builds if needed)
|
||||
make down # Stop services
|
||||
make restart # Restart services
|
||||
make logs # View application logs
|
||||
make stop # Stop and remove containers
|
||||
make clean # Remove all containers and volumes
|
||||
```
|
||||
|
||||
### Type Checking
|
||||
|
||||
```bash
|
||||
pyright
|
||||
```
|
||||
|
||||
### Code Style
|
||||
|
||||
Follow PEP 8 conventions. Configuration in `.editorconfig`.
|
||||
|
||||
## Key Design Principles
|
||||
|
||||
1. **Kernel is Core**: Infrastructure lives in `kernel/`, never import from `feature/` into `kernel/`
|
||||
2. **Shared for Common Code**: Cross-cutting concerns and shared implementations belong in `shared/`
|
||||
3. **Features are Independent**: Features should not depend on other features directly
|
||||
4. **DI Over Coupling**: Use the DI container to manage dependencies
|
||||
5. **Async-First**: Embrace async/await throughout the codebase
|
||||
6. **Events for Communication**: Features communicate through events, not direct calls
|
||||
7. **Provider Pattern**: Each feature must implement the Provider pattern with `register()` and `bootstrap()` methods
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- **Graceful Shutdown**: Application responds to SIGTERM and SIGINT signals
|
||||
- **Resource Cleanup**: Set `stop_grace_period: 3s` in Docker Compose for clean shutdown
|
||||
- **Connection Pooling**: RabbitMQ and Redis connections are pooled for efficiency
|
||||
- **Event-Driven**: Non-blocking event handling allows processing many messages concurrently
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Application doesn't stop gracefully
|
||||
|
||||
Ensure `stop_grace_period` is set in `docker-compose.yaml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
application:
|
||||
stop_grace_period: 3s
|
||||
```
|
||||
|
||||
### Messages not being processed
|
||||
|
||||
1. Check RabbitMQ is healthy: `docker ps`
|
||||
2. Verify queue and routing key match the listeners
|
||||
3. View logs: `make logs`
|
||||
4. Check environment variables in `.env`
|
||||
|
||||
### Type checking errors
|
||||
|
||||
Ensure typings are installed:
|
||||
|
||||
```bash
|
||||
ls typings/
|
||||
# Should contain stubs for aioredis, aioredlock, punq
|
||||
```
|
||||
|
||||
### ConnectionError to RabbitMQ
|
||||
|
||||
1. Verify `depends_on` conditions in compose.yaml:
|
||||
```yaml
|
||||
depends_on:
|
||||
rabbitmq:
|
||||
condition: service_healthy
|
||||
```
|
||||
2. Check healthcheck passes: `docker ps`
|
||||
3. Verify credentials in `.env`
|
||||
|
||||
## Dependencies
|
||||
|
||||
Core dependencies include:
|
||||
|
||||
- `punq` - Dependency injection container
|
||||
- `aio-pika` - RabbitMQ client
|
||||
- `redis` - Redis client
|
||||
- `aioredlock` - Distributed locking
|
||||
- `pyee` - Event emitter
|
||||
|
||||
See `requirements.txt` for complete list and versions.
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Follow the feature-oriented structure
|
||||
2. Each feature should be self-contained
|
||||
3. Use dependency injection for all dependencies
|
||||
4. Add type annotations to all functions
|
||||
5. Write docstrings for public APIs
|
||||
6. Test features in isolation using the DI container
|
||||
|
||||
## License
|
||||
|
||||
[Add your license here]
|
||||
|
||||
---
|
||||
|
||||
**For more information on feature-oriented architecture and event-driven systems, see the examples in the `examples/` directory.**
|
||||
Reference in New Issue
Block a user