531 lines
14 KiB
Markdown
531 lines
14 KiB
Markdown
# Clear Docker Registry v2
|
|
|
|
A comprehensive automation toolkit for managing and maintaining Docker registries. This project provides two powerful scripts for cleaning up unused container images and optimizing registry storage.
|
|
|
|
## Overview
|
|
|
|
Docker registries can accumulate many image versions over time, consuming significant disk space. This project offers automated solutions to:
|
|
|
|
1. **Clean up old image tags** in remote registries
|
|
2. **Run garbage collection** on local Docker registry containers
|
|
3. **Free up storage space** by removing unused layers and manifests
|
|
|
|
## Scripts
|
|
|
|
### 1. `clear-registry`
|
|
|
|
**Purpose**: Delete old image tags from a Docker registry while preserving recent versions.
|
|
|
|
This script helps manage image versioning by automatically removing old tags, keeping only the latest N versions plus the `latest` tag.
|
|
|
|
#### Features
|
|
|
|
- Automatically detects and installs required dependencies
|
|
- Supports multiple Linux distributions (Debian, Ubuntu, Alpine, Arch, CentOS, Fedora, openSUSE)
|
|
- Preserves the `latest` tag automatically
|
|
- Provides detailed progress output with visual indicators
|
|
- Works with any Docker-compatible registry
|
|
|
|
#### Requirements
|
|
|
|
- **skopeo** - Tool for working with remote registries
|
|
- **jq** - JSON query processor
|
|
- **grep** - Text pattern matching
|
|
- **sort** - Text sorting
|
|
- **head** - Display file/input head
|
|
|
|
The script automatically installs missing dependencies if your system's package manager is detected.
|
|
|
|
#### Usage
|
|
|
|
```bash
|
|
./clear-registry <repository> [number_of_tags]
|
|
```
|
|
|
|
**Arguments:**
|
|
- `<repository>` (required) - Docker repository URL or name
|
|
- Examples: `docker.io/library/alpine`, `myregistry.local:5000/myapp`
|
|
- `[number_of_tags]` (optional) - Number of recent tags to keep (default: 3)
|
|
|
|
#### Examples
|
|
|
|
```bash
|
|
# Keep 3 most recent tags of alpine image
|
|
./clear-registry docker.io/library/alpine
|
|
|
|
# Keep 5 most recent tags from private registry
|
|
./clear-registry myregistry.local:5000/myapp 5
|
|
|
|
# Keep 10 versions of specific image
|
|
./clear-registry registry.example.com/project/image 10
|
|
```
|
|
|
|
#### How It Works
|
|
|
|
1. Validates input arguments
|
|
2. Checks and installs required utilities if needed
|
|
3. Fetches all tags from the specified repository using skopeo
|
|
4. Calculates which tags to delete (keeping N newest + latest)
|
|
5. Deletes old tags one by one with confirmation
|
|
6. Reports completion with summary
|
|
|
|
#### Output Example
|
|
|
|
```
|
|
🔍 Fetching list of tags for docker.io/library/alpine...
|
|
📦 Will keep 3 recent tags (excluding latest)
|
|
📊 Total tags: 10 (including latest)
|
|
🗑️ Will be deleted: 6 tags
|
|
|
|
➜ Deleting: v1.0.0
|
|
✓ Successfully deleted
|
|
➜ Deleting: v1.1.0
|
|
✓ Successfully deleted
|
|
...
|
|
✅ Done!
|
|
📝 Kept: 3 recent tags and latest
|
|
```
|
|
|
|
#### Error Handling
|
|
|
|
- **Repository not specified**: Shows usage help and exits
|
|
- **Repository unreachable**: Reports connection error
|
|
- **Empty repository**: Informs user and exits safely
|
|
- **Invalid tag count**: Validates numeric input and provides error message
|
|
- **Missing dependencies**: Attempts automatic installation
|
|
|
|
---
|
|
|
|
### 2. `collect-registry`
|
|
|
|
**Purpose**: Run garbage collection on a Docker registry container to free up disk space.
|
|
|
|
Docker registry containers accumulate unused layers and manifests. This script safely runs the built-in garbage collection tool and restarts the registry.
|
|
|
|
#### Features
|
|
|
|
- Comprehensive container validation (exists and running)
|
|
- Automatic configuration file detection with fallback search
|
|
- Interactive configuration selection
|
|
- Color-coded output for easy reading
|
|
- Container status verification after completion
|
|
- Support for custom configuration paths
|
|
- Handles permission requirements gracefully
|
|
|
|
#### Requirements
|
|
|
|
- **Docker** - Docker daemon must be running
|
|
- **Access** - Sufficient permissions to run `docker exec` and `docker restart`
|
|
- **Registry container** - Must have `registry garbage-collect` command available
|
|
- **Configuration file** - Must be accessible inside the container
|
|
|
|
#### Usage
|
|
|
|
```bash
|
|
./collect-registry <container_name> [config_path]
|
|
```
|
|
|
|
**Arguments:**
|
|
- `<container_name>` (required) - Name or ID of Docker registry container
|
|
- `[config_path]` (optional) - Path to configuration file inside container
|
|
|
|
#### Default Configuration Paths
|
|
|
|
The script searches for configuration in this order:
|
|
1. User-specified path (if provided)
|
|
2. `/etc/docker/registry/config.yml`
|
|
3. `/etc/distribution/config.yml`
|
|
4. Extended search through entire container (if allowed)
|
|
|
|
#### Examples
|
|
|
|
```bash
|
|
# Run GC on container named 'registry'
|
|
./collect-registry registry
|
|
|
|
# Specify custom configuration path
|
|
./collect-registry my-registry-container /etc/distribution/config.yml
|
|
|
|
# Use container ID from docker ps
|
|
./collect-registry $(docker ps -qf 'name=registry')
|
|
|
|
# Custom path inside container
|
|
./collect-registry registry /custom/path/config.yml
|
|
```
|
|
|
|
#### How It Works
|
|
|
|
1. **Validation Phase:**
|
|
- Verifies container name argument provided
|
|
- Checks container exists
|
|
- Verifies container is running
|
|
- Reports status and available containers if errors occur
|
|
|
|
2. **Configuration Detection Phase:**
|
|
- Uses provided path if specified
|
|
- Searches standard registry paths
|
|
- Offers extended filesystem search if needed
|
|
- Allows manual path entry if multiple options found
|
|
|
|
3. **Garbage Collection Phase:**
|
|
- Executes `registry garbage-collect` command
|
|
- Handles execution errors with troubleshooting hints
|
|
|
|
4. **Restart Phase:**
|
|
- Safely restarts Docker container
|
|
- Verifies restart success
|
|
|
|
5. **Verification Phase:**
|
|
- Shows container status
|
|
- Reports configuration file used
|
|
|
|
#### Output Example
|
|
|
|
```
|
|
[Search] Looking for configuration file in container...
|
|
[OK] Configuration file found: /etc/docker/registry/config.yml
|
|
[OK] All checks passed
|
|
[Info] Using config: /etc/docker/registry/config.yml
|
|
[Start] Running garbage collector in container 'registry'...
|
|
[OK] Garbage collector executed successfully
|
|
[Restart] Restarting container...
|
|
[OK] Container successfully restarted
|
|
[OK] DONE! Garbage collector completed, container restarted
|
|
[Status] Current container status:
|
|
registry Up 2 seconds Up 2 seconds
|
|
[Info] Configuration file used: /etc/docker/registry/config.yml
|
|
```
|
|
|
|
#### Color-Coded Output
|
|
|
|
- 🟢 **GREEN [OK]** - Successful operations
|
|
- 🔴 **RED [ERR]** - Errors requiring attention
|
|
- 🟡 **YELLOW [Warning/List]** - Warnings or information lists
|
|
- 🔵 **BLUE [Hint]** - Helpful suggestions
|
|
|
|
#### Error Handling
|
|
|
|
- **Container not specified**: Shows usage help
|
|
- **Container not found**: Lists available containers
|
|
- **Container not running**: Suggests starting the container
|
|
- **Configuration not found**: Offers extended search option
|
|
- **Garbage collection error**: Provides troubleshooting hints
|
|
- **Restart failure**: Reports error with next steps
|
|
|
|
#### Permission Issues
|
|
|
|
The script uses `sudo` for package installation but **NOT** for Docker operations. If you get permission errors:
|
|
|
|
```bash
|
|
# Add user to docker group
|
|
sudo usermod -aG docker $USER
|
|
|
|
# Apply new permissions
|
|
newgrp docker
|
|
|
|
# Or run with sudo
|
|
sudo ./collect-registry registry
|
|
```
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Quick Start
|
|
|
|
1. Clone or download the scripts:
|
|
|
|
```bash
|
|
cd /path/to/ClearDockerRegistry_v2
|
|
```
|
|
|
|
2. Make scripts executable:
|
|
|
|
```bash
|
|
chmod +x clear-registry collect-registry
|
|
```
|
|
|
|
3. (Optional) Add to PATH:
|
|
|
|
```bash
|
|
sudo cp clear-registry collect-registry /usr/local/bin/
|
|
```
|
|
|
|
### System Requirements
|
|
|
|
- Linux/macOS/WSL environment
|
|
- Bash shell (version 4+)
|
|
- Docker installed and running (for `collect-registry`)
|
|
- `sudo` access for package installation (first run only)
|
|
|
|
### Supported Package Managers
|
|
|
|
- **apt/apt-get** - Debian, Ubuntu, Linux Mint
|
|
- **pacman** - Arch Linux, Manjaro
|
|
- **apk** - Alpine Linux
|
|
- **yum** - CentOS 7, RHEL 7
|
|
- **dnf** - Fedora, CentOS 8+, RHEL 8+
|
|
- **zypper** - openSUSE
|
|
|
|
---
|
|
|
|
## Usage Examples
|
|
|
|
### Scenario 1: Clean Up Public Registry
|
|
|
|
Clean up old Alpine image versions, keeping only 3 most recent:
|
|
|
|
```bash
|
|
./clear-registry docker.io/library/alpine 3
|
|
```
|
|
|
|
### Scenario 2: Maintain Private Registry Images
|
|
|
|
Clean up multiple images in a private registry:
|
|
|
|
```bash
|
|
# Keep 5 versions of web app
|
|
./clear-registry myregistry.local:5000/myapp/web 5
|
|
|
|
# Keep 10 versions of database
|
|
./clear-registry myregistry.local:5000/myapp/db 10
|
|
|
|
# Keep 2 versions of worker (aggressive cleanup)
|
|
./clear-registry myregistry.local:5000/myapp/worker 2
|
|
```
|
|
|
|
### Scenario 3: Run Registry Garbage Collection
|
|
|
|
After cleaning images, run garbage collection on the registry container:
|
|
|
|
```bash
|
|
# Find registry container
|
|
docker ps | grep registry
|
|
|
|
# Run garbage collection and restart
|
|
./collect-registry my-docker-registry
|
|
|
|
# Or specify config path if non-standard
|
|
./collect-registry my-docker-registry /etc/registry/config.yml
|
|
```
|
|
|
|
### Scenario 4: Automated Cleanup (Cron Job)
|
|
|
|
Add both scripts to cron for periodic maintenance:
|
|
|
|
```bash
|
|
# Crontab entry - run at 2:00 AM daily
|
|
0 2 * * * /usr/local/bin/clear-registry docker.io/library/myapp 5
|
|
5 2 * * * /usr/local/bin/collect-registry my-docker-registry
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### clear-registry Issues
|
|
|
|
#### Error: "Could not determine package manager"
|
|
|
|
**Cause**: System uses an unsupported package manager.
|
|
|
|
**Solution**: Install dependencies manually:
|
|
|
|
```bash
|
|
# Install required packages (Ubuntu example)
|
|
sudo apt install -y skopeo jq coreutils
|
|
```
|
|
|
|
#### Error: "Could not fetch tag list or repository is empty"
|
|
|
|
**Cause**:
|
|
- Repository URL is incorrect
|
|
- Registry is unreachable
|
|
- Repository doesn't exist
|
|
|
|
**Solution**:
|
|
- Verify URL: `skopeo list-tags docker://your-repo`
|
|
- Check registry connectivity: `curl -I https://registry-url/v2/`
|
|
|
|
#### Error: "permission denied" during deletion
|
|
|
|
**Cause**: Insufficient permissions to delete tags.
|
|
|
|
**Solution**:
|
|
- Verify registry credentials if using private registry
|
|
- Check registry permissions for your account
|
|
|
|
### collect-registry Issues
|
|
|
|
#### Error: "container not found"
|
|
|
|
**Cause**: Container name doesn't match.
|
|
|
|
**Solution**:
|
|
```bash
|
|
# List all containers
|
|
docker ps -a | grep registry
|
|
|
|
# Use correct container name
|
|
./collect-registry correct_name
|
|
```
|
|
|
|
#### Error: "container not running"
|
|
|
|
**Cause**: Registry container is stopped.
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Start container
|
|
docker start registry_name
|
|
|
|
# Then run script
|
|
./collect-registry registry_name
|
|
```
|
|
|
|
#### Error: "Could not find configuration file"
|
|
|
|
**Cause**: Registry config not in standard locations.
|
|
|
|
**Solution**:
|
|
- When prompted, select "Yes" for extended search
|
|
- If that fails, find it manually:
|
|
|
|
```bash
|
|
# Find config inside container
|
|
docker exec registry_name find / -name "config.yml" 2>/dev/null
|
|
|
|
# Run script with explicit path
|
|
./collect-registry registry_name /path/to/config.yml
|
|
```
|
|
|
|
#### Error: "Error executing garbage collector"
|
|
|
|
**Causes**:
|
|
- Insufficient container permissions
|
|
- Corrupted registry database
|
|
- Registry version incompatibility
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Check registry logs
|
|
docker logs registry_name | tail -100
|
|
|
|
# Verify registry health
|
|
docker exec registry_name registry --version
|
|
|
|
# Try manual garbage collection
|
|
docker exec registry_name registry garbage-collect /etc/docker/registry/config.yml
|
|
```
|
|
|
|
#### Error: "Error restarting container"
|
|
|
|
**Cause**: Docker daemon issues.
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Restart Docker daemon
|
|
sudo systemctl restart docker
|
|
|
|
# Manually restart container
|
|
docker restart registry_name
|
|
|
|
# Check container status
|
|
docker ps | grep registry_name
|
|
```
|
|
|
|
---
|
|
|
|
## Performance Considerations
|
|
|
|
### clear-registry
|
|
|
|
- **Speed**: Depends on registry size and network connectivity
|
|
- **Risk**: Deletion is permanent; deleted tags cannot be recovered
|
|
- **Recommendation**: Start with a higher keep count (e.g., 10) and adjust
|
|
|
|
### collect-registry
|
|
|
|
- **Duration**: Garbage collection time depends on registry size
|
|
- Small registries: 1-5 minutes
|
|
- Medium registries (10-100GB): 5-30 minutes
|
|
- Large registries (>100GB): 30+ minutes
|
|
- **Registry Availability**: Registry will be unavailable during restart (~10-30 seconds)
|
|
- **Disk Space**: GC may require temporary space equal to max blob size
|
|
|
|
---
|
|
|
|
## Best Practices
|
|
|
|
1. **Backup First**: Always back up registry data before running GC
|
|
|
|
2. **Test on Small Registries**: Verify scripts work with non-critical registries first
|
|
|
|
3. **Schedule Off-Peak**: Run garbage collection during low-traffic periods
|
|
|
|
4. **Start Conservative**: Keep more tags initially, adjust based on storage trends
|
|
|
|
5. **Monitor Logs**: Check container logs after GC completion
|
|
|
|
6. **Version Control**: Keep builds/tags with proper versioning scheme to avoid confusion
|
|
|
|
7. **Document Changes**: Log which images were cleaned and when
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
|
|
Permission is granted for personal and commercial use, modification, and distribution.
|
|
|
|
## Support
|
|
|
|
For issues or questions:
|
|
1. Check the Troubleshooting section above
|
|
2. Review script output for specific error messages
|
|
3. Check Docker and registry logs for additional context
|
|
4. Verify all prerequisites are installed and configured
|
|
|
|
---
|
|
|
|
## Changelog
|
|
|
|
### Version 2.0
|
|
- Complete rewrite with improved error handling
|
|
- Added configuration validation and auto-detection
|
|
- Support for multiple Linux distributions
|
|
- Enhanced output with color coding and progress indicators
|
|
- Extended configuration search capability
|
|
- Better permission and error messaging
|
|
|
|
---
|
|
|
|
## Related Commands
|
|
|
|
Useful Docker registry commands:
|
|
|
|
```bash
|
|
# List all running containers
|
|
docker ps
|
|
|
|
# View registry logs
|
|
docker logs registry_name
|
|
|
|
# Get container details
|
|
docker inspect registry_name
|
|
|
|
# Verify registry health
|
|
curl https://registry-url/v2/
|
|
|
|
# List all images/tags in registry (using skopeo)
|
|
skopeo list-tags docker://registry/image
|
|
|
|
# Manual garbage collection
|
|
docker exec registry_name registry garbage-collect /etc/docker/registry/config.yml
|
|
```
|
|
|
|
---
|
|
|
|
**Last Updated**: March 2026
|