Posted inPython

Pro-Level Python Config Management with Dynaconf: From .env Files to Redis/Vault

Python tutorial - IT technology blog
Python tutorial - IT technology blog

The Pain of Manual Config Management

If you often write Python scripts for automation or build monitoring systems, you’ve likely experienced “code runs fine locally but fails on the server.” Even worse is accidentally leaking API keys on GitHub because they were hardcoded into the source code.

Previously, I used python-dotenv for small projects. However, as systems grow to 5-10 microservices across multiple environments (Dev, Staging, Prod), .env files start showing their weaknesses. They only support strings, lack nested structures, and managing dozens of .env.production files is prone to error.

After 6 months of using Dynaconf in production, I believe it’s the best configuration management solution for Python today. It handles everything seamlessly, from static files (TOML, YAML) to dynamic systems like Redis or HashiCorp Vault.

Comparing Common Config Management Methods

Let’s look back at how we usually do things to see why Dynaconf is worth it:

  • Direct Hardcoding: The fastest way to create security vulnerabilities. Never put passwords or tokens in code unless you want to be featured on security forums.
  • Using os.environ: Safer, but your code will be littered with os.getenv("DB_URL", "localhost"). When a project has over 50 environment variables, management becomes a nightmare.
  • Manual JSON/YAML Parsing: You have to write your own logic to merge configurations across environments. This is time-consuming and prone to unnecessary bugs.
  • Dynaconf: Automatically casts types (numbers, lists, booleans), supports multiple environments, and prioritizes values using a smart hierarchy.

Why I Chose Dynaconf for Real-World Projects

When deploying complex scripts, I need a tool smart enough to understand priority. For example: default settings are in settings.toml, but if an environment variable exists on the server, that value must take top priority. Dynaconf handles this very cleanly.

The biggest advantage is Validation. You can enforce data types or require a variable to exist. If a config is missing, the app will error out at startup instead of crashing unexpectedly while processing customer data.

Professional Dynaconf Implementation Guide

1. Installation

Installing Dynaconf via pip is simple. I recommend installing YAML and TOML support to optimize the experience.

pip install dynaconf[yaml,toml]

2. Initializing Project Structure

Don’t create files manually. Use the init command to let Dynaconf set up the standard structure for you:

dynaconf init -v TOML

This command creates settings.toml, .secrets.toml, and config.py. Your directory structure will look like this:

.
├── config.py
├── settings.toml
├── .secrets.toml
└── main.py

3. Setting Up Multi-Environment Config Files

In the settings.toml file, you can define default values and override them for specific environments.

[default]
app_name = "My Automation Tool"
debug = false

[development]
debug = true
database = "localhost"

[production]
database = "prod-db-server"
port = 5432

In the config.py file, we initialize the settings object as follows:

from dynaconf import Dynaconf

settings = Dynaconf(
    envvar_prefix="DYNACONF",
    settings_files=['settings.toml', '.secrets.toml'],
    environments=True,
    load_dotenv=True,
)

4. Usage in Code

Now, you just need to import settings into any file to use it. Dynaconf will automatically detect which environment you are in and retrieve the corresponding values.

from config import settings

print(f"Running: {settings.app_name}")
if settings.get("debug"):
    print("Debug mode is ON")

Flexible Environment Switching

This is the “killer feature” when deploying. To switch from Dev to Production, you don’t need to change a single line of code. Just set an environment variable on the server:

export ENV_FOR_DYNACONF=production
python main.py

Immediately, Dynaconf will ignore values in [development] and load configurations from [production].

Upgrade: Hot-Reloading Config with Redis

For distributed systems, changing config without restarting the app (Hot reload) is a major advantage. Dynaconf supports direct connections to Redis for dynamic configuration.

First, install the redis client:

pip install redis

Then, update config.py to connect to your Redis server:

settings = Dynaconf(
    # ... keep existing configs ...
    redis_enabled=True,
    redis_host="localhost",
    redis_port=6379,
)

When you change a value on Redis, the application will update with the new configuration without needing a restart.

Real-World Experience Using Dynaconf

After many projects, I’ve gathered 3 important notes to save you time:

  1. Security is Priority #1: Always add .secrets.toml to .gitignore. Never commit this file to Git.
  2. Use Validators: Use settings.validators.register to check for mandatory variables like DATABASE_URL. If missing, the app will stop immediately with a clear error message.
  3. Understand Priority Rules: The rule is: System environment variables > .secrets file > settings file. If you set export DYNACONF_PORT=8080, it will override any port value in the config files.

Since switching to Dynaconf, I’ve saved 30% of my time whenever I need to set up a new environment. I hope this article helps you standardize config management for your Python projects.

Share:
Tags: